Merge main into refactor modules checkpoint

Keep the refactor_modules branch current with main and preserve the real Obsidian E2E progress.

Build and unit tests pass; real Obsidian hidden-file create/delete now pass, while hidden JSON conflict round-trip still needs follow-up.
This commit is contained in:
vorotamoroz
2026-07-03 08:35:48 +00:00
451 changed files with 5429 additions and 502 deletions
+118
View File
@@ -0,0 +1,118 @@
# Real Obsidian E2E Runner
This directory contains the experimental real Obsidian end-to-end runner.
The current smoke runner verifies only the launch path:
1. create a temporary vault,
2. install the built Self-hosted LiveSync plug-in artifacts,
3. launch real Obsidian,
4. open the temporary vault through `obsidian-cli`,
5. enable Obsidian community plug-ins for the temporary app profile,
6. reload Self-hosted LiveSync through `obsidian-cli`,
7. verify through `obsidian-cli eval` that the plug-in is loaded,
8. optionally drive a real vault or CouchDB workflow through Obsidian's own API,
9. terminate Obsidian and remove the temporary vault.
The runner does not require Self-hosted LiveSync to expose an E2E-only bridge. Readiness is checked from outside the plug-in through Obsidian's own CLI.
Obsidian 1.12 stores the global community plug-in switch outside `.obsidian/community-plugins.json`. The smoke runner enables it through `app.plugins.setEnable(true)` after the vault window is available.
Future workflows should use `startObsidianLiveSyncSession()` from `runner/session.ts` rather than repeating the launch and plug-in readiness sequence.
Each test vault uses an isolated Obsidian profile. The runner creates temporary directories for `HOME`, `XDG_CONFIG_HOME`, `XDG_CACHE_HOME`, `XDG_DATA_HOME`, and Electron `--user-data-dir`, writes the vault registry into those directories, pre-seeds the temporary Chromium local storage so community plug-ins are trusted for that generated vault ID, and passes the same environment to `obsidian-cli`. This is intended to keep real Obsidian E2E runs separate from a developer's daily Obsidian profile and vault registry.
## Local Setup
Set `OBSIDIAN_BINARY` when Obsidian is not installed in a standard location.
For an AppImage on Linux without FUSE, use the helper script:
```bash
npm run test:e2e:obsidian:install-appimage
```
The script downloads Obsidian `1.12.7` for the current architecture, stores it in `_testdata/obsidian`, and extracts it to `_testdata/obsidian/squashfs-root`. The runner checks `_testdata/obsidian/squashfs-root/obsidian` before the AppImage path.
These tests are intended for local verification, not the default CI gate. Reuse the installed Obsidian application, or reuse the extracted AppImage directory between local runs:
- set `OBSIDIAN_BINARY` to an installed Obsidian executable,
- keep `_testdata/obsidian/squashfs-root` after running the AppImage installer, or
- run `test:e2e:obsidian:install-appimage` again only when the local Obsidian version should change.
## Commands
```bash
npm run test:e2e:obsidian:install-appimage
npm run test:e2e:obsidian:discover
npm run test:e2e:obsidian:cli-help -- vaults verbose
npm run test:e2e:obsidian:smoke
npm run test:e2e:obsidian:vault-reflection
npm run test:e2e:obsidian:couchdb-upload
npm run test:e2e:obsidian:minio-upload
npm run test:e2e:obsidian:startup-scan
npm run test:e2e:obsidian:two-vault-sync
npm run test:e2e:obsidian:hidden-file-snippet-sync
npm run test:e2e:obsidian:customisation-sync
npm run test:e2e:obsidian:setting-markdown-export
npm run test:e2e:obsidian:local-suite
npm run test:e2e:obsidian:local-suite:services
```
`test:e2e:obsidian:local-suite` runs `npm run build`, discovery, smoke, vault reflection, CouchDB upload, Object Storage upload, startup scan, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export in sequence. Start the local CouchDB and MinIO fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run.
`test:e2e:obsidian:couchdb-upload` reuses the CouchDB variables from `.test.env` or the process environment. It expects a reachable CouchDB service, creates a unique database, configures Self-hosted LiveSync through `obsidian-cli eval`, creates a note in real Obsidian, commits the note into the local database, runs one-shot synchronisation, and verifies that the remote database contains both the metadata document and its chunk documents.
`test:e2e:obsidian:minio-upload` reuses the Object Storage variables from `.test.env` or the process environment. It expects a reachable S3-compatible service, configures Self-hosted LiveSync for Object Storage through `obsidian-cli eval`, creates a note in real Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written under a unique bucket prefix.
`test:e2e:obsidian:startup-scan` configures a temporary CouchDB database, stops Obsidian, writes a note directly into the vault, restarts Obsidian, and verifies from CouchDB that the boot-time scan picked up the offline file.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, rename, deletion, per-device target filters where one vault ignores a note that the other vault synchronises, and a separate encrypted round-trip with Path Obfuscation enabled. The optional Markdown conflict automatic merge check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`, but it is not part of the default local suite.
`test:e2e:obsidian:hidden-file-snippet-sync` runs a two-vault hidden file round-trip. It verifies creation and deletion of a real `.obsidian/snippets/*.css` file, automatic JSON conflict merging for a hidden file with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target patterns where one vault ignores a hidden file that the other vault synchronises.
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`.
Start the local fixtures first when they are not already running:
```bash
npm run test:docker-couchdb:start
npm run test:docker-s3:start
npm run test:e2e:obsidian:local-suite
```
Or let the wrapper manage both fixtures:
```bash
npm run test:e2e:obsidian:local-suite:services
```
Useful environment variables:
- `OBSIDIAN_BINARY`: explicit Obsidian executable path.
- `E2E_OBSIDIAN_VERSION`: Obsidian AppImage version for `test:e2e:obsidian:install-appimage`; default is `1.12.7`.
- `E2E_OBSIDIAN_APPIMAGE_ARCH`: AppImage architecture override, such as `arm64` or `x86_64`.
- `E2E_OBSIDIAN_APPIMAGE_URL`: explicit AppImage URL override.
- `E2E_OBSIDIAN_DOWNLOAD_DIR`: AppImage download and extraction directory; default is `_testdata/obsidian`.
- `E2E_OBSIDIAN_FORCE_DOWNLOAD=true`: re-download the AppImage even when it exists.
- `E2E_OBSIDIAN_SKIP_EXTRACT=true`: download the AppImage without extracting it.
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS`: smoke timeout in milliseconds.
- `E2E_OBSIDIAN_READY_TIMEOUT_MS`: plug-in readiness timeout in milliseconds.
- `E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS`: timeout for waiting until the vault-side Obsidian CLI exposes the plug-in catalogue.
- `E2E_OBSIDIAN_CLI_TIMEOUT_MS`: timeout for each `obsidian-cli` invocation.
- `E2E_OBSIDIAN_FILE_TIMEOUT_MS`: timeout for waiting until a note created through Obsidian's vault API is reflected to disk.
- `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready.
- `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database.
- `E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS`: timeout for waiting until CouchDB contains uploaded E2E documents.
- `E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS`: timeout for waiting until Object Storage contains uploaded E2E objects.
- `E2E_OBSIDIAN_KEEP_COUCHDB=true`: keep the temporary CouchDB database for inspection.
- `E2E_OBSIDIAN_KEEP_OBJECT_STORAGE=true`: keep the temporary Object Storage prefix for inspection.
- `E2E_OBSIDIAN_STARTUP_GRACE_MS`: early process-exit detection window in milliseconds.
- `E2E_OBSIDIAN_KEEP_VAULT=true`: keep the temporary vault for inspection.
- `E2E_OBSIDIAN_USE_XVFB=false`: disable automatic `xvfb-run` on headless Linux.
- `E2E_OBSIDIAN_USE_USER_DATA_DIR=false`: disable the isolated Electron `--user-data-dir` argument. This is not recommended for normal local testing.
- `E2E_OBSIDIAN_ARGS`: override the default Obsidian launch arguments.
On headless Linux, the runner automatically uses `/usr/bin/xvfb-run` when no `DISPLAY` or `WAYLAND_DISPLAY` is present.
+103
View File
@@ -0,0 +1,103 @@
import { spawn } from "node:child_process";
export type ObsidianCliResult = {
code: number | null;
signal: NodeJS.Signals | null;
stdout: string;
stderr: string;
};
function parseEvalJson(stdout: string): unknown {
const marker = "=> ";
const markerIndex = stdout.indexOf(marker);
const text = markerIndex >= 0 ? stdout.slice(markerIndex + marker.length) : stdout;
return JSON.parse(text.trim());
}
export async function runObsidianCli(
cliBinary: string,
args: string[],
env: NodeJS.ProcessEnv = process.env,
timeoutMs = Number(process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ?? 10000)
): Promise<ObsidianCliResult> {
return await new Promise((resolve, reject) => {
const child = spawn(cliBinary, args, {
stdio: ["ignore", "pipe", "pipe"],
env,
});
let stdout = "";
let stderr = "";
const timeout = setTimeout(() => {
child.kill("SIGKILL");
reject(new Error(`Obsidian CLI timed out: ${cliBinary} ${args.join(" ")}`));
}, timeoutMs);
child.stdout?.on("data", (chunk: Buffer) => {
stdout += chunk.toString();
});
child.stderr?.on("data", (chunk: Buffer) => {
stderr += chunk.toString();
});
child.on("error", (error) => {
clearTimeout(timeout);
reject(error);
});
child.on("exit", (code, signal) => {
clearTimeout(timeout);
resolve({ code, signal, stdout, stderr });
});
});
}
export async function openVaultWithObsidianCli(
cliBinary: string,
vaultPath: string,
env: NodeJS.ProcessEnv = process.env
): Promise<void> {
const result = await runObsidianCli(cliBinary, [`obsidian://open?path=${encodeURIComponent(vaultPath)}`], env);
if (result.code !== 0) {
throw new Error(
[
`Failed to open Obsidian vault through CLI. code=${result.code}, signal=${result.signal}`,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
export async function evalObsidianJson<T>(
cliBinary: string,
code: string,
env: NodeJS.ProcessEnv = process.env,
timeoutMs?: number
): Promise<T> {
const result = await runObsidianCli(cliBinary, ["eval", `code=${code}`], env, timeoutMs);
if (result.code !== 0) {
throw new Error(
[
`Failed to evaluate Obsidian JavaScript through CLI. code=${result.code}, signal=${result.signal}`,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
try {
return parseEvalJson(result.stdout) as T;
} catch (error) {
throw new Error(
[
`Failed to parse Obsidian CLI eval JSON. code=${result.code}, signal=${result.signal}`,
error instanceof Error ? `parse error: ${error.message}` : undefined,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
+174
View File
@@ -0,0 +1,174 @@
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
export type CouchDbConfig = {
uri: string;
username: string;
password: string;
dbPrefix: string;
};
export type CouchDbDocument = {
_id: string;
_rev?: string;
type?: string;
path?: string;
children?: string[];
[key: string]: unknown;
};
export type CouchDbAllDocsResponse = {
rows: Array<{
id: string;
key: string;
value: { rev: string; deleted?: boolean };
doc?: CouchDbDocument;
}>;
};
function parseEnvFile(content: string): Record<string, string> {
const entries = content
.split(/\r?\n/u)
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("#"))
.map((line) => {
const equalsAt = line.indexOf("=");
if (equalsAt < 0) {
return undefined;
}
const key = line.slice(0, equalsAt).trim();
const rawValue = line.slice(equalsAt + 1).trim();
const value = rawValue.replace(/^['"]|['"]$/gu, "");
return [key, value] as const;
})
.filter((entry): entry is readonly [string, string] => entry !== undefined);
return Object.fromEntries(entries);
}
function getEnvValue(values: Record<string, string | undefined>, ...keys: string[]): string {
for (const key of keys) {
const value = values[key]?.trim();
if (value) {
return value;
}
}
throw new Error(`Required CouchDB environment value is missing: ${keys.join(" or ")}`);
}
function authHeader(config: Pick<CouchDbConfig, "username" | "password">): string {
return `Basic ${Buffer.from(`${config.username}:${config.password}`).toString("base64")}`;
}
function databaseUrl(config: Pick<CouchDbConfig, "uri">, dbName: string, suffix = ""): string {
return `${config.uri.replace(/\/+$/u, "")}/${encodeURIComponent(dbName)}${suffix}`;
}
async function couchDbRequest(
config: Pick<CouchDbConfig, "uri" | "username" | "password">,
path: string,
init: RequestInit = {}
): Promise<Response> {
const response = await fetch(`${config.uri.replace(/\/+$/u, "")}${path}`, {
...init,
headers: {
authorization: authHeader(config),
...init.headers,
},
});
return response;
}
export async function loadCouchDbConfig(envFile = ".test.env"): Promise<CouchDbConfig> {
let fileValues: Record<string, string> = {};
try {
fileValues = parseEnvFile(await readFile(resolve(envFile), "utf-8"));
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
throw error;
}
}
const values = { ...fileValues, ...process.env };
return {
uri: getEnvValue(values, "COUCHDB_URI", "hostname").replace(/\/+$/u, ""),
username: getEnvValue(values, "COUCHDB_USER", "username"),
password: getEnvValue(values, "COUCHDB_PASSWORD", "password"),
dbPrefix: getEnvValue(values, "COUCHDB_DBNAME", "dbname"),
};
}
export function makeUniqueDatabaseName(prefix: string, label: string): string {
const safePrefix = prefix
.toLowerCase()
.replace(/[^a-z0-9_$()+/-]+/gu, "-")
.replace(/^-+/u, "")
.slice(0, 80);
const random = Math.random().toString(36).slice(2, 8);
return `${safePrefix || "livesync-e2e"}-${label}-${Date.now()}-${random}`;
}
export async function assertCouchDbReachable(config: CouchDbConfig): Promise<void> {
const response = await couchDbRequest(config, "/_up");
if (!response.ok) {
throw new Error(`CouchDB is not reachable at ${config.uri}. HTTP ${response.status}: ${await response.text()}`);
}
}
export async function createCouchDbDatabase(config: CouchDbConfig, dbName: string): Promise<void> {
const response = await fetch(databaseUrl(config, dbName), {
method: "PUT",
headers: { authorization: authHeader(config) },
});
if (!response.ok && response.status !== 412) {
throw new Error(
`Failed to create CouchDB database ${dbName}. HTTP ${response.status}: ${await response.text()}`
);
}
}
export async function deleteCouchDbDatabase(config: CouchDbConfig, dbName: string): Promise<void> {
const response = await fetch(databaseUrl(config, dbName), {
method: "DELETE",
headers: { authorization: authHeader(config) },
});
if (!response.ok && response.status !== 404) {
throw new Error(
`Failed to delete CouchDB database ${dbName}. HTTP ${response.status}: ${await response.text()}`
);
}
}
export async function fetchAllCouchDbDocs(config: CouchDbConfig, dbName: string): Promise<CouchDbAllDocsResponse> {
const response = await fetch(databaseUrl(config, dbName, "/_all_docs?include_docs=true"), {
headers: { authorization: authHeader(config) },
});
if (!response.ok) {
throw new Error(
`Failed to read CouchDB documents from ${dbName}. HTTP ${response.status}: ${await response.text()}`
);
}
return (await response.json()) as CouchDbAllDocsResponse;
}
export async function waitForCouchDbDocs(
config: CouchDbConfig,
dbName: string,
predicate: (docs: CouchDbDocument[]) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ?? 15000)
): Promise<CouchDbDocument[]> {
const deadline = Date.now() + timeoutMs;
let lastDocs: CouchDbDocument[] = [];
while (Date.now() < deadline) {
const response = await fetchAllCouchDbDocs(config, dbName);
lastDocs = response.rows.flatMap((row) => (row.doc ? [row.doc] : []));
if (predicate(lastDocs)) {
return lastDocs;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(
`Timed out waiting for CouchDB documents in ${dbName}. Last document IDs: ${lastDocs
.map((doc) => doc._id)
.join(", ")}`
);
}
+149
View File
@@ -0,0 +1,149 @@
import { accessSync, constants, existsSync } from "node:fs";
import { resolve } from "node:path";
import { platform } from "node:process";
export type ObsidianDiscoveryResult = {
binary?: string;
source?: string;
checked: string[];
};
const defaultCandidatesByPlatform: Record<NodeJS.Platform, string[]> = {
aix: [],
android: [],
darwin: [
"/Applications/Obsidian.app/Contents/MacOS/Obsidian",
"/Applications/Obsidian.app/Contents/MacOS/obsidian",
],
freebsd: [],
haiku: [],
linux: [
"_testdata/obsidian/squashfs-root/obsidian",
"_testdata/obsidian/squashfs-root/AppRun",
"_testdata/obsidian/Obsidian-1.12.7-arm64.AppImage",
"_testdata/obsidian/Obsidian-1.12.7-x86_64.AppImage",
"/usr/bin/obsidian",
"/usr/local/bin/obsidian",
"/snap/bin/obsidian",
"/opt/Obsidian/obsidian",
"/opt/obsidian/obsidian",
"/app/bin/obsidian",
],
openbsd: [],
sunos: [],
win32: ["C:\\Program Files\\Obsidian\\Obsidian.exe", "C:\\Program Files (x86)\\Obsidian\\Obsidian.exe"],
cygwin: [],
netbsd: [],
};
const defaultCliCandidatesByPlatform: Record<NodeJS.Platform, string[]> = {
aix: [],
android: [],
darwin: [
"/Applications/Obsidian.app/Contents/MacOS/obsidian-cli",
"/Applications/Obsidian.app/Contents/Resources/obsidian-cli",
],
freebsd: [],
haiku: [],
linux: [
"_testdata/obsidian/squashfs-root/obsidian-cli",
"/usr/bin/obsidian-cli",
"/usr/local/bin/obsidian-cli",
"/snap/bin/obsidian-cli",
"/opt/Obsidian/obsidian-cli",
"/opt/obsidian/obsidian-cli",
],
openbsd: [],
sunos: [],
win32: ["C:\\Program Files\\Obsidian\\obsidian-cli.exe", "C:\\Program Files (x86)\\Obsidian\\obsidian-cli.exe"],
cygwin: [],
netbsd: [],
};
function isUsableFile(path: string): boolean {
const resolvedPath = resolve(path);
if (!existsSync(resolvedPath)) {
return false;
}
if (platform === "win32") {
return true;
}
try {
accessSync(resolvedPath, constants.X_OK);
return true;
} catch {
return false;
}
}
export function discoverObsidianBinary(env: NodeJS.ProcessEnv = process.env): ObsidianDiscoveryResult {
const checked: string[] = [];
const envBinary = env.OBSIDIAN_BINARY?.trim();
if (envBinary) {
checked.push(envBinary);
if (isUsableFile(envBinary)) {
return {
binary: resolve(envBinary),
source: "OBSIDIAN_BINARY",
checked,
};
}
}
const candidates = defaultCandidatesByPlatform[platform] ?? [];
for (const candidate of candidates) {
checked.push(candidate);
if (isUsableFile(candidate)) {
return {
binary: resolve(candidate),
source: "default-path",
checked,
};
}
}
return { checked };
}
export function requireObsidianBinary(env: NodeJS.ProcessEnv = process.env): string {
const result = discoverObsidianBinary(env);
if (!result.binary) {
throw new Error(
[
"Could not find an Obsidian executable.",
"Set OBSIDIAN_BINARY to the installed Obsidian executable path.",
`Checked paths: ${result.checked.length > 0 ? result.checked.join(", ") : "(none)"}`,
].join("\n")
);
}
return result.binary;
}
export function discoverObsidianCli(env: NodeJS.ProcessEnv = process.env): ObsidianDiscoveryResult {
const checked: string[] = [];
const envBinary = env.OBSIDIAN_CLI?.trim();
if (envBinary) {
checked.push(envBinary);
if (isUsableFile(envBinary)) {
return {
binary: resolve(envBinary),
source: "OBSIDIAN_CLI",
checked,
};
}
}
const candidates = defaultCliCandidatesByPlatform[platform] ?? [];
for (const candidate of candidates) {
checked.push(candidate);
if (isUsableFile(candidate)) {
return {
binary: resolve(candidate),
source: "default-path",
checked,
};
}
}
return { checked };
}
+196
View File
@@ -0,0 +1,196 @@
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { once } from "node:events";
import { existsSync } from "node:fs";
import { dirname } from "node:path";
import { platform } from "node:process";
import { promisify } from "node:util";
export type ObsidianProcess = {
process: ChildProcess;
output: () => { stdout: string; stderr: string };
stop: () => Promise<void>;
};
export type LaunchObsidianOptions = {
binary: string;
vaultPath: string;
homePath?: string;
xdgConfigPath?: string;
xdgCachePath?: string;
xdgDataPath?: string;
userDataPath?: string;
startupGraceMs?: number;
};
const execFileAsync = promisify(execFile);
function splitArgs(args: string): string[] {
return args.split(" ").filter((arg) => arg.length > 0);
}
function launchArgs(options: LaunchObsidianOptions): string[] {
const explicitArgs = process.env.E2E_OBSIDIAN_ARGS;
if (explicitArgs) {
return splitArgs(explicitArgs);
}
return [
"--no-sandbox",
"--disable-gpu",
"--disable-software-rasterizer",
...(process.env.E2E_OBSIDIAN_USE_USER_DATA_DIR !== "false" && options.userDataPath
? [`--user-data-dir=${options.userDataPath}`]
: []),
...(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT
? [`--remote-debugging-port=${process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT}`]
: []),
`obsidian://open?path=${encodeURIComponent(options.vaultPath)}`,
];
}
function shouldUseXvfb(): boolean {
if (process.env.E2E_OBSIDIAN_USE_XVFB === "false") {
return false;
}
if (process.env.DISPLAY || process.env.WAYLAND_DISPLAY) {
return false;
}
return platform === "linux" && existsSync("/usr/bin/xvfb-run");
}
async function listChildPids(pid: number): Promise<number[]> {
if (platform === "win32") {
return [];
}
const { stdout } = await execFileAsync("ps", ["-o", "pid=", "--ppid", String(pid)]).catch(() => ({
stdout: "",
}));
const directChildren = stdout
.split("\n")
.map((line) => Number(line.trim()))
.filter((childPid) => Number.isInteger(childPid) && childPid > 0);
const descendants = await Promise.all(directChildren.map((childPid) => listChildPids(childPid)));
return [...directChildren, ...descendants.flat()];
}
async function killPids(pids: number[], signal: NodeJS.Signals): Promise<void> {
for (const pid of pids) {
if (pid === process.pid) {
continue;
}
try {
process.kill(pid, signal);
} catch {
// The process may have exited between discovery and signalling.
}
}
}
async function waitForExit(exitPromise: Promise<unknown>, timeoutMs: number): Promise<"exited" | "timeout"> {
const stopTimer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), timeoutMs);
});
const stopResult = await Promise.race([exitPromise.then(() => "exited" as const), stopTimer]);
return stopResult;
}
export async function cleanupStaleObsidianE2EProcesses(): Promise<void> {
if (process.env.E2E_OBSIDIAN_CLEANUP_STALE_PROCESSES === "false" || platform === "win32") {
return;
}
const { stdout } = await execFileAsync("pgrep", ["-f", "obsidian-livesync-e2e-state"]).catch(() => ({
stdout: "",
}));
const pids = stdout
.split("\n")
.map((line) => Number(line.trim()))
.filter((pid) => Number.isInteger(pid) && pid > 0 && pid !== process.pid);
if (pids.length === 0) {
return;
}
await killPids(pids, "SIGTERM");
await new Promise((resolve) => setTimeout(resolve, 1000));
await killPids(pids, "SIGKILL");
}
export async function launchObsidian(options: LaunchObsidianOptions): Promise<ObsidianProcess> {
await cleanupStaleObsidianE2EProcesses();
const startupGraceMs = options.startupGraceMs ?? 1000;
const args = launchArgs(options);
const useXvfb = shouldUseXvfb();
const command = useXvfb ? "/usr/bin/xvfb-run" : options.binary;
const commandArgs = useXvfb ? ["-a", options.binary, ...args] : args;
const child = spawn(command, commandArgs, {
cwd: dirname(options.binary),
detached: true,
stdio: ["ignore", "pipe", "pipe"],
env: {
...process.env,
...(options.homePath ? { HOME: options.homePath } : {}),
...(options.xdgConfigPath ? { XDG_CONFIG_HOME: options.xdgConfigPath } : {}),
...(options.xdgCachePath ? { XDG_CACHE_HOME: options.xdgCachePath } : {}),
...(options.xdgDataPath ? { XDG_DATA_HOME: options.xdgDataPath } : {}),
OBSIDIAN_DISABLE_GPU: process.env.OBSIDIAN_DISABLE_GPU ?? "1",
},
});
let stderr = "";
let stdout = "";
child.stderr?.on("data", (chunk: Buffer) => {
stderr += chunk.toString();
});
child.stdout?.on("data", (chunk: Buffer) => {
stdout += chunk.toString();
});
const exitPromise = once(child, "exit").then(([code, signal]) => ({ code, signal }));
const timer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), startupGraceMs);
});
const firstResult = await Promise.race([exitPromise, timer]);
if (firstResult !== "timeout") {
throw new Error(
[
`Obsidian exited before the smoke timeout. code=${firstResult.code}, signal=${firstResult.signal}`,
stdout ? `stdout:\n${stdout}` : undefined,
stderr ? `stderr:\n${stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
return {
process: child,
output: () => ({ stdout, stderr }),
stop: async () => {
if (child.exitCode !== null || child.signalCode !== null) {
return;
}
const descendantPids = child.pid ? await listChildPids(child.pid) : [];
if (child.pid) {
try {
process.kill(-child.pid, "SIGTERM");
} catch {
child.kill("SIGTERM");
}
} else {
child.kill("SIGTERM");
}
await killPids(descendantPids.reverse(), "SIGTERM");
const stopResult = await waitForExit(exitPromise, 5000);
if (stopResult === "timeout") {
if (child.pid) {
try {
process.kill(-child.pid, "SIGKILL");
} catch {
child.kill("SIGKILL");
}
} else {
child.kill("SIGKILL");
}
await killPids(descendantPids, "SIGKILL");
await exitPromise;
}
},
};
}
@@ -0,0 +1,247 @@
import { evalObsidianJson } from "./cli.ts";
import type { CouchDbConfig } from "./couchdb.ts";
import type { ObjectStorageConfig } from "./objectStorage.ts";
export type ConfiguredSettings = {
isConfigured: boolean;
liveSync: boolean;
syncOnStart: boolean;
syncOnSave: boolean;
remoteType: string;
couchDB_URI: string;
couchDB_DBNAME: string;
endpoint?: string;
bucket?: string;
bucketPrefix?: string;
};
export type CoreReadiness = {
databaseReady: boolean;
appReady: boolean;
};
export type LocalDatabaseEntry = {
id: string;
rev: string;
path: string;
type: string;
children: string[];
};
function e2ePreferredSettingsSource(): string[] {
return [
"liveSync:false,",
"syncOnStart:false,",
"syncOnSave:false,",
"usePluginSync:false,",
"usePluginSyncV2:true,",
"useEden:false,",
"customChunkSize:60,",
"sendChunksBulk:false,",
"sendChunksBulkMaxSize:1,",
"chunkSplitterVersion:'v3-rabin-karp',",
"readChunksOnline:true,",
"disableCheckingConfigMismatch:false,",
"enableCompression:false,",
"hashAlg:'xxhash64',",
"handleFilenameCaseSensitive:false,",
"doNotUseFixedRevisionForChunks:true,",
"E2EEAlgorithm:'v2',",
"doctorProcessedVersion:'0.25.27',",
"isConfigured:true,",
];
}
export function assertEqual(actual: unknown, expected: unknown, message: string): void {
if (actual !== expected) {
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
}
}
export async function configureCouchDb(
cliBinary: string,
env: NodeJS.ProcessEnv,
settings: Pick<CouchDbConfig, "uri" | "username" | "password"> & { dbName: string },
overrides: Record<string, unknown> = {}
): Promise<ConfiguredSettings> {
return await evalObsidianJson<ConfiguredSettings>(
cliBinary,
[
"(async()=>{",
"const plugin=app.plugins.plugins['obsidian-livesync'];",
"const core=plugin.core;",
"const nextSettings={",
`couchDB_URI:${JSON.stringify(settings.uri)},`,
`couchDB_USER:${JSON.stringify(settings.username)},`,
`couchDB_PASSWORD:${JSON.stringify(settings.password)},`,
`couchDB_DBNAME:${JSON.stringify(settings.dbName)},`,
"remoteType:'',",
...e2ePreferredSettingsSource(),
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
"};",
"await core.services.setting.applyExternalSettings(nextSettings,true);",
"await core.services.control.applySettings();",
"const current=core.services.setting.currentSettings();",
"return JSON.stringify({",
"isConfigured:current.isConfigured,",
"liveSync:current.liveSync,",
"syncOnStart:current.syncOnStart,",
"syncOnSave:current.syncOnSave,",
"remoteType:current.remoteType,",
"couchDB_URI:current.couchDB_URI,",
"couchDB_DBNAME:current.couchDB_DBNAME,",
"});",
"})()",
].join(""),
env
);
}
export async function configureObjectStorage(
cliBinary: string,
env: NodeJS.ProcessEnv,
settings: ObjectStorageConfig & { bucketPrefix: string },
overrides: Record<string, unknown> = {}
): Promise<ConfiguredSettings> {
return await evalObsidianJson<ConfiguredSettings>(
cliBinary,
[
"(async()=>{",
"const plugin=app.plugins.plugins['obsidian-livesync'];",
"const core=plugin.core;",
"const nextSettings={",
"remoteType:'MINIO',",
`endpoint:${JSON.stringify(settings.endpoint)},`,
`accessKey:${JSON.stringify(settings.accessKey)},`,
`secretKey:${JSON.stringify(settings.secretKey)},`,
`bucket:${JSON.stringify(settings.bucket)},`,
`region:${JSON.stringify(settings.region)},`,
`forcePathStyle:${JSON.stringify(settings.forcePathStyle)},`,
`bucketPrefix:${JSON.stringify(settings.bucketPrefix)},`,
"bucketCustomHeaders:'',",
...e2ePreferredSettingsSource(),
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
"};",
"await core.services.setting.applyExternalSettings(nextSettings,true);",
"await core.services.control.applySettings();",
"const current=core.services.setting.currentSettings();",
"return JSON.stringify({",
"isConfigured:current.isConfigured,",
"liveSync:current.liveSync,",
"syncOnStart:current.syncOnStart,",
"syncOnSave:current.syncOnSave,",
"remoteType:current.remoteType,",
"couchDB_URI:current.couchDB_URI,",
"couchDB_DBNAME:current.couchDB_DBNAME,",
"endpoint:current.endpoint,",
"bucket:current.bucket,",
"bucketPrefix:current.bucketPrefix,",
"});",
"})()",
].join(""),
env
);
}
export async function waitForLiveSyncCoreReady(
cliBinary: string,
env: NodeJS.ProcessEnv,
timeoutMs = Number(process.env.E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS ?? 20000)
): Promise<CoreReadiness> {
const deadline = Date.now() + timeoutMs;
let lastReadiness: CoreReadiness | undefined;
while (Date.now() < deadline) {
lastReadiness = await evalObsidianJson<CoreReadiness>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"return JSON.stringify({",
"databaseReady:core.services.database.isDatabaseReady(),",
"appReady:core.services.appLifecycle.isReady(),",
"});",
"})()",
].join(""),
env
);
if (lastReadiness.databaseReady && lastReadiness.appReady) {
return lastReadiness;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Self-hosted LiveSync core readiness: ${JSON.stringify(lastReadiness)}`);
}
export async function prepareRemote(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const settings=core.services.setting.currentSettings();",
"const replicator=core.services.replicator.getActiveReplicator();",
"await replicator.tryCreateRemoteDatabase(settings);",
"await replicator.markRemoteResolved(settings);",
"const status=await replicator.getRemoteStatus(settings);",
"return JSON.stringify({status});",
"})()",
].join(""),
env
);
}
export async function pushLocalChanges(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"await core.services.fileProcessing.commitPendingFileEvents();",
"const result=await core.services.replication.replicate(true);",
"return JSON.stringify({result:!!result});",
"})()",
].join(""),
env
);
}
export async function waitForLocalDatabaseEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
options: { hidden?: boolean; timeoutMs?: number } = {}
): Promise<LocalDatabaseEntry> {
const timeoutMs = options.timeoutMs ?? Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000);
return await evalObsidianJson<LocalDatabaseEntry>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const hidden=${JSON.stringify(options.hidden === true)};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const deadline=Date.now()+timeoutMs;",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"while(Date.now()<deadline){",
"await core.services.fileProcessing.commitPendingFileEvents();",
"const dbPath=hidden?`i:${path}`:path;",
"entry=await core.localDatabase.getDBEntry(dbPath,undefined,false,true).catch(()=>false);",
"if(!entry||!entry._id){",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"entry=rows.map((row)=>row.doc).find((doc)=>doc&&(",
"doc._id===dbPath||doc._id===path||doc.path===dbPath||doc.path===path||",
"(typeof doc.path==='string'&&doc.path.endsWith(path))||",
"(typeof doc._id==='string'&&doc._id.endsWith(path))",
"))||false;",
"}",
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
"await sleep(250);",
"}",
"if(!entry||!entry._id) throw new Error(`Timed out waiting for local database entry: ${path}`);",
"return JSON.stringify({id:entry._id,rev:entry._rev,path:entry.path,type:entry.type,children:entry.children||[]});",
"})()",
].join(""),
env
);
}
+142
View File
@@ -0,0 +1,142 @@
import {
CreateBucketCommand,
DeleteObjectsCommand,
ListObjectsV2Command,
S3Client,
type _Object,
} from "@aws-sdk/client-s3";
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
export type ObjectStorageConfig = {
endpoint: string;
accessKey: string;
secretKey: string;
bucket: string;
region: string;
forcePathStyle: boolean;
};
function parseEnvFile(content: string): Record<string, string> {
const entries = content
.split(/\r?\n/u)
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("#"))
.map((line) => {
const equalsAt = line.indexOf("=");
if (equalsAt < 0) {
return undefined;
}
const key = line.slice(0, equalsAt).trim();
const rawValue = line.slice(equalsAt + 1).trim();
const value = rawValue.replace(/^['"]|['"]$/gu, "");
return [key, value] as const;
})
.filter((entry): entry is readonly [string, string] => entry !== undefined);
return Object.fromEntries(entries);
}
function getEnvValue(values: Record<string, string | undefined>, ...keys: string[]): string {
for (const key of keys) {
const value = values[key]?.trim();
if (value) {
return value;
}
}
throw new Error(`Required Object Storage environment value is missing: ${keys.join(" or ")}`);
}
export async function loadObjectStorageConfig(envFile = ".test.env"): Promise<ObjectStorageConfig> {
let fileValues: Record<string, string> = {};
try {
fileValues = parseEnvFile(await readFile(resolve(envFile), "utf-8"));
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
throw error;
}
}
const values = { ...fileValues, ...process.env };
return {
endpoint: getEnvValue(values, "MINIO_ENDPOINT", "minioEndpoint").replace(/\/+$/u, ""),
accessKey: getEnvValue(values, "MINIO_ACCESS_KEY", "accessKey"),
secretKey: getEnvValue(values, "MINIO_SECRET_KEY", "secretKey"),
bucket: getEnvValue(values, "MINIO_BUCKET", "bucketName"),
region: values.MINIO_REGION?.trim() || values.region?.trim() || "us-east-1",
forcePathStyle: values.MINIO_FORCE_PATH_STYLE?.trim() !== "false",
};
}
export function makeUniqueBucketPrefix(label: string): string {
const random = Math.random().toString(36).slice(2, 8);
return `obsidian-e2e/${label}-${Date.now()}-${random}/`;
}
export function createObjectStorageClient(config: ObjectStorageConfig): S3Client {
return new S3Client({
endpoint: config.endpoint,
region: config.region,
forcePathStyle: config.forcePathStyle,
credentials: {
accessKeyId: config.accessKey,
secretAccessKey: config.secretKey,
},
});
}
export async function ensureObjectStorageBucket(config: ObjectStorageConfig): Promise<void> {
const client = createObjectStorageClient(config);
try {
await client.send(new CreateBucketCommand({ Bucket: config.bucket }));
} catch (error) {
const name = (error as { name?: string }).name;
if (name !== "BucketAlreadyOwnedByYou" && name !== "BucketAlreadyExists") {
throw error;
}
} finally {
client.destroy();
}
}
export async function listObjectStorageObjects(config: ObjectStorageConfig, prefix: string): Promise<_Object[]> {
const client = createObjectStorageClient(config);
try {
const objects: _Object[] = [];
let continuationToken: string | undefined;
do {
const response = await client.send(
new ListObjectsV2Command({
Bucket: config.bucket,
Prefix: prefix,
ContinuationToken: continuationToken,
})
);
objects.push(...(response.Contents ?? []));
continuationToken = response.NextContinuationToken;
} while (continuationToken);
return objects;
} finally {
client.destroy();
}
}
export async function deleteObjectStoragePrefix(config: ObjectStorageConfig, prefix: string): Promise<void> {
const client = createObjectStorageClient(config);
try {
const objects = await listObjectStorageObjects(config, prefix);
const keys = objects.flatMap((object) => (object.Key ? [{ Key: object.Key }] : []));
for (let index = 0; index < keys.length; index += 1000) {
await client.send(
new DeleteObjectsCommand({
Bucket: config.bucket,
Delete: {
Objects: keys.slice(index, index + 1000),
Quiet: true,
},
})
);
}
} finally {
client.destroy();
}
}
@@ -0,0 +1,39 @@
import { copyFile, mkdir, writeFile } from "node:fs/promises";
import { existsSync } from "node:fs";
import { join, resolve } from "node:path";
export type PluginInstallResult = {
pluginDir: string;
copied: string[];
};
const pluginId = "obsidian-livesync";
export async function installBuiltPlugin(vaultPath: string, rootDir = process.cwd()): Promise<PluginInstallResult> {
const pluginDir = join(vaultPath, ".obsidian", "plugins", pluginId);
const copied: string[] = [];
await mkdir(pluginDir, { recursive: true });
const requiredArtifacts = ["main.js", "manifest.json"];
for (const artifact of requiredArtifacts) {
const source = resolve(rootDir, artifact);
if (!existsSync(source)) {
throw new Error(`Required plug-in artifact is missing: ${source}`);
}
await copyFile(source, join(pluginDir, artifact));
copied.push(artifact);
}
const optionalArtifacts = ["styles.css"];
for (const artifact of optionalArtifacts) {
const source = resolve(rootDir, artifact);
if (!existsSync(source)) {
continue;
}
await copyFile(source, join(pluginDir, artifact));
copied.push(artifact);
}
await writeFile(join(vaultPath, ".obsidian", "community-plugins.json"), JSON.stringify([pluginId], null, 4));
return { pluginDir, copied };
}
+48
View File
@@ -0,0 +1,48 @@
import { evalObsidianJson } from "./cli.ts";
export type PluginReadiness = {
status: "ready" | "pending";
pluginId: string;
pluginVersion: string;
vaultName: string;
enabled?: boolean;
pluginKeys?: string[];
loadingPluginId?: string;
};
export async function waitForPluginReady(
cliBinary: string,
env: NodeJS.ProcessEnv,
timeoutMs = Number(process.env.E2E_OBSIDIAN_READY_TIMEOUT_MS ?? 20000)
): Promise<PluginReadiness> {
const deadline = Date.now() + timeoutMs;
let lastOutput = "";
while (Date.now() < deadline) {
try {
const readiness = await evalObsidianJson<PluginReadiness>(
cliBinary,
[
"(async()=>JSON.stringify({",
"status:!!app.plugins.plugins['obsidian-livesync']?'ready':'pending',",
"pluginId:'obsidian-livesync',",
"pluginVersion:app.plugins.manifests['obsidian-livesync']?.version,",
"vaultName:app.vault.getName(),",
"enabled:app.plugins.enabledPlugins?.has?.('obsidian-livesync'),",
"pluginKeys:Object.keys(app.plugins.plugins),",
"loadingPluginId:app.plugins.loadingPluginId",
"}))()",
].join(""),
env
);
lastOutput = JSON.stringify(readiness);
if (readiness.status === "ready") {
return readiness as PluginReadiness & { status: "ready" };
}
} catch (error) {
lastOutput = error instanceof Error ? error.message : String(error);
// Keep polling until Obsidian exposes the vault-side CLI and plug-in state.
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Self-hosted LiveSync readiness through Obsidian CLI.\n${lastOutput}`);
}
+188
View File
@@ -0,0 +1,188 @@
import { evalObsidianJson, openVaultWithObsidianCli, runObsidianCli } from "./cli.ts";
import { launchObsidian, type ObsidianProcess } from "./launch.ts";
import { installBuiltPlugin, type PluginInstallResult } from "./pluginInstaller.ts";
import { waitForPluginReady, type PluginReadiness } from "./readiness.ts";
import type { TemporaryVault } from "./vault.ts";
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, trustVaultIfPrompted } from "./ui.ts";
export type ObsidianLiveSyncSession = {
app: ObsidianProcess;
cliEnv: NodeJS.ProcessEnv;
install: PluginInstallResult;
readiness: PluginReadiness;
};
export type StartObsidianLiveSyncSessionOptions = {
binary: string;
cliBinary: string;
vault: TemporaryVault;
startupGraceMs?: number;
};
async function waitForPluginCatalogue(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS ?? 60000);
let lastOutput = "";
while (Date.now() < deadline) {
try {
const result = await evalObsidianJson<{ hasLiveSync: boolean }>(
cliBinary,
["JSON.stringify({", "hasLiveSync:!!app.plugins?.manifests?.['obsidian-livesync']", "})"].join(""),
env
);
lastOutput = JSON.stringify(result);
if (result.hasLiveSync) {
return;
}
} catch (error) {
lastOutput = error instanceof Error ? error.message : String(error);
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Obsidian plug-in catalogue through CLI.\n${lastOutput}`);
}
async function enableCommunityPlugins(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
const result = await runObsidianCli(cliBinary, ["eval", "code=(async()=>app.plugins.setEnable(true))()"], env);
if (result.code !== 0 || result.stdout.includes("Error:")) {
throw new Error(
[
`Failed to enable Obsidian community plug-ins through CLI. code=${result.code}, signal=${result.signal}`,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
async function reloadLiveSyncPlugin(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
let enableState: { enabled?: boolean; loaded?: boolean; pluginKeys?: string[] };
try {
enableState = await evalObsidianJson<{ enabled?: boolean; loaded?: boolean; pluginKeys?: string[] }>(
cliBinary,
[
"(async()=>{",
"await app.plugins.setEnable(true);",
"await app.plugins.enablePlugin('obsidian-livesync');",
"return JSON.stringify({",
"enabled:app.plugins.enabledPlugins?.has?.('obsidian-livesync'),",
"loaded:!!app.plugins.plugins['obsidian-livesync'],",
"pluginKeys:Object.keys(app.plugins.plugins)",
"});",
"})()",
].join(""),
env
);
} catch (error) {
throw new Error(
[
"Failed to enable Self-hosted LiveSync through Obsidian CLI.",
error instanceof Error ? error.message : String(error),
]
.filter(Boolean)
.join("\n")
);
}
if (!enableState.enabled) {
throw new Error(
[
`Failed to mark Self-hosted LiveSync enabled through Obsidian CLI: ${JSON.stringify(enableState)}`,
]
.filter(Boolean)
.join("\n")
);
}
}
async function loadLiveSyncPlugin(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
try {
await evalObsidianJson(
cliBinary,
[
"(async()=>{",
"if(!app.plugins.plugins['obsidian-livesync']){",
"await app.plugins.loadPlugin('obsidian-livesync');",
"}",
"return JSON.stringify({",
"loaded:!!app.plugins.plugins['obsidian-livesync'],",
"pluginKeys:Object.keys(app.plugins.plugins)",
"});",
"})()",
].join(""),
env
);
} catch (error) {
throw new Error(
[
"Failed to load Self-hosted LiveSync through Obsidian CLI.",
error instanceof Error ? error.message : String(error),
]
.filter(Boolean)
.join("\n")
);
}
}
export async function startObsidianLiveSyncSession(
options: StartObsidianLiveSyncSessionOptions
): Promise<ObsidianLiveSyncSession> {
const install = await installBuiltPlugin(options.vault.path);
const remoteDebuggingPort = obsidianRemoteDebuggingPort();
let app = await launchObsidian({
binary: options.binary,
vaultPath: options.vault.path,
homePath: options.vault.homePath,
xdgConfigPath: options.vault.xdgConfigPath,
xdgCachePath: options.vault.xdgCachePath,
xdgDataPath: options.vault.xdgDataPath,
userDataPath: options.vault.userDataPath,
startupGraceMs: options.startupGraceMs,
});
const cliEnv = {
...process.env,
HOME: options.vault.homePath,
XDG_CONFIG_HOME: options.vault.xdgConfigPath,
XDG_CACHE_HOME: options.vault.xdgCachePath,
XDG_DATA_HOME: options.vault.xdgDataPath,
};
try {
await preseedTrustedVaultState(remoteDebuggingPort, options.vault.id);
await openVaultWithObsidianCli(options.cliBinary, options.vault.path, cliEnv);
await trustVaultIfPrompted(remoteDebuggingPort);
await waitForPluginCatalogue(options.cliBinary, cliEnv);
await enableCommunityPlugins(options.cliBinary, cliEnv);
await reloadLiveSyncPlugin(options.cliBinary, cliEnv);
await app.stop();
app = await launchObsidian({
binary: options.binary,
vaultPath: options.vault.path,
homePath: options.vault.homePath,
xdgConfigPath: options.vault.xdgConfigPath,
xdgCachePath: options.vault.xdgCachePath,
xdgDataPath: options.vault.xdgDataPath,
userDataPath: options.vault.userDataPath,
startupGraceMs: options.startupGraceMs,
});
await preseedTrustedVaultState(remoteDebuggingPort, options.vault.id);
await openVaultWithObsidianCli(options.cliBinary, options.vault.path, cliEnv);
await trustVaultIfPrompted(remoteDebuggingPort);
await waitForPluginCatalogue(options.cliBinary, cliEnv);
await loadLiveSyncPlugin(options.cliBinary, cliEnv);
const readiness = await waitForPluginReady(options.cliBinary, cliEnv);
return { app, cliEnv, install, readiness };
} catch (error) {
const output = app.output();
await app.stop();
throw new Error(
[
error instanceof Error ? error.message : String(error),
output.stdout ? `Obsidian stdout:\n${output.stdout}` : undefined,
output.stderr ? `Obsidian stderr:\n${output.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
+83
View File
@@ -0,0 +1,83 @@
import { chromium, type Page } from "playwright";
export function obsidianRemoteDebuggingPort(): number {
const port = Number(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT ?? 9222);
process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT = String(port);
return port;
}
async function waitForCdp(port: number): Promise<void> {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CDP_TIMEOUT_MS ?? 30000);
while (Date.now() < deadline) {
try {
const response = await fetch(`http://127.0.0.1:${port}/json/version`);
if (response.ok) {
return;
}
} catch {
// Keep polling until Obsidian exposes the debugging endpoint.
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Obsidian DevTools endpoint on port ${port}`);
}
export async function withObsidianPage<T>(port: number, operation: (page: Page) => Promise<T>): Promise<T> {
await waitForCdp(port);
const browser = await chromium.connectOverCDP(`http://127.0.0.1:${port}`);
try {
const context = browser.contexts()[0];
const page = context.pages()[0] ?? (await context.waitForEvent("page", { timeout: 10000 }));
return await operation(page);
} finally {
await browser.close();
}
}
export async function preseedTrustedVaultState(port: number, vaultId: string): Promise<void> {
await withObsidianPage(port, async (page) => {
await page.evaluate((id) => {
localStorage.setItem(`enable-plugin-${id}`, "true");
}, vaultId);
await page.reload({ waitUntil: "domcontentloaded", timeout: 10000 }).catch(() => undefined);
await page.waitForTimeout(1000);
});
}
export async function trustVaultIfPrompted(port: number): Promise<void> {
await withObsidianPage(port, async (page) => {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_TRUST_PROMPT_TIMEOUT_MS ?? 30000);
while (Date.now() < deadline) {
const yesButton = page.getByRole("button", { name: "Yes" });
if (await yesButton.isVisible({ timeout: 1000 }).catch(() => false)) {
await yesButton.click();
await page.waitForTimeout(500);
continue;
}
const trustButton = page.getByText("Trust author and enable plugins");
if (await trustButton.isVisible({ timeout: 1000 }).catch(() => false)) {
await trustButton.click();
await page.waitForTimeout(500);
continue;
}
const workspace = page.locator(".workspace");
if (await workspace.isVisible({ timeout: 1000 }).catch(() => false)) {
return;
}
}
});
}
export async function clickJsonResolveOption(port: number, mode: "AB" | "BA"): Promise<void> {
await withObsidianPage(port, async (page) => {
const option = page.locator(`label:has(input[name="disp"][value="${mode}"])`);
await option.click({ timeout: 10000 });
const checked = await page.locator(`input[name="disp"][value="${mode}"]`).isChecked({ timeout: 10000 });
if (!checked) {
throw new Error(`JSON Resolve option was not selected: ${mode}`);
}
await page.getByRole("button", { name: "Apply" }).click({ timeout: 10000 });
});
}
+94
View File
@@ -0,0 +1,94 @@
import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { tmpdir } from "node:os";
export type TemporaryVault = {
path: string;
name: string;
id: string;
homePath: string;
xdgConfigPath: string;
xdgCachePath: string;
xdgDataPath: string;
userDataPath: string;
dispose: () => Promise<void>;
};
export async function createTemporaryVault(prefix = "obsidian-livesync-e2e-"): Promise<TemporaryVault> {
const vaultPath = await mkdtemp(join(tmpdir(), prefix));
const statePath = await mkdtemp(join(tmpdir(), `${prefix}state-`));
const name = vaultPath.split(/[\\/]/).pop() ?? "obsidian-livesync-e2e";
await mkdir(join(vaultPath, ".obsidian"), { recursive: true });
const homePath = join(statePath, "home");
const xdgConfigPath = join(statePath, "xdg-config");
const xdgCachePath = join(statePath, "xdg-cache");
const xdgDataPath = join(statePath, "xdg-data");
const userDataPath = join(statePath, "user-data");
const id = `livesync-e2e-${Date.now()}`;
await mkdir(homePath, { recursive: true });
await mkdir(xdgConfigPath, { recursive: true });
await mkdir(xdgCachePath, { recursive: true });
await mkdir(xdgDataPath, { recursive: true });
await mkdir(userDataPath, { recursive: true });
await writeFile(
join(vaultPath, ".obsidian", "app.json"),
JSON.stringify({ legacyEditor: false, safeMode: false }, null, 4)
);
await writeFile(
join(vaultPath, ".obsidian", "community-plugins.json"),
JSON.stringify(["obsidian-livesync"], null, 4)
);
await writeObsidianVaultRegistry(id, vaultPath, name, homePath, xdgConfigPath, userDataPath);
return {
path: vaultPath,
name,
id,
homePath,
xdgConfigPath,
xdgCachePath,
xdgDataPath,
userDataPath,
dispose: async () => {
if (process.env.E2E_OBSIDIAN_KEEP_VAULT === "true") {
console.log(`Keeping temporary vault: ${vaultPath}`);
console.log(`Keeping temporary Obsidian state: ${statePath}`);
return;
}
await Promise.all([
rm(vaultPath, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 }),
rm(statePath, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 }),
]);
},
};
}
async function writeObsidianVaultRegistry(
vaultId: string,
vaultPath: string,
vaultName: string,
homePath: string,
xdgConfigPath: string,
userDataPath: string
): Promise<void> {
const vaultRecord = {
path: vaultPath,
ts: Date.now(),
open: true,
name: vaultName,
};
const registry = {
cli: true,
vaults: {
[vaultId]: vaultRecord,
},
};
const registryText = JSON.stringify(registry, null, 4);
for (const configRoot of [join(homePath, ".config"), xdgConfigPath]) {
const obsidianConfigDir = join(configRoot, "obsidian");
await mkdir(obsidianConfigDir, { recursive: true });
await writeFile(join(obsidianConfigDir, "obsidian.json"), registryText);
}
await writeFile(join(userDataPath, "obsidian.json"), registryText);
await writeFile(join(userDataPath, `${vaultId}.json`), JSON.stringify(vaultRecord, null, 4));
}
+53
View File
@@ -0,0 +1,53 @@
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { launchObsidian } from "../runner/launch.ts";
import { runObsidianCli } from "../runner/cli.ts";
import { createTemporaryVault } from "../runner/vault.ts";
import { installBuiltPlugin } from "../runner/pluginInstaller.ts";
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let app;
try {
await installBuiltPlugin(vault.path);
app = await launchObsidian({
binary,
vaultPath: vault.path,
homePath: vault.homePath,
xdgConfigPath: vault.xdgConfigPath,
userDataPath: vault.userDataPath,
});
const cliEnv = {
...process.env,
HOME: vault.homePath,
XDG_CONFIG_HOME: vault.xdgConfigPath,
XDG_CACHE_HOME: vault.xdgCachePath,
XDG_DATA_HOME: vault.xdgDataPath,
};
await runObsidianCli(cli.binary, [`obsidian://open?path=${encodeURIComponent(vault.path)}`], cliEnv);
await new Promise((resolve) => setTimeout(resolve, 3000));
if (process.env.E2E_OBSIDIAN_RELOAD_PLUGIN === "true") {
await runObsidianCli(cli.binary, ["eval", "code=(async()=>app.plugins.setEnable(true))()"], cliEnv);
await runObsidianCli(cli.binary, ["plugin:reload", "id=obsidian-livesync"], cliEnv);
}
const cliArgs = process.argv.slice(2);
const result = await runObsidianCli(cli.binary, cliArgs.length > 0 ? cliArgs : ["--help"], cliEnv);
console.log(result.stdout);
console.error(result.stderr);
process.exitCode = result.code ?? 1;
} finally {
if (app) {
await app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+140
View File
@@ -0,0 +1,140 @@
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/couchdb-upload.md";
const noteContent = [
"# CouchDB upload from real Obsidian",
"",
"This note is created through Obsidian and uploaded by Self-hosted LiveSync.",
"The content is intentionally long enough to require chunk metadata in the local database.",
"0123456789 abcdefghijklmnopqrstuvwxyz 0123456789 abcdefghijklmnopqrstuvwxyz",
"0123456789 abcdefghijklmnopqrstuvwxyz 0123456789 abcdefghijklmnopqrstuvwxyz",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.ProcessEnv): Promise<LocalDatabaseEntry> {
return await evalObsidianJson<LocalDatabaseEntry>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
`const content=${JSON.stringify(noteContent)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"await app.vault.create(path,content);",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"for(let i=0;i<40;i++){",
"await core.services.fileProcessing.commitPendingFileEvents();",
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true).catch(()=>false);",
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
"await sleep(250);",
"}",
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
"})()",
].join(""),
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "obsidian-upload");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureCouchDb(cli.binary, session.cliEnv, {
uri: couchDb.uri,
username: couchDb.username,
password: couchDb.password,
dbName,
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
assertEqual(configured.couchDB_URI, couchDb.uri, "Configured CouchDB URI did not match.");
assertEqual(configured.couchDB_DBNAME, dbName, "Configured CouchDB database name did not match.");
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during this one-shot workflow.");
assertEqual(configured.syncOnStart, false, "Sync on start should remain disabled during this workflow.");
assertEqual(configured.syncOnSave, false, "Sync on save should remain disabled during this workflow.");
await prepareRemote(cli.binary, session.cliEnv);
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
await pushLocalChanges(cli.binary, session.cliEnv);
const remoteDocs = await waitForCouchDbDocs(couchDb, dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(localEntry.id) && localEntry.children.every((childId) => ids.has(childId));
});
const remoteMetadata = remoteDocs.find((doc) => doc._id === localEntry.id);
assertEqual(
remoteMetadata?.path,
localEntry.path,
"Remote metadata path did not match the local database entry."
);
console.log(
`Uploaded metadata ${localEntry.id} and ${localEntry.children.length} chunk(s) to CouchDB database ${dbName}`
);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,603 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
type CouchDbConfig,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const snippetPath = ".obsidian/snippets/livesync-customisation-e2e.css";
const snippetContent = [
"body {",
" --livesync-customisation-e2e-colour: #3d6f54;",
"}",
"",
".livesync-customisation-e2e {",
" color: var(--livesync-customisation-e2e-colour);",
"}",
"",
].join("\n");
const snippetUpdatedContent = [
"body {",
" --livesync-customisation-e2e-colour: #73548f;",
"}",
"",
".livesync-customisation-e2e {",
" background-color: var(--livesync-customisation-e2e-colour);",
"}",
"",
].join("\n");
const configPath = ".obsidian/livesync-customisation-e2e.json";
const configContent = JSON.stringify({ source: "customisation-sync", enabled: true }, null, 4) + "\n";
const pluginDir = ".obsidian/plugins/livesync-e2e-sample";
const pluginManifestPath = `${pluginDir}/manifest.json`;
const pluginMainPath = `${pluginDir}/main.js`;
const pluginStylesPath = `${pluginDir}/styles.css`;
const pluginManifestContent =
JSON.stringify(
{
id: "livesync-e2e-sample",
name: "LiveSync E2E Sample",
version: "0.0.1",
minAppVersion: "1.0.0",
description: "A sample plug-in fixture for real Obsidian E2E.",
author: "Self-hosted LiveSync",
isDesktopOnly: false,
},
null,
4
) + "\n";
const pluginMainContent = [
"module.exports = class LiveSyncE2ESamplePlugin extends Plugin {",
" async onload() {",
" this.register(() => undefined);",
" }",
"};",
"",
].join("\n");
const pluginStylesContent = ".livesync-e2e-sample { color: #73548f; }\n";
const sourceDeviceName = "customisation-sync-a";
const targetDeviceName = "customisation-sync-b";
type RunnerContext = {
binary: string;
cliBinary: string;
couchDb: CouchDbConfig;
dbName: string;
};
type CustomisationEntry = {
id: string;
path: string;
children: string[];
};
type CustomisationScanResult = {
enabled: boolean;
useV2: boolean;
device: string;
configDir: string;
files: string[];
};
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
await rm(join(vaultPath, path), { force: true });
}
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
return await readFile(join(vaultPath, path), "utf-8");
}
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
try {
await readFile(join(vaultPath, path));
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
async function waitForPathContent(
vaultPath: string,
path: string,
predicate: (content: string) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const deadline = Date.now() + timeoutMs;
let lastContent = "";
while (Date.now() < deadline) {
if (await pathExists(vaultPath, path)) {
lastContent = await readVaultFile(vaultPath, path);
if (predicate(lastContent)) {
return lastContent;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
deviceName: string
): Promise<ObsidianLiveSyncSession> {
const session = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await configureCouchDb(
context.cliBinary,
session.cliEnv,
{
uri: context.couchDb.uri,
username: context.couchDb.username,
password: context.couchDb.password,
dbName: context.dbName,
},
{
deviceAndVaultName: deviceName,
usePluginSync: true,
usePluginSyncV2: true,
autoSweepPlugins: false,
autoSweepPluginsPeriodic: false,
syncInternalFiles: false,
}
);
await evalObsidianJson<unknown>(
context.cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
`core.services.setting.setDeviceAndVaultName(${JSON.stringify(deviceName)});`,
"await core.services.setting.saveSettingData();",
"return JSON.stringify({device:core.services.setting.getDeviceAndVaultName()});",
"})()",
].join(""),
session.cliEnv
);
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await prepareRemote(context.cliBinary, session.cliEnv);
return session;
}
async function scanCustomisations(cliBinary: string, env: NodeJS.ProcessEnv): Promise<CustomisationScanResult> {
return await evalObsidianJson<CustomisationScanResult>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const before=await addOn.scanInternalFiles();",
"await addOn.scanAllConfigFiles(false);",
"return JSON.stringify({",
"ok:true,",
"enabled:core.settings.usePluginSync,",
"useV2:core.settings.usePluginSyncV2,",
"device:core.services.setting.getDeviceAndVaultName(),",
"configDir:addOn.configDir,",
"files:before,",
"});",
"})()",
].join(""),
env
);
}
async function storeCustomisationFile(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const term=core.services.setting.getDeviceAndVaultName();",
"const stat=await core.storageAccess.statHidden(path);",
"const category=addOn.getFileCategory(path);",
"const result=await addOn.storeCustomizationFiles(path,term);",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>doc.path);",
"const filename=path.split('/').pop();",
"const existing=entries.some((entry)=>entry.startsWith(`ix:${term}/${category}/`)&&entry.endsWith(`%${filename}`));",
"if(!result&&!existing){",
" throw new Error(`Could not store Customisation Sync file: path=${path}; term=${term}; category=${category}; stat=${JSON.stringify(stat)}; result=${JSON.stringify(result)}; entries=${JSON.stringify(entries)}`);",
"}",
"return JSON.stringify({ok:true,path,term,category,result:!!result,existing,entries});",
"})()",
].join(""),
env
);
}
async function deleteCustomisationSyncEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&!doc.deleted&&!doc._deleted)||false;",
"if(!entry) throw new Error(`Could not find customisation sync entry to delete: ${category}/${name}`);",
"if(!(await addOn.deleteConfigOnDatabase(entry.path))){",
" throw new Error(`Could not delete Customisation Sync entry: ${entry.path}`);",
"}",
"return JSON.stringify({ok:true,path:entry.path});",
"})()",
].join(""),
env
);
}
async function waitForCustomisationEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
): Promise<CustomisationEntry> {
const entries = await waitForCustomisationEntries(cliBinary, env, category, name, 1, term, timeoutMs);
return entries[0];
}
async function waitForCustomisationEntries(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
count: number,
term?: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
): Promise<CustomisationEntry[]> {
return await evalObsidianJson<CustomisationEntry[]>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const count=${JSON.stringify(count)};`,
`const term=${JSON.stringify(term ?? "")};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const deadline=Date.now()+timeoutMs;",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entries=[];",
"while(Date.now()<deadline){",
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
" entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&Array.isArray(doc.children)&&doc.children.length>0);",
" if(entries.length>=count) break;",
" await sleep(250);",
"}",
"if(entries.length<count){",
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
" const found=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>({id:doc._id,path:doc.path,children:doc.children?.length??0}));",
" throw new Error(`Timed out waiting for customisation sync entries: ${category}/${name}; expected=${count}; entries=${JSON.stringify(found)}`);",
"}",
"return JSON.stringify(entries.map((entry)=>({id:entry._id,path:entry.path,children:entry.children||[]})));",
"})()",
].join(""),
env
);
}
async function waitForCustomisationEntryAbsent(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const deadline=Date.now()+timeoutMs;",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"while(Date.now()<deadline){",
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
" entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&!doc.deleted&&!doc._deleted)||false;",
" if(!entry) return JSON.stringify({ok:true});",
" await sleep(250);",
"}",
"throw new Error(`Timed out waiting for customisation sync entry deletion: ${category}/${name}; entry=${JSON.stringify(entry)}`);",
"})()",
].join(""),
env
);
}
async function applyRemoteCustomisationEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`)))||false;",
"if(!entry) throw new Error(`Could not find remote customisation entry: ${category}/${name}`);",
"const display=addOn.createPluginDataFromV2(entry.path);",
"if(!display) throw new Error(`Could not create Customisation Sync display entry: ${entry.path}`);",
"const file=await addOn.createPluginDataExFileV2(entry.path);",
"if(!file) throw new Error(`Could not load Customisation Sync file entry: ${entry.path}`);",
"await display.setFile(file);",
"if(!(await addOn.applyDataV2(display))){",
" throw new Error(`Could not apply Customisation Sync entry: ${entry.path}`);",
"}",
"return JSON.stringify({ok:true,path:entry.path});",
"})()",
].join(""),
env
);
}
async function applyRemoteCustomisationGroup(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "PLUGIN_MAIN",
name: string,
term?: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`)));",
"if(entries.length===0) throw new Error(`Could not find remote customisation entries: ${category}/${name}`);",
"const display=addOn.createPluginDataFromV2(entries[0].path);",
"if(!display) throw new Error(`Could not create Customisation Sync display entry: ${entries[0].path}`);",
"for(const entry of entries){",
" const file=await addOn.createPluginDataExFileV2(entry.path);",
" if(!file) throw new Error(`Could not load Customisation Sync file entry: ${entry.path}`);",
" await display.setFile(file);",
"}",
"if(!(await addOn.applyDataV2(display))){",
" throw new Error(`Could not apply Customisation Sync group: ${category}/${name}`);",
"}",
"return JSON.stringify({ok:true,count:entries.length});",
"})()",
].join(""),
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "customisation-sync");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
const snippetPathParts = snippetPath.split("/");
const snippetName = snippetPathParts[snippetPathParts.length - 1] ?? snippetPath;
const configName = configPath.split("/").pop() ?? configPath;
const pluginName = pluginDir.split("/").pop() ?? pluginDir;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
await writeVaultFile(vaultA.path, configPath, configContent);
await writeVaultFile(vaultA.path, pluginManifestPath, pluginManifestContent);
await writeVaultFile(vaultA.path, pluginMainPath, pluginMainContent);
await writeVaultFile(vaultA.path, pluginStylesPath, pluginStylesContent);
let session = await startConfiguredSession(context, vaultA, sourceDeviceName);
const scanResult = await scanCustomisations(context.cliBinary, session.cliEnv);
console.log(`Customisation scan files: ${scanResult.files.join(", ") || "(none)"}`);
await storeCustomisationFile(context.cliBinary, session.cliEnv, snippetPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, configPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginManifestPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginMainPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginStylesPath);
const entry = await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName);
const configEntry = await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "CONFIG", configName);
const pluginEntries = await waitForCustomisationEntries(
context.cliBinary,
session.cliEnv,
"PLUGIN_MAIN",
pluginName,
3
);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
const entries = [entry, configEntry, ...pluginEntries];
return entries.every(
(target) => ids.has(target.id) && target.children.every((childId) => ids.has(childId))
);
});
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
assertEqual(
await pathExists(vaultB.path, snippetPath),
false,
"Customisation Sync snippet was reflected before explicit application."
);
await applyRemoteCustomisationEntry(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
const applied = await waitForPathContent(vaultB.path, snippetPath, (content) => content === snippetContent);
await applyRemoteCustomisationEntry(context.cliBinary, session.cliEnv, "CONFIG", configName, sourceDeviceName);
const appliedConfig = await waitForPathContent(vaultB.path, configPath, (content) => content === configContent);
await applyRemoteCustomisationGroup(
context.cliBinary,
session.cliEnv,
"PLUGIN_MAIN",
pluginName,
sourceDeviceName
);
const appliedPluginManifest = await waitForPathContent(
vaultB.path,
pluginManifestPath,
(content) => content === pluginManifestContent
);
const appliedPluginMain = await waitForPathContent(
vaultB.path,
pluginMainPath,
(content) => content === pluginMainContent
);
const appliedPluginStyles = await waitForPathContent(
vaultB.path,
pluginStylesPath,
(content) => content === pluginStylesContent
);
await session.app.stop();
assertEqual(applied, snippetContent, "Customisation Sync snippet content did not match after application.");
assertEqual(appliedConfig, configContent, "Customisation Sync config content did not match after application.");
assertEqual(
appliedPluginManifest,
pluginManifestContent,
"Customisation Sync plug-in manifest did not match after application."
);
assertEqual(appliedPluginMain, pluginMainContent, "Customisation Sync plug-in main file did not match.");
assertEqual(appliedPluginStyles, pluginStylesContent, "Customisation Sync plug-in stylesheet did not match.");
await writeVaultFile(vaultA.path, snippetPath, snippetUpdatedContent);
session = await startConfiguredSession(context, vaultA, sourceDeviceName);
await storeCustomisationFile(context.cliBinary, session.cliEnv, snippetPath);
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await applyRemoteCustomisationEntry(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
const updated = await waitForPathContent(
vaultB.path,
snippetPath,
(content) => content === snippetUpdatedContent
);
await session.app.stop();
assertEqual(updated, snippetUpdatedContent, "Updated Customisation Sync snippet did not apply.");
await removeVaultFile(vaultA.path, snippetPath);
session = await startConfiguredSession(context, vaultA, sourceDeviceName);
await deleteCustomisationSyncEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
await waitForCustomisationEntryAbsent(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCustomisationEntryAbsent(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
await session.app.stop();
console.log(
`Customisation Sync applied snippet, config, and plug-in fixtures, then propagated snippet update and sync-data deletion.`
);
} finally {
await vaultA.dispose();
await vaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+60
View File
@@ -0,0 +1,60 @@
import { launchObsidian } from "../runner/launch.ts";
import { installBuiltPlugin } from "../runner/pluginInstaller.ts";
import { createTemporaryVault } from "../runner/vault.ts";
import { requireObsidianBinary } from "../runner/environment.ts";
import { writeFile } from "node:fs/promises";
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, withObsidianPage } from "../runner/ui.ts";
const port = obsidianRemoteDebuggingPort();
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const vault = await createTemporaryVault();
await installBuiltPlugin(vault.path);
const app = await launchObsidian({
binary,
vaultPath: vault.path,
homePath: vault.homePath,
xdgConfigPath: vault.xdgConfigPath,
xdgCachePath: vault.xdgCachePath,
xdgDataPath: vault.xdgDataPath,
userDataPath: vault.userDataPath,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
try {
await preseedTrustedVaultState(port, vault.id);
const { screenshotPath, textPath } = await withObsidianPage(port, async (page) => {
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_WAIT_MS ?? 5000));
const title = await page.title().catch((error: unknown) => `title error: ${String(error)}`);
const url = page.url();
const text = await page
.locator("body")
.innerText({ timeout: 5000 })
.catch((error: unknown) => {
return `body text error: ${String(error)}`;
});
if (process.env.E2E_OBSIDIAN_DEBUG_CLICK_TRUST === "true") {
await page.getByText("Trust author and enable plugins").click({ timeout: 10000 });
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_AFTER_CLICK_WAIT_MS ?? 3000));
}
const screenshotPath = process.env.E2E_OBSIDIAN_DEBUG_SCREENSHOT ?? "/tmp/obsidian-e2e-debug.png";
const textPath = process.env.E2E_OBSIDIAN_DEBUG_TEXT ?? "/tmp/obsidian-e2e-debug.txt";
await page.screenshot({ path: screenshotPath, fullPage: true });
await writeFile(textPath, [`title: ${title}`, `url: ${url}`, "", text].join("\n"), "utf-8");
return { screenshotPath, textPath };
});
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary Obsidian state: ${vault.userDataPath}`);
console.log(`Debug text: ${textPath}`);
console.log(`Debug screenshot: ${screenshotPath}`);
} finally {
await app.stop();
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+13
View File
@@ -0,0 +1,13 @@
import { discoverObsidianBinary } from "../runner/environment.ts";
const result = discoverObsidianBinary();
if (result.binary) {
console.log(`Obsidian executable: ${result.binary}`);
console.log(`Source: ${result.source}`);
process.exit(0);
}
console.error("Obsidian executable was not found.");
console.error("Set OBSIDIAN_BINARY to the installed Obsidian executable path.");
console.error(`Checked paths: ${result.checked.length > 0 ? result.checked.join(", ") : "(none)"}`);
process.exit(1);
@@ -0,0 +1,557 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
type CouchDbConfig,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { clickJsonResolveOption, obsidianRemoteDebuggingPort } from "../runner/ui.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const snippetPath = ".obsidian/snippets/livesync-e2e.css";
const snippetContent = [
"body {",
" --livesync-e2e-snippet-colour: #245a70;",
"}",
"",
".livesync-e2e-snippet {",
" color: var(--livesync-e2e-snippet-colour);",
"}",
"",
].join("\n");
const mergeJsonPath = ".obsidian/livesync-e2e-merge.json";
const manualMergeJsonPath = ".obsidian/livesync-e2e-manual-merge.json";
const targetPath = ".obsidian/livesync-targeted/only-a.json";
const hiddenFileCliTimeoutMs = Number(process.env.E2E_OBSIDIAN_HIDDEN_FILE_CLI_TIMEOUT_MS ?? 90000);
type RunnerContext = {
binary: string;
cliBinary: string;
couchDb: CouchDbConfig;
dbName: string;
};
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
await rm(join(vaultPath, path), { force: true });
}
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
return await readFile(join(vaultPath, path), "utf-8");
}
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
try {
await readFile(join(vaultPath, path));
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
async function waitForPathContent(
vaultPath: string,
path: string,
predicate: (content: string) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const deadline = Date.now() + timeoutMs;
let lastContent = "";
while (Date.now() < deadline) {
if (await pathExists(vaultPath, path)) {
lastContent = await readVaultFile(vaultPath, path);
if (predicate(lastContent)) {
return lastContent;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
}
async function waitForPathDeleted(
vaultPath: string,
path: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (!(await pathExists(vaultPath, path))) {
return;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
}
function hasJsonValues(content: string, values: Record<string, unknown>): boolean {
try {
const parsed = JSON.parse(content) as Record<string, unknown>;
return Object.entries(values).every(([key, value]) => parsed[key] === value);
} catch {
return false;
}
}
async function scanHiddenStorage(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"await app.commands.commands['obsidian-livesync:livesync-scaninternal-storage'].callback();",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function scanHiddenDatabase(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"await app.commands.commands['obsidian-livesync:livesync-scaninternal-database'].callback();",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env,
hiddenFileCliTimeoutMs
);
}
async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"await app.commands.commands['obsidian-livesync:livesync-resolveinternal-conflicts'].callback();",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env,
hiddenFileCliTimeoutMs
);
}
async function autoMergeHiddenJsonConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"let doc=false;",
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
" if(entry.path===prefixedPath){ doc=entry; break; }",
"}",
"if(!doc) throw new Error(`Could not find hidden conflict candidate: ${path}`);",
"if(!doc._conflicts?.length) throw new Error(`Hidden file has no conflicts: ${path}`);",
"const conflicts=doc._conflicts.sort((a,b)=>Number(a.split('-')[0])-Number(b.split('-')[0]));",
"const conflictedRev=conflicts[0];",
"const conflictedRevNo=Number(conflictedRev.split('-')[0]);",
"const revFrom=await core.localDatabase.getRaw(doc._id,{revs_info:true});",
"const commonBase=(revFrom._revs_info||[])",
" .filter((rev)=>rev.status==='available'&&Number(rev.rev.split('-')[0])<conflictedRevNo)",
" .map((rev)=>rev.rev)[0]||'';",
"const result=await core.localDatabase.managers.conflictManager.mergeObject(",
" doc.path, commonBase, doc._rev, conflictedRev",
");",
"if(!result){",
" throw new Error(`Hidden JSON conflict was not auto-mergeable: ${path}; base=${commonBase}; current=${doc._rev}; conflict=${conflictedRev}`);",
"}",
"await app.vault.adapter.write(path,result);",
"const stat=await app.vault.adapter.stat(path);",
"if(!stat) throw new Error(`Could not stat merged hidden file: ${path}`);",
"const baseData=await core.localDatabase.getDBEntry(prefixedPath,{conflicts:true},false,true);",
"if(baseData===false) throw new Error(`Could not load base save data: ${path}`);",
"const saveData={",
" ...baseData,",
" data:new Blob([result]),",
" mtime:stat.mtime,",
" ctime:stat.ctime,",
" size:stat.size,",
" children:[],",
" deleted:false,",
" type:baseData.datatype,",
"};",
"delete saveData._conflicts;",
"const stored=await core.localDatabase.putDBEntry(saveData);",
"if(!stored?.ok) throw new Error(`Could not store merged hidden file: ${path}`);",
"await core.localDatabase.removeRevision(doc._id,conflictedRev);",
"await app.commands.commands['obsidian-livesync:livesync-scaninternal-storage'].callback();",
"await app.commands.commands['obsidian-livesync:livesync-scaninternal-database'].callback();",
"const content=await app.vault.adapter.read(path);",
"return JSON.stringify({ok:true,merged:JSON.parse(content)});",
"})()",
].join(""),
env
);
}
async function openHiddenJsonResolveModal(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"let doc=false;",
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
" if(entry.path===prefixedPath){ doc=entry; break; }",
"}",
"if(!doc?._conflicts?.length) throw new Error(`Could not find hidden JSON conflict: ${path}`);",
"void app.commands.commands['obsidian-livesync:livesync-resolveinternal-conflicts'].callback();",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function storeHiddenFileAsConflict(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
baseRev: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const baseRev=${JSON.stringify(baseRev)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const prefixedPath=`i:${path}`;",
"const stat=await app.vault.adapter.stat(path);",
"if(!stat||stat.type!=='file') throw new Error(`Hidden file was unexpectedly missing: ${path}`);",
"const content=await app.vault.adapter.readBinary(path);",
"const oldFile=await core.localDatabase.getDBEntry(prefixedPath,{conflicts:true},false,true);",
"const baseData=oldFile===false?{",
" _id:prefixedPath,",
" path:prefixedPath,",
" mtime:0,",
" ctime:Date.now(),",
" size:0,",
" children:[],",
" deleted:false,",
" type:'newnote',",
" datatype:'newnote',",
" data:'',",
" eden:{},",
"}:oldFile;",
"const saveData={",
" ...baseData,",
" data:new Blob([content]),",
" mtime:stat.mtime,",
" ctime:stat.ctime,",
" size:stat.size,",
" children:[],",
" deleted:false,",
" type:baseData.datatype,",
"};",
"const result=await core.localDatabase.putDBEntry(saveData,false,baseRev);",
"if(!result?.ok) throw new Error(`Could not store conflicted hidden file: ${path}`);",
"return JSON.stringify({ok:true,rev:result.rev});",
"})()",
].join(""),
env
);
}
async function createHiddenJsonConflict(
context: RunnerContext,
session: ObsidianLiveSyncSession,
vault: TemporaryVault,
path: string,
base: string,
left: string,
right: string
): Promise<void> {
await writeVaultFile(vault.path, path, base);
await scanHiddenStorage(context.cliBinary, session.cliEnv);
const baseEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
await writeVaultFile(vault.path, path, left);
await scanHiddenStorage(context.cliBinary, session.cliEnv);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
await writeVaultFile(vault.path, path, right);
await storeHiddenFileAsConflict(context.cliBinary, session.cliEnv, path, baseEntry.rev);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
overrides: Record<string, unknown> = {}
): Promise<ObsidianLiveSyncSession> {
const session = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await configureCouchDb(
context.cliBinary,
session.cliEnv,
{
uri: context.couchDb.uri,
username: context.couchDb.username,
password: context.couchDb.password,
dbName: context.dbName,
},
{
syncInternalFiles: true,
syncInternalFilesBeforeReplication: true,
watchInternalFileChanges: false,
syncInternalFilesTargetPatterns: "",
...overrides,
}
);
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await prepareRemote(context.cliBinary, session.cliEnv);
return session;
}
async function uploadHiddenFile(
context: RunnerContext,
session: ObsidianLiveSyncSession,
path: string
): Promise<LocalDatabaseEntry> {
await scanHiddenStorage(context.cliBinary, session.cliEnv);
const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
});
return entry;
}
async function pullAndApplyHiddenFiles(
context: RunnerContext,
session: ObsidianLiveSyncSession,
options: { resolveConflicts?: boolean } = {}
): Promise<void> {
await pushLocalChanges(context.cliBinary, session.cliEnv);
if (options.resolveConflicts === true) {
await resolveHiddenConflicts(context.cliBinary, session.cliEnv);
}
await scanHiddenDatabase(context.cliBinary, session.cliEnv);
}
async function runCreateRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
let session = await startConfiguredSession(context, vaultA);
const entry = await uploadHiddenFile(context, session, snippetPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await pullAndApplyHiddenFiles(context, session);
const received = await waitForPathContent(vaultB.path, snippetPath, (content) => content === snippetContent);
await session.app.stop();
assertEqual(received, snippetContent, "Hidden snippet content did not round-trip to the second vault.");
console.log(`Hidden create round-trip copied ${entry.id} to the second vault.`);
}
async function runDeleteRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
await removeVaultFile(vaultA.path, snippetPath);
let session = await startConfiguredSession(context, vaultA);
await scanHiddenStorage(context.cliBinary, session.cliEnv);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await pullAndApplyHiddenFiles(context, session);
await waitForPathDeleted(vaultB.path, snippetPath);
await session.app.stop();
console.log("Hidden delete round-trip removed the snippet from the second vault.");
}
async function runJsonConflictRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const base = JSON.stringify({ base: true, fromA: false, fromB: false }, null, 4) + "\n";
const left = JSON.stringify({ base: true, fromA: true, fromB: false }, null, 4) + "\n";
const right = JSON.stringify({ base: true, fromA: false, fromB: true }, null, 4) + "\n";
let session = await startConfiguredSession(context, vaultB);
await createHiddenJsonConflict(context, session, vaultB, mergeJsonPath, base, left, right);
await autoMergeHiddenJsonConflict(context.cliBinary, session.cliEnv, mergeJsonPath);
const mergedEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, mergeJsonPath, {
hidden: true,
});
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(mergedEntry.id) && mergedEntry.children.every((childId) => ids.has(childId));
});
const mergedOnB = await waitForPathContent(vaultB.path, mergeJsonPath, (content) =>
hasJsonValues(content, { fromA: true, fromB: true })
);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, mergeJsonPath, { hidden: true });
await scanHiddenDatabase(context.cliBinary, session.cliEnv);
const mergedOnA = await waitForPathContent(vaultA.path, mergeJsonPath, (content) =>
hasJsonValues(content, { fromA: true, fromB: true })
);
await session.app.stop();
assertEqual(mergedOnA, mergedOnB, "Merged hidden JSON content was not consistent across both vaults.");
console.log("Hidden JSON conflict was automatically merged and round-tripped.");
}
async function runJsonManualConflictResolution(context: RunnerContext, vault: TemporaryVault): Promise<void> {
const base = JSON.stringify({ shared: "base" }, null, 4) + "\n";
const left = JSON.stringify({ shared: "left", fromA: true }, null, 4) + "\n";
const right = JSON.stringify({ shared: "right", fromB: true }, null, 4) + "\n";
const session = await startConfiguredSession(context, vault);
await createHiddenJsonConflict(context, session, vault, manualMergeJsonPath, base, left, right);
await openHiddenJsonResolveModal(context.cliBinary, session.cliEnv, manualMergeJsonPath);
await clickJsonResolveOption(obsidianRemoteDebuggingPort(), "AB");
const merged = await waitForPathContent(vault.path, manualMergeJsonPath, (content) =>
hasJsonValues(content, { shared: "right", fromA: true, fromB: true })
);
await session.app.stop();
const parsed = JSON.parse(merged);
assertEqual(parsed.shared, "right", "Manual JSON conflict resolution did not apply the selected merged result.");
assertEqual(parsed.fromA, true, "Manual JSON conflict resolution lost the first-side value.");
assertEqual(parsed.fromB, true, "Manual JSON conflict resolution lost the second-side value.");
console.log("Hidden JSON conflict modal applied the selected merged result.");
}
async function runTargetMismatch(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const targetContent = JSON.stringify({ onlyA: true, targetMismatch: true }, null, 4) + "\n";
await writeVaultFile(vaultA.path, targetPath, targetContent);
let session = await startConfiguredSession(context, vaultA);
try {
await uploadHiddenFile(context, session, targetPath);
} finally {
await session.app.stop();
}
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "snippets",
});
try {
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
assertEqual(
await pathExists(vaultB.path, targetPath),
false,
"Hidden file was applied on a device where it was not a target file."
);
} finally {
await session.app.stop();
}
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "",
});
let received = "";
try {
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
received = await waitForPathContent(vaultB.path, targetPath, (content) => content === targetContent);
} finally {
await session.app.stop();
}
assertEqual(received, targetContent, "Hidden file was not applied after it became a target file.");
console.log("Hidden target mismatch respected per-device target patterns, then applied after enabling the target.");
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "hidden-roundtrip");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
await runCreateRoundTrip(context, vaultA, vaultB);
await runDeleteRoundTrip(context, vaultA, vaultB);
await runJsonConflictRoundTrip(context, vaultA, vaultB);
await runJsonManualConflictResolution(context, vaultB);
await runTargetMismatch(context, vaultA, vaultB);
} finally {
await vaultA.dispose();
await vaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,121 @@
import { createWriteStream, existsSync } from "node:fs";
import { chmod, mkdir } from "node:fs/promises";
import { get } from "node:https";
import { arch } from "node:process";
import { basename, join, resolve } from "node:path";
import { spawn } from "node:child_process";
const defaultVersion = "1.12.7";
function appImageArch(): string {
const requestedArch = process.env.E2E_OBSIDIAN_APPIMAGE_ARCH?.trim();
if (requestedArch) {
return requestedArch;
}
if (arch === "arm64") {
return "arm64";
}
if (arch === "x64") {
return "x86_64";
}
throw new Error(`Unsupported architecture for Obsidian AppImage: ${arch}`);
}
function appImageUrl(version: string, imageArch: string): string {
return `https://github.com/obsidianmd/obsidian-releases/releases/download/v${version}/Obsidian-${version}-${imageArch}.AppImage`;
}
function download(url: string, destination: string, redirectsLeft = 5): Promise<void> {
return new Promise((resolveDownload, reject) => {
const request = get(url, (response) => {
const statusCode = response.statusCode ?? 0;
const location = response.headers.location;
if (statusCode >= 300 && statusCode < 400 && location) {
response.resume();
if (redirectsLeft <= 0) {
reject(new Error(`Too many redirects while downloading ${url}`));
return;
}
download(new URL(location, url).toString(), destination, redirectsLeft - 1)
.then(resolveDownload)
.catch(reject);
return;
}
if (statusCode !== 200) {
response.resume();
reject(new Error(`Failed to download ${url}: HTTP ${statusCode}`));
return;
}
const file = createWriteStream(destination, { mode: 0o755 });
response.pipe(file);
file.on("finish", () => {
file.close((error) => {
if (error) {
reject(error);
} else {
resolveDownload();
}
});
});
file.on("error", reject);
});
request.on("error", reject);
});
}
function extractAppImage(appImagePath: string, cwd: string): Promise<void> {
return new Promise((resolveExtract, reject) => {
const child = spawn(appImagePath, ["--appimage-extract"], {
cwd,
stdio: "inherit",
});
child.on("error", reject);
child.on("exit", (code, signal) => {
if (code === 0) {
resolveExtract();
return;
}
reject(new Error(`AppImage extraction failed. code=${code}, signal=${signal}`));
});
});
}
async function main(): Promise<void> {
const version = process.env.E2E_OBSIDIAN_VERSION?.trim() || defaultVersion;
const imageArch = appImageArch();
const targetDir = resolve(process.env.E2E_OBSIDIAN_DOWNLOAD_DIR?.trim() || "_testdata/obsidian");
const url = process.env.E2E_OBSIDIAN_APPIMAGE_URL?.trim() || appImageUrl(version, imageArch);
const appImagePath = join(targetDir, basename(new URL(url).pathname));
const extractedBinary = join(targetDir, "squashfs-root", "obsidian");
const forceDownload = process.env.E2E_OBSIDIAN_FORCE_DOWNLOAD === "true";
const skipExtract = process.env.E2E_OBSIDIAN_SKIP_EXTRACT === "true";
await mkdir(targetDir, { recursive: true });
if (!existsSync(appImagePath) || forceDownload) {
console.log(`Downloading Obsidian AppImage: ${url}`);
console.log(`Destination: ${appImagePath}`);
await download(url, appImagePath);
await chmod(appImagePath, 0o755);
} else {
console.log(`Using existing Obsidian AppImage: ${appImagePath}`);
}
if (!skipExtract) {
if (existsSync(extractedBinary)) {
console.log(`Using existing extracted Obsidian binary: ${extractedBinary}`);
} else {
console.log(`Extracting Obsidian AppImage in ${targetDir}`);
await extractAppImage(appImagePath, targetDir);
console.log(`Extracted Obsidian binary: ${extractedBinary}`);
}
}
console.log(`Set OBSIDIAN_BINARY=${extractedBinary} to use the extracted binary explicitly.`);
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+105
View File
@@ -0,0 +1,105 @@
import { spawn } from "node:child_process";
type Step = {
name: string;
args: string[];
optional?: boolean;
};
const testSteps: Step[] = [
{ name: "build", args: ["run", "build"] },
{ name: "discover", args: ["run", "test:e2e:obsidian:discover"] },
{ name: "smoke", args: ["run", "test:e2e:obsidian:smoke"] },
{ name: "vault reflection", args: ["run", "test:e2e:obsidian:vault-reflection"] },
{ name: "CouchDB upload", args: ["run", "test:e2e:obsidian:couchdb-upload"] },
{ name: "Object Storage upload", args: ["run", "test:e2e:obsidian:minio-upload"] },
{ name: "startup scan", args: ["run", "test:e2e:obsidian:startup-scan"] },
{ name: "two-vault synchronisation", args: ["run", "test:e2e:obsidian:two-vault-sync"] },
{ name: "hidden file snippet synchronisation", args: ["run", "test:e2e:obsidian:hidden-file-snippet-sync"] },
{ name: "Customisation Sync", args: ["run", "test:e2e:obsidian:customisation-sync"] },
{ name: "setting Markdown export", args: ["run", "test:e2e:obsidian:setting-markdown-export"] },
];
const manageCouchDb = process.argv.includes("--manage-couchdb") || process.argv.includes("--manage-services");
const manageMinio = process.argv.includes("--manage-minio") || process.argv.includes("--manage-services");
const keepServices = process.argv.includes("--keep-services");
const keepCouchDb = keepServices || process.argv.includes("--keep-couchdb");
const keepMinio = keepServices || process.argv.includes("--keep-minio");
function npmBinary(): string {
return process.platform === "win32" ? "npm.cmd" : "npm";
}
function runStep(step: Step): Promise<void> {
return new Promise((resolve, reject) => {
console.log(`\n# ${step.name}`);
const child = spawn(npmBinary(), step.args, {
cwd: process.cwd(),
env: process.env,
stdio: "inherit",
});
child.on("error", reject);
child.on("exit", (code, signal) => {
if (code === 0) {
resolve();
return;
}
const message = `${step.name} failed with ${signal ? `signal ${signal}` : `exit code ${code}`}`;
if (step.optional) {
console.warn(message);
resolve();
return;
}
reject(new Error(message));
});
});
}
async function stopManagedCouchDb(): Promise<void> {
await runStep({
name: "stop CouchDB fixture",
args: ["run", "test:docker-couchdb:stop"],
optional: true,
});
}
async function stopManagedMinio(): Promise<void> {
await runStep({
name: "stop MinIO fixture",
args: ["run", "test:docker-s3:stop"],
optional: true,
});
}
async function main(): Promise<void> {
let shouldStopCouchDb = false;
let shouldStopMinio = false;
try {
if (manageCouchDb) {
await stopManagedCouchDb();
await runStep({ name: "start CouchDB fixture", args: ["run", "test:docker-couchdb:start"] });
shouldStopCouchDb = !keepCouchDb;
}
if (manageMinio) {
await stopManagedMinio();
await runStep({ name: "start MinIO fixture", args: ["run", "test:docker-s3:start"] });
shouldStopMinio = !keepMinio;
}
for (const step of testSteps) {
await runStep(step);
}
} finally {
if (shouldStopMinio) {
await stopManagedMinio();
}
if (shouldStopCouchDb) {
await stopManagedCouchDb();
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+142
View File
@@ -0,0 +1,142 @@
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureObjectStorage,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import {
deleteObjectStoragePrefix,
ensureObjectStorageBucket,
listObjectStorageObjects,
loadObjectStorageConfig,
makeUniqueBucketPrefix,
} from "../runner/objectStorage.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/minio-upload.md";
const noteContent = [
"# Object Storage upload from real Obsidian",
"",
"This note is created through Obsidian and uploaded by Self-hosted LiveSync to S3-compatible Object Storage.",
"The test is intentionally small, but it crosses the real Obsidian, Journal Sync, and AWS SDK boundary.",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.ProcessEnv): Promise<LocalDatabaseEntry> {
return await evalObsidianJson<LocalDatabaseEntry>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
`const content=${JSON.stringify(noteContent)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"await app.vault.create(path,content);",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"for(let i=0;i<40;i++){",
"await core.services.fileProcessing.commitPendingFileEvents();",
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true).catch(()=>false);",
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
"await sleep(250);",
"}",
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
"})()",
].join(""),
env
);
}
async function waitForObjectStorageObjects(prefix: string): Promise<string[]> {
const objectStorage = await loadObjectStorageConfig();
const timeoutMs = Number(process.env.E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS ?? 20000);
const deadline = Date.now() + timeoutMs;
let keys: string[] = [];
while (Date.now() < deadline) {
const objects = await listObjectStorageObjects(objectStorage, prefix);
keys = objects.flatMap((object) => (object.Key ? [object.Key] : []));
if (keys.length > 0) {
return keys;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Object Storage objects under ${prefix}. Last keys: ${keys.join(", ")}`);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const objectStorage = await loadObjectStorageConfig();
const bucketPrefix = makeUniqueBucketPrefix("minio-upload");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await ensureObjectStorageBucket(objectStorage);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary Object Storage bucket: ${objectStorage.bucket}`);
console.log(`Temporary Object Storage prefix: ${bucketPrefix}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureObjectStorage(cli.binary, session.cliEnv, {
...objectStorage,
bucketPrefix,
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
assertEqual(configured.remoteType, "MINIO", "Remote type was not Object Storage.");
assertEqual(configured.endpoint, objectStorage.endpoint, "Configured Object Storage endpoint did not match.");
assertEqual(configured.bucket, objectStorage.bucket, "Configured Object Storage bucket did not match.");
assertEqual(configured.bucketPrefix, bucketPrefix, "Configured Object Storage bucket prefix did not match.");
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during this one-shot workflow.");
await prepareRemote(cli.binary, session.cliEnv);
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
await pushLocalChanges(cli.binary, session.cliEnv);
const keys = await waitForObjectStorageObjects(bucketPrefix);
console.log(
`Uploaded ${localEntry.path} through Journal Sync to ${objectStorage.bucket}/${bucketPrefix} (${keys.length} object(s))`
);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_OBJECT_STORAGE !== "true") {
await deleteObjectStoragePrefix(objectStorage, bucketPrefix).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,106 @@
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { assertEqual, waitForLiveSyncCoreReady } from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const settingPath = "LiveSync/settings-export.md";
async function waitForFileContaining(
vaultPath: string,
path: string,
predicates: ((content: string) => boolean)[],
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const fullPath = join(vaultPath, path);
const deadline = Date.now() + timeoutMs;
let lastContent = "";
let lastError: unknown;
while (Date.now() < deadline) {
try {
lastContent = await readFile(fullPath, "utf-8");
if (predicates.every((predicate) => predicate(lastContent))) {
return lastContent;
}
} catch (error) {
lastError = error;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for setting Markdown: ${fullPath}\nLast error: ${String(lastError)}`);
}
async function configureSettingMarkdown(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"await core.services.setting.applyExternalSettings({",
`settingSyncFile:${JSON.stringify(settingPath)},`,
"writeCredentialsForSettingSync:false,",
"couchDB_USER:'e2e-user',",
"couchDB_PASSWORD:'e2e-password',",
"passphrase:'e2e-passphrase',",
"showVerboseLog:true,",
"},true);",
"await core.services.setting.saveSettingData();",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
await configureSettingMarkdown(cli.binary, session.cliEnv);
const content = await waitForFileContaining(vault.path, settingPath, [
(value) => value.includes("````yaml:livesync-setting"),
(value) => value.includes(`settingSyncFile: ${settingPath}`),
(value) => value.includes("showVerboseLog: true"),
]);
assertEqual(
content.includes("couchDB_PASSWORD: e2e-password"),
false,
"Credential leaked into setting Markdown."
);
assertEqual(content.includes("passphrase: e2e-passphrase"), false, "Passphrase leaked into setting Markdown.");
console.log(`Generated setting Markdown without credentials: ${settingPath}`);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+41
View File
@@ -0,0 +1,41 @@
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
console.log(`Installed plug-in artifacts: ${session.install.copied.join(", ")}`);
const { readiness } = session;
console.log(
`Obsidian plug-in ready: ${readiness.pluginId}@${readiness.pluginVersion} in ${readiness.vaultName}`
);
await new Promise((resolve) => setTimeout(resolve, Number(process.env.E2E_OBSIDIAN_SMOKE_TIMEOUT_MS ?? 1000)));
console.log("Obsidian stayed alive after the plug-in readiness check.");
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+116
View File
@@ -0,0 +1,116 @@
import { mkdir, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/startup-scan.md";
const noteContent = [
"# Startup scan",
"",
"This note was written while Obsidian was stopped.",
"The test verifies that the next real Obsidian boot scans it into the local database.",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "startup-scan");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureCouchDb(cli.binary, session.cliEnv, {
uri: couchDb.uri,
username: couchDb.username,
password: couchDb.password,
dbName,
});
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not configured.");
await prepareRemote(cli.binary, session.cliEnv);
await session.app.stop();
session = undefined;
await writeVaultFile(vault.path, notePath, noteContent);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const localEntry = await waitForLocalDatabaseEntry(cli.binary, session.cliEnv, notePath);
await pushLocalChanges(cli.binary, session.cliEnv);
const remoteDocs = await waitForCouchDbDocs(couchDb, dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(localEntry.id) && localEntry.children.every((childId) => ids.has(childId));
});
const remoteMetadata = remoteDocs.find((doc) => doc._id === localEntry.id);
assertEqual(remoteMetadata?.path, localEntry.path, "Startup-scanned remote metadata path did not match.");
console.log(`Startup scan uploaded metadata ${localEntry.id} and ${localEntry.children.length} chunk(s).`);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+525
View File
@@ -0,0 +1,525 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
type CouchDbConfig,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const createPath = "E2E/two-vault/create.md";
const updatePath = "E2E/two-vault/update.md";
const deletePath = "E2E/two-vault/delete.md";
const renameFromPath = "E2E/two-vault/rename-source.md";
const renameToPath = "E2E/two-vault/renamed/rename-target.md";
const conflictPath = "E2E/two-vault/conflict.md";
const targetMismatchPath = "E2E/two-vault/target-mismatch.md";
const encryptedPath = "E2E/two-vault/encrypted.md";
type RunnerContext = {
binary: string;
cliBinary: string;
couchDb: CouchDbConfig;
dbName: string;
};
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
await rm(join(vaultPath, path), { force: true });
}
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
return await readFile(join(vaultPath, path), "utf-8");
}
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
try {
await readFile(join(vaultPath, path));
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
async function waitForPathContent(
vaultPath: string,
path: string,
predicate: (content: string) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const deadline = Date.now() + timeoutMs;
let lastContent = "";
while (Date.now() < deadline) {
if (await pathExists(vaultPath, path)) {
lastContent = await readVaultFile(vaultPath, path);
if (predicate(lastContent)) {
return lastContent;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
}
async function waitForPathDeleted(
vaultPath: string,
path: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (!(await pathExists(vaultPath, path))) {
return;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
}
async function writeNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string, content: string) {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(content)};`,
"const folder=path.split('/').slice(0,-1).join('/');",
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.modify(existing,content);",
"else await app.vault.create(path,content);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function deleteNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string) {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function renameNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, fromPath: string, toPath: string) {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const fromPath=${JSON.stringify(fromPath)};`,
`const toPath=${JSON.stringify(toPath)};`,
"const folder=toPath.split('/').slice(0,-1).join('/');",
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
"const existing=app.vault.getAbstractFileByPath(fromPath);",
"if(!existing) throw new Error(`Could not find note to rename: ${fromPath}`);",
"await app.vault.rename(existing,toPath);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
overrides: Record<string, unknown> = {}
): Promise<ObsidianLiveSyncSession> {
const session = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await configureCouchDb(
context.cliBinary,
session.cliEnv,
{
uri: context.couchDb.uri,
username: context.couchDb.username,
password: context.couchDb.password,
dbName: context.dbName,
},
overrides
);
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await prepareRemote(context.cliBinary, session.cliEnv);
return session;
}
async function uploadNote(
context: RunnerContext,
session: ObsidianLiveSyncSession,
path: string
): Promise<LocalDatabaseEntry> {
const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
});
return entry;
}
async function syncAndApply(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
await pushLocalChanges(context.cliBinary, session.cliEnv);
}
async function storeFileRevision(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
content: string,
baseRev?: string
): Promise<string> {
const result = await evalObsidianJson<{ rev: string }>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(content)};`,
`const baseRev=${JSON.stringify(baseRev ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const blob=new Blob([content],{type:'text/plain'});",
"const id=await core.services.path.path2id(path);",
"const now=Date.now();",
"const result=await core.localDatabase.putDBEntry({",
" _id:id,",
" path,",
" data:blob,",
" ctime:now,",
" mtime:now,",
" size:(await blob.arrayBuffer()).byteLength,",
" children:[],",
" datatype:'plain',",
" type:'plain',",
" eden:{},",
"},false,baseRev||undefined);",
"if(!result?.ok) throw new Error(`Could not store file revision: ${path}`);",
"return JSON.stringify({ok:true,rev:result.rev});",
"})()",
].join(""),
env
);
return result.rev;
}
async function createMarkdownConflict(
context: RunnerContext,
session: ObsidianLiveSyncSession,
vault: TemporaryVault,
path: string,
base: string,
left: string,
right: string
): Promise<void> {
const baseRev = await storeFileRevision(context.cliBinary, session.cliEnv, path, base);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path);
await storeFileRevision(context.cliBinary, session.cliEnv, path, left, baseRev);
await storeFileRevision(context.cliBinary, session.cliEnv, path, right, baseRev);
await writeVaultFile(vault.path, path, right);
}
async function autoMergeMarkdownConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const result=await core.localDatabase.managers.conflictManager.tryAutoMerge(path,true);",
"if(!('result' in result)){",
" throw new Error(`Markdown conflict was not auto-mergeable: ${path}; ${JSON.stringify(result)}`);",
"}",
"if(!(await core.databaseFileAccess.storeContent(path,result.result))){",
" throw new Error(`Could not store merged Markdown content: ${path}`);",
"}",
"if(!(await core.fileHandler.deleteRevisionFromDB(path,result.conflictedRev))){",
" throw new Error(`Could not delete conflicted revision: ${path}`);",
"}",
"if(!(await core.fileHandler.dbToStorage(path,path,true))){",
" throw new Error(`Could not reflect merged Markdown content: ${path}`);",
"}",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function runCreateUpdateDelete(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const createdContent = "# Created on A\n\nThis note should appear on B.\n";
let session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, createPath, createdContent);
await uploadNote(context, session, createPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
const createdOnB = await waitForPathContent(vaultB.path, createPath, (content) => content === createdContent);
await session.app.stop();
assertEqual(createdOnB, createdContent, "Created note did not round-trip to the second vault.");
const initialUpdateContent = "# Update target\n\nInitial content.\n";
const updatedContent = "# Update target\n\nUpdated content from A.\n";
session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, updatePath, initialUpdateContent);
await uploadNote(context, session, updatePath);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, updatePath, updatedContent);
await uploadNote(context, session, updatePath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
const updatedOnB = await waitForPathContent(vaultB.path, updatePath, (content) => content === updatedContent);
await session.app.stop();
assertEqual(updatedOnB, updatedContent, "Updated note content did not round-trip to the second vault.");
const deleteContent = "# Delete target\n\nThis note should be removed from B.\n";
session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, deletePath, deleteContent);
await uploadNote(context, session, deletePath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
await waitForPathContent(vaultB.path, deletePath, (content) => content === deleteContent);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await deleteNoteViaObsidian(context.cliBinary, session.cliEnv, deletePath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
await waitForPathDeleted(vaultB.path, deletePath);
await session.app.stop();
console.log("Two-vault note creation, update, and deletion round-tripped.");
}
async function runRename(context: RunnerContext, vaultA: TemporaryVault, vaultB: TemporaryVault): Promise<void> {
const renamedContent = "# Rename target\n\nThis note should move from A to B.\n";
let session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renamedContent);
await uploadNote(context, session, renameFromPath);
await renameNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renameToPath);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, renameToPath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
const renamedOnB = await waitForPathContent(vaultB.path, renameToPath, (content) => content === renamedContent);
await waitForPathDeleted(vaultB.path, renameFromPath);
await session.app.stop();
assertEqual(renamedOnB, renamedContent, "Renamed note content did not round-trip to the second vault.");
console.log("Two-vault note rename round-tripped.");
}
async function runEncryptedRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const encryptedContent = "# Encrypted round-trip\n\nThis note should synchronise with E2EE enabled.\n";
const encryptedOverrides = {
encrypt: true,
passphrase: "real-obsidian-e2e-passphrase",
usePathObfuscation: true,
E2EEAlgorithm: "v2",
};
let session = await startConfiguredSession(context, vaultA, encryptedOverrides);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, encryptedPath, encryptedContent);
await uploadNote(context, session, encryptedPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, encryptedOverrides);
await syncAndApply(context, session);
const received = await waitForPathContent(vaultB.path, encryptedPath, (content) => content === encryptedContent);
await session.app.stop();
assertEqual(received, encryptedContent, "Encrypted note did not round-trip to the second vault.");
console.log("Two-vault encrypted note synchronisation round-tripped.");
}
async function runMarkdownAutoMerge(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const base = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nBottom anchor\n";
const left = "# Conflict\n\nTop anchor\n\nLeft line\n\nMiddle anchor\n\nBottom anchor\n";
const right = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nRight tail\n\nBottom anchor\n";
let session = await startConfiguredSession(context, vaultB);
await createMarkdownConflict(context, session, vaultB, conflictPath, base, left, right);
await autoMergeMarkdownConflict(context.cliBinary, session.cliEnv, conflictPath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
const mergedOnB = await waitForPathContent(
vaultB.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail"),
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await syncAndApply(context, session);
const mergedOnA = await waitForPathContent(
vaultA.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail"),
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
);
await session.app.stop();
assertEqual(mergedOnA, mergedOnB, "Merged Markdown content was not consistent across both vaults.");
console.log("Markdown conflict was automatically merged and propagated by the next synchronisation.");
}
async function runTargetMismatch(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const ignoredContent = "# Target mismatch\n\nB should ignore this revision.\n";
const acceptedContent = "# Target mismatch\n\nB should accept this revision after its target filter changes.\n";
let session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, targetMismatchPath, ignoredContent);
await uploadNote(context, session, targetMismatchPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, {
syncOnlyRegEx: "^E2E/two-vault/allowed/.*",
});
await syncAndApply(context, session);
assertEqual(
await pathExists(vaultB.path, targetMismatchPath),
false,
"A note was reflected on a device where it was not a target file."
);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, targetMismatchPath, acceptedContent);
await uploadNote(context, session, targetMismatchPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, {
syncOnlyRegEx: "",
});
await syncAndApply(context, session);
const received = await waitForPathContent(
vaultB.path,
targetMismatchPath,
(content) => content === acceptedContent
);
await session.app.stop();
assertEqual(received, acceptedContent, "Target file was not reflected after the device accepted the path.");
console.log("Two-vault target mismatch skipped a non-target note, then reflected it after enabling the target.");
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync");
const encryptedDbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync-e2ee");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const encryptedVaultA = await createTemporaryVault();
const encryptedVaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
const encryptedContext: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName: encryptedDbName };
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
await createCouchDbDatabase(couchDb, encryptedDbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
console.log(`Temporary encrypted CouchDB database: ${encryptedDbName}`);
await runCreateUpdateDelete(context, vaultA, vaultB);
await runRename(context, vaultA, vaultB);
if (process.env.E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT === "true") {
await runMarkdownAutoMerge(context, vaultA, vaultB);
}
await runTargetMismatch(context, vaultA, vaultB);
await runEncryptedRoundTrip(encryptedContext, encryptedVaultA, encryptedVaultB);
} finally {
await vaultA.dispose();
await vaultB.dispose();
await encryptedVaultA.dispose();
await encryptedVaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
await deleteCouchDbDatabase(couchDb, encryptedDbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,122 @@
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
type CreatedNote = {
path: string;
read: string;
exists: boolean;
};
type ReadNote = {
exists: boolean;
read: string | null;
};
const notePath = "E2E/real-vault-reflection.md";
const noteContent = [
"# Real Obsidian E2E",
"",
"This note was created through Obsidian's own vault API.",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function waitForFileContent(vaultPath: string, path: string, expectedContent: string): Promise<void> {
const fullPath = join(vaultPath, path);
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000);
let lastError: unknown;
while (Date.now() < deadline) {
try {
const content = await readFile(fullPath, "utf-8");
if (content === expectedContent) {
return;
}
lastError = new Error(`Unexpected content in ${fullPath}`);
} catch (error) {
lastError = error;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for reflected vault file: ${fullPath}\nLast error: ${String(lastError)}`);
}
function assertEqual(actual: unknown, expected: unknown, message: string): void {
if (actual !== expected) {
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
}
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
const created = await evalObsidianJson<CreatedNote>(
cli.binary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
`const content=${JSON.stringify(noteContent)};`,
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"const file=await app.vault.create(path,content);",
"const read=await app.vault.read(file);",
"return JSON.stringify({path:file.path,read,exists:await app.vault.adapter.exists(path)});",
"})()",
].join(""),
session.cliEnv
);
assertEqual(created.path, notePath, "Obsidian created the note at an unexpected path.");
assertEqual(created.exists, true, "Obsidian adapter did not report the created note.");
assertEqual(created.read, noteContent, "Obsidian did not read back the created note content.");
await waitForFileContent(vault.path, notePath, noteContent);
const readBack = await evalObsidianJson<ReadNote>(
cli.binary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
"const file=app.vault.getAbstractFileByPath(path);",
"return JSON.stringify({exists:!!file,read:file?await app.vault.read(file):null});",
"})()",
].join(""),
session.cliEnv
);
assertEqual(readBack.exists, true, "Obsidian did not find the reflected note on read-back.");
assertEqual(readBack.read, noteContent, "Obsidian read-back content did not match the reflected file.");
console.log(`Created and verified reflected note: ${notePath}`);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});