From cc3d30dbcf2b92b580ffbdfa6603a98dec0ba3df Mon Sep 17 00:00:00 2001 From: vorotamoroz Date: Thu, 7 May 2026 11:06:12 +0100 Subject: [PATCH] feat(tests): add Deno-based tests for checking CLI functionality in the same-codebase between platforms. --- src/apps/cli/testdeno/CONTRIBUTING_TESTS.md | 150 +++++ src/apps/cli/testdeno/deno.json | 22 + src/apps/cli/testdeno/deno.lock | 31 + .../cli/testdeno/helpers/backgroundCli.ts | 112 ++++ src/apps/cli/testdeno/helpers/cli.ts | 231 ++++++++ src/apps/cli/testdeno/helpers/docker.ts | 530 ++++++++++++++++++ src/apps/cli/testdeno/helpers/env.ts | 26 + src/apps/cli/testdeno/helpers/p2p.ts | 52 ++ src/apps/cli/testdeno/helpers/settings.ts | 205 +++++++ src/apps/cli/testdeno/helpers/temp.ts | 33 ++ .../testdeno/test-e2e-two-vaults-couchdb.ts | 279 +++++++++ .../testdeno/test-e2e-two-vaults-matrix.ts | 20 + src/apps/cli/testdeno/test-mirror.ts | 196 +++++++ src/apps/cli/testdeno/test-p2p-host.ts | 40 ++ .../testdeno/test-p2p-peers-local-relay.ts | 42 ++ src/apps/cli/testdeno/test-p2p-sync.ts | 59 ++ .../testdeno/test-p2p-three-nodes-conflict.ts | 118 ++++ .../test-p2p-upload-download-repro.ts | 111 ++++ src/apps/cli/testdeno/test-push-pull.ts | 64 +++ src/apps/cli/testdeno/test-setup-put-cat.ts | 214 +++++++ .../cli/testdeno/test-sync-locked-remote.ts | 97 ++++ .../testdeno/test-sync-two-local-databases.ts | 287 ++++++++++ src/apps/cli/testdeno/test_dev_deno.md | 292 ++++++++++ 23 files changed, 3211 insertions(+) create mode 100644 src/apps/cli/testdeno/CONTRIBUTING_TESTS.md create mode 100644 src/apps/cli/testdeno/deno.json create mode 100644 src/apps/cli/testdeno/deno.lock create mode 100644 src/apps/cli/testdeno/helpers/backgroundCli.ts create mode 100644 src/apps/cli/testdeno/helpers/cli.ts create mode 100644 src/apps/cli/testdeno/helpers/docker.ts create mode 100644 src/apps/cli/testdeno/helpers/env.ts create mode 100644 src/apps/cli/testdeno/helpers/p2p.ts create mode 100644 src/apps/cli/testdeno/helpers/settings.ts create mode 100644 src/apps/cli/testdeno/helpers/temp.ts create mode 100644 src/apps/cli/testdeno/test-e2e-two-vaults-couchdb.ts create mode 100644 src/apps/cli/testdeno/test-e2e-two-vaults-matrix.ts create mode 100644 src/apps/cli/testdeno/test-mirror.ts create mode 100644 src/apps/cli/testdeno/test-p2p-host.ts create mode 100644 src/apps/cli/testdeno/test-p2p-peers-local-relay.ts create mode 100644 src/apps/cli/testdeno/test-p2p-sync.ts create mode 100644 src/apps/cli/testdeno/test-p2p-three-nodes-conflict.ts create mode 100644 src/apps/cli/testdeno/test-p2p-upload-download-repro.ts create mode 100644 src/apps/cli/testdeno/test-push-pull.ts create mode 100644 src/apps/cli/testdeno/test-setup-put-cat.ts create mode 100644 src/apps/cli/testdeno/test-sync-locked-remote.ts create mode 100644 src/apps/cli/testdeno/test-sync-two-local-databases.ts create mode 100644 src/apps/cli/testdeno/test_dev_deno.md diff --git a/src/apps/cli/testdeno/CONTRIBUTING_TESTS.md b/src/apps/cli/testdeno/CONTRIBUTING_TESTS.md new file mode 100644 index 0000000..b244807 --- /dev/null +++ b/src/apps/cli/testdeno/CONTRIBUTING_TESTS.md @@ -0,0 +1,150 @@ +# Writing CLI Tests on Deno + +This guide explains how to add or update tests under `src/apps/cli/testdeno/`. +Note that new tests should be added to the Deno suite rather than the existing bash suite due to the cross-platform execution and TypeScript benefits. + +## Scope + +The Deno suite is designed for cross-platform execution, with a strong focus on Windows compatibility while keeping behaviour equivalent to existing bash tests. + +## Principles + +- Keep one scenario per file when practical. +- Reuse helpers from `helpers/` rather than duplicating process, Docker, or settings logic. +- Prefer deterministic data over random inputs unless randomness is explicitly required. +- Ensure every test can clean up automatically. +- Keep assertions actionable with clear failure messages. + +## Directory structure + +``` +src/apps/cli/testdeno/ + helpers/ + backgroundCli.ts + cli.ts + docker.ts + env.ts + p2p.ts + settings.ts + temp.ts + test-*.ts + deno.json +``` + +## Test file naming + +- Use `test-.ts`. +- Use names aligned with existing bash tests when porting, for example: + - `test-sync-locked-remote.ts` + - `test-p2p-sync.ts` + +## Core helper usage + +### Temporary workspace + +Use `TempDir` and `await using` so cleanup is automatic: + +```ts +await using workDir = await TempDir.create("livesync-cli-my-test"); +``` + +### CLI execution + +- `runCli(...)`: returns code and combined output. +- `runCliOrFail(...)`: throws on non-zero exit. +- `runCliWithInputOrFail(input, ...)`: for `put` and stdin-driven commands. + +### Settings + +- `initSettingsFile(...)`: creates a baseline settings file. +- `applyCouchdbSettings(...)`: applies CouchDB fields. +- `applyRemoteSyncSettings(...)`: applies remote and encryption fields. +- `applyP2pSettings(...)`: applies P2P fields. +- `applyP2pTestTweaks(...)`: enables P2P-only test profile. + +### Docker services + +- `startCouchdb(...)`, `stopCouchdb()` +- `startP2pRelay()`, `stopP2pRelay()` + +### P2P discovery + +- `discoverPeer(...)` +- `maybeStartLocalRelay(...)` +- `stopLocalRelayIfStarted(...)` + +### Background host process + +Use `startCliInBackground(...)` for long-running host mode such as `p2p-host`. + +## Recommended test structure + +1. Arrange +2. Act +3. Assert +4. Cleanup in `finally` + +Example skeleton: + +```ts +Deno.test("feature: behaviour", async () => { + await using workDir = await TempDir.create("example"); + // Arrange + + try { + // Act + + // Assert + } finally { + // Optional explicit cleanup + } +}); +``` + +## Reliability guidelines + +- Use explicit waits only when needed for eventual consistency. +- Re-run sync operations where the protocol is eventually consistent. +- For network-sensitive commands, use `LIVESYNC_CLI_RETRY` during debugging. +- Keep Docker container reuse disabled by default unless debugging. + +## Environment variables + +Common variables: + +- `LIVESYNC_DOCKER_MODE` +- `LIVESYNC_DOCKER_COMMAND` +- `LIVESYNC_TEST_TEE` +- `LIVESYNC_DOCKER_TEE` +- `LIVESYNC_CLI_DEBUG` +- `LIVESYNC_CLI_VERBOSE` +- `LIVESYNC_CLI_RETRY` +- `LIVESYNC_DEBUG_KEEP_DOCKER` + +P2P variables: + +- `RELAY` +- `ROOM_ID` +- `PASSPHRASE` +- `APP_ID` +- `PEERS_TIMEOUT` +- `SYNC_TIMEOUT` +- `USE_INTERNAL_RELAY` + +## Adding a new test task + +1. Add the test file under `src/apps/cli/testdeno/`. +2. Add a task in `src/apps/cli/testdeno/deno.json`. +3. Update `src/apps/cli/testdeno/test_dev_deno.md`. +4. Run the new task locally. + +## Validation checklist + +- The test passes on a clean workspace. +- The test does not leave persistent artefacts unless explicitly requested. +- Failure messages identify both expected and actual behaviour. +- The corresponding task is documented. + +## Out of scope for this suite + +- One-off reproduction scripts that are not intended as stable regression tests. diff --git a/src/apps/cli/testdeno/deno.json b/src/apps/cli/testdeno/deno.json new file mode 100644 index 0000000..18797ae --- /dev/null +++ b/src/apps/cli/testdeno/deno.json @@ -0,0 +1,22 @@ +{ + "tasks": { + "test": "deno test -A --no-check .", + "test:local": "deno test -A --no-check test-setup-put-cat.ts test-mirror.ts", + "test:push-pull": "deno test -A --no-check test-push-pull.ts", + "test:setup-put-cat": "deno test -A --no-check test-setup-put-cat.ts", + "test:mirror": "deno test -A --no-check test-mirror.ts", + "test:sync-two-local": "deno test -A --no-check test-sync-two-local-databases.ts", + "test:sync-locked-remote": "deno test -A --no-check test-sync-locked-remote.ts", + "test:p2p-host": "deno test -A --no-check test-p2p-host.ts", + "test:p2p-peers": "deno test -A --no-check test-p2p-peers-local-relay.ts", + "test:p2p-sync": "deno test -A --no-check test-p2p-sync.ts", + "test:p2p-three-nodes": "deno test -A --no-check test-p2p-three-nodes-conflict.ts", + "test:p2p-upload-download": "deno test -A --no-check test-p2p-upload-download-repro.ts", + "test:e2e-couchdb": "deno test -A --no-check test-e2e-two-vaults-couchdb.ts", + "test:e2e-matrix": "deno test -A --no-check test-e2e-two-vaults-matrix.ts" + }, + "imports": { + "@std/assert": "jsr:@std/assert@^1.0.13", + "@std/path": "jsr:@std/path@^1.0.9" + } +} diff --git a/src/apps/cli/testdeno/deno.lock b/src/apps/cli/testdeno/deno.lock new file mode 100644 index 0000000..a65fb54 --- /dev/null +++ b/src/apps/cli/testdeno/deno.lock @@ -0,0 +1,31 @@ +{ + "version": "5", + "specifiers": { + "jsr:@std/assert@^1.0.13": "1.0.19", + "jsr:@std/internal@^1.0.12": "1.0.12", + "jsr:@std/path@^1.0.9": "1.1.4" + }, + "jsr": { + "@std/assert@1.0.19": { + "integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e", + "dependencies": [ + "jsr:@std/internal" + ] + }, + "@std/internal@1.0.12": { + "integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027" + }, + "@std/path@1.1.4": { + "integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5", + "dependencies": [ + "jsr:@std/internal" + ] + } + }, + "workspace": { + "dependencies": [ + "jsr:@std/assert@^1.0.13", + "jsr:@std/path@^1.0.9" + ] + } +} diff --git a/src/apps/cli/testdeno/helpers/backgroundCli.ts b/src/apps/cli/testdeno/helpers/backgroundCli.ts new file mode 100644 index 0000000..1269d0f --- /dev/null +++ b/src/apps/cli/testdeno/helpers/backgroundCli.ts @@ -0,0 +1,112 @@ +import { CLI_DIR } from "./cli.ts"; +import { join } from "@std/path"; + +const CLI_DIST = join(CLI_DIR, "dist", "index.cjs"); +const VERBOSE_ENABLED = Deno.env.get("LIVESYNC_CLI_VERBOSE") === "1"; +const DEBUG_ENABLED = Deno.env.get("LIVESYNC_CLI_DEBUG") === "1"; + +function decorateArgs(args: string[]): string[] { + return DEBUG_ENABLED ? ["-d", ...args] : VERBOSE_ENABLED ? ["-v", ...args] : args; +} + +async function pump( + stream: ReadableStream, + sink: (text: string) => void, + teeTarget: WritableStream | null +): Promise { + const reader = stream.getReader(); + const writer = teeTarget?.getWriter(); + const dec = new TextDecoder(); + try { + while (true) { + const { done, value } = await reader.read(); + if (done) break; + if (!value) continue; + sink(dec.decode(value, { stream: true })); + if (writer) { + await writer.write(value); + } + } + } finally { + if (writer) writer.releaseLock(); + reader.releaseLock(); + } +} + +export class BackgroundCliProcess { + #stdout = ""; + #stderr = ""; + #stdoutDone: Promise; + #stderrDone: Promise; + + constructor( + readonly child: Deno.ChildProcess, + readonly args: string[] + ) { + this.#stdoutDone = pump( + child.stdout, + (text) => { + this.#stdout += text; + }, + null + ); + this.#stderrDone = pump( + child.stderr, + (text) => { + this.#stderr += text; + }, + null + ); + } + + get stdout(): string { + return this.#stdout; + } + + get stderr(): string { + return this.#stderr; + } + + get combined(): string { + return this.#stdout + this.#stderr; + } + + async waitUntilContains(needle: string, timeoutMs = 15000): Promise { + const started = Date.now(); + while (Date.now() - started < timeoutMs) { + if (this.combined.includes(needle)) return; + const status = await Promise.race([ + this.child.status.then((s) => ({ type: "status" as const, status: s })), + new Promise<{ type: "tick" }>((resolve) => setTimeout(() => resolve({ type: "tick" }), 100)), + ]); + if (status.type === "status") { + throw new Error( + `Background CLI exited before '${needle}' appeared (code ${status.status.code})\n${this.combined}` + ); + } + } + throw new Error(`Timed out waiting for '${needle}'\n${this.combined}`); + } + + async stop(): Promise { + try { + this.child.kill("SIGTERM"); + } catch { + // ignore already-exited processes + } + const status = await this.child.status; + await Promise.all([this.#stdoutDone, this.#stderrDone]); + return status.code; + } +} + +export function startCliInBackground(...args: string[]): BackgroundCliProcess { + const child = new Deno.Command("node", { + args: [CLI_DIST, ...decorateArgs(args)], + cwd: CLI_DIR, + stdin: "null", + stdout: "piped", + stderr: "piped", + }).spawn(); + return new BackgroundCliProcess(child, args); +} diff --git a/src/apps/cli/testdeno/helpers/cli.ts b/src/apps/cli/testdeno/helpers/cli.ts new file mode 100644 index 0000000..8c78b52 --- /dev/null +++ b/src/apps/cli/testdeno/helpers/cli.ts @@ -0,0 +1,231 @@ +import { join } from "@std/path"; + +// --------------------------------------------------------------------------- +// Path resolution +// --------------------------------------------------------------------------- +// This file lives at: src/apps/cli/testdeno/helpers/cli.ts +// CLI root (src/apps/cli/) is two levels up. +// import.meta.dirname is available in Deno 1.40+ as an OS-native path string. +export const CLI_DIR: string = join(import.meta.dirname!, "..", ".."); +const CLI_DIST = join(CLI_DIR, "dist", "index.cjs"); + +// --------------------------------------------------------------------------- +// Result type +// --------------------------------------------------------------------------- +export interface CliResult { + stdout: string; + stderr: string; + /** stdout + stderr concatenated — useful for assertion messages. */ + combined: string; + code: number; +} + +const TEE_ENABLED = Deno.env.get("LIVESYNC_TEST_TEE") === "1"; +const VERBOSE_ENABLED = Deno.env.get("LIVESYNC_CLI_VERBOSE") === "1"; +const DEBUG_ENABLED = Deno.env.get("LIVESYNC_CLI_DEBUG") === "1"; + +function sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +function concatChunks(chunks: Uint8Array[]): Uint8Array { + const total = chunks.reduce((n, c) => n + c.length, 0); + const out = new Uint8Array(total); + let offset = 0; + for (const c of chunks) { + out.set(c, offset); + offset += c.length; + } + return out; +} + +async function collectStream( + stream: ReadableStream, + teeTarget: WritableStream | null +): Promise { + const reader = stream.getReader(); + const chunks: Uint8Array[] = []; + const writer = teeTarget?.getWriter(); + try { + while (true) { + const { done, value } = await reader.read(); + if (done) break; + if (value) { + chunks.push(value); + if (writer) { + await writer.write(value); + } + } + } + } finally { + if (writer) { + writer.releaseLock(); + } + reader.releaseLock(); + } + return concatChunks(chunks); +} + +async function runNodeCommand(args: string[], stdinData?: Uint8Array): Promise { + const cliArgs = DEBUG_ENABLED ? ["-d", ...args] : VERBOSE_ENABLED ? ["-v", ...args] : args; + const child = new Deno.Command("node", { + args: [CLI_DIST, ...cliArgs], + cwd: CLI_DIR, + stdin: stdinData ? "piped" : "null", + stdout: "piped", + stderr: "piped", + }).spawn(); + + const stdoutPromise = collectStream(child.stdout, TEE_ENABLED ? Deno.stdout.writable : null); + const stderrPromise = collectStream(child.stderr, TEE_ENABLED ? Deno.stderr.writable : null); + + if (stdinData) { + const w = child.stdin.getWriter(); + await w.write(stdinData); + await w.close(); + } + + const [status, stdout, stderr] = await Promise.all([child.status, stdoutPromise, stderrPromise]); + + const dec = new TextDecoder(); + const out = dec.decode(stdout); + const err = dec.decode(stderr); + return { stdout: out, stderr: err, combined: out + err, code: status.code }; +} + +function isTransientNetworkError(message: string): boolean { + const m = message.toLowerCase(); + return ( + m.includes("fetch failed") || + m.includes("econnreset") || + m.includes("econnrefused") || + m.includes("und_err_socket") || + m.includes("other side closed") + ); +} + +// --------------------------------------------------------------------------- +// Core runners +// --------------------------------------------------------------------------- + +/** + * Run the CLI (node dist/index.cjs) with the supplied arguments. + * Pass the vault / DB path as the first argument, exactly as the bash helpers + * do. Does NOT throw on non-zero exit — check `.code` yourself. + */ +export async function runCli(...args: string[]): Promise { + const retries = Number(Deno.env.get("LIVESYNC_CLI_RETRY") ?? "0"); + for (let attempt = 0; ; attempt++) { + const result = await runNodeCommand(args); + if (result.code === 0) return result; + + if (attempt >= retries || !isTransientNetworkError(result.combined)) { + return result; + } + const waitMs = 400 * (attempt + 1); + console.warn(`[WARN] transient CLI failure, retrying (${attempt + 1}/${retries}) in ${waitMs}ms`); + await sleep(waitMs); + } +} + +/** + * Run the CLI and throw if it exits non-zero. Returns stdout. + */ +export async function runCliOrFail(...args: string[]): Promise { + const r = await runCli(...args); + if (r.code !== 0) { + throw new Error(`CLI exited with code ${r.code}\nstdout: ${r.stdout}\nstderr: ${r.stderr}`); + } + return r.stdout; +} + +/** + * Run the CLI with data piped to stdin (equivalent to `echo … | run_cli …` + * or `cat file | run_cli …`). + */ +export async function runCliWithInput(input: string | Uint8Array, ...args: string[]): Promise { + const data = typeof input === "string" ? new TextEncoder().encode(input) : input; + + const retries = Number(Deno.env.get("LIVESYNC_CLI_RETRY") ?? "0"); + for (let attempt = 0; ; attempt++) { + const result = await runNodeCommand(args, data); + if (result.code === 0) return result; + + if (attempt >= retries || !isTransientNetworkError(result.combined)) { + return result; + } + const waitMs = 400 * (attempt + 1); + console.warn(`[WARN] transient CLI(stdin) failure, retrying (${attempt + 1}/${retries}) in ${waitMs}ms`); + await sleep(waitMs); + } +} + +/** + * runCliWithInput — throws on non-zero exit, returns stdout. + */ +export async function runCliWithInputOrFail(input: string | Uint8Array, ...args: string[]): Promise { + const r = await runCliWithInput(input, ...args); + if (r.code !== 0) { + throw new Error(`CLI (with stdin) exited with code ${r.code}\nstdout: ${r.stdout}\nstderr: ${r.stderr}`); + } + return r.stdout; +} + +// --------------------------------------------------------------------------- +// Output helpers +// --------------------------------------------------------------------------- + +/** Strip the CLIWatchAdapter banner line that `cat` emits. */ +export function sanitiseCatStdout(raw: string): string { + return raw + .split("\n") + .filter((l) => l !== "[CLIWatchAdapter] File watching is not enabled in CLI version") + .join("\n"); +} + +// --------------------------------------------------------------------------- +// Assertions (parity with test-helpers.sh) +// --------------------------------------------------------------------------- + +export function assertContains(haystack: string, needle: string, message: string): void { + if (!haystack.includes(needle)) { + throw new Error(`[FAIL] ${message}\nExpected to find: ${JSON.stringify(needle)}\nActual output:\n${haystack}`); + } +} + +export function assertNotContains(haystack: string, needle: string, message: string): void { + if (haystack.includes(needle)) { + throw new Error(`[FAIL] ${message}\nDid NOT expect: ${JSON.stringify(needle)}\nActual output:\n${haystack}`); + } +} + +export async function assertFilesEqual(expectedPath: string, actualPath: string, message: string): Promise { + const [expected, actual] = await Promise.all([Deno.readFile(expectedPath), Deno.readFile(actualPath)]); + if (expected.length !== actual.length || expected.some((b, i) => b !== actual[i])) { + const hex = async (d: Uint8Array) => { + const h = await crypto.subtle.digest("SHA-256", d); + return [...new Uint8Array(h)].map((b) => b.toString(16).padStart(2, "0")).join(""); + }; + throw new Error( + `[FAIL] ${message}\nexpected SHA-256: ${await hex(expected)}\nactual SHA-256: ${await hex(actual)}` + ); + } +} + +// --------------------------------------------------------------------------- +// JSON helpers +// --------------------------------------------------------------------------- + +export async function readJsonFile>(filePath: string): Promise { + return JSON.parse(await Deno.readTextFile(filePath)) as T; +} + +export function jsonStringField(jsonText: string, field: string): string { + const data = JSON.parse(jsonText) as Record; + const value = data[field]; + return typeof value === "string" ? value : ""; +} + +export function jsonFieldIsNa(data: Record, field: string): boolean { + return data[field] === "N/A"; +} diff --git a/src/apps/cli/testdeno/helpers/docker.ts b/src/apps/cli/testdeno/helpers/docker.ts new file mode 100644 index 0000000..5ecea1f --- /dev/null +++ b/src/apps/cli/testdeno/helpers/docker.ts @@ -0,0 +1,530 @@ +/** + * Docker service management for tests. + * + * CouchDB start/stop/init is implemented directly using `docker` CLI commands + * and the Fetch API, so it works on any platform where Docker (Desktop) is + * available — including Windows — without needing bash. + */ + +type DockerInvoker = { + bin: string; + prefix: string[]; + label: string; +}; + +let dockerInvokerPromise: Promise | null = null; +const DOCKER_TEE = Deno.env.get("LIVESYNC_DOCKER_TEE") === "1" || Deno.env.get("LIVESYNC_TEST_TEE") === "1"; + +// --------------------------------------------------------------------------- +// Low-level docker wrapper +// --------------------------------------------------------------------------- + +function parseCommand(command: string): { bin: string; prefix: string[] } { + const parts = command.trim().split(/\s+/).filter(Boolean); + if (parts.length === 0) { + throw new Error("LIVESYNC_DOCKER_COMMAND is empty"); + } + return { bin: parts[0], prefix: parts.slice(1) }; +} + +async function runCommand(bin: string, args: string[]): Promise<{ code: number; stdout: string; stderr: string }> { + const cmd = new Deno.Command(bin, { + args, + stdin: "null", + stdout: "piped", + stderr: "piped", + }); + try { + const { code, stdout, stderr } = await cmd.output(); + const dec = new TextDecoder(); + const result = { + code, + stdout: dec.decode(stdout), + stderr: dec.decode(stderr), + }; + if (DOCKER_TEE) { + if (result.stdout.trim().length > 0) { + console.log(`[docker:${bin}] ${result.stdout.trimEnd()}`); + } + if (result.stderr.trim().length > 0) { + console.error(`[docker:${bin}] ${result.stderr.trimEnd()}`); + } + } + return result; + } catch (err) { + if (err instanceof Deno.errors.NotFound) { + return { + code: 127, + stdout: "", + stderr: `Command not found: ${bin}`, + }; + } + throw err; + } +} + +async function resolveDockerInvoker(): Promise { + const custom = Deno.env.get("LIVESYNC_DOCKER_COMMAND")?.trim(); + if (custom) { + const parsed = parseCommand(custom); + const runner: DockerInvoker = { + ...parsed, + label: `custom(${custom})`, + }; + + // Validate custom command eagerly so misconfiguration fails fast. + const checkArgs = runner.prefix.length === 0 ? ["--version"] : [...runner.prefix, "docker", "--version"]; + const check = await runCommand(runner.bin, checkArgs); + if (check.code !== 0) { + throw new Error(`LIVESYNC_DOCKER_COMMAND is not usable: ${custom}\n${check.stderr || check.stdout}`); + } + return runner; + } + + const mode = (Deno.env.get("LIVESYNC_DOCKER_MODE") ?? "auto").toLowerCase(); + const onWindows = Deno.build.os === "windows"; + + const native: DockerInvoker = { bin: "docker", prefix: [], label: "docker" }; + const wsl: DockerInvoker = { bin: "wsl", prefix: [], label: "wsl docker" }; + + if (mode === "native") { + return native; + } + if (mode === "wsl") { + return wsl; + } + if (mode !== "auto") { + throw new Error(`Unsupported LIVESYNC_DOCKER_MODE='${mode}'. Use auto, native, or wsl.`); + } + + // On Windows we prefer `wsl docker` first, then native docker. + // This typically works better in setups where Docker is installed only in + // WSL and not exposed as docker.exe on PATH. + const candidates = onWindows ? [wsl, native] : [native, wsl]; + for (const c of candidates) { + if (c.bin === "docker") { + const r = await runCommand("docker", ["--version"]); + if (r.code === 0) return c; + continue; + } + const r = await runCommand("wsl", ["docker", "--version"]); + if (r.code === 0) return c; + } + + throw new Error( + [ + "Docker command is not available.", + "Set one of:", + "- LIVESYNC_DOCKER_MODE=native", + "- LIVESYNC_DOCKER_MODE=wsl", + "- LIVESYNC_DOCKER_COMMAND='docker'", + "- LIVESYNC_DOCKER_COMMAND='wsl docker'", + ].join("\n") + ); +} + +async function getDockerInvoker(): Promise { + if (!dockerInvokerPromise) { + dockerInvokerPromise = resolveDockerInvoker().then((r) => { + console.log(`[INFO] docker runner: ${r.label}`); + return r; + }); + } + return await dockerInvokerPromise; +} + +async function docker(...args: string[]): Promise<{ code: number; stdout: string; stderr: string }> { + const invoker = await getDockerInvoker(); + + // Either: + // docker + // Or: + // wsl docker + const finalArgs = + invoker.prefix.length === 0 + ? invoker.bin === "wsl" + ? ["docker", ...args] + : args + : [...invoker.prefix, ...args]; + + const r = await runCommand(invoker.bin, finalArgs); + return { code: r.code, stdout: r.stdout, stderr: r.stderr }; +} + +async function dockerOrFail(...args: string[]): Promise { + const r = await docker(...args); + if (r.code !== 0) { + throw new Error(`docker ${args[0]} failed (code ${r.code}): ${r.stderr.trim()}`); + } + return r.stdout; +} + +function sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +async function waitForCouchdbStable(hostname: string, user: string, password: string): Promise { + const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1"); + const auth = btoa(`${user}:${password}`); + const headers = { Authorization: `Basic ${auth}` }; + let consecutive = 0; + for (let i = 0; i < 30; i++) { + try { + const r = await fetch(`${h}/_up`, { + headers, + signal: AbortSignal.timeout(3000), + }); + if (r.ok) { + consecutive++; + if (consecutive >= 3) return; + } else { + consecutive = 0; + } + } catch { + consecutive = 0; + } + await sleep(500); + } + throw new Error("CouchDB did not become stable in time"); +} + +// --------------------------------------------------------------------------- +// Fetch with retry (mirrors cli_test_curl_json() retry loop) +// --------------------------------------------------------------------------- + +async function fetchRetry( + url: string, + init: RequestInit, + retries = 30, + delayMs = 2000, + allowStatus: number[] = [] +): Promise { + let lastError: unknown; + let lastStatus: number | undefined; + for (let i = 0; i < retries; i++) { + try { + const r = await fetch(url, { + signal: AbortSignal.timeout(5000), + ...init, + }); + lastStatus = r.status; + await r.body?.cancel().catch(() => {}); + if (r.ok || allowStatus.includes(r.status)) return; + lastError = `HTTP ${r.status}`; + } catch (e) { + lastError = e; + } + await sleep(delayMs); + } + throw new Error( + `Could not reach ${url} after ${retries} retries: ${lastError} (last status: ${lastStatus ?? "N/A"})` + ); +} + +// --------------------------------------------------------------------------- +// CouchDB +// --------------------------------------------------------------------------- +// +// TODO: these values could be configurable via environment variables. +// +const COUCHDB_CONTAINER = "couchdb-test"; +const COUCHDB_IMAGE = "couchdb:3.5.0"; + +const MINIO_CONTAINER = "minio-test"; +const MINIO_IMAGE = "minio/minio"; +const MINIO_MC_IMAGE = "minio/mc"; + +export async function stopCouchdb(): Promise { + await docker("stop", COUCHDB_CONTAINER); + await docker("rm", COUCHDB_CONTAINER); +} + +/** + * Start a CouchDB test container, initialise it, and create the test DB. + * Mirrors cli_test_start_couchdb() from test-helpers.sh, using direct + * docker / fetch calls instead of the bash util scripts. + */ +export async function startCouchdb(couchdbUri: string, user: string, password: string, dbname: string): Promise { + console.log("[INFO] stopping leftover CouchDB container if present"); + await stopCouchdb().catch(() => {}); + + console.log("[INFO] starting CouchDB test container"); + await dockerOrFail( + "run", + "-d", + "--name", + COUCHDB_CONTAINER, + "-p", + // TODO: port mapping should be configurable. + "5989:5984", + "-e", + `COUCHDB_USER=${user}`, + "-e", + `COUCHDB_PASSWORD=${password}`, + "-e", + "COUCHDB_SINGLE_NODE=y", + COUCHDB_IMAGE + ); + + console.log("[INFO] initialising CouchDB"); + await initCouchdb(couchdbUri, user, password); + + console.log("[INFO] waiting for CouchDB to become stable"); + await waitForCouchdbStable(couchdbUri, user, password); + + console.log(`[INFO] creating test database: ${dbname}`); + await createCouchdbDatabase(couchdbUri, user, password, dbname); +} + +/** + * Mirror couchdb-init.sh: configure single-node CouchDB via its REST API. + */ +async function initCouchdb(hostname: string, user: string, password: string, node = "_local"): Promise { + // Podman environments often resolve localhost to ::1; use 127.0.0.1 like + // the bash script does. + const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1"); + const auth = btoa(`${user}:${password}`); + const headers = { + "Content-Type": "application/json", + Authorization: `Basic ${auth}`, + }; + + const calls: Array<[string, string, string]> = [ + [ + "POST", + `${h}/_cluster_setup`, + JSON.stringify({ + action: "enable_single_node", + username: user, + password, + bind_address: "0.0.0.0", + port: 5984, + singlenode: true, + }), + ], + ["PUT", `${h}/_node/${node}/_config/chttpd/require_valid_user`, '"true"'], + ["PUT", `${h}/_node/${node}/_config/chttpd_auth/require_valid_user`, '"true"'], + ["PUT", `${h}/_node/${node}/_config/httpd/WWW-Authenticate`, '"Basic realm=\\"couchdb\\""'], + ["PUT", `${h}/_node/${node}/_config/httpd/enable_cors`, '"true"'], + ["PUT", `${h}/_node/${node}/_config/chttpd/enable_cors`, '"true"'], + ["PUT", `${h}/_node/${node}/_config/chttpd/max_http_request_size`, '"4294967296"'], + ["PUT", `${h}/_node/${node}/_config/couchdb/max_document_size`, '"50000000"'], + ["PUT", `${h}/_node/${node}/_config/cors/credentials`, '"true"'], + ["PUT", `${h}/_node/${node}/_config/cors/origins`, '"*"'], + ]; + + for (const [method, url, body] of calls) { + await fetchRetry(url, { method, headers, body }); + } +} + +export async function createCouchdbDatabase( + hostname: string, + user: string, + password: string, + dbname: string +): Promise { + const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1"); + const auth = btoa(`${user}:${password}`); + await fetchRetry(`${h}/${dbname}`, { + method: "PUT", + headers: { Authorization: `Basic ${auth}` }, + }); +} + +/** Update a CouchDB document via PUT. Returns the updated document. */ +export async function updateCouchdbDoc( + hostname: string, + user: string, + password: string, + docUrl: string, + updater: (doc: Record) => Record +): Promise { + const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1"); + const auth = btoa(`${user}:${password}`); + const headers = { + "Content-Type": "application/json", + Authorization: `Basic ${auth}`, + }; + const getRes = await fetch(`${h}/${docUrl}`, { headers }); + const current = (await getRes.json()) as Record; + const updated = updater(current); + await fetchRetry(`${h}/${docUrl}`, { + method: "PUT", + headers, + body: JSON.stringify(updated), + }); +} + +// --------------------------------------------------------------------------- +// MinIO +// --------------------------------------------------------------------------- + +function shQuote(value: string): string { + return `'${value.split("'").join(`'"'"'`)}'`; +} + +export async function stopMinio(): Promise { + await docker("stop", MINIO_CONTAINER); + await docker("rm", MINIO_CONTAINER); +} + +async function initMinioBucket( + minioEndpoint: string, + accessKey: string, + secretKey: string, + bucket: string +): Promise { + const cmd = + `mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` + + `mc mb --ignore-existing myminio/${shQuote(bucket)} >/dev/null 2>&1`; + const r = await docker("run", "--rm", "--network", "host", "--entrypoint", "/bin/sh", MINIO_MC_IMAGE, "-c", cmd); + return r.code === 0; +} + +async function waitForMinioBucket( + minioEndpoint: string, + accessKey: string, + secretKey: string, + bucket: string +): Promise { + for (let i = 0; i < 30; i++) { + const checkCmd = + `mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` + + `mc ls myminio/${shQuote(bucket)} >/dev/null 2>&1`; + const check = await docker( + "run", + "--rm", + "--network", + // Now I used host networking to access the container via localhost for some environments (Docker Desktop on Windows). + // We need something good idea to work across all environments. + "host", + "--entrypoint", + "/bin/sh", + MINIO_MC_IMAGE, + "-c", + checkCmd + ); + if (check.code === 0) { + return; + } + await initMinioBucket(minioEndpoint, accessKey, secretKey, bucket); + await sleep(2000); + } + throw new Error(`MinIO bucket not ready: ${bucket}`); +} + +export async function startMinio( + minioEndpoint: string, + accessKey: string, + secretKey: string, + bucket: string +): Promise { + console.log("[INFO] stopping leftover MinIO container if present"); + await stopMinio().catch(() => {}); + + console.log("[INFO] starting MinIO test container"); + await dockerOrFail( + "run", + "-d", + "--name", + MINIO_CONTAINER, + // TODO: Ports should be configurable. + "-p", + "9000:9000", + "-p", + "9001:9001", + "-e", + `MINIO_ROOT_USER=${accessKey}`, + "-e", + `MINIO_ROOT_PASSWORD=${secretKey}`, + "-e", + `MINIO_SERVER_URL=${minioEndpoint}`, + MINIO_IMAGE, + "server", + "/data", + "--console-address", + ":9001" + ); + + console.log(`[INFO] initialising MinIO test bucket: ${bucket}`); + let initialised = false; + for (let i = 0; i < 5; i++) { + if (await initMinioBucket(minioEndpoint, accessKey, secretKey, bucket)) { + initialised = true; + break; + } + await sleep(2000); + } + if (!initialised) { + throw new Error(`Could not initialise MinIO bucket after retries: ${bucket}`); + } + + await waitForMinioBucket(minioEndpoint, accessKey, secretKey, bucket); +} + +// --------------------------------------------------------------------------- +// P2P relay (strfry) +// --------------------------------------------------------------------------- +// TODO: these values could be configurable via environment variables. +const P2P_RELAY_CONTAINER = "relay-test"; +const P2P_RELAY_IMAGE = "ghcr.io/hoytech/strfry:latest"; +const STRFRY_BOOTSTRAP_SH = String.raw`cat > /tmp/strfry.conf <<"EOF" +db = "./strfry-db/" + +relay { + bind = "0.0.0.0" + port = 7777 + nofiles = 100000 + + info { + name = "livesync test relay" + description = "local relay for livesync p2p tests" + } + + maxWebsocketPayloadSize = 131072 + autoPingSeconds = 55 + + writePolicy { + plugin = "" + } +} +EOF +exec /app/strfry --config /tmp/strfry.conf relay`; + +export async function stopP2pRelay(): Promise { + await docker("stop", P2P_RELAY_CONTAINER); + await docker("rm", P2P_RELAY_CONTAINER); +} + +/** + * Start the local P2P relay container through the same docker runner used + * by CouchDB helpers. This keeps process ownership consistent across + * start/stop on Windows, WSL, and native Linux/macOS. + */ +export async function startP2pRelay(): Promise { + console.log("[INFO] stopping leftover P2P relay container if present"); + await stopP2pRelay().catch(() => {}); + + console.log("[INFO] starting local P2P relay container"); + await dockerOrFail( + "run", + "-d", + "--name", + P2P_RELAY_CONTAINER, + "-p", + //TODO: port mapping should be configurable. + "4000:7777", + "--tmpfs", + "/app/strfry-db:rw,size=256m", + "--entrypoint", + "sh", + P2P_RELAY_IMAGE, + "-lc", + STRFRY_BOOTSTRAP_SH + ); +} + +export function isLocalP2pRelay(relayUrl: string): boolean { + return relayUrl === "ws://localhost:4000" || relayUrl === "ws://localhost:4000/"; +} diff --git a/src/apps/cli/testdeno/helpers/env.ts b/src/apps/cli/testdeno/helpers/env.ts new file mode 100644 index 0000000..656a2f4 --- /dev/null +++ b/src/apps/cli/testdeno/helpers/env.ts @@ -0,0 +1,26 @@ +/** + * Load a .env-style file (KEY=value per line) into a plain object. + * Equivalent to `source $TEST_ENV_FILE; set -a` in bash. + * Maybe we should use some library... now it is just the minimal implementation that covers our use cases. + * + * Supported value formats: + * KEY=value + * KEY='single quoted' + * KEY="double quoted" + * # comment lines are ignored + */ +export async function loadEnvFile(filePath: string): Promise> { + const text = await Deno.readTextFile(filePath); + const result: Record = {}; + for (const line of text.split("\n")) { + const trimmed = line.trim(); + if (!trimmed || trimmed.startsWith("#")) continue; + const idx = trimmed.indexOf("="); + if (idx < 0) continue; + const key = trimmed.slice(0, idx).trim(); + const raw = trimmed.slice(idx + 1).trim(); + // Strip surrounding single or double quotes + result[key] = raw.replace(/^(['"])(.*)\1$/, "$2"); + } + return result; +} diff --git a/src/apps/cli/testdeno/helpers/p2p.ts b/src/apps/cli/testdeno/helpers/p2p.ts new file mode 100644 index 0000000..28741c4 --- /dev/null +++ b/src/apps/cli/testdeno/helpers/p2p.ts @@ -0,0 +1,52 @@ +import { runCli } from "./cli.ts"; +import { isLocalP2pRelay, startP2pRelay, stopP2pRelay } from "./docker.ts"; + +export type PeerEntry = { + id: string; + name: string; +}; + +export function parsePeerLines(output: string): PeerEntry[] { + return output + .split(/\r?\n/) + .map((line) => line.split("\t")) + .filter((parts) => parts.length >= 3 && parts[0] === "[peer]") + .map((parts) => ({ id: parts[1], name: parts[2] })); +} + +export async function discoverPeer( + vaultDir: string, + settingsFile: string, + timeoutSeconds: number, + targetPeer?: string +): Promise { + const result = await runCli(vaultDir, "--settings", settingsFile, "p2p-peers", String(timeoutSeconds)); + if (result.code !== 0) { + throw new Error(`p2p-peers failed\n${result.combined}`); + } + const peers = parsePeerLines(result.stdout); + if (targetPeer) { + const matched = peers.find((peer) => peer.id === targetPeer || peer.name === targetPeer); + if (matched) return matched; + } + if (peers.length === 0) { + const fallback = result.combined.match(/Advertisement from\s+([^\s]+)/); + if (fallback?.[1]) { + return { id: fallback[1], name: fallback[1] }; + } + throw new Error(`No peers discovered\n${result.combined}`); + } + return peers[0]; +} + +export async function maybeStartLocalRelay(relay: string): Promise { + if (!isLocalP2pRelay(relay)) return false; + await startP2pRelay(); + return true; +} + +export async function stopLocalRelayIfStarted(started: boolean): Promise { + if (started) { + await stopP2pRelay().catch(() => {}); + } +} diff --git a/src/apps/cli/testdeno/helpers/settings.ts b/src/apps/cli/testdeno/helpers/settings.ts new file mode 100644 index 0000000..6f88a03 --- /dev/null +++ b/src/apps/cli/testdeno/helpers/settings.ts @@ -0,0 +1,205 @@ +import { join } from "@std/path"; +import { CLI_DIR, runCliOrFail } from "./cli.ts"; + +// --------------------------------------------------------------------------- +// Settings file initialisation +// --------------------------------------------------------------------------- + +/** Generate a default settings file using the CLI's init-settings command. */ +export async function initSettingsFile(settingsFile: string): Promise { + await runCliOrFail("init-settings", "--force", settingsFile); +} + +/** + * Generate a full setup URI from a settings file via src/lib API. + * Mirrors the bash flow in test-setup-put-cat-linux.sh. + */ +export async function generateSetupUriFromSettings(settingsFile: string, setupPassphrase: string): Promise { + const repoRoot = join(CLI_DIR, "..", "..", ".."); + const script = [ + "import fs from 'node:fs';", + "import { pathToFileURL } from 'node:url';", + "(async () => {", + " const modulePath = process.env.REPO_ROOT + '/src/lib/src/API/processSetting.ts';", + " const moduleUrl = pathToFileURL(modulePath).href;", + " const { encodeSettingsToSetupURI } = await import(moduleUrl);", + " const settingsPath = process.env.SETTINGS_FILE;", + " const passphrase = process.env.SETUP_PASSPHRASE;", + " const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf-8'));", + " settings.couchDB_DBNAME = 'setup-put-cat-db';", + " settings.couchDB_URI = 'http://127.0.0.1:5999';", + " settings.couchDB_USER = 'dummy';", + " settings.couchDB_PASSWORD = 'dummy';", + " settings.liveSync = false;", + " settings.syncOnStart = false;", + " settings.syncOnSave = false;", + " const uri = await encodeSettingsToSetupURI(settings, passphrase);", + " process.stdout.write(uri.trim());", + "})();", + ].join("\n"); + + const scriptPath = await Deno.makeTempFile({ + prefix: "livesync-setup-uri-", + suffix: ".mts", + }); + await Deno.writeTextFile(scriptPath, script); + + try { + const cmd = new Deno.Command("npx", { + args: ["tsx", scriptPath], + cwd: CLI_DIR, + env: { + REPO_ROOT: repoRoot, + SETTINGS_FILE: settingsFile, + SETUP_PASSPHRASE: setupPassphrase, + }, + stdin: "null", + stdout: "piped", + stderr: "piped", + }); + + const { code, stdout, stderr } = await cmd.output(); + const dec = new TextDecoder(); + if (code !== 0) { + throw new Error( + `Failed to generate setup URI (code ${code})\nstdout: ${dec.decode(stdout)}\nstderr: ${dec.decode(stderr)}` + ); + } + + const uri = dec.decode(stdout).trim(); + if (!uri) { + throw new Error("Failed to generate setup URI: output is empty"); + } + return uri; + } finally { + await Deno.remove(scriptPath).catch(() => {}); + } +} + +/** Set isConfigured=true in a settings file (required for mirror / scan). */ +export async function markSettingsConfigured(settingsFile: string): Promise { + const data = JSON.parse(await Deno.readTextFile(settingsFile)); + data.isConfigured = true; + await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2)); +} + +// --------------------------------------------------------------------------- +// CouchDB remote settings +// --------------------------------------------------------------------------- + +/** + * Apply CouchDB connection details to a settings file. + * Mirrors cli_test_apply_couchdb_settings() from test-helpers.sh. + */ +export async function applyCouchdbSettings( + settingsFile: string, + couchdbUri: string, + couchdbUser: string, + couchdbPassword: string, + couchdbDbname: string, + liveSync = false +): Promise { + const data = JSON.parse(await Deno.readTextFile(settingsFile)); + data.couchDB_URI = couchdbUri; + data.couchDB_USER = couchdbUser; + data.couchDB_PASSWORD = couchdbPassword; + data.couchDB_DBNAME = couchdbDbname; + if (liveSync) { + data.liveSync = true; + data.syncOnStart = false; + data.syncOnSave = false; + data.usePluginSync = false; + } + data.isConfigured = true; + await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2)); +} + +export async function applyRemoteSyncSettings( + settingsFile: string, + options: { + remoteType: "COUCHDB" | "MINIO"; + couchdbUri?: string; + couchdbUser?: string; + couchdbPassword?: string; + couchdbDbname?: string; + minioBucket?: string; + minioEndpoint?: string; + minioAccessKey?: string; + minioSecretKey?: string; + encrypt?: boolean; + passphrase?: string; + } +): Promise { + const data = JSON.parse(await Deno.readTextFile(settingsFile)); + + if (options.remoteType === "COUCHDB") { + data.remoteType = ""; + data.couchDB_URI = options.couchdbUri; + data.couchDB_USER = options.couchdbUser; + data.couchDB_PASSWORD = options.couchdbPassword; + data.couchDB_DBNAME = options.couchdbDbname; + } else { + data.remoteType = "MINIO"; + data.bucket = options.minioBucket; + data.endpoint = options.minioEndpoint; + data.accessKey = options.minioAccessKey; + data.secretKey = options.minioSecretKey; + data.region = "auto"; + data.forcePathStyle = true; + } + + data.liveSync = true; + data.syncOnStart = false; + data.syncOnSave = false; + data.usePluginSync = false; + data.encrypt = options.encrypt === true; + data.passphrase = options.encrypt ? (options.passphrase ?? "") : ""; + data.isConfigured = true; + await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2)); +} + +// --------------------------------------------------------------------------- +// P2P settings +// --------------------------------------------------------------------------- + +/** + * Apply P2P connection details to a settings file. + * Mirrors cli_test_apply_p2p_settings() from test-helpers.sh. + */ +export async function applyP2pSettings( + settingsFile: string, + roomId: string, + passphrase: string, + appId = "self-hosted-livesync-cli-tests", + relays = "ws://localhost:4000/", + autoAccept = "~.*" +): Promise { + const data = JSON.parse(await Deno.readTextFile(settingsFile)); + data.P2P_Enabled = true; + data.P2P_AutoStart = false; + data.P2P_AutoBroadcast = false; + data.P2P_AppID = appId; + data.P2P_roomID = roomId; + data.P2P_passphrase = passphrase; + data.P2P_relays = relays; + data.P2P_AutoAcceptingPeers = autoAccept; + data.P2P_AutoDenyingPeers = ""; + data.P2P_IsHeadless = true; + data.isConfigured = true; + await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2)); +} + +export async function applyP2pTestTweaks(settingsFile: string, deviceName: string, passphrase: string): Promise { + const data = JSON.parse(await Deno.readTextFile(settingsFile)); + data.remoteType = "ONLY_P2P"; + data.encrypt = true; + data.passphrase = passphrase; + data.usePathObfuscation = true; + data.handleFilenameCaseSensitive = false; + data.customChunkSize = 50; + data.usePluginSyncV2 = true; + data.doNotUseFixedRevisionForChunks = false; + data.P2P_DevicePeerName = deviceName; + data.isConfigured = true; + await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2)); +} diff --git a/src/apps/cli/testdeno/helpers/temp.ts b/src/apps/cli/testdeno/helpers/temp.ts new file mode 100644 index 0000000..655d18d --- /dev/null +++ b/src/apps/cli/testdeno/helpers/temp.ts @@ -0,0 +1,33 @@ +import { join } from "@std/path"; + +/** + * A temporary directory that cleans itself up via `await using`. + * Requires TypeScript 5.2+ / Deno 1.40+ for the AsyncDisposable protocol. + * + * @example + * ```ts + * await using tmp = await TempDir.create(); + * const file = tmp.join("data.json"); + * ``` + */ +export class TempDir implements AsyncDisposable { + readonly path: string; + + private constructor(path: string) { + this.path = path; + } + + static async create(prefix = "livesync-deno-test"): Promise { + const path = await Deno.makeTempDir({ prefix: `${prefix}.` }); + return new TempDir(path); + } + + /** Return an OS path joined to the temp directory root. */ + join(...parts: string[]): string { + return join(this.path, ...parts); + } + + async [Symbol.asyncDispose](): Promise { + await Deno.remove(this.path, { recursive: true }).catch(() => {}); + } +} diff --git a/src/apps/cli/testdeno/test-e2e-two-vaults-couchdb.ts b/src/apps/cli/testdeno/test-e2e-two-vaults-couchdb.ts new file mode 100644 index 0000000..f1b60f1 --- /dev/null +++ b/src/apps/cli/testdeno/test-e2e-two-vaults-couchdb.ts @@ -0,0 +1,279 @@ +import { assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { loadEnvFile } from "./helpers/env.ts"; +import { + runCli, + runCliOrFail, + runCliWithInputOrFail, + sanitiseCatStdout, + assertFilesEqual, + jsonStringField, +} from "./helpers/cli.ts"; +import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts"; +import { startCouchdb, startMinio, stopCouchdb, stopMinio } from "./helpers/docker.ts"; +import { join } from "@std/path"; + +const TEST_ENV = join(import.meta.dirname!, "..", ".test.env"); +type RemoteType = "COUCHDB" | "MINIO"; + +function requireEnv(env: Record, key: string): string { + const value = env[key]?.trim(); + if (!value) throw new Error(`Required env var is missing: ${key}`); + return value; +} + +export async function runScenario(remoteType: RemoteType, encrypt: boolean): Promise { + const env = await loadEnvFile(TEST_ENV); + const dbSuffix = `${Date.now()}-${Math.floor(Math.random() * 100000)}`; + + const couchdbUri = remoteType === "COUCHDB" ? requireEnv(env, "hostname").replace(/\/$/, "") : ""; + const couchdbUser = remoteType === "COUCHDB" ? requireEnv(env, "username") : ""; + const couchdbPassword = remoteType === "COUCHDB" ? requireEnv(env, "password") : ""; + const dbPrefix = remoteType === "COUCHDB" ? requireEnv(env, "dbname") : ""; + const dbname = remoteType === "COUCHDB" ? `${dbPrefix}-${dbSuffix}` : ""; + + const minioEndpoint = remoteType === "MINIO" ? requireEnv(env, "minioEndpoint").replace(/\/$/, "") : ""; + const minioAccessKey = remoteType === "MINIO" ? requireEnv(env, "accessKey") : ""; + const minioSecretKey = remoteType === "MINIO" ? requireEnv(env, "secretKey") : ""; + const minioBucketBase = remoteType === "MINIO" ? requireEnv(env, "bucketName") : ""; + const minioBucket = remoteType === "MINIO" ? `${minioBucketBase}-${dbSuffix}` : ""; + + const passphrase = "e2e-passphrase"; + + await using workDir = await TempDir.create( + `livesync-cli-e2e-${remoteType.toLowerCase()}-${encrypt ? "enc1" : "enc0"}` + ); + const vaultA = workDir.join("testvault_a"); + const vaultB = workDir.join("testvault_b"); + const settingsA = workDir.join("test-settings-a.json"); + const settingsB = workDir.join("test-settings-b.json"); + const pushSrc = workDir.join("push-source.txt"); + const pullDst = workDir.join("pull-destination.txt"); + const pushBinarySrc = workDir.join("push-source.bin"); + const pullBinaryDst = workDir.join("pull-destination.bin"); + await Deno.mkdir(vaultA, { recursive: true }); + await Deno.mkdir(vaultB, { recursive: true }); + + const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1"; + if (remoteType === "COUCHDB") { + await startCouchdb(couchdbUri, couchdbUser, couchdbPassword, dbname); + } else { + await startMinio(minioEndpoint, minioAccessKey, minioSecretKey, minioBucket); + } + + try { + await initSettingsFile(settingsA); + await initSettingsFile(settingsB); + await applyRemoteSyncSettings(settingsA, { + remoteType, + couchdbUri, + couchdbUser, + couchdbPassword, + couchdbDbname: dbname, + minioBucket, + minioEndpoint, + minioAccessKey, + minioSecretKey, + encrypt, + passphrase, + }); + await applyRemoteSyncSettings(settingsB, { + remoteType, + couchdbUri, + couchdbUser, + couchdbPassword, + couchdbDbname: dbname, + minioBucket, + minioEndpoint, + minioAccessKey, + minioSecretKey, + encrypt, + passphrase, + }); + + const syncBoth = async () => { + await runCliOrFail(vaultA, "--settings", settingsA, "sync"); + await runCliOrFail(vaultB, "--settings", settingsB, "sync"); + }; + + const targetAOnly = "e2e/a-only-info.md"; + const targetSync = "e2e/sync-info.md"; + const targetSyncTwiceFirst = "e2e/sync-twice-first.md"; + const targetSyncTwiceSecond = "e2e/sync-twice-second.md"; + const targetPush = "e2e/pushed-from-a.md"; + const targetPut = "e2e/put-from-a.md"; + const targetPushBinary = "e2e/pushed-from-a.bin"; + const targetConflict = "e2e/conflict.md"; + + await runCliWithInputOrFail("alpha-from-a\n", vaultA, "--settings", settingsA, "put", targetAOnly); + const infoAOnly = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetAOnly); + assert(infoAOnly.includes(`"path": "${targetAOnly}"`)); + + await runCliWithInputOrFail("visible-after-sync\n", vaultA, "--settings", settingsA, "put", targetSync); + await syncBoth(); + const infoBSync = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetSync); + assert(infoBSync.includes(`"path": "${targetSync}"`)); + + await runCliWithInputOrFail( + `first-sync-round-${dbSuffix}\n`, + vaultA, + "--settings", + settingsA, + "put", + targetSyncTwiceFirst + ); + await runCliOrFail(vaultA, "--settings", settingsA, "sync"); + await runCliOrFail(vaultB, "--settings", settingsB, "sync"); + const firstVisible = sanitiseCatStdout( + await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetSyncTwiceFirst) + ).trimEnd(); + assert(firstVisible === `first-sync-round-${dbSuffix}`); + + await runCliWithInputOrFail( + `second-sync-round-${dbSuffix}\n`, + vaultA, + "--settings", + settingsA, + "put", + targetSyncTwiceSecond + ); + await runCliOrFail(vaultA, "--settings", settingsA, "sync"); + await runCliOrFail(vaultB, "--settings", settingsB, "sync"); + const secondVisible = sanitiseCatStdout( + await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetSyncTwiceSecond) + ).trimEnd(); + assert(secondVisible === `second-sync-round-${dbSuffix}`); + + await Deno.writeTextFile(pushSrc, `pushed-content-${dbSuffix}\n`); + await runCliOrFail(vaultA, "--settings", settingsA, "push", pushSrc, targetPush); + await runCliWithInputOrFail(`put-content-${dbSuffix}\n`, vaultA, "--settings", settingsA, "put", targetPut); + await syncBoth(); + await runCliOrFail(vaultB, "--settings", settingsB, "pull", targetPush, pullDst); + await assertFilesEqual(pushSrc, pullDst, "B pull result does not match pushed source"); + const catBPut = sanitiseCatStdout( + await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetPut) + ).trimEnd(); + assert(catBPut === `put-content-${dbSuffix}`); + + const binary = new Uint8Array(4096); + binary.fill(0x61); + await Deno.writeFile(pushBinarySrc, binary); + await runCliOrFail(vaultA, "--settings", settingsA, "push", pushBinarySrc, targetPushBinary); + await syncBoth(); + await runCliOrFail(vaultB, "--settings", settingsB, "pull", targetPushBinary, pullBinaryDst); + await assertFilesEqual(pushBinarySrc, pullBinaryDst, "B pull result does not match pushed binary source"); + + await runCliOrFail(vaultA, "--settings", settingsA, "rm", targetPut); + await syncBoth(); + const removed = await runCli(vaultB, "--settings", settingsB, "cat", targetPut); + assert(removed.code !== 0, `B cat should fail after A removed the file\n${removed.combined}`); + + await runCliWithInputOrFail("conflict-base\n", vaultA, "--settings", settingsA, "put", targetConflict); + await syncBoth(); + await runCliWithInputOrFail( + `conflict-from-a-${dbSuffix}\n`, + vaultA, + "--settings", + settingsA, + "put", + targetConflict + ); + await runCliWithInputOrFail( + `conflict-from-b-${dbSuffix}\n`, + vaultB, + "--settings", + settingsB, + "put", + targetConflict + ); + + let infoAConflict = ""; + let infoBConflict = ""; + let conflictDetected = false; + for (const side of ["a", "b", "a"] as const) { + await runCliOrFail( + side === "a" ? vaultA : vaultB, + "--settings", + side === "a" ? settingsA : settingsB, + "sync" + ); + infoAConflict = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetConflict); + infoBConflict = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetConflict); + if ( + jsonStringField(infoAConflict, "conflicts") !== "N/A" || + jsonStringField(infoBConflict, "conflicts") !== "N/A" + ) { + conflictDetected = true; + break; + } + } + assert(conflictDetected, `conflict was expected\nA: ${infoAConflict}\nB: ${infoBConflict}`); + + const lsAConflict = + (await runCliOrFail(vaultA, "--settings", settingsA, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? ""; + const lsBConflict = + (await runCliOrFail(vaultB, "--settings", settingsB, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? ""; + const revA = lsAConflict.split("\t")[3] ?? ""; + const revB = lsBConflict.split("\t")[3] ?? ""; + assert( + revA.includes("*") || revB.includes("*"), + `conflicted entry should be marked with '*'\nA: ${lsAConflict}\nB: ${lsBConflict}` + ); + + const keepRevision = jsonStringField(infoAConflict, "revision"); + assert(keepRevision.length > 0, `could not extract revision\n${infoAConflict}`); + await runCliOrFail(vaultA, "--settings", settingsA, "resolve", targetConflict, keepRevision); + + let resolved = false; + let infoAResolved = ""; + let infoBResolved = ""; + for (let i = 0; i < 6; i++) { + await syncBoth(); + infoAResolved = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetConflict); + infoBResolved = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetConflict); + if ( + jsonStringField(infoAResolved, "conflicts") === "N/A" && + jsonStringField(infoBResolved, "conflicts") === "N/A" + ) { + resolved = true; + break; + } + const retryRevision = jsonStringField(infoAResolved, "revision"); + if (retryRevision) { + await runCli(vaultA, "--settings", settingsA, "resolve", targetConflict, retryRevision); + } + } + assert(resolved, `conflicts should be resolved\nA: ${infoAResolved}\nB: ${infoBResolved}`); + + const lsAResolved = + (await runCliOrFail(vaultA, "--settings", settingsA, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? ""; + const lsBResolved = + (await runCliOrFail(vaultB, "--settings", settingsB, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? ""; + assert(!(lsAResolved.split("\t")[3] ?? "").includes("*")); + assert(!(lsBResolved.split("\t")[3] ?? "").includes("*")); + + const catAResolved = sanitiseCatStdout( + await runCliOrFail(vaultA, "--settings", settingsA, "cat", targetConflict) + ).trimEnd(); + const catBResolved = sanitiseCatStdout( + await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetConflict) + ).trimEnd(); + assert(catAResolved === catBResolved, `resolved content should match\nA: ${catAResolved}\nB: ${catBResolved}`); + } finally { + if (!keepDocker) { + if (remoteType === "COUCHDB") { + await stopCouchdb().catch(() => {}); + } else { + await stopMinio().catch(() => {}); + } + } + } +} + +Deno.test("e2e: two vaults over CouchDB without encryption", async () => { + await runScenario("COUCHDB", false); +}); + +Deno.test("e2e: two vaults over CouchDB with encryption", async () => { + await runScenario("COUCHDB", true); +}); diff --git a/src/apps/cli/testdeno/test-e2e-two-vaults-matrix.ts b/src/apps/cli/testdeno/test-e2e-two-vaults-matrix.ts new file mode 100644 index 0000000..757d6b5 --- /dev/null +++ b/src/apps/cli/testdeno/test-e2e-two-vaults-matrix.ts @@ -0,0 +1,20 @@ +import { runScenario } from "./test-e2e-two-vaults-couchdb.ts"; + +type MatrixCase = { + remoteType: "COUCHDB" | "MINIO"; + encrypt: boolean; + label: string; +}; + +const matrixCases: MatrixCase[] = [ + { remoteType: "COUCHDB", encrypt: false, label: "COUCHDB-enc0" }, + { remoteType: "COUCHDB", encrypt: true, label: "COUCHDB-enc1" }, + { remoteType: "MINIO", encrypt: false, label: "MINIO-enc0" }, + { remoteType: "MINIO", encrypt: true, label: "MINIO-enc1" }, +]; + +for (const tc of matrixCases) { + Deno.test(`e2e matrix: ${tc.label}`, async () => { + await runScenario(tc.remoteType, tc.encrypt); + }); +} diff --git a/src/apps/cli/testdeno/test-mirror.ts b/src/apps/cli/testdeno/test-mirror.ts new file mode 100644 index 0000000..b6eae4d --- /dev/null +++ b/src/apps/cli/testdeno/test-mirror.ts @@ -0,0 +1,196 @@ +/** + * Deno port of test-mirror-linux.sh + * + * Tests the `mirror` command — bidirectional synchronisation between a local + * storage directory (vault) and an in-process database. + * + * Covered cases (identical to the bash test): + * 1. Storage-only file -> synced into DB (UPDATE DATABASE) + * 2. DB-only file -> restored to storage (UPDATE STORAGE) + * 3. DB-deleted file -> NOT restored to storage (UPDATE STORAGE skip) + * 4. Both, storage newer -> DB updated (SYNC: STORAGE -> DB) + * 5. Both, DB newer -> storage updated (SYNC: DB -> STORAGE) + * 6. Compatibility mode -> omitted vault-path works (same DB + vault path) + * + * No external services are required. + * + * Run: + * deno test -A test-mirror.ts + */ + +import { assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { runCliOrFail } from "./helpers/cli.ts"; +import { initSettingsFile, markSettingsConfigured } from "./helpers/settings.ts"; + +Deno.test("mirror: storage <-> DB synchronisation", async (t) => { + await using workDir = await TempDir.create("livesync-cli-mirror"); + + // ------------------------------------------------------------------- + // Shared setup + // ------------------------------------------------------------------- + const settingsFile = workDir.join("data.json"); + const vaultDir = workDir.join("vault"); + const dbDir = workDir.join("db"); + await Deno.mkdir(workDir.join("vault", "test"), { recursive: true }); + await Deno.mkdir(dbDir, { recursive: true }); + + await initSettingsFile(settingsFile); + // isConfigured=true is required for canProceedScan in the mirror command. + await markSettingsConfigured(settingsFile); + + // Copy settings to the DB directory (separated-path mode) + const dbSettings = workDir.join("db", "settings.json"); + await Deno.copyFile(settingsFile, dbSettings); + + /** Run mirror in separated-path mode: DB dir ≠ vault dir. */ + const runMirror = () => runCliOrFail(dbDir, "--settings", dbSettings, "mirror", vaultDir); + + /** Run mirror in compatibility mode: DB path = vault path. */ + const runMirrorCompat = () => runCliOrFail(vaultDir, "--settings", settingsFile, "mirror"); + + // Helper wrappers + const dbRun = (...args: string[]) => runCliOrFail(dbDir, "--settings", dbSettings, ...args); + const compatRun = (...args: string[]) => runCliOrFail(vaultDir, "--settings", settingsFile, ...args); + + // ------------------------------------------------------------------- + // Case 1: storage-only -> DB (UPDATE DATABASE) + // ------------------------------------------------------------------- + await t.step("case 1: storage-only file is synced into DB", async () => { + const storageFile = workDir.join("vault", "test", "storage-only.md"); + await Deno.writeTextFile(storageFile, "storage-only content\n"); + + await runMirror(); + + const resultFile = workDir.join("case1-pull.txt"); + await dbRun("pull", "test/storage-only.md", resultFile); + + const storageContent = await Deno.readTextFile(storageFile); + const pulledContent = await Deno.readTextFile(resultFile); + assert( + storageContent === pulledContent, + `storage-only file NOT synced into DB\nexpected: ${storageContent}\ngot: ${pulledContent}` + ); + console.log("[PASS] case 1: storage-only file was synced into DB"); + }); + + // ------------------------------------------------------------------- + // Case 2: DB-only -> storage (UPDATE STORAGE) + // ------------------------------------------------------------------- + await t.step("case 2: DB-only file is restored to storage", async () => { + await dbRun( + "push", + // write inline via push (pipe not needed — push takes a file path) + // create a temp file with content and push it + await (async () => { + const tmp = workDir.join("db-only-src.txt"); + await Deno.writeTextFile(tmp, "db-only content\n"); + return tmp; + })(), + "test/db-only.md" + ); + + const storagePath = workDir.join("vault", "test", "db-only.md"); + assert(!(await exists(storagePath)), "db-only.md unexpectedly exists in storage before mirror"); + + await runMirror(); + + assert(await exists(storagePath), "DB-only file NOT restored to storage after mirror"); + const content = await Deno.readTextFile(storagePath); + assert(content === "db-only content\n", `DB-only file restored but content mismatch: '${content}'`); + console.log("[PASS] case 2: DB-only file was restored to storage"); + }); + + // ------------------------------------------------------------------- + // Case 3: DB-deleted -> storage untouched + // ------------------------------------------------------------------- + await t.step("case 3: DB-deleted entry is NOT restored to storage", async () => { + const deletedSrc = workDir.join("deleted-src.txt"); + await Deno.writeTextFile(deletedSrc, "to-be-deleted\n"); + await dbRun("push", deletedSrc, "test/deleted.md"); + await dbRun("rm", "test/deleted.md"); + + await runMirror(); + + const storagePath = workDir.join("vault", "test", "deleted.md"); + assert(!(await exists(storagePath)), "deleted DB entry was incorrectly restored to storage"); + console.log("[PASS] case 3: deleted DB entry was NOT restored to storage"); + }); + + // ------------------------------------------------------------------- + // Case 4: storage newer -> DB updated (SYNC: STORAGE -> DB) + // ------------------------------------------------------------------- + await t.step("case 4: storage newer than DB -> DB is updated", async () => { + // Seed DB with old content (mtime ~ now) + const seedFile = workDir.join("case4-seed.txt"); + await Deno.writeTextFile(seedFile, "old content\n"); + await dbRun("push", seedFile, "test/sync-storage-newer.md"); + + // Write new content to storage with a timestamp 1 hour in the future + const storageFile = workDir.join("vault", "test", "sync-storage-newer.md"); + await Deno.writeTextFile(storageFile, "new content\n"); + await Deno.utime(storageFile, new Date(), new Date(Date.now() + 3600_000)); + + await runMirror(); + + const resultFile = workDir.join("case4-pull.txt"); + await dbRun("pull", "test/sync-storage-newer.md", resultFile); + const storageContent = await Deno.readTextFile(storageFile); + const pulledContent = await Deno.readTextFile(resultFile); + assert( + storageContent === pulledContent, + `DB NOT updated to match newer storage file\nexpected: ${storageContent}\ngot: ${pulledContent}` + ); + console.log("[PASS] case 4: DB updated to match newer storage file"); + }); + + // ------------------------------------------------------------------- + // Case 5: DB newer -> storage updated (SYNC: DB -> STORAGE) + // ------------------------------------------------------------------- + await t.step("case 5: DB newer than storage -> storage is updated", async () => { + // Write old content to storage with a timestamp 1 hour in the past + const storageFile = workDir.join("vault", "test", "sync-db-newer.md"); + await Deno.writeTextFile(storageFile, "old storage content\n"); + await Deno.utime(storageFile, new Date(), new Date(Date.now() - 3600_000)); + + // Write new content to DB only (mtime ~ now, newer than the storage file) + const dbNewFile = workDir.join("case5-db-new.txt"); + await Deno.writeTextFile(dbNewFile, "new db content\n"); + await dbRun("push", dbNewFile, "test/sync-db-newer.md"); + + await runMirror(); + + const content = await Deno.readTextFile(storageFile); + assert(content === "new db content\n", `storage NOT updated to match newer DB entry (got: '${content}')`); + console.log("[PASS] case 5: storage updated to match newer DB entry"); + }); + + // ------------------------------------------------------------------- + // Case 6: compatibility mode (vault path = DB path) + // ------------------------------------------------------------------- + await t.step("case 6: compatibility mode (omitted vault-path)", async () => { + const compatFile = workDir.join("vault", "compat.md"); + await Deno.writeTextFile(compatFile, "compat-content\n"); + + await runMirrorCompat(); + + const resultFile = workDir.join("case6-pull.txt"); + await compatRun("pull", "compat.md", resultFile); + const pulled = await Deno.readTextFile(resultFile); + assert(pulled === "compat-content\n", `Compatibility mode failed to sync file into DB (got: '${pulled}')`); + console.log("[PASS] case 6: compatibility mode works"); + }); +}); + +// --------------------------------------------------------------------------- +// Utility +// --------------------------------------------------------------------------- + +async function exists(path: string): Promise { + try { + await Deno.stat(path); + return true; + } catch { + return false; + } +} diff --git a/src/apps/cli/testdeno/test-p2p-host.ts b/src/apps/cli/testdeno/test-p2p-host.ts new file mode 100644 index 0000000..7c67c5f --- /dev/null +++ b/src/apps/cli/testdeno/test-p2p-host.ts @@ -0,0 +1,40 @@ +import { assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { initSettingsFile, applyP2pSettings } from "./helpers/settings.ts"; +import { startP2pRelay, stopP2pRelay, isLocalP2pRelay } from "./helpers/docker.ts"; +import { startCliInBackground } from "./helpers/backgroundCli.ts"; + +Deno.test("p2p-host: starts and becomes ready", async () => { + const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/"; + const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`; + const passphrase = Deno.env.get("PASSPHRASE") ?? "test"; + const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests"; + const useInternalRelay = Deno.env.get("USE_INTERNAL_RELAY") !== "0"; + + await using workDir = await TempDir.create("livesync-cli-p2p-host"); + const vaultDir = workDir.join("vault-host"); + const settingsFile = workDir.join("settings-host.json"); + await Deno.mkdir(vaultDir, { recursive: true }); + + let relayStarted = false; + if (useInternalRelay && isLocalP2pRelay(relay)) { + await startP2pRelay(); + relayStarted = true; + } + + try { + await initSettingsFile(settingsFile); + await applyP2pSettings(settingsFile, roomId, passphrase, appId, relay); + const host = startCliInBackground(vaultDir, "--settings", settingsFile, "p2p-host"); + try { + await host.waitUntilContains("P2P host is running", 20000); + assert(host.combined.includes("P2P host is running")); + } finally { + await host.stop(); + } + } finally { + if (relayStarted) { + await stopP2pRelay().catch(() => {}); + } + } +}); diff --git a/src/apps/cli/testdeno/test-p2p-peers-local-relay.ts b/src/apps/cli/testdeno/test-p2p-peers-local-relay.ts new file mode 100644 index 0000000..f73fe35 --- /dev/null +++ b/src/apps/cli/testdeno/test-p2p-peers-local-relay.ts @@ -0,0 +1,42 @@ +import { assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { initSettingsFile, applyP2pSettings, applyP2pTestTweaks } from "./helpers/settings.ts"; +import { startCliInBackground } from "./helpers/backgroundCli.ts"; +import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts"; + +Deno.test("p2p-peers: discovers host through local relay", async () => { + const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/"; + const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`; + const passphrase = Deno.env.get("PASSPHRASE") ?? "test"; + const timeoutSeconds = Number(Deno.env.get("TIMEOUT_SECONDS") ?? "8"); + + await using workDir = await TempDir.create("livesync-cli-p2p-peers-local-relay"); + const hostVault = workDir.join("vault-host"); + const hostSettings = workDir.join("settings-host.json"); + const clientVault = workDir.join("vault"); + const clientSettings = workDir.join("settings.json"); + await Deno.mkdir(hostVault, { recursive: true }); + await Deno.mkdir(clientVault, { recursive: true }); + + const relayStarted = await maybeStartLocalRelay(relay); + try { + await initSettingsFile(hostSettings); + await initSettingsFile(clientSettings); + await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay); + await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay); + await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase); + await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase); + + const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host"); + try { + await host.waitUntilContains("P2P host is running", 20000); + const peer = await discoverPeer(clientVault, clientSettings, timeoutSeconds); + assert(peer.id.length > 0); + assert(peer.name.length > 0); + } finally { + await host.stop(); + } + } finally { + await stopLocalRelayIfStarted(relayStarted); + } +}); diff --git a/src/apps/cli/testdeno/test-p2p-sync.ts b/src/apps/cli/testdeno/test-p2p-sync.ts new file mode 100644 index 0000000..970b4e9 --- /dev/null +++ b/src/apps/cli/testdeno/test-p2p-sync.ts @@ -0,0 +1,59 @@ +import { assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { initSettingsFile, applyP2pSettings, applyP2pTestTweaks } from "./helpers/settings.ts"; +import { startCliInBackground } from "./helpers/backgroundCli.ts"; +import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts"; +import { runCli } from "./helpers/cli.ts"; + +Deno.test("p2p-sync: discovers peer and completes sync", async () => { + const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/"; + const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`; + const passphrase = Deno.env.get("PASSPHRASE") ?? "test"; + const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "12"); + const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15"); + + await using workDir = await TempDir.create("livesync-cli-p2p-sync"); + const hostVault = workDir.join("vault-host"); + const hostSettings = workDir.join("settings-host.json"); + const clientVault = workDir.join("vault-sync"); + const clientSettings = workDir.join("settings-sync.json"); + await Deno.mkdir(hostVault, { recursive: true }); + await Deno.mkdir(clientVault, { recursive: true }); + + const relayStarted = await maybeStartLocalRelay(relay); + try { + await initSettingsFile(hostSettings); + await initSettingsFile(clientSettings); + await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay); + await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay); + await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase); + await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase); + + const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host"); + try { + await host.waitUntilContains("P2P host is running", 20000); + const peer = await discoverPeer( + clientVault, + clientSettings, + peersTimeout, + Deno.env.get("TARGET_PEER") ?? undefined + ); + const syncResult = await runCli( + clientVault, + "--settings", + clientSettings, + "p2p-sync", + peer.id, + String(syncTimeout) + ); + assert( + syncResult.code === 0, + `p2p-sync failed\nstdout: ${syncResult.stdout}\nstderr: ${syncResult.stderr}` + ); + } finally { + await host.stop(); + } + } finally { + await stopLocalRelayIfStarted(relayStarted); + } +}); diff --git a/src/apps/cli/testdeno/test-p2p-three-nodes-conflict.ts b/src/apps/cli/testdeno/test-p2p-three-nodes-conflict.ts new file mode 100644 index 0000000..744692e --- /dev/null +++ b/src/apps/cli/testdeno/test-p2p-three-nodes-conflict.ts @@ -0,0 +1,118 @@ +import { assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { applyP2pSettings, initSettingsFile } from "./helpers/settings.ts"; +import { startCliInBackground } from "./helpers/backgroundCli.ts"; +import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts"; +import { jsonStringField, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts"; + +Deno.test("p2p: three nodes detect and resolve conflicts", async () => { + const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/"; + const roomId = `${Deno.env.get("ROOM_ID_PREFIX") ?? "p2p-room"}-${Date.now()}`; + const passphrase = `${Deno.env.get("PASSPHRASE_PREFIX") ?? "p2p-pass"}-${Date.now()}`; + const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests"; + const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "10"); + const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15"); + + await using workDir = await TempDir.create("livesync-cli-p2p-3nodes"); + const vaultA = workDir.join("vault-a"); + const vaultB = workDir.join("vault-b"); + const vaultC = workDir.join("vault-c"); + const settingsA = workDir.join("settings-a.json"); + const settingsB = workDir.join("settings-b.json"); + const settingsC = workDir.join("settings-c.json"); + await Deno.mkdir(vaultA, { recursive: true }); + await Deno.mkdir(vaultB, { recursive: true }); + await Deno.mkdir(vaultC, { recursive: true }); + + const relayStarted = await maybeStartLocalRelay(relay); + try { + for (const settings of [settingsA, settingsB, settingsC]) { + await initSettingsFile(settings); + await applyP2pSettings(settings, roomId, passphrase, appId, relay); + } + + const host = startCliInBackground(vaultA, "--settings", settingsA, "p2p-host"); + try { + await host.waitUntilContains("P2P host is running", 20000); + const peerFromB = await discoverPeer(vaultB, settingsB, peersTimeout); + const peerFromC = await discoverPeer(vaultC, settingsC, peersTimeout); + const targetPath = "p2p/conflicted-from-two-clients.txt"; + + await runCliWithInputOrFail("from-client-b-v1\n", vaultB, "--settings", settingsB, "put", targetPath); + await runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout)); + await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout)); + + let visibleOnC = ""; + for (let i = 0; i < 5; i++) { + try { + visibleOnC = sanitiseCatStdout( + await runCliOrFail(vaultC, "--settings", settingsC, "cat", targetPath) + ).trimEnd(); + if (visibleOnC === "from-client-b-v1") break; + } catch { + // retry below + } + await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout)); + } + assert(visibleOnC === "from-client-b-v1", `C should see file created by B, got: ${visibleOnC}`); + + await runCliWithInputOrFail("from-client-b-v2\n", vaultB, "--settings", settingsB, "put", targetPath); + await runCliWithInputOrFail("from-client-c-v2\n", vaultC, "--settings", settingsC, "put", targetPath); + + const [syncB, syncC] = await Promise.all([ + runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout)), + runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout)), + ]); + void syncB; + void syncC; + + await runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout)); + await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout)); + + const infoBBefore = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetPath); + const conflictsBBefore = jsonStringField(infoBBefore, "conflicts"); + const keepRevB = jsonStringField(infoBBefore, "revision"); + assert( + conflictsBBefore !== "N/A" && conflictsBBefore.length > 0, + `expected conflicts on B\n${infoBBefore}` + ); + assert(keepRevB.length > 0, `could not read revision on B\n${infoBBefore}`); + + const infoCBefore = await runCliOrFail(vaultC, "--settings", settingsC, "info", targetPath); + const conflictsCBefore = jsonStringField(infoCBefore, "conflicts"); + const keepRevC = jsonStringField(infoCBefore, "revision"); + assert( + conflictsCBefore !== "N/A" && conflictsCBefore.length > 0, + `expected conflicts on C\n${infoCBefore}` + ); + assert(keepRevC.length > 0, `could not read revision on C\n${infoCBefore}`); + + await runCliOrFail(vaultB, "--settings", settingsB, "resolve", targetPath, keepRevB); + await runCliOrFail(vaultC, "--settings", settingsC, "resolve", targetPath, keepRevC); + + const infoBAfter = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetPath); + const infoCAfter = await runCliOrFail(vaultC, "--settings", settingsC, "info", targetPath); + assert(jsonStringField(infoBAfter, "conflicts") === "N/A", `conflict still remains on B\n${infoBAfter}`); + assert(jsonStringField(infoCAfter, "conflicts") === "N/A", `conflict still remains on C\n${infoCAfter}`); + + const finalContentB = sanitiseCatStdout( + await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetPath) + ).trimEnd(); + const finalContentC = sanitiseCatStdout( + await runCliOrFail(vaultC, "--settings", settingsC, "cat", targetPath) + ).trimEnd(); + assert( + finalContentB === "from-client-b-v2" || finalContentB === "from-client-c-v2", + `unexpected final content on B: ${finalContentB}` + ); + assert( + finalContentC === "from-client-b-v2" || finalContentC === "from-client-c-v2", + `unexpected final content on C: ${finalContentC}` + ); + } finally { + await host.stop(); + } + } finally { + await stopLocalRelayIfStarted(relayStarted); + } +}); diff --git a/src/apps/cli/testdeno/test-p2p-upload-download-repro.ts b/src/apps/cli/testdeno/test-p2p-upload-download-repro.ts new file mode 100644 index 0000000..ba07fd5 --- /dev/null +++ b/src/apps/cli/testdeno/test-p2p-upload-download-repro.ts @@ -0,0 +1,111 @@ +import { TempDir } from "./helpers/temp.ts"; +import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts"; +import { startCliInBackground } from "./helpers/backgroundCli.ts"; +import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts"; +import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts"; + +async function writeFilledFile(path: string, size: number, byte: number): Promise { + const data = new Uint8Array(size); + data.fill(byte); + await Deno.writeFile(path, data); +} + +Deno.test("p2p: upload/download reproduction scenario", async () => { + const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/"; + const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests"; + const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "20"); + const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "240"); + const roomId = `p2p-room-${Date.now()}`; + const passphrase = `p2p-pass-${Date.now()}`; + + await using workDir = await TempDir.create("livesync-cli-p2p-upload-download"); + const vaultHost = workDir.join("vault-host"); + const vaultUp = workDir.join("vault-up"); + const vaultDown = workDir.join("vault-down"); + const settingsHost = workDir.join("settings-host.json"); + const settingsUp = workDir.join("settings-up.json"); + const settingsDown = workDir.join("settings-down.json"); + for (const dir of [vaultHost, vaultUp, vaultDown]) { + await Deno.mkdir(dir, { recursive: true }); + } + + const relayStarted = await maybeStartLocalRelay(relay); + try { + for (const settings of [settingsHost, settingsUp, settingsDown]) { + await initSettingsFile(settings); + await applyP2pSettings(settings, roomId, passphrase, appId, relay, "~.*"); + } + await applyP2pTestTweaks(settingsHost, "p2p-cli-host", passphrase); + await applyP2pTestTweaks(settingsUp, `p2p-cli-upload-${Date.now()}`, passphrase); + await applyP2pTestTweaks(settingsDown, `p2p-cli-download-${Date.now()}`, passphrase); + + const host = startCliInBackground(vaultHost, "--settings", settingsHost, "p2p-host"); + try { + await host.waitUntilContains("P2P host is running", 20000); + const uploadPeer = await discoverPeer(vaultUp, settingsUp, peersTimeout); + + const storeText = workDir.join("store-file.md"); + const diffA = workDir.join("test-diff-1.md"); + const diffB = workDir.join("test-diff-2.md"); + const diffC = workDir.join("test-diff-3.md"); + await Deno.writeTextFile(storeText, "Hello, World!\n"); + await Deno.writeTextFile(diffA, "Content A\n"); + await Deno.writeTextFile(diffB, "Content B\n"); + await Deno.writeTextFile(diffC, "Content C\n"); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", storeText, "p2p/store-file.md"); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffA, "p2p/test-diff-1.md"); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffB, "p2p/test-diff-2.md"); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffC, "p2p/test-diff-3.md"); + + const large100k = workDir.join("large-100k.txt"); + const large1m = workDir.join("large-1m.txt"); + const binary100k = workDir.join("binary-100k.bin"); + const binary5m = workDir.join("binary-5m.bin"); + await Deno.writeTextFile(large100k, "a".repeat(100000)); + await Deno.writeTextFile(large1m, "b".repeat(1000000)); + await writeFilledFile(binary100k, 100000, 0x5a); + await writeFilledFile(binary5m, 5000000, 0x7c); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", large100k, "p2p/large-100000.md"); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", large1m, "p2p/large-1000000.md"); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", binary100k, "p2p/binary-100000.bin"); + await runCliOrFail(vaultUp, "--settings", settingsUp, "push", binary5m, "p2p/binary-5000000.bin"); + + await runCliOrFail(vaultUp, "--settings", settingsUp, "p2p-sync", uploadPeer.id, String(syncTimeout)); + await runCliOrFail(vaultUp, "--settings", settingsUp, "p2p-sync", uploadPeer.id, String(syncTimeout)); + + const downloadPeer = await discoverPeer(vaultDown, settingsDown, peersTimeout); + await runCliOrFail(vaultDown, "--settings", settingsDown, "p2p-sync", downloadPeer.id, String(syncTimeout)); + await runCliOrFail(vaultDown, "--settings", settingsDown, "p2p-sync", downloadPeer.id, String(syncTimeout)); + + const downStoreText = workDir.join("down-store-file.md"); + const downDiffA = workDir.join("down-test-diff-1.md"); + const downDiffB = workDir.join("down-test-diff-2.md"); + const downDiffC = workDir.join("down-test-diff-3.md"); + const downLarge100k = workDir.join("down-large-100k.txt"); + const downLarge1m = workDir.join("down-large-1m.txt"); + const downBinary100k = workDir.join("down-binary-100k.bin"); + const downBinary5m = workDir.join("down-binary-5m.bin"); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/store-file.md", downStoreText); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-1.md", downDiffA); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-2.md", downDiffB); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-3.md", downDiffC); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/large-100000.md", downLarge100k); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/large-1000000.md", downLarge1m); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/binary-100000.bin", downBinary100k); + await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/binary-5000000.bin", downBinary5m); + + await assertFilesEqual(storeText, downStoreText, "store-file mismatch"); + await assertFilesEqual(diffA, downDiffA, "test-diff-1 mismatch"); + await assertFilesEqual(diffB, downDiffB, "test-diff-2 mismatch"); + await assertFilesEqual(diffC, downDiffC, "test-diff-3 mismatch"); + await assertFilesEqual(large100k, downLarge100k, "large-100000 mismatch"); + await assertFilesEqual(large1m, downLarge1m, "large-1000000 mismatch"); + await assertFilesEqual(binary100k, downBinary100k, "binary-100000 mismatch"); + await assertFilesEqual(binary5m, downBinary5m, "binary-5000000 mismatch"); + } finally { + await host.stop(); + } + } finally { + await stopLocalRelayIfStarted(relayStarted); + } +}); diff --git a/src/apps/cli/testdeno/test-push-pull.ts b/src/apps/cli/testdeno/test-push-pull.ts new file mode 100644 index 0000000..3b04a1e --- /dev/null +++ b/src/apps/cli/testdeno/test-push-pull.ts @@ -0,0 +1,64 @@ +/** + * Deno port of test-push-pull-linux.sh + * + * Requires CouchDB connection details either via environment variables or a + * .test.env file. If neither is present the test logs a warning and the + * CLI will likely fail at the push step. + * + * Run: + * deno test -A test-push-pull.ts + * + * With explicit CouchDB: + * COUCHDB_URI=http://127.0.0.1:5984 \ + * COUCHDB_USER=admin \ + * COUCHDB_PASSWORD=password \ + * COUCHDB_DBNAME=livesync-test \ + * deno test -A test-push-pull.ts + */ + +import { join } from "@std/path"; +import { assertEquals } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { runCliOrFail } from "./helpers/cli.ts"; +import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts"; + +const REMOTE_PATH = Deno.env.get("REMOTE_PATH") ?? "test/push-pull.txt"; + +Deno.test("push/pull roundtrip", async () => { + await using workDir = await TempDir.create("livesync-cli-push-pull"); + + const settingsFile = workDir.join("data.json"); + const vaultDir = workDir.join("vault"); + await Deno.mkdir(join(vaultDir, "test"), { recursive: true }); + + await initSettingsFile(settingsFile); + + const uri = Deno.env.get("COUCHDB_URI") ?? ""; + const user = Deno.env.get("COUCHDB_USER") ?? ""; + const password = Deno.env.get("COUCHDB_PASSWORD") ?? ""; + const dbname = Deno.env.get("COUCHDB_DBNAME") ?? ""; + + if (uri && user && password && dbname) { + console.log("[INFO] applying CouchDB env vars to settings"); + await applyCouchdbSettings(settingsFile, uri, user, password, dbname); + } else { + console.warn( + "[WARN] CouchDB env vars not fully set — push/pull may fail unless the generated settings already contain connection details" + ); + } + + const srcFile = workDir.join("push-source.txt"); + const pulledFile = workDir.join("pull-result.txt"); + const content = `push-pull-test ${new Date().toISOString()}\n`; + await Deno.writeTextFile(srcFile, content); + + console.log(`[INFO] push -> ${REMOTE_PATH}`); + await runCliOrFail(vaultDir, "--settings", settingsFile, "push", srcFile, REMOTE_PATH); + + console.log(`[INFO] pull <- ${REMOTE_PATH}`); + await runCliOrFail(vaultDir, "--settings", settingsFile, "pull", REMOTE_PATH, pulledFile); + + const pulled = await Deno.readTextFile(pulledFile); + assertEquals(content, pulled, "push/pull roundtrip content mismatch"); + console.log("[PASS] push/pull roundtrip matched"); +}); diff --git a/src/apps/cli/testdeno/test-setup-put-cat.ts b/src/apps/cli/testdeno/test-setup-put-cat.ts new file mode 100644 index 0000000..f2a7696 --- /dev/null +++ b/src/apps/cli/testdeno/test-setup-put-cat.ts @@ -0,0 +1,214 @@ +/** + * Deno port of test-setup-put-cat-linux.sh + * + * Tests all local-DB file operations that require no external remote: + * setup / + * push / cat / ls / info / rm / resolve / cat-rev / pull-rev + * + * Run (no external services needed): + * deno test -A test-setup-put-cat.ts + */ + +import { join } from "@std/path"; +import { assertEquals, assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { runCli, runCliOrFail, runCliWithInput, sanitiseCatStdout } from "./helpers/cli.ts"; +import { generateSetupUriFromSettings, initSettingsFile } from "./helpers/settings.ts"; + +const REMOTE_PATH = Deno.env.get("REMOTE_PATH") ?? "test/setup-put-cat.txt"; +const SETUP_PASSPHRASE = Deno.env.get("SETUP_PASSPHRASE") ?? "setup-passphrase"; + +Deno.test("CLI file operations: push / cat / ls / info / rm / resolve / cat-rev / pull-rev", async (t) => { + await using workDir = await TempDir.create("livesync-cli-setup-put-cat"); + + const settingsFile = workDir.join("data.json"); + const vaultDir = workDir.join("vault"); + await Deno.mkdir(join(vaultDir, "test"), { recursive: true }); + + await initSettingsFile(settingsFile); + + const setupUri = await generateSetupUriFromSettings(settingsFile, SETUP_PASSPHRASE); + const setupResult = await runCliWithInput( + `${SETUP_PASSPHRASE}\n`, + vaultDir, + "--settings", + settingsFile, + "setup", + setupUri + ); + assert(setupResult.code === 0, `setup command exited with ${setupResult.code}\n${setupResult.combined}`); + assert( + setupResult.combined.includes("[Command] setup ->"), + `setup command did not execute expected code path\n${setupResult.combined}` + ); + + const run = (...args: string[]) => runCliOrFail(vaultDir, "--settings", settingsFile, ...args); + + // ------------------------------------------------------------------ + // push / cat roundtrip + // ------------------------------------------------------------------ + await t.step("push/cat roundtrip", async () => { + const srcFile = workDir.join("put-source.txt"); + const content = `setup-put-cat-test ${new Date().toISOString()}\nline-2\n`; + await Deno.writeTextFile(srcFile, content); + + console.log(`[INFO] push -> ${REMOTE_PATH}`); + await runCliWithInput(content, vaultDir, "--settings", settingsFile, "put", REMOTE_PATH); + + console.log(`[INFO] cat <- ${REMOTE_PATH}`); + const rawOutput = await run("cat", REMOTE_PATH); + const catOutput = sanitiseCatStdout(rawOutput); + + assertEquals(content, catOutput, "push/cat roundtrip content mismatch"); + console.log("[PASS] push/cat roundtrip matched"); + }); + + // ------------------------------------------------------------------ + // ls: single file + // ------------------------------------------------------------------ + await t.step("ls output format (single file)", async () => { + const lsOutput = await run("ls", REMOTE_PATH); + const line = lsOutput + .trim() + .split("\n") + .find((l) => l.startsWith(REMOTE_PATH + "\t")); + assert(line, `ls output did not include ${REMOTE_PATH}`); + + const [lsPath, lsSize, lsMtime, lsRev] = line.split("\t"); + assertEquals(lsPath, REMOTE_PATH, "ls path column mismatch"); + assert(/^\d+$/.test(lsSize), `ls size not numeric: ${lsSize}`); + assert(/^\d+$/.test(lsMtime), `ls mtime not numeric: ${lsMtime}`); + assert(lsRev?.length > 0, "ls revision column is empty"); + console.log("[PASS] ls output format matched"); + }); + + // ------------------------------------------------------------------ + // ls: prefix filter and sort order + // ------------------------------------------------------------------ + await t.step("ls prefix filter and sort order", async () => { + await runCliWithInput("file-a\n", vaultDir, "--settings", settingsFile, "put", "test/a-first.txt"); + await runCliWithInput("file-z\n", vaultDir, "--settings", settingsFile, "put", "test/z-last.txt"); + + const lsOut = await run("ls", "test/"); + const lines = lsOut.trim().split("\n").filter(Boolean); + assert(lines.length >= 3, "ls prefix output expected at least 3 rows"); + + // Verify sorted ascending by path + const paths = lines.map((l) => l.split("\t")[0]); + for (let i = 1; i < paths.length; i++) { + assert(paths[i - 1] <= paths[i], `ls output not sorted: ${paths[i - 1]} > ${paths[i]}`); + } + assert( + lines.some((l) => l.startsWith("test/a-first.txt\t")), + "ls prefix output missing test/a-first.txt" + ); + assert( + lines.some((l) => l.startsWith("test/z-last.txt\t")), + "ls prefix output missing test/z-last.txt" + ); + console.log("[PASS] ls prefix and sorting matched"); + }); + + // ------------------------------------------------------------------ + // ls: no-match prefix returns empty output + // ------------------------------------------------------------------ + await t.step("ls no-match prefix returns empty", async () => { + const lsOut = await run("ls", "no-such-prefix/"); + assertEquals(lsOut.trim(), "", "ls no-match prefix should produce empty output"); + console.log("[PASS] ls no-match prefix matched"); + }); + + // ------------------------------------------------------------------ + // info: JSON output format + // ------------------------------------------------------------------ + await t.step("info output JSON format", async () => { + const infoOut = await run("info", REMOTE_PATH); + let data: Record; + try { + data = JSON.parse(infoOut); + } catch { + throw new Error(`info output is not valid JSON:\n${infoOut}`); + } + assertEquals(data.path, REMOTE_PATH, "info .path mismatch"); + assertEquals(data.filename, REMOTE_PATH.split("/").at(-1), "info .filename mismatch"); + assert(typeof data.size === "number" && data.size >= 0, `info .size invalid: ${data.size}`); + assert(typeof data.chunks === "number" && (data.chunks as number) >= 1, `info .chunks invalid: ${data.chunks}`); + assertEquals(data.conflicts, "N/A", "info .conflicts should be N/A"); + console.log("[PASS] info output format matched"); + }); + + // ------------------------------------------------------------------ + // info: non-existent path exits non-zero + // ------------------------------------------------------------------ + await t.step("info non-existent path returns non-zero", async () => { + const r = await runCli(vaultDir, "--settings", settingsFile, "info", "no-such-file.md"); + assert(r.code !== 0, "info on non-existent file should exit non-zero"); + console.log("[PASS] info non-existent path returns non-zero"); + }); + + // ------------------------------------------------------------------ + // rm: removes file from ls and makes cat fail + // ------------------------------------------------------------------ + await t.step("rm removes target from ls and cat", async () => { + await run("rm", "test/z-last.txt"); + + const catResult = await runCli(vaultDir, "--settings", settingsFile, "cat", "test/z-last.txt"); + assert(catResult.code !== 0, "rm target should not be readable by cat"); + + const lsOut = await run("ls", "test/"); + assert(!lsOut.includes("test/z-last.txt\t"), "rm target should not appear in ls output"); + console.log("[PASS] rm removed target from visible entries"); + }); + + // ------------------------------------------------------------------ + // resolve: accepts current revision, rejects invalid revision + // ------------------------------------------------------------------ + await t.step("resolve: valid and invalid revisions", async () => { + const lsLine = (await run("ls", "test/a-first.txt")).trim().split("\n")[0]; + assert(lsLine, "could not fetch revision for resolve test"); + const rev = lsLine.split("\t")[3]; + assert(rev?.length > 0, "revision was empty for resolve test"); + + await run("resolve", "test/a-first.txt", rev); + console.log("[PASS] resolve accepted current revision"); + + const badR = await runCli(vaultDir, "--settings", settingsFile, "resolve", "test/a-first.txt", "9-no-such-rev"); + assert(badR.code !== 0, "resolve with non-existent revision should exit non-zero"); + console.log("[PASS] resolve non-existent revision returns non-zero"); + }); + + // ------------------------------------------------------------------ + // cat-rev / pull-rev: retrieve a past revision + // ------------------------------------------------------------------ + await t.step("cat-rev / pull-rev: retrieve past revision", async () => { + const revPath = "test/revision-history.txt"; + await runCliWithInput("revision-v1\n", vaultDir, "--settings", settingsFile, "put", revPath); + await runCliWithInput("revision-v2\n", vaultDir, "--settings", settingsFile, "put", revPath); + await runCliWithInput("revision-v3\n", vaultDir, "--settings", settingsFile, "put", revPath); + + const infoOut = await run("info", revPath); + const infoData = JSON.parse(infoOut) as { + revisions?: string[]; + }; + const revisions = Array.isArray(infoData.revisions) ? infoData.revisions : []; + const pastRev = revisions.find((r): r is string => typeof r === "string" && r !== "N/A"); + assert(pastRev, "info output did not include any past revision"); + + const catRevOut = await run("cat-rev", revPath, pastRev); + const catRevClean = sanitiseCatStdout(catRevOut); + assert( + catRevClean === "revision-v1\n" || catRevClean === "revision-v2\n", + `cat-rev output did not match expected past revision:\n${catRevClean}` + ); + console.log("[PASS] cat-rev matched one of the past revisions from info"); + + const pullRevFile = workDir.join("rev-pull-output.txt"); + await run("pull-rev", revPath, pullRevFile, pastRev); + const pullRevContent = await Deno.readTextFile(pullRevFile); + assert( + pullRevContent === "revision-v1\n" || pullRevContent === "revision-v2\n", + `pull-rev output did not match expected past revision:\n${pullRevContent}` + ); + console.log("[PASS] pull-rev matched one of the past revisions from info"); + }); +}); diff --git a/src/apps/cli/testdeno/test-sync-locked-remote.ts b/src/apps/cli/testdeno/test-sync-locked-remote.ts new file mode 100644 index 0000000..1dfc568 --- /dev/null +++ b/src/apps/cli/testdeno/test-sync-locked-remote.ts @@ -0,0 +1,97 @@ +/** + * Deno port of test-sync-locked-remote-linux.sh + * + * Verifies CLI sync behaviour when the remote milestone document is unlocked + * versus locked. + */ + +import { assert, assertStringIncludes } from "@std/assert"; +import { join } from "@std/path"; +import { loadEnvFile } from "./helpers/env.ts"; +import { TempDir } from "./helpers/temp.ts"; +import { runCli } from "./helpers/cli.ts"; +import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts"; +import { createCouchdbDatabase, startCouchdb, stopCouchdb, updateCouchdbDoc } from "./helpers/docker.ts"; + +const TEST_ENV = join(import.meta.dirname!, "..", ".test.env"); +const MILESTONE_DOC = "_local/obsydian_livesync_milestone"; + +function requireEnv(env: Record, key: string): string { + const value = env[key]?.trim(); + if (!value) { + throw new Error(`Required env var is missing: ${key}`); + } + return value; +} + +Deno.test("sync: actionable error against locked remote DB", async () => { + const env = await loadEnvFile(TEST_ENV); + const couchdbUri = requireEnv(env, "hostname").replace(/\/$/, ""); + const couchdbUser = requireEnv(env, "username"); + const couchdbPassword = requireEnv(env, "password"); + const dbPrefix = requireEnv(env, "dbname"); + const dbname = `${dbPrefix}-locked-${Date.now()}-${Math.floor(Math.random() * 100000)}`; + + await using workDir = await TempDir.create("livesync-cli-locked-test"); + const vaultDir = workDir.join("vault"); + const settingsFile = workDir.join("settings.json"); + await Deno.mkdir(vaultDir, { recursive: true }); + + const shouldStartDocker = Deno.env.get("LIVESYNC_START_DOCKER") !== "0"; + const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1"; + + if (shouldStartDocker) { + console.log(`[INFO] starting CouchDB and creating test database: ${dbname}`); + await startCouchdb(couchdbUri, couchdbUser, couchdbPassword, dbname); + } else { + console.log(`[INFO] using existing CouchDB and creating test database: ${dbname}`); + await createCouchdbDatabase(couchdbUri, couchdbUser, couchdbPassword, dbname); + } + + try { + await initSettingsFile(settingsFile); + await applyCouchdbSettings(settingsFile, couchdbUri, couchdbUser, couchdbPassword, dbname, true); + + console.log("[CASE] initial sync to create milestone document"); + const initialSync = await runCli(vaultDir, "--settings", settingsFile, "sync"); + assert( + initialSync.code === 0, + `initial sync failed\nstdout: ${initialSync.stdout}\nstderr: ${initialSync.stderr}` + ); + + const updateMilestone = async (locked: boolean) => { + await updateCouchdbDoc(couchdbUri, couchdbUser, couchdbPassword, `${dbname}/${MILESTONE_DOC}`, (doc) => ({ + ...doc, + locked, + accepted_nodes: [], + })); + }; + + console.log("[CASE] sync should succeed when remote is not locked"); + await updateMilestone(false); + const unlockedSync = await runCli(vaultDir, "--settings", settingsFile, "sync"); + assert( + unlockedSync.code === 0, + `sync should succeed when remote is not locked\nstdout: ${unlockedSync.stdout}\nstderr: ${unlockedSync.stderr}` + ); + assert( + !unlockedSync.combined.includes("The remote database is locked"), + `locked error should not appear when remote is not locked\n${unlockedSync.combined}` + ); + console.log("[PASS] unlocked remote DB syncs successfully"); + + console.log("[CASE] sync should fail with actionable error when remote is locked"); + await updateMilestone(true); + const lockedSync = await runCli(vaultDir, "--settings", settingsFile, "sync"); + assert( + lockedSync.code !== 0, + `sync should fail when remote is locked\nstdout: ${lockedSync.stdout}\nstderr: ${lockedSync.stderr}` + ); + assertStringIncludes(lockedSync.combined, "The remote database is locked and this device is not yet accepted"); + console.log("[PASS] locked remote DB produces actionable CLI error"); + } finally { + if (shouldStartDocker && !keepDocker) { + await stopCouchdb().catch(() => {}); + } + } +}); diff --git a/src/apps/cli/testdeno/test-sync-two-local-databases.ts b/src/apps/cli/testdeno/test-sync-two-local-databases.ts new file mode 100644 index 0000000..c14ee08 --- /dev/null +++ b/src/apps/cli/testdeno/test-sync-two-local-databases.ts @@ -0,0 +1,287 @@ +/** + * Deno port of test-sync-two-local-databases-linux.sh + * + * Tests two-vault synchronisation via CouchDB including conflict detection + * and resolution. + * + * Requires CouchDB connection details. Provide them via environment variables + * OR place a .test.env file at src/apps/cli/.test.env. + * + * By default, a CouchDB Docker container is started automatically + * (LIVESYNC_START_DOCKER=1). Set LIVESYNC_START_DOCKER=0 to use an existing + * CouchDB instance instead. + * + * Run: + * deno test -A test-sync-two-local-databases.ts + * + * With an existing CouchDB: + * COUCHDB_URI=http://127.0.0.1:5984 \ + * COUCHDB_USER=admin \ + * COUCHDB_PASSWORD=password \ + * COUCHDB_DBNAME=livesync-test \ + * LIVESYNC_START_DOCKER=0 \ + * deno test -A test-sync-two-local-databases.ts + */ + +import { join } from "@std/path"; +import { assertEquals, assert } from "@std/assert"; +import { TempDir } from "./helpers/temp.ts"; +import { CLI_DIR, runCliOrFail, jsonFieldIsNa } from "./helpers/cli.ts"; +import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts"; +import { startCouchdb, stopCouchdb } from "./helpers/docker.ts"; +import { loadEnvFile } from "./helpers/env.ts"; + +// --------------------------------------------------------------------------- +// Load configuration +// --------------------------------------------------------------------------- + +async function resolveConfig(): Promise<{ + uri: string; + user: string; + password: string; + baseDbname: string; +} | null> { + let env: Record = {}; + + // 1. Explicit environment variables take priority + if (Deno.env.get("COUCHDB_URI")) { + env = Object.fromEntries(Deno.env.toObject()); + } else { + // 2. TEST_ENV_FILE env var + const envFile = Deno.env.get("TEST_ENV_FILE") ?? join(CLI_DIR, ".test.env"); + try { + env = await loadEnvFile(envFile); + } catch { + return null; // no config available — skip + } + } + + const uri = (env["COUCHDB_URI"] ?? env["hostname"] ?? "").replace(/\/$/, ""); + const user = env["COUCHDB_USER"] ?? env["username"] ?? ""; + const password = env["COUCHDB_PASSWORD"] ?? env["password"] ?? ""; + const baseDbname = env["COUCHDB_DBNAME"] ?? env["dbname"] ?? "livesync-test"; + + if (!uri || !user || !password) return null; + return { uri, user, password, baseDbname }; +} + +const config = await resolveConfig(); +const START_DOCKER = Deno.env.get("LIVESYNC_START_DOCKER") !== "0"; +const KEEP_DOCKER = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1"; +const SYNC_RETRY = Number(Deno.env.get("LIVESYNC_SYNC_RETRY") ?? "8"); + +// Provide a sane default for flaky remote connectivity in Docker-on-WSL +// environments. Users can override explicitly if needed. +if (!Deno.env.has("LIVESYNC_CLI_RETRY")) { + Deno.env.set("LIVESYNC_CLI_RETRY", "2"); +} + +// --------------------------------------------------------------------------- +// Test suite +// --------------------------------------------------------------------------- + +Deno.test( + { + name: "sync two local databases: sync + conflict detection + resolution", + ignore: config === null, + }, + async (t) => { + if (!config) return; // narrowing for TypeScript + + const suffix = `${Date.now()}-${Math.floor(Math.random() * 65535)}`; + const dbname = `${config.baseDbname}-${suffix}`; + + await using workDir = await TempDir.create("livesync-cli-two-db-test"); + + // ------------------------------------------------------------------ + // Docker lifecycle + // ------------------------------------------------------------------ + if (START_DOCKER) { + await startCouchdb(config.uri, config.user, config.password, dbname); + } + + try { + await runSuite(t, workDir, config, dbname); + } finally { + if (START_DOCKER && !KEEP_DOCKER) { + await stopCouchdb().catch(() => {}); + } + if (START_DOCKER && KEEP_DOCKER) { + console.log("[INFO] LIVESYNC_DEBUG_KEEP_DOCKER=1, keeping couchdb-test container"); + } + console.log(`[INFO] test database '${dbname}' is preserved for debugging.`); + } + } +); + +// --------------------------------------------------------------------------- +// Suite implementation +// --------------------------------------------------------------------------- + +async function runSuite( + t: Deno.TestContext, + workDir: TempDir, + config: { uri: string; user: string; password: string }, + dbname: string +): Promise { + const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)); + const runWithRetry = async (label: string, fn: () => Promise, retries = SYNC_RETRY): Promise => { + let lastErr: unknown; + for (let i = 0; i <= retries; i++) { + try { + return await fn(); + } catch (err) { + lastErr = err; + if (i === retries) break; + const delayMs = 500 * (i + 1); + console.warn(`[WARN] ${label} failed, retrying (${i + 1}/${retries}) in ${delayMs}ms`); + await sleep(delayMs); + } + } + throw lastErr; + }; + + const vaultA = workDir.join("vault-a"); + const vaultB = workDir.join("vault-b"); + const settingsA = workDir.join("a-settings.json"); + const settingsB = workDir.join("b-settings.json"); + await Deno.mkdir(vaultA, { recursive: true }); + await Deno.mkdir(vaultB, { recursive: true }); + + await initSettingsFile(settingsA); + await initSettingsFile(settingsB); + + const applySettings = async (f: string) => + applyCouchdbSettings(f, config.uri, config.user, config.password, dbname, /* liveSync */ true); + await applySettings(settingsA); + await applySettings(settingsB); + + const runA = (...args: string[]) => runCliOrFail(vaultA, "--settings", settingsA, ...args); + const runB = (...args: string[]) => runCliOrFail(vaultB, "--settings", settingsB, ...args); + + const syncA = () => runWithRetry("syncA", () => runA("sync")); + const syncB = () => runWithRetry("syncB", () => runB("sync")); + const catA = (path: string) => runA("cat", path); + const catB = (path: string) => runB("cat", path); + + // ------------------------------------------------------------------ + // Case 1: A creates file, B reads after sync + // ------------------------------------------------------------------ + await t.step("case 1: A creates file -> B can read after sync", async () => { + const srcA = workDir.join("from-a-src.txt"); + await Deno.writeTextFile(srcA, "from-a\n"); + await runA("push", srcA, "shared/from-a.txt"); + await syncA(); + await syncB(); + const value = (await catB("shared/from-a.txt")).replace(/\r\n/g, "\n").trimEnd(); + assertEquals(value, "from-a", "B could not read file created on A"); + console.log("[PASS] case 1 passed"); + }); + + // ------------------------------------------------------------------ + // Case 2: B creates file, A reads after sync + // ------------------------------------------------------------------ + await t.step("case 2: B creates file -> A can read after sync", async () => { + const srcB = workDir.join("from-b-src.txt"); + await Deno.writeTextFile(srcB, "from-b\n"); + await runB("push", srcB, "shared/from-b.txt"); + await syncB(); + await syncA(); + const value = (await catA("shared/from-b.txt")).replace(/\r\n/g, "\n").trimEnd(); + assertEquals(value, "from-b", "A could not read file created on B"); + console.log("[PASS] case 2 passed"); + }); + + // ------------------------------------------------------------------ + // Case 3: concurrent edits create a conflict + // ------------------------------------------------------------------ + await t.step("case 3: concurrent edits create conflict", async () => { + const baseSrc = workDir.join("base-src.txt"); + await Deno.writeTextFile(baseSrc, "base\n"); + await runA("push", baseSrc, "shared/conflicted.txt"); + await syncA(); + await syncB(); + + const aEdit = workDir.join("edit-a.txt"); + const bEdit = workDir.join("edit-b.txt"); + await Deno.writeTextFile(aEdit, "edit-from-a\n"); + await Deno.writeTextFile(bEdit, "edit-from-b\n"); + await runA("push", aEdit, "shared/conflicted.txt"); + await runB("push", bEdit, "shared/conflicted.txt"); + + const infoFileA = workDir.join("info-a.json"); + const infoFileB = workDir.join("info-b.json"); + + let conflictDetected = false; + for (const side of ["a", "b"] as const) { + if (side === "a") await syncA(); + else await syncB(); + await Deno.writeTextFile(infoFileA, await runA("info", "shared/conflicted.txt")); + await Deno.writeTextFile(infoFileB, await runB("info", "shared/conflicted.txt")); + const da = JSON.parse(await Deno.readTextFile(infoFileA)) as Record; + const db = JSON.parse(await Deno.readTextFile(infoFileB)) as Record; + if (!jsonFieldIsNa(da, "conflicts") || !jsonFieldIsNa(db, "conflicts")) { + conflictDetected = true; + break; + } + } + assert(conflictDetected, "expected conflict after concurrent edits, but both sides show N/A"); + console.log("[PASS] case 3 conflict detected"); + }); + + // ------------------------------------------------------------------ + // Case 4: resolve on A, verify B has no conflict after sync + // ------------------------------------------------------------------ + await t.step("case 4: resolve on A propagates to B", async () => { + const infoFileA = workDir.join("info-a-resolve.json"); + const infoFileB = workDir.join("info-b-resolve.json"); + + // Ensure A sees the conflict + for (let i = 0; i < 5; i++) { + const raw = await runA("info", "shared/conflicted.txt"); + await Deno.writeTextFile(infoFileA, raw); + const da = JSON.parse(raw) as Record; + if (!jsonFieldIsNa(da, "conflicts")) break; + await syncB(); + await syncA(); + } + + const rawA = await runA("info", "shared/conflicted.txt"); + await Deno.writeTextFile(infoFileA, rawA); + const dataA = JSON.parse(rawA) as Record; + assert(!jsonFieldIsNa(dataA, "conflicts"), "A does not see conflict, cannot resolve from A only"); + + const keepRev = dataA["revision"] as string; + assert(keepRev?.length > 0, "could not read revision from A info output"); + + await runA("resolve", "shared/conflicted.txt", keepRev); + + let resolved = false; + for (let i = 0; i < 6; i++) { + await syncA(); + await syncB(); + const rawA2 = await runA("info", "shared/conflicted.txt"); + const rawB2 = await runB("info", "shared/conflicted.txt"); + await Deno.writeTextFile(infoFileA, rawA2); + await Deno.writeTextFile(infoFileB, rawB2); + const da2 = JSON.parse(rawA2) as Record; + const db2 = JSON.parse(rawB2) as Record; + if (jsonFieldIsNa(da2, "conflicts") && jsonFieldIsNa(db2, "conflicts")) { + resolved = true; + break; + } + // If A still sees a conflict, resolve it again + if (!jsonFieldIsNa(da2, "conflicts")) { + const rev2 = da2["revision"] as string; + if (rev2) await runA("resolve", "shared/conflicted.txt", rev2).catch(() => {}); + } + } + assert(resolved, "conflicts should be resolved on both A and B"); + + const contentA = (await catA("shared/conflicted.txt")).replace(/\r\n/g, "\n"); + const contentB = (await catB("shared/conflicted.txt")).replace(/\r\n/g, "\n"); + assertEquals(contentA, contentB, "resolved content mismatch between A and B"); + console.log("[PASS] case 4 passed"); + console.log("[PASS] all sync/resolve scenarios passed"); + }); +} diff --git a/src/apps/cli/testdeno/test_dev_deno.md b/src/apps/cli/testdeno/test_dev_deno.md new file mode 100644 index 0000000..521661a --- /dev/null +++ b/src/apps/cli/testdeno/test_dev_deno.md @@ -0,0 +1,292 @@ +# CLI Deno Test Development Notes + +This document provides an overview of the Deno-based compatibility tests under `src/apps/cli/testdeno/`. +The existing bash tests under `src/apps/cli/test/` are preserved, while a Windows-friendly suite is maintained in parallel. + +--- + +## Goals + +- Keep existing bash tests intact. +- Provide direct execution from Windows PowerShell. +- Establish a TypeScript (Deno) foundation for core end-to-end and integration scenarios. + +--- + +## Directory structure + +``` +src/apps/cli/testdeno/ + deno.json + CONTRIBUTING_TESTS.md + helpers/ + backgroundCli.ts + cli.ts + docker.ts + env.ts + p2p.ts + settings.ts + temp.ts + test-e2e-two-vaults-couchdb.ts + test-push-pull.ts + test-p2p-host.ts + test-p2p-peers-local-relay.ts + test-p2p-sync.ts + test-p2p-three-nodes-conflict.ts + test-p2p-upload-download-repro.ts + test-e2e-two-vaults-matrix.ts + test-setup-put-cat.ts + test-mirror.ts + test-sync-two-local-databases.ts + test-sync-locked-remote.ts +``` + +--- + +## Key files + +### `deno.json` + +- Defines Deno tasks. +- Defines import maps for `@std/assert` and `@std/path`. + +Main tasks: + +- `deno task test` +- `deno task test:local` +- `deno task test:push-pull` +- `deno task test:setup-put-cat` +- `deno task test:mirror` +- `deno task test:sync-two-local` +- `deno task test:sync-locked-remote` +- `deno task test:p2p-host` +- `deno task test:p2p-peers` +- `deno task test:p2p-sync` +- `deno task test:p2p-three-nodes` +- `deno task test:p2p-upload-download` +- `deno task test:e2e-couchdb` +- `deno task test:e2e-matrix` + +### `helpers/cli.ts` + +- CLI execution wrappers. +- `runCli`, `runCliOrFail`, `runCliWithInput`. +- Output normalisation via `sanitiseCatStdout`. +- Comparison utilities, including `assertFilesEqual`. + +This file corresponds to `run_cli` and common assertions in `test-helpers.sh`. + +### `helpers/settings.ts` + +- Executes `init-settings --force`. +- Marks `isConfigured = true`. +- Applies CouchDB and P2P settings. +- Applies remote synchronisation settings and P2P test tweaks. + +This file corresponds to settings helpers in `test-helpers.sh`. + +### `helpers/docker.ts` + +- Starts, stops, and initialises CouchDB directly from Deno. +- Configures CouchDB via `fetch + retry`. +- Starts and stops the P2P relay through the same Docker runner. + +Both CouchDB and P2P relay flows are bash-independent. + +### `helpers/backgroundCli.ts` + +- Starts long-running commands such as `p2p-host` in the background. +- Waits for readiness logs and handles termination. + +### `helpers/p2p.ts` + +- Determines whether a local relay should be started. +- Parses `p2p-peers` output. +- Discovers peer IDs with a fallback based on advertisement logs. + +### `helpers/env.ts` + +- Loads `.test.env`. +- Supports `KEY=value`, single-quoted values, and double-quoted values. + +### `helpers/temp.ts` + +- Provides `TempDir`. +- Uses `await using` to auto-clean temporary directories. + +--- + +## Implemented tests + +### `test-push-pull.ts` + +- Verifies push and pull round trips. +- Uses environment variables or `.test.env` for CouchDB values. + +### `test-setup-put-cat.ts` + +- Verifies `setup` with full setup URI generation via `encodeSettingsToSetupURI`. +- Verifies `push`, `cat`, `ls`, `info`, `rm`, `resolve`, `cat-rev`, and `pull-rev`. +- Does not require an external remote. + +### `test-mirror.ts` + +- Verifies six core mirror scenarios. +- Does not require an external remote. + +### `test-sync-two-local-databases.ts` + +- Verifies sync between two vaults and CouchDB. +- Verifies conflict detection and resolve propagation. +- Starts Docker CouchDB by default when `LIVESYNC_START_DOCKER != 0`. + +### `test-sync-locked-remote.ts` + +- Updates the CouchDB milestone `locked` flag. +- Verifies sync success when unlocked. +- Verifies actionable CLI error when locked. + +### `test-p2p-host.ts` + +- Verifies that `p2p-host` starts and emits readiness output. + +### `test-p2p-peers-local-relay.ts` + +- Verifies peer discovery through a local relay. + +### `test-p2p-sync.ts` + +- Verifies that `p2p-sync` completes after peer discovery. + +### `test-p2p-three-nodes-conflict.ts` + +- Uses one host and two clients. +- Verifies conflict creation, detection via `info`, and resolution via `resolve`. + +### `test-p2p-upload-download-repro.ts` + +- Uses host, upload, and download nodes. +- Verifies transfer of text files and binary files, including larger files. + +### `test-e2e-two-vaults-couchdb.ts` + +- Verifies two-vault end-to-end scenarios on CouchDB. +- Runs both encryption-off and encryption-on cases. +- Includes conflict marker checks in `ls` and resolve propagation checks. + +### `test-e2e-two-vaults-matrix.ts` + +- Verifies the matrix equivalent of the bash script. +- Runs four combinations: + - `COUCHDB-enc0` + - `COUCHDB-enc1` + - `MINIO-enc0` + - `MINIO-enc1` + +--- + +## Running tests (PowerShell) + +From `src/apps/cli/testdeno`: + +```powershell +cd src/apps/cli/testdeno + +# Local-only set +deno task test:local + +# Individual tests +deno task test:setup-put-cat +deno task test:mirror +deno task test:push-pull +deno task test:sync-locked-remote + +# CouchDB-based tests +deno task test:sync-two-local +deno task test:e2e-couchdb + +# P2P-based tests +deno task test:p2p-host +deno task test:p2p-peers +deno task test:p2p-sync +deno task test:p2p-three-nodes +deno task test:p2p-upload-download +deno task test:e2e-matrix +``` + +--- + +## Environment variables + +### CouchDB + +- `COUCHDB_URI` +- `COUCHDB_USER` +- `COUCHDB_PASSWORD` +- `COUCHDB_DBNAME` + +Equivalent keys in `src/apps/cli/.test.env`: + +- `hostname` +- `username` +- `password` +- `dbname` + +### Behaviour switches + +- `LIVESYNC_START_DOCKER=0`: use existing CouchDB. +- `REMOTE_PATH`: override target path for selected tests. +- `LIVESYNC_TEST_TEE=1`: stream CLI stdout and stderr during execution. +- `LIVESYNC_DOCKER_TEE=1`: stream Docker stdout and stderr. +- `LIVESYNC_CLI_RETRY=`: retry transient network failures. +- `LIVESYNC_DEBUG_KEEP_DOCKER=1`: keep `couchdb-test` after test completion. + +### Docker command selection + +`helpers/docker.ts` supports command selection via environment variables. + +- `LIVESYNC_DOCKER_MODE=auto` (default) + - Windows: tries `wsl docker` first, then `docker`. + - Non-Windows: tries `docker` first, then `wsl docker`. +- `LIVESYNC_DOCKER_MODE=native`: always uses `docker`. +- `LIVESYNC_DOCKER_MODE=wsl`: always uses `wsl docker`. +- `LIVESYNC_DOCKER_COMMAND="..."`: custom command, for example `wsl docker`. + +`LIVESYNC_DOCKER_COMMAND` has priority over `LIVESYNC_DOCKER_MODE`. + +PowerShell examples: + +```powershell +# Use Docker in WSL explicitly +$env:LIVESYNC_DOCKER_MODE = "wsl" +deno task test:sync-two-local + +# Full custom command +$env:LIVESYNC_DOCKER_COMMAND = "wsl docker" +deno task test:sync-two-local +``` + +### P2P + +- `RELAY` +- `ROOM_ID` +- `PASSPHRASE` +- `APP_ID` +- `PEERS_TIMEOUT` +- `SYNC_TIMEOUT` +- `USE_INTERNAL_RELAY=0|1` +- `TIMEOUT_SECONDS` + +--- + +## Current limitations + +- MinIO startup and matrix coverage are ported. Current limits are elsewhere, not setup URI generation. + +--- + +## Maintenance policy + +- Existing bash tests remain available. +- Deno tests are expanded in parallel for cross-platform usage. +- New scenarios should be added through reusable helpers in `helpers/`.