mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-05-07 16:21:51 +00:00
feat(tests): add Deno-based tests for checking CLI functionality in the same-codebase between platforms.
This commit is contained in:
150
src/apps/cli/testdeno/CONTRIBUTING_TESTS.md
Normal file
150
src/apps/cli/testdeno/CONTRIBUTING_TESTS.md
Normal file
@@ -0,0 +1,150 @@
|
||||
# Writing CLI Tests on Deno
|
||||
|
||||
This guide explains how to add or update tests under `src/apps/cli/testdeno/`.
|
||||
Note that new tests should be added to the Deno suite rather than the existing bash suite due to the cross-platform execution and TypeScript benefits.
|
||||
|
||||
## Scope
|
||||
|
||||
The Deno suite is designed for cross-platform execution, with a strong focus on Windows compatibility while keeping behaviour equivalent to existing bash tests.
|
||||
|
||||
## Principles
|
||||
|
||||
- Keep one scenario per file when practical.
|
||||
- Reuse helpers from `helpers/` rather than duplicating process, Docker, or settings logic.
|
||||
- Prefer deterministic data over random inputs unless randomness is explicitly required.
|
||||
- Ensure every test can clean up automatically.
|
||||
- Keep assertions actionable with clear failure messages.
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
src/apps/cli/testdeno/
|
||||
helpers/
|
||||
backgroundCli.ts
|
||||
cli.ts
|
||||
docker.ts
|
||||
env.ts
|
||||
p2p.ts
|
||||
settings.ts
|
||||
temp.ts
|
||||
test-*.ts
|
||||
deno.json
|
||||
```
|
||||
|
||||
## Test file naming
|
||||
|
||||
- Use `test-<feature>.ts`.
|
||||
- Use names aligned with existing bash tests when porting, for example:
|
||||
- `test-sync-locked-remote.ts`
|
||||
- `test-p2p-sync.ts`
|
||||
|
||||
## Core helper usage
|
||||
|
||||
### Temporary workspace
|
||||
|
||||
Use `TempDir` and `await using` so cleanup is automatic:
|
||||
|
||||
```ts
|
||||
await using workDir = await TempDir.create("livesync-cli-my-test");
|
||||
```
|
||||
|
||||
### CLI execution
|
||||
|
||||
- `runCli(...)`: returns code and combined output.
|
||||
- `runCliOrFail(...)`: throws on non-zero exit.
|
||||
- `runCliWithInputOrFail(input, ...)`: for `put` and stdin-driven commands.
|
||||
|
||||
### Settings
|
||||
|
||||
- `initSettingsFile(...)`: creates a baseline settings file.
|
||||
- `applyCouchdbSettings(...)`: applies CouchDB fields.
|
||||
- `applyRemoteSyncSettings(...)`: applies remote and encryption fields.
|
||||
- `applyP2pSettings(...)`: applies P2P fields.
|
||||
- `applyP2pTestTweaks(...)`: enables P2P-only test profile.
|
||||
|
||||
### Docker services
|
||||
|
||||
- `startCouchdb(...)`, `stopCouchdb()`
|
||||
- `startP2pRelay()`, `stopP2pRelay()`
|
||||
|
||||
### P2P discovery
|
||||
|
||||
- `discoverPeer(...)`
|
||||
- `maybeStartLocalRelay(...)`
|
||||
- `stopLocalRelayIfStarted(...)`
|
||||
|
||||
### Background host process
|
||||
|
||||
Use `startCliInBackground(...)` for long-running host mode such as `p2p-host`.
|
||||
|
||||
## Recommended test structure
|
||||
|
||||
1. Arrange
|
||||
2. Act
|
||||
3. Assert
|
||||
4. Cleanup in `finally`
|
||||
|
||||
Example skeleton:
|
||||
|
||||
```ts
|
||||
Deno.test("feature: behaviour", async () => {
|
||||
await using workDir = await TempDir.create("example");
|
||||
// Arrange
|
||||
|
||||
try {
|
||||
// Act
|
||||
|
||||
// Assert
|
||||
} finally {
|
||||
// Optional explicit cleanup
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Reliability guidelines
|
||||
|
||||
- Use explicit waits only when needed for eventual consistency.
|
||||
- Re-run sync operations where the protocol is eventually consistent.
|
||||
- For network-sensitive commands, use `LIVESYNC_CLI_RETRY` during debugging.
|
||||
- Keep Docker container reuse disabled by default unless debugging.
|
||||
|
||||
## Environment variables
|
||||
|
||||
Common variables:
|
||||
|
||||
- `LIVESYNC_DOCKER_MODE`
|
||||
- `LIVESYNC_DOCKER_COMMAND`
|
||||
- `LIVESYNC_TEST_TEE`
|
||||
- `LIVESYNC_DOCKER_TEE`
|
||||
- `LIVESYNC_CLI_DEBUG`
|
||||
- `LIVESYNC_CLI_VERBOSE`
|
||||
- `LIVESYNC_CLI_RETRY`
|
||||
- `LIVESYNC_DEBUG_KEEP_DOCKER`
|
||||
|
||||
P2P variables:
|
||||
|
||||
- `RELAY`
|
||||
- `ROOM_ID`
|
||||
- `PASSPHRASE`
|
||||
- `APP_ID`
|
||||
- `PEERS_TIMEOUT`
|
||||
- `SYNC_TIMEOUT`
|
||||
- `USE_INTERNAL_RELAY`
|
||||
|
||||
## Adding a new test task
|
||||
|
||||
1. Add the test file under `src/apps/cli/testdeno/`.
|
||||
2. Add a task in `src/apps/cli/testdeno/deno.json`.
|
||||
3. Update `src/apps/cli/testdeno/test_dev_deno.md`.
|
||||
4. Run the new task locally.
|
||||
|
||||
## Validation checklist
|
||||
|
||||
- The test passes on a clean workspace.
|
||||
- The test does not leave persistent artefacts unless explicitly requested.
|
||||
- Failure messages identify both expected and actual behaviour.
|
||||
- The corresponding task is documented.
|
||||
|
||||
## Out of scope for this suite
|
||||
|
||||
- One-off reproduction scripts that are not intended as stable regression tests.
|
||||
22
src/apps/cli/testdeno/deno.json
Normal file
22
src/apps/cli/testdeno/deno.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"tasks": {
|
||||
"test": "deno test -A --no-check .",
|
||||
"test:local": "deno test -A --no-check test-setup-put-cat.ts test-mirror.ts",
|
||||
"test:push-pull": "deno test -A --no-check test-push-pull.ts",
|
||||
"test:setup-put-cat": "deno test -A --no-check test-setup-put-cat.ts",
|
||||
"test:mirror": "deno test -A --no-check test-mirror.ts",
|
||||
"test:sync-two-local": "deno test -A --no-check test-sync-two-local-databases.ts",
|
||||
"test:sync-locked-remote": "deno test -A --no-check test-sync-locked-remote.ts",
|
||||
"test:p2p-host": "deno test -A --no-check test-p2p-host.ts",
|
||||
"test:p2p-peers": "deno test -A --no-check test-p2p-peers-local-relay.ts",
|
||||
"test:p2p-sync": "deno test -A --no-check test-p2p-sync.ts",
|
||||
"test:p2p-three-nodes": "deno test -A --no-check test-p2p-three-nodes-conflict.ts",
|
||||
"test:p2p-upload-download": "deno test -A --no-check test-p2p-upload-download-repro.ts",
|
||||
"test:e2e-couchdb": "deno test -A --no-check test-e2e-two-vaults-couchdb.ts",
|
||||
"test:e2e-matrix": "deno test -A --no-check test-e2e-two-vaults-matrix.ts"
|
||||
},
|
||||
"imports": {
|
||||
"@std/assert": "jsr:@std/assert@^1.0.13",
|
||||
"@std/path": "jsr:@std/path@^1.0.9"
|
||||
}
|
||||
}
|
||||
31
src/apps/cli/testdeno/deno.lock
generated
Normal file
31
src/apps/cli/testdeno/deno.lock
generated
Normal file
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"version": "5",
|
||||
"specifiers": {
|
||||
"jsr:@std/assert@^1.0.13": "1.0.19",
|
||||
"jsr:@std/internal@^1.0.12": "1.0.12",
|
||||
"jsr:@std/path@^1.0.9": "1.1.4"
|
||||
},
|
||||
"jsr": {
|
||||
"@std/assert@1.0.19": {
|
||||
"integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
|
||||
"dependencies": [
|
||||
"jsr:@std/internal"
|
||||
]
|
||||
},
|
||||
"@std/internal@1.0.12": {
|
||||
"integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027"
|
||||
},
|
||||
"@std/path@1.1.4": {
|
||||
"integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
|
||||
"dependencies": [
|
||||
"jsr:@std/internal"
|
||||
]
|
||||
}
|
||||
},
|
||||
"workspace": {
|
||||
"dependencies": [
|
||||
"jsr:@std/assert@^1.0.13",
|
||||
"jsr:@std/path@^1.0.9"
|
||||
]
|
||||
}
|
||||
}
|
||||
112
src/apps/cli/testdeno/helpers/backgroundCli.ts
Normal file
112
src/apps/cli/testdeno/helpers/backgroundCli.ts
Normal file
@@ -0,0 +1,112 @@
|
||||
import { CLI_DIR } from "./cli.ts";
|
||||
import { join } from "@std/path";
|
||||
|
||||
const CLI_DIST = join(CLI_DIR, "dist", "index.cjs");
|
||||
const VERBOSE_ENABLED = Deno.env.get("LIVESYNC_CLI_VERBOSE") === "1";
|
||||
const DEBUG_ENABLED = Deno.env.get("LIVESYNC_CLI_DEBUG") === "1";
|
||||
|
||||
function decorateArgs(args: string[]): string[] {
|
||||
return DEBUG_ENABLED ? ["-d", ...args] : VERBOSE_ENABLED ? ["-v", ...args] : args;
|
||||
}
|
||||
|
||||
async function pump(
|
||||
stream: ReadableStream<Uint8Array>,
|
||||
sink: (text: string) => void,
|
||||
teeTarget: WritableStream<Uint8Array> | null
|
||||
): Promise<void> {
|
||||
const reader = stream.getReader();
|
||||
const writer = teeTarget?.getWriter();
|
||||
const dec = new TextDecoder();
|
||||
try {
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
if (!value) continue;
|
||||
sink(dec.decode(value, { stream: true }));
|
||||
if (writer) {
|
||||
await writer.write(value);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
if (writer) writer.releaseLock();
|
||||
reader.releaseLock();
|
||||
}
|
||||
}
|
||||
|
||||
export class BackgroundCliProcess {
|
||||
#stdout = "";
|
||||
#stderr = "";
|
||||
#stdoutDone: Promise<void>;
|
||||
#stderrDone: Promise<void>;
|
||||
|
||||
constructor(
|
||||
readonly child: Deno.ChildProcess,
|
||||
readonly args: string[]
|
||||
) {
|
||||
this.#stdoutDone = pump(
|
||||
child.stdout,
|
||||
(text) => {
|
||||
this.#stdout += text;
|
||||
},
|
||||
null
|
||||
);
|
||||
this.#stderrDone = pump(
|
||||
child.stderr,
|
||||
(text) => {
|
||||
this.#stderr += text;
|
||||
},
|
||||
null
|
||||
);
|
||||
}
|
||||
|
||||
get stdout(): string {
|
||||
return this.#stdout;
|
||||
}
|
||||
|
||||
get stderr(): string {
|
||||
return this.#stderr;
|
||||
}
|
||||
|
||||
get combined(): string {
|
||||
return this.#stdout + this.#stderr;
|
||||
}
|
||||
|
||||
async waitUntilContains(needle: string, timeoutMs = 15000): Promise<void> {
|
||||
const started = Date.now();
|
||||
while (Date.now() - started < timeoutMs) {
|
||||
if (this.combined.includes(needle)) return;
|
||||
const status = await Promise.race([
|
||||
this.child.status.then((s) => ({ type: "status" as const, status: s })),
|
||||
new Promise<{ type: "tick" }>((resolve) => setTimeout(() => resolve({ type: "tick" }), 100)),
|
||||
]);
|
||||
if (status.type === "status") {
|
||||
throw new Error(
|
||||
`Background CLI exited before '${needle}' appeared (code ${status.status.code})\n${this.combined}`
|
||||
);
|
||||
}
|
||||
}
|
||||
throw new Error(`Timed out waiting for '${needle}'\n${this.combined}`);
|
||||
}
|
||||
|
||||
async stop(): Promise<number> {
|
||||
try {
|
||||
this.child.kill("SIGTERM");
|
||||
} catch {
|
||||
// ignore already-exited processes
|
||||
}
|
||||
const status = await this.child.status;
|
||||
await Promise.all([this.#stdoutDone, this.#stderrDone]);
|
||||
return status.code;
|
||||
}
|
||||
}
|
||||
|
||||
export function startCliInBackground(...args: string[]): BackgroundCliProcess {
|
||||
const child = new Deno.Command("node", {
|
||||
args: [CLI_DIST, ...decorateArgs(args)],
|
||||
cwd: CLI_DIR,
|
||||
stdin: "null",
|
||||
stdout: "piped",
|
||||
stderr: "piped",
|
||||
}).spawn();
|
||||
return new BackgroundCliProcess(child, args);
|
||||
}
|
||||
231
src/apps/cli/testdeno/helpers/cli.ts
Normal file
231
src/apps/cli/testdeno/helpers/cli.ts
Normal file
@@ -0,0 +1,231 @@
|
||||
import { join } from "@std/path";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Path resolution
|
||||
// ---------------------------------------------------------------------------
|
||||
// This file lives at: src/apps/cli/testdeno/helpers/cli.ts
|
||||
// CLI root (src/apps/cli/) is two levels up.
|
||||
// import.meta.dirname is available in Deno 1.40+ as an OS-native path string.
|
||||
export const CLI_DIR: string = join(import.meta.dirname!, "..", "..");
|
||||
const CLI_DIST = join(CLI_DIR, "dist", "index.cjs");
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Result type
|
||||
// ---------------------------------------------------------------------------
|
||||
export interface CliResult {
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
/** stdout + stderr concatenated — useful for assertion messages. */
|
||||
combined: string;
|
||||
code: number;
|
||||
}
|
||||
|
||||
const TEE_ENABLED = Deno.env.get("LIVESYNC_TEST_TEE") === "1";
|
||||
const VERBOSE_ENABLED = Deno.env.get("LIVESYNC_CLI_VERBOSE") === "1";
|
||||
const DEBUG_ENABLED = Deno.env.get("LIVESYNC_CLI_DEBUG") === "1";
|
||||
|
||||
function sleep(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
function concatChunks(chunks: Uint8Array[]): Uint8Array {
|
||||
const total = chunks.reduce((n, c) => n + c.length, 0);
|
||||
const out = new Uint8Array(total);
|
||||
let offset = 0;
|
||||
for (const c of chunks) {
|
||||
out.set(c, offset);
|
||||
offset += c.length;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async function collectStream(
|
||||
stream: ReadableStream<Uint8Array>,
|
||||
teeTarget: WritableStream<Uint8Array> | null
|
||||
): Promise<Uint8Array> {
|
||||
const reader = stream.getReader();
|
||||
const chunks: Uint8Array[] = [];
|
||||
const writer = teeTarget?.getWriter();
|
||||
try {
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
if (value) {
|
||||
chunks.push(value);
|
||||
if (writer) {
|
||||
await writer.write(value);
|
||||
}
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
if (writer) {
|
||||
writer.releaseLock();
|
||||
}
|
||||
reader.releaseLock();
|
||||
}
|
||||
return concatChunks(chunks);
|
||||
}
|
||||
|
||||
async function runNodeCommand(args: string[], stdinData?: Uint8Array): Promise<CliResult> {
|
||||
const cliArgs = DEBUG_ENABLED ? ["-d", ...args] : VERBOSE_ENABLED ? ["-v", ...args] : args;
|
||||
const child = new Deno.Command("node", {
|
||||
args: [CLI_DIST, ...cliArgs],
|
||||
cwd: CLI_DIR,
|
||||
stdin: stdinData ? "piped" : "null",
|
||||
stdout: "piped",
|
||||
stderr: "piped",
|
||||
}).spawn();
|
||||
|
||||
const stdoutPromise = collectStream(child.stdout, TEE_ENABLED ? Deno.stdout.writable : null);
|
||||
const stderrPromise = collectStream(child.stderr, TEE_ENABLED ? Deno.stderr.writable : null);
|
||||
|
||||
if (stdinData) {
|
||||
const w = child.stdin.getWriter();
|
||||
await w.write(stdinData);
|
||||
await w.close();
|
||||
}
|
||||
|
||||
const [status, stdout, stderr] = await Promise.all([child.status, stdoutPromise, stderrPromise]);
|
||||
|
||||
const dec = new TextDecoder();
|
||||
const out = dec.decode(stdout);
|
||||
const err = dec.decode(stderr);
|
||||
return { stdout: out, stderr: err, combined: out + err, code: status.code };
|
||||
}
|
||||
|
||||
function isTransientNetworkError(message: string): boolean {
|
||||
const m = message.toLowerCase();
|
||||
return (
|
||||
m.includes("fetch failed") ||
|
||||
m.includes("econnreset") ||
|
||||
m.includes("econnrefused") ||
|
||||
m.includes("und_err_socket") ||
|
||||
m.includes("other side closed")
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Core runners
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Run the CLI (node dist/index.cjs) with the supplied arguments.
|
||||
* Pass the vault / DB path as the first argument, exactly as the bash helpers
|
||||
* do. Does NOT throw on non-zero exit — check `.code` yourself.
|
||||
*/
|
||||
export async function runCli(...args: string[]): Promise<CliResult> {
|
||||
const retries = Number(Deno.env.get("LIVESYNC_CLI_RETRY") ?? "0");
|
||||
for (let attempt = 0; ; attempt++) {
|
||||
const result = await runNodeCommand(args);
|
||||
if (result.code === 0) return result;
|
||||
|
||||
if (attempt >= retries || !isTransientNetworkError(result.combined)) {
|
||||
return result;
|
||||
}
|
||||
const waitMs = 400 * (attempt + 1);
|
||||
console.warn(`[WARN] transient CLI failure, retrying (${attempt + 1}/${retries}) in ${waitMs}ms`);
|
||||
await sleep(waitMs);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the CLI and throw if it exits non-zero. Returns stdout.
|
||||
*/
|
||||
export async function runCliOrFail(...args: string[]): Promise<string> {
|
||||
const r = await runCli(...args);
|
||||
if (r.code !== 0) {
|
||||
throw new Error(`CLI exited with code ${r.code}\nstdout: ${r.stdout}\nstderr: ${r.stderr}`);
|
||||
}
|
||||
return r.stdout;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the CLI with data piped to stdin (equivalent to `echo … | run_cli …`
|
||||
* or `cat file | run_cli …`).
|
||||
*/
|
||||
export async function runCliWithInput(input: string | Uint8Array, ...args: string[]): Promise<CliResult> {
|
||||
const data = typeof input === "string" ? new TextEncoder().encode(input) : input;
|
||||
|
||||
const retries = Number(Deno.env.get("LIVESYNC_CLI_RETRY") ?? "0");
|
||||
for (let attempt = 0; ; attempt++) {
|
||||
const result = await runNodeCommand(args, data);
|
||||
if (result.code === 0) return result;
|
||||
|
||||
if (attempt >= retries || !isTransientNetworkError(result.combined)) {
|
||||
return result;
|
||||
}
|
||||
const waitMs = 400 * (attempt + 1);
|
||||
console.warn(`[WARN] transient CLI(stdin) failure, retrying (${attempt + 1}/${retries}) in ${waitMs}ms`);
|
||||
await sleep(waitMs);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* runCliWithInput — throws on non-zero exit, returns stdout.
|
||||
*/
|
||||
export async function runCliWithInputOrFail(input: string | Uint8Array, ...args: string[]): Promise<string> {
|
||||
const r = await runCliWithInput(input, ...args);
|
||||
if (r.code !== 0) {
|
||||
throw new Error(`CLI (with stdin) exited with code ${r.code}\nstdout: ${r.stdout}\nstderr: ${r.stderr}`);
|
||||
}
|
||||
return r.stdout;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Output helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Strip the CLIWatchAdapter banner line that `cat` emits. */
|
||||
export function sanitiseCatStdout(raw: string): string {
|
||||
return raw
|
||||
.split("\n")
|
||||
.filter((l) => l !== "[CLIWatchAdapter] File watching is not enabled in CLI version")
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Assertions (parity with test-helpers.sh)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function assertContains(haystack: string, needle: string, message: string): void {
|
||||
if (!haystack.includes(needle)) {
|
||||
throw new Error(`[FAIL] ${message}\nExpected to find: ${JSON.stringify(needle)}\nActual output:\n${haystack}`);
|
||||
}
|
||||
}
|
||||
|
||||
export function assertNotContains(haystack: string, needle: string, message: string): void {
|
||||
if (haystack.includes(needle)) {
|
||||
throw new Error(`[FAIL] ${message}\nDid NOT expect: ${JSON.stringify(needle)}\nActual output:\n${haystack}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function assertFilesEqual(expectedPath: string, actualPath: string, message: string): Promise<void> {
|
||||
const [expected, actual] = await Promise.all([Deno.readFile(expectedPath), Deno.readFile(actualPath)]);
|
||||
if (expected.length !== actual.length || expected.some((b, i) => b !== actual[i])) {
|
||||
const hex = async (d: Uint8Array<ArrayBuffer>) => {
|
||||
const h = await crypto.subtle.digest("SHA-256", d);
|
||||
return [...new Uint8Array(h)].map((b) => b.toString(16).padStart(2, "0")).join("");
|
||||
};
|
||||
throw new Error(
|
||||
`[FAIL] ${message}\nexpected SHA-256: ${await hex(expected)}\nactual SHA-256: ${await hex(actual)}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// JSON helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export async function readJsonFile<T = Record<string, unknown>>(filePath: string): Promise<T> {
|
||||
return JSON.parse(await Deno.readTextFile(filePath)) as T;
|
||||
}
|
||||
|
||||
export function jsonStringField(jsonText: string, field: string): string {
|
||||
const data = JSON.parse(jsonText) as Record<string, unknown>;
|
||||
const value = data[field];
|
||||
return typeof value === "string" ? value : "";
|
||||
}
|
||||
|
||||
export function jsonFieldIsNa(data: Record<string, unknown>, field: string): boolean {
|
||||
return data[field] === "N/A";
|
||||
}
|
||||
530
src/apps/cli/testdeno/helpers/docker.ts
Normal file
530
src/apps/cli/testdeno/helpers/docker.ts
Normal file
@@ -0,0 +1,530 @@
|
||||
/**
|
||||
* Docker service management for tests.
|
||||
*
|
||||
* CouchDB start/stop/init is implemented directly using `docker` CLI commands
|
||||
* and the Fetch API, so it works on any platform where Docker (Desktop) is
|
||||
* available — including Windows — without needing bash.
|
||||
*/
|
||||
|
||||
type DockerInvoker = {
|
||||
bin: string;
|
||||
prefix: string[];
|
||||
label: string;
|
||||
};
|
||||
|
||||
let dockerInvokerPromise: Promise<DockerInvoker> | null = null;
|
||||
const DOCKER_TEE = Deno.env.get("LIVESYNC_DOCKER_TEE") === "1" || Deno.env.get("LIVESYNC_TEST_TEE") === "1";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Low-level docker wrapper
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function parseCommand(command: string): { bin: string; prefix: string[] } {
|
||||
const parts = command.trim().split(/\s+/).filter(Boolean);
|
||||
if (parts.length === 0) {
|
||||
throw new Error("LIVESYNC_DOCKER_COMMAND is empty");
|
||||
}
|
||||
return { bin: parts[0], prefix: parts.slice(1) };
|
||||
}
|
||||
|
||||
async function runCommand(bin: string, args: string[]): Promise<{ code: number; stdout: string; stderr: string }> {
|
||||
const cmd = new Deno.Command(bin, {
|
||||
args,
|
||||
stdin: "null",
|
||||
stdout: "piped",
|
||||
stderr: "piped",
|
||||
});
|
||||
try {
|
||||
const { code, stdout, stderr } = await cmd.output();
|
||||
const dec = new TextDecoder();
|
||||
const result = {
|
||||
code,
|
||||
stdout: dec.decode(stdout),
|
||||
stderr: dec.decode(stderr),
|
||||
};
|
||||
if (DOCKER_TEE) {
|
||||
if (result.stdout.trim().length > 0) {
|
||||
console.log(`[docker:${bin}] ${result.stdout.trimEnd()}`);
|
||||
}
|
||||
if (result.stderr.trim().length > 0) {
|
||||
console.error(`[docker:${bin}] ${result.stderr.trimEnd()}`);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
} catch (err) {
|
||||
if (err instanceof Deno.errors.NotFound) {
|
||||
return {
|
||||
code: 127,
|
||||
stdout: "",
|
||||
stderr: `Command not found: ${bin}`,
|
||||
};
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
async function resolveDockerInvoker(): Promise<DockerInvoker> {
|
||||
const custom = Deno.env.get("LIVESYNC_DOCKER_COMMAND")?.trim();
|
||||
if (custom) {
|
||||
const parsed = parseCommand(custom);
|
||||
const runner: DockerInvoker = {
|
||||
...parsed,
|
||||
label: `custom(${custom})`,
|
||||
};
|
||||
|
||||
// Validate custom command eagerly so misconfiguration fails fast.
|
||||
const checkArgs = runner.prefix.length === 0 ? ["--version"] : [...runner.prefix, "docker", "--version"];
|
||||
const check = await runCommand(runner.bin, checkArgs);
|
||||
if (check.code !== 0) {
|
||||
throw new Error(`LIVESYNC_DOCKER_COMMAND is not usable: ${custom}\n${check.stderr || check.stdout}`);
|
||||
}
|
||||
return runner;
|
||||
}
|
||||
|
||||
const mode = (Deno.env.get("LIVESYNC_DOCKER_MODE") ?? "auto").toLowerCase();
|
||||
const onWindows = Deno.build.os === "windows";
|
||||
|
||||
const native: DockerInvoker = { bin: "docker", prefix: [], label: "docker" };
|
||||
const wsl: DockerInvoker = { bin: "wsl", prefix: [], label: "wsl docker" };
|
||||
|
||||
if (mode === "native") {
|
||||
return native;
|
||||
}
|
||||
if (mode === "wsl") {
|
||||
return wsl;
|
||||
}
|
||||
if (mode !== "auto") {
|
||||
throw new Error(`Unsupported LIVESYNC_DOCKER_MODE='${mode}'. Use auto, native, or wsl.`);
|
||||
}
|
||||
|
||||
// On Windows we prefer `wsl docker` first, then native docker.
|
||||
// This typically works better in setups where Docker is installed only in
|
||||
// WSL and not exposed as docker.exe on PATH.
|
||||
const candidates = onWindows ? [wsl, native] : [native, wsl];
|
||||
for (const c of candidates) {
|
||||
if (c.bin === "docker") {
|
||||
const r = await runCommand("docker", ["--version"]);
|
||||
if (r.code === 0) return c;
|
||||
continue;
|
||||
}
|
||||
const r = await runCommand("wsl", ["docker", "--version"]);
|
||||
if (r.code === 0) return c;
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
[
|
||||
"Docker command is not available.",
|
||||
"Set one of:",
|
||||
"- LIVESYNC_DOCKER_MODE=native",
|
||||
"- LIVESYNC_DOCKER_MODE=wsl",
|
||||
"- LIVESYNC_DOCKER_COMMAND='docker'",
|
||||
"- LIVESYNC_DOCKER_COMMAND='wsl docker'",
|
||||
].join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
async function getDockerInvoker(): Promise<DockerInvoker> {
|
||||
if (!dockerInvokerPromise) {
|
||||
dockerInvokerPromise = resolveDockerInvoker().then((r) => {
|
||||
console.log(`[INFO] docker runner: ${r.label}`);
|
||||
return r;
|
||||
});
|
||||
}
|
||||
return await dockerInvokerPromise;
|
||||
}
|
||||
|
||||
async function docker(...args: string[]): Promise<{ code: number; stdout: string; stderr: string }> {
|
||||
const invoker = await getDockerInvoker();
|
||||
|
||||
// Either:
|
||||
// docker <args>
|
||||
// Or:
|
||||
// wsl docker <args>
|
||||
const finalArgs =
|
||||
invoker.prefix.length === 0
|
||||
? invoker.bin === "wsl"
|
||||
? ["docker", ...args]
|
||||
: args
|
||||
: [...invoker.prefix, ...args];
|
||||
|
||||
const r = await runCommand(invoker.bin, finalArgs);
|
||||
return { code: r.code, stdout: r.stdout, stderr: r.stderr };
|
||||
}
|
||||
|
||||
async function dockerOrFail(...args: string[]): Promise<string> {
|
||||
const r = await docker(...args);
|
||||
if (r.code !== 0) {
|
||||
throw new Error(`docker ${args[0]} failed (code ${r.code}): ${r.stderr.trim()}`);
|
||||
}
|
||||
return r.stdout;
|
||||
}
|
||||
|
||||
function sleep(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
async function waitForCouchdbStable(hostname: string, user: string, password: string): Promise<void> {
|
||||
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
|
||||
const auth = btoa(`${user}:${password}`);
|
||||
const headers = { Authorization: `Basic ${auth}` };
|
||||
let consecutive = 0;
|
||||
for (let i = 0; i < 30; i++) {
|
||||
try {
|
||||
const r = await fetch(`${h}/_up`, {
|
||||
headers,
|
||||
signal: AbortSignal.timeout(3000),
|
||||
});
|
||||
if (r.ok) {
|
||||
consecutive++;
|
||||
if (consecutive >= 3) return;
|
||||
} else {
|
||||
consecutive = 0;
|
||||
}
|
||||
} catch {
|
||||
consecutive = 0;
|
||||
}
|
||||
await sleep(500);
|
||||
}
|
||||
throw new Error("CouchDB did not become stable in time");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fetch with retry (mirrors cli_test_curl_json() retry loop)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function fetchRetry(
|
||||
url: string,
|
||||
init: RequestInit,
|
||||
retries = 30,
|
||||
delayMs = 2000,
|
||||
allowStatus: number[] = []
|
||||
): Promise<void> {
|
||||
let lastError: unknown;
|
||||
let lastStatus: number | undefined;
|
||||
for (let i = 0; i < retries; i++) {
|
||||
try {
|
||||
const r = await fetch(url, {
|
||||
signal: AbortSignal.timeout(5000),
|
||||
...init,
|
||||
});
|
||||
lastStatus = r.status;
|
||||
await r.body?.cancel().catch(() => {});
|
||||
if (r.ok || allowStatus.includes(r.status)) return;
|
||||
lastError = `HTTP ${r.status}`;
|
||||
} catch (e) {
|
||||
lastError = e;
|
||||
}
|
||||
await sleep(delayMs);
|
||||
}
|
||||
throw new Error(
|
||||
`Could not reach ${url} after ${retries} retries: ${lastError} (last status: ${lastStatus ?? "N/A"})`
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// CouchDB
|
||||
// ---------------------------------------------------------------------------
|
||||
//
|
||||
// TODO: these values could be configurable via environment variables.
|
||||
//
|
||||
const COUCHDB_CONTAINER = "couchdb-test";
|
||||
const COUCHDB_IMAGE = "couchdb:3.5.0";
|
||||
|
||||
const MINIO_CONTAINER = "minio-test";
|
||||
const MINIO_IMAGE = "minio/minio";
|
||||
const MINIO_MC_IMAGE = "minio/mc";
|
||||
|
||||
export async function stopCouchdb(): Promise<void> {
|
||||
await docker("stop", COUCHDB_CONTAINER);
|
||||
await docker("rm", COUCHDB_CONTAINER);
|
||||
}
|
||||
|
||||
/**
|
||||
* Start a CouchDB test container, initialise it, and create the test DB.
|
||||
* Mirrors cli_test_start_couchdb() from test-helpers.sh, using direct
|
||||
* docker / fetch calls instead of the bash util scripts.
|
||||
*/
|
||||
export async function startCouchdb(couchdbUri: string, user: string, password: string, dbname: string): Promise<void> {
|
||||
console.log("[INFO] stopping leftover CouchDB container if present");
|
||||
await stopCouchdb().catch(() => {});
|
||||
|
||||
console.log("[INFO] starting CouchDB test container");
|
||||
await dockerOrFail(
|
||||
"run",
|
||||
"-d",
|
||||
"--name",
|
||||
COUCHDB_CONTAINER,
|
||||
"-p",
|
||||
// TODO: port mapping should be configurable.
|
||||
"5989:5984",
|
||||
"-e",
|
||||
`COUCHDB_USER=${user}`,
|
||||
"-e",
|
||||
`COUCHDB_PASSWORD=${password}`,
|
||||
"-e",
|
||||
"COUCHDB_SINGLE_NODE=y",
|
||||
COUCHDB_IMAGE
|
||||
);
|
||||
|
||||
console.log("[INFO] initialising CouchDB");
|
||||
await initCouchdb(couchdbUri, user, password);
|
||||
|
||||
console.log("[INFO] waiting for CouchDB to become stable");
|
||||
await waitForCouchdbStable(couchdbUri, user, password);
|
||||
|
||||
console.log(`[INFO] creating test database: ${dbname}`);
|
||||
await createCouchdbDatabase(couchdbUri, user, password, dbname);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mirror couchdb-init.sh: configure single-node CouchDB via its REST API.
|
||||
*/
|
||||
async function initCouchdb(hostname: string, user: string, password: string, node = "_local"): Promise<void> {
|
||||
// Podman environments often resolve localhost to ::1; use 127.0.0.1 like
|
||||
// the bash script does.
|
||||
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
|
||||
const auth = btoa(`${user}:${password}`);
|
||||
const headers = {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Basic ${auth}`,
|
||||
};
|
||||
|
||||
const calls: Array<[string, string, string]> = [
|
||||
[
|
||||
"POST",
|
||||
`${h}/_cluster_setup`,
|
||||
JSON.stringify({
|
||||
action: "enable_single_node",
|
||||
username: user,
|
||||
password,
|
||||
bind_address: "0.0.0.0",
|
||||
port: 5984,
|
||||
singlenode: true,
|
||||
}),
|
||||
],
|
||||
["PUT", `${h}/_node/${node}/_config/chttpd/require_valid_user`, '"true"'],
|
||||
["PUT", `${h}/_node/${node}/_config/chttpd_auth/require_valid_user`, '"true"'],
|
||||
["PUT", `${h}/_node/${node}/_config/httpd/WWW-Authenticate`, '"Basic realm=\\"couchdb\\""'],
|
||||
["PUT", `${h}/_node/${node}/_config/httpd/enable_cors`, '"true"'],
|
||||
["PUT", `${h}/_node/${node}/_config/chttpd/enable_cors`, '"true"'],
|
||||
["PUT", `${h}/_node/${node}/_config/chttpd/max_http_request_size`, '"4294967296"'],
|
||||
["PUT", `${h}/_node/${node}/_config/couchdb/max_document_size`, '"50000000"'],
|
||||
["PUT", `${h}/_node/${node}/_config/cors/credentials`, '"true"'],
|
||||
["PUT", `${h}/_node/${node}/_config/cors/origins`, '"*"'],
|
||||
];
|
||||
|
||||
for (const [method, url, body] of calls) {
|
||||
await fetchRetry(url, { method, headers, body });
|
||||
}
|
||||
}
|
||||
|
||||
export async function createCouchdbDatabase(
|
||||
hostname: string,
|
||||
user: string,
|
||||
password: string,
|
||||
dbname: string
|
||||
): Promise<void> {
|
||||
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
|
||||
const auth = btoa(`${user}:${password}`);
|
||||
await fetchRetry(`${h}/${dbname}`, {
|
||||
method: "PUT",
|
||||
headers: { Authorization: `Basic ${auth}` },
|
||||
});
|
||||
}
|
||||
|
||||
/** Update a CouchDB document via PUT. Returns the updated document. */
|
||||
export async function updateCouchdbDoc(
|
||||
hostname: string,
|
||||
user: string,
|
||||
password: string,
|
||||
docUrl: string,
|
||||
updater: (doc: Record<string, unknown>) => Record<string, unknown>
|
||||
): Promise<void> {
|
||||
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
|
||||
const auth = btoa(`${user}:${password}`);
|
||||
const headers = {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Basic ${auth}`,
|
||||
};
|
||||
const getRes = await fetch(`${h}/${docUrl}`, { headers });
|
||||
const current = (await getRes.json()) as Record<string, unknown>;
|
||||
const updated = updater(current);
|
||||
await fetchRetry(`${h}/${docUrl}`, {
|
||||
method: "PUT",
|
||||
headers,
|
||||
body: JSON.stringify(updated),
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MinIO
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function shQuote(value: string): string {
|
||||
return `'${value.split("'").join(`'"'"'`)}'`;
|
||||
}
|
||||
|
||||
export async function stopMinio(): Promise<void> {
|
||||
await docker("stop", MINIO_CONTAINER);
|
||||
await docker("rm", MINIO_CONTAINER);
|
||||
}
|
||||
|
||||
async function initMinioBucket(
|
||||
minioEndpoint: string,
|
||||
accessKey: string,
|
||||
secretKey: string,
|
||||
bucket: string
|
||||
): Promise<boolean> {
|
||||
const cmd =
|
||||
`mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` +
|
||||
`mc mb --ignore-existing myminio/${shQuote(bucket)} >/dev/null 2>&1`;
|
||||
const r = await docker("run", "--rm", "--network", "host", "--entrypoint", "/bin/sh", MINIO_MC_IMAGE, "-c", cmd);
|
||||
return r.code === 0;
|
||||
}
|
||||
|
||||
async function waitForMinioBucket(
|
||||
minioEndpoint: string,
|
||||
accessKey: string,
|
||||
secretKey: string,
|
||||
bucket: string
|
||||
): Promise<void> {
|
||||
for (let i = 0; i < 30; i++) {
|
||||
const checkCmd =
|
||||
`mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` +
|
||||
`mc ls myminio/${shQuote(bucket)} >/dev/null 2>&1`;
|
||||
const check = await docker(
|
||||
"run",
|
||||
"--rm",
|
||||
"--network",
|
||||
// Now I used host networking to access the container via localhost for some environments (Docker Desktop on Windows).
|
||||
// We need something good idea to work across all environments.
|
||||
"host",
|
||||
"--entrypoint",
|
||||
"/bin/sh",
|
||||
MINIO_MC_IMAGE,
|
||||
"-c",
|
||||
checkCmd
|
||||
);
|
||||
if (check.code === 0) {
|
||||
return;
|
||||
}
|
||||
await initMinioBucket(minioEndpoint, accessKey, secretKey, bucket);
|
||||
await sleep(2000);
|
||||
}
|
||||
throw new Error(`MinIO bucket not ready: ${bucket}`);
|
||||
}
|
||||
|
||||
export async function startMinio(
|
||||
minioEndpoint: string,
|
||||
accessKey: string,
|
||||
secretKey: string,
|
||||
bucket: string
|
||||
): Promise<void> {
|
||||
console.log("[INFO] stopping leftover MinIO container if present");
|
||||
await stopMinio().catch(() => {});
|
||||
|
||||
console.log("[INFO] starting MinIO test container");
|
||||
await dockerOrFail(
|
||||
"run",
|
||||
"-d",
|
||||
"--name",
|
||||
MINIO_CONTAINER,
|
||||
// TODO: Ports should be configurable.
|
||||
"-p",
|
||||
"9000:9000",
|
||||
"-p",
|
||||
"9001:9001",
|
||||
"-e",
|
||||
`MINIO_ROOT_USER=${accessKey}`,
|
||||
"-e",
|
||||
`MINIO_ROOT_PASSWORD=${secretKey}`,
|
||||
"-e",
|
||||
`MINIO_SERVER_URL=${minioEndpoint}`,
|
||||
MINIO_IMAGE,
|
||||
"server",
|
||||
"/data",
|
||||
"--console-address",
|
||||
":9001"
|
||||
);
|
||||
|
||||
console.log(`[INFO] initialising MinIO test bucket: ${bucket}`);
|
||||
let initialised = false;
|
||||
for (let i = 0; i < 5; i++) {
|
||||
if (await initMinioBucket(minioEndpoint, accessKey, secretKey, bucket)) {
|
||||
initialised = true;
|
||||
break;
|
||||
}
|
||||
await sleep(2000);
|
||||
}
|
||||
if (!initialised) {
|
||||
throw new Error(`Could not initialise MinIO bucket after retries: ${bucket}`);
|
||||
}
|
||||
|
||||
await waitForMinioBucket(minioEndpoint, accessKey, secretKey, bucket);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2P relay (strfry)
|
||||
// ---------------------------------------------------------------------------
|
||||
// TODO: these values could be configurable via environment variables.
|
||||
const P2P_RELAY_CONTAINER = "relay-test";
|
||||
const P2P_RELAY_IMAGE = "ghcr.io/hoytech/strfry:latest";
|
||||
const STRFRY_BOOTSTRAP_SH = String.raw`cat > /tmp/strfry.conf <<"EOF"
|
||||
db = "./strfry-db/"
|
||||
|
||||
relay {
|
||||
bind = "0.0.0.0"
|
||||
port = 7777
|
||||
nofiles = 100000
|
||||
|
||||
info {
|
||||
name = "livesync test relay"
|
||||
description = "local relay for livesync p2p tests"
|
||||
}
|
||||
|
||||
maxWebsocketPayloadSize = 131072
|
||||
autoPingSeconds = 55
|
||||
|
||||
writePolicy {
|
||||
plugin = ""
|
||||
}
|
||||
}
|
||||
EOF
|
||||
exec /app/strfry --config /tmp/strfry.conf relay`;
|
||||
|
||||
export async function stopP2pRelay(): Promise<void> {
|
||||
await docker("stop", P2P_RELAY_CONTAINER);
|
||||
await docker("rm", P2P_RELAY_CONTAINER);
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the local P2P relay container through the same docker runner used
|
||||
* by CouchDB helpers. This keeps process ownership consistent across
|
||||
* start/stop on Windows, WSL, and native Linux/macOS.
|
||||
*/
|
||||
export async function startP2pRelay(): Promise<void> {
|
||||
console.log("[INFO] stopping leftover P2P relay container if present");
|
||||
await stopP2pRelay().catch(() => {});
|
||||
|
||||
console.log("[INFO] starting local P2P relay container");
|
||||
await dockerOrFail(
|
||||
"run",
|
||||
"-d",
|
||||
"--name",
|
||||
P2P_RELAY_CONTAINER,
|
||||
"-p",
|
||||
//TODO: port mapping should be configurable.
|
||||
"4000:7777",
|
||||
"--tmpfs",
|
||||
"/app/strfry-db:rw,size=256m",
|
||||
"--entrypoint",
|
||||
"sh",
|
||||
P2P_RELAY_IMAGE,
|
||||
"-lc",
|
||||
STRFRY_BOOTSTRAP_SH
|
||||
);
|
||||
}
|
||||
|
||||
export function isLocalP2pRelay(relayUrl: string): boolean {
|
||||
return relayUrl === "ws://localhost:4000" || relayUrl === "ws://localhost:4000/";
|
||||
}
|
||||
26
src/apps/cli/testdeno/helpers/env.ts
Normal file
26
src/apps/cli/testdeno/helpers/env.ts
Normal file
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Load a .env-style file (KEY=value per line) into a plain object.
|
||||
* Equivalent to `source $TEST_ENV_FILE; set -a` in bash.
|
||||
* Maybe we should use some library... now it is just the minimal implementation that covers our use cases.
|
||||
*
|
||||
* Supported value formats:
|
||||
* KEY=value
|
||||
* KEY='single quoted'
|
||||
* KEY="double quoted"
|
||||
* # comment lines are ignored
|
||||
*/
|
||||
export async function loadEnvFile(filePath: string): Promise<Record<string, string>> {
|
||||
const text = await Deno.readTextFile(filePath);
|
||||
const result: Record<string, string> = {};
|
||||
for (const line of text.split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed || trimmed.startsWith("#")) continue;
|
||||
const idx = trimmed.indexOf("=");
|
||||
if (idx < 0) continue;
|
||||
const key = trimmed.slice(0, idx).trim();
|
||||
const raw = trimmed.slice(idx + 1).trim();
|
||||
// Strip surrounding single or double quotes
|
||||
result[key] = raw.replace(/^(['"])(.*)\1$/, "$2");
|
||||
}
|
||||
return result;
|
||||
}
|
||||
52
src/apps/cli/testdeno/helpers/p2p.ts
Normal file
52
src/apps/cli/testdeno/helpers/p2p.ts
Normal file
@@ -0,0 +1,52 @@
|
||||
import { runCli } from "./cli.ts";
|
||||
import { isLocalP2pRelay, startP2pRelay, stopP2pRelay } from "./docker.ts";
|
||||
|
||||
export type PeerEntry = {
|
||||
id: string;
|
||||
name: string;
|
||||
};
|
||||
|
||||
export function parsePeerLines(output: string): PeerEntry[] {
|
||||
return output
|
||||
.split(/\r?\n/)
|
||||
.map((line) => line.split("\t"))
|
||||
.filter((parts) => parts.length >= 3 && parts[0] === "[peer]")
|
||||
.map((parts) => ({ id: parts[1], name: parts[2] }));
|
||||
}
|
||||
|
||||
export async function discoverPeer(
|
||||
vaultDir: string,
|
||||
settingsFile: string,
|
||||
timeoutSeconds: number,
|
||||
targetPeer?: string
|
||||
): Promise<PeerEntry> {
|
||||
const result = await runCli(vaultDir, "--settings", settingsFile, "p2p-peers", String(timeoutSeconds));
|
||||
if (result.code !== 0) {
|
||||
throw new Error(`p2p-peers failed\n${result.combined}`);
|
||||
}
|
||||
const peers = parsePeerLines(result.stdout);
|
||||
if (targetPeer) {
|
||||
const matched = peers.find((peer) => peer.id === targetPeer || peer.name === targetPeer);
|
||||
if (matched) return matched;
|
||||
}
|
||||
if (peers.length === 0) {
|
||||
const fallback = result.combined.match(/Advertisement from\s+([^\s]+)/);
|
||||
if (fallback?.[1]) {
|
||||
return { id: fallback[1], name: fallback[1] };
|
||||
}
|
||||
throw new Error(`No peers discovered\n${result.combined}`);
|
||||
}
|
||||
return peers[0];
|
||||
}
|
||||
|
||||
export async function maybeStartLocalRelay(relay: string): Promise<boolean> {
|
||||
if (!isLocalP2pRelay(relay)) return false;
|
||||
await startP2pRelay();
|
||||
return true;
|
||||
}
|
||||
|
||||
export async function stopLocalRelayIfStarted(started: boolean): Promise<void> {
|
||||
if (started) {
|
||||
await stopP2pRelay().catch(() => {});
|
||||
}
|
||||
}
|
||||
205
src/apps/cli/testdeno/helpers/settings.ts
Normal file
205
src/apps/cli/testdeno/helpers/settings.ts
Normal file
@@ -0,0 +1,205 @@
|
||||
import { join } from "@std/path";
|
||||
import { CLI_DIR, runCliOrFail } from "./cli.ts";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Settings file initialisation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Generate a default settings file using the CLI's init-settings command. */
|
||||
export async function initSettingsFile(settingsFile: string): Promise<void> {
|
||||
await runCliOrFail("init-settings", "--force", settingsFile);
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a full setup URI from a settings file via src/lib API.
|
||||
* Mirrors the bash flow in test-setup-put-cat-linux.sh.
|
||||
*/
|
||||
export async function generateSetupUriFromSettings(settingsFile: string, setupPassphrase: string): Promise<string> {
|
||||
const repoRoot = join(CLI_DIR, "..", "..", "..");
|
||||
const script = [
|
||||
"import fs from 'node:fs';",
|
||||
"import { pathToFileURL } from 'node:url';",
|
||||
"(async () => {",
|
||||
" const modulePath = process.env.REPO_ROOT + '/src/lib/src/API/processSetting.ts';",
|
||||
" const moduleUrl = pathToFileURL(modulePath).href;",
|
||||
" const { encodeSettingsToSetupURI } = await import(moduleUrl);",
|
||||
" const settingsPath = process.env.SETTINGS_FILE;",
|
||||
" const passphrase = process.env.SETUP_PASSPHRASE;",
|
||||
" const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf-8'));",
|
||||
" settings.couchDB_DBNAME = 'setup-put-cat-db';",
|
||||
" settings.couchDB_URI = 'http://127.0.0.1:5999';",
|
||||
" settings.couchDB_USER = 'dummy';",
|
||||
" settings.couchDB_PASSWORD = 'dummy';",
|
||||
" settings.liveSync = false;",
|
||||
" settings.syncOnStart = false;",
|
||||
" settings.syncOnSave = false;",
|
||||
" const uri = await encodeSettingsToSetupURI(settings, passphrase);",
|
||||
" process.stdout.write(uri.trim());",
|
||||
"})();",
|
||||
].join("\n");
|
||||
|
||||
const scriptPath = await Deno.makeTempFile({
|
||||
prefix: "livesync-setup-uri-",
|
||||
suffix: ".mts",
|
||||
});
|
||||
await Deno.writeTextFile(scriptPath, script);
|
||||
|
||||
try {
|
||||
const cmd = new Deno.Command("npx", {
|
||||
args: ["tsx", scriptPath],
|
||||
cwd: CLI_DIR,
|
||||
env: {
|
||||
REPO_ROOT: repoRoot,
|
||||
SETTINGS_FILE: settingsFile,
|
||||
SETUP_PASSPHRASE: setupPassphrase,
|
||||
},
|
||||
stdin: "null",
|
||||
stdout: "piped",
|
||||
stderr: "piped",
|
||||
});
|
||||
|
||||
const { code, stdout, stderr } = await cmd.output();
|
||||
const dec = new TextDecoder();
|
||||
if (code !== 0) {
|
||||
throw new Error(
|
||||
`Failed to generate setup URI (code ${code})\nstdout: ${dec.decode(stdout)}\nstderr: ${dec.decode(stderr)}`
|
||||
);
|
||||
}
|
||||
|
||||
const uri = dec.decode(stdout).trim();
|
||||
if (!uri) {
|
||||
throw new Error("Failed to generate setup URI: output is empty");
|
||||
}
|
||||
return uri;
|
||||
} finally {
|
||||
await Deno.remove(scriptPath).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
/** Set isConfigured=true in a settings file (required for mirror / scan). */
|
||||
export async function markSettingsConfigured(settingsFile: string): Promise<void> {
|
||||
const data = JSON.parse(await Deno.readTextFile(settingsFile));
|
||||
data.isConfigured = true;
|
||||
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// CouchDB remote settings
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Apply CouchDB connection details to a settings file.
|
||||
* Mirrors cli_test_apply_couchdb_settings() from test-helpers.sh.
|
||||
*/
|
||||
export async function applyCouchdbSettings(
|
||||
settingsFile: string,
|
||||
couchdbUri: string,
|
||||
couchdbUser: string,
|
||||
couchdbPassword: string,
|
||||
couchdbDbname: string,
|
||||
liveSync = false
|
||||
): Promise<void> {
|
||||
const data = JSON.parse(await Deno.readTextFile(settingsFile));
|
||||
data.couchDB_URI = couchdbUri;
|
||||
data.couchDB_USER = couchdbUser;
|
||||
data.couchDB_PASSWORD = couchdbPassword;
|
||||
data.couchDB_DBNAME = couchdbDbname;
|
||||
if (liveSync) {
|
||||
data.liveSync = true;
|
||||
data.syncOnStart = false;
|
||||
data.syncOnSave = false;
|
||||
data.usePluginSync = false;
|
||||
}
|
||||
data.isConfigured = true;
|
||||
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
|
||||
}
|
||||
|
||||
export async function applyRemoteSyncSettings(
|
||||
settingsFile: string,
|
||||
options: {
|
||||
remoteType: "COUCHDB" | "MINIO";
|
||||
couchdbUri?: string;
|
||||
couchdbUser?: string;
|
||||
couchdbPassword?: string;
|
||||
couchdbDbname?: string;
|
||||
minioBucket?: string;
|
||||
minioEndpoint?: string;
|
||||
minioAccessKey?: string;
|
||||
minioSecretKey?: string;
|
||||
encrypt?: boolean;
|
||||
passphrase?: string;
|
||||
}
|
||||
): Promise<void> {
|
||||
const data = JSON.parse(await Deno.readTextFile(settingsFile));
|
||||
|
||||
if (options.remoteType === "COUCHDB") {
|
||||
data.remoteType = "";
|
||||
data.couchDB_URI = options.couchdbUri;
|
||||
data.couchDB_USER = options.couchdbUser;
|
||||
data.couchDB_PASSWORD = options.couchdbPassword;
|
||||
data.couchDB_DBNAME = options.couchdbDbname;
|
||||
} else {
|
||||
data.remoteType = "MINIO";
|
||||
data.bucket = options.minioBucket;
|
||||
data.endpoint = options.minioEndpoint;
|
||||
data.accessKey = options.minioAccessKey;
|
||||
data.secretKey = options.minioSecretKey;
|
||||
data.region = "auto";
|
||||
data.forcePathStyle = true;
|
||||
}
|
||||
|
||||
data.liveSync = true;
|
||||
data.syncOnStart = false;
|
||||
data.syncOnSave = false;
|
||||
data.usePluginSync = false;
|
||||
data.encrypt = options.encrypt === true;
|
||||
data.passphrase = options.encrypt ? (options.passphrase ?? "") : "";
|
||||
data.isConfigured = true;
|
||||
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// P2P settings
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Apply P2P connection details to a settings file.
|
||||
* Mirrors cli_test_apply_p2p_settings() from test-helpers.sh.
|
||||
*/
|
||||
export async function applyP2pSettings(
|
||||
settingsFile: string,
|
||||
roomId: string,
|
||||
passphrase: string,
|
||||
appId = "self-hosted-livesync-cli-tests",
|
||||
relays = "ws://localhost:4000/",
|
||||
autoAccept = "~.*"
|
||||
): Promise<void> {
|
||||
const data = JSON.parse(await Deno.readTextFile(settingsFile));
|
||||
data.P2P_Enabled = true;
|
||||
data.P2P_AutoStart = false;
|
||||
data.P2P_AutoBroadcast = false;
|
||||
data.P2P_AppID = appId;
|
||||
data.P2P_roomID = roomId;
|
||||
data.P2P_passphrase = passphrase;
|
||||
data.P2P_relays = relays;
|
||||
data.P2P_AutoAcceptingPeers = autoAccept;
|
||||
data.P2P_AutoDenyingPeers = "";
|
||||
data.P2P_IsHeadless = true;
|
||||
data.isConfigured = true;
|
||||
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
|
||||
}
|
||||
|
||||
export async function applyP2pTestTweaks(settingsFile: string, deviceName: string, passphrase: string): Promise<void> {
|
||||
const data = JSON.parse(await Deno.readTextFile(settingsFile));
|
||||
data.remoteType = "ONLY_P2P";
|
||||
data.encrypt = true;
|
||||
data.passphrase = passphrase;
|
||||
data.usePathObfuscation = true;
|
||||
data.handleFilenameCaseSensitive = false;
|
||||
data.customChunkSize = 50;
|
||||
data.usePluginSyncV2 = true;
|
||||
data.doNotUseFixedRevisionForChunks = false;
|
||||
data.P2P_DevicePeerName = deviceName;
|
||||
data.isConfigured = true;
|
||||
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
|
||||
}
|
||||
33
src/apps/cli/testdeno/helpers/temp.ts
Normal file
33
src/apps/cli/testdeno/helpers/temp.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { join } from "@std/path";
|
||||
|
||||
/**
|
||||
* A temporary directory that cleans itself up via `await using`.
|
||||
* Requires TypeScript 5.2+ / Deno 1.40+ for the AsyncDisposable protocol.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* await using tmp = await TempDir.create();
|
||||
* const file = tmp.join("data.json");
|
||||
* ```
|
||||
*/
|
||||
export class TempDir implements AsyncDisposable {
|
||||
readonly path: string;
|
||||
|
||||
private constructor(path: string) {
|
||||
this.path = path;
|
||||
}
|
||||
|
||||
static async create(prefix = "livesync-deno-test"): Promise<TempDir> {
|
||||
const path = await Deno.makeTempDir({ prefix: `${prefix}.` });
|
||||
return new TempDir(path);
|
||||
}
|
||||
|
||||
/** Return an OS path joined to the temp directory root. */
|
||||
join(...parts: string[]): string {
|
||||
return join(this.path, ...parts);
|
||||
}
|
||||
|
||||
async [Symbol.asyncDispose](): Promise<void> {
|
||||
await Deno.remove(this.path, { recursive: true }).catch(() => {});
|
||||
}
|
||||
}
|
||||
279
src/apps/cli/testdeno/test-e2e-two-vaults-couchdb.ts
Normal file
279
src/apps/cli/testdeno/test-e2e-two-vaults-couchdb.ts
Normal file
@@ -0,0 +1,279 @@
|
||||
import { assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { loadEnvFile } from "./helpers/env.ts";
|
||||
import {
|
||||
runCli,
|
||||
runCliOrFail,
|
||||
runCliWithInputOrFail,
|
||||
sanitiseCatStdout,
|
||||
assertFilesEqual,
|
||||
jsonStringField,
|
||||
} from "./helpers/cli.ts";
|
||||
import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||
import { startCouchdb, startMinio, stopCouchdb, stopMinio } from "./helpers/docker.ts";
|
||||
import { join } from "@std/path";
|
||||
|
||||
const TEST_ENV = join(import.meta.dirname!, "..", ".test.env");
|
||||
type RemoteType = "COUCHDB" | "MINIO";
|
||||
|
||||
function requireEnv(env: Record<string, string>, key: string): string {
|
||||
const value = env[key]?.trim();
|
||||
if (!value) throw new Error(`Required env var is missing: ${key}`);
|
||||
return value;
|
||||
}
|
||||
|
||||
export async function runScenario(remoteType: RemoteType, encrypt: boolean): Promise<void> {
|
||||
const env = await loadEnvFile(TEST_ENV);
|
||||
const dbSuffix = `${Date.now()}-${Math.floor(Math.random() * 100000)}`;
|
||||
|
||||
const couchdbUri = remoteType === "COUCHDB" ? requireEnv(env, "hostname").replace(/\/$/, "") : "";
|
||||
const couchdbUser = remoteType === "COUCHDB" ? requireEnv(env, "username") : "";
|
||||
const couchdbPassword = remoteType === "COUCHDB" ? requireEnv(env, "password") : "";
|
||||
const dbPrefix = remoteType === "COUCHDB" ? requireEnv(env, "dbname") : "";
|
||||
const dbname = remoteType === "COUCHDB" ? `${dbPrefix}-${dbSuffix}` : "";
|
||||
|
||||
const minioEndpoint = remoteType === "MINIO" ? requireEnv(env, "minioEndpoint").replace(/\/$/, "") : "";
|
||||
const minioAccessKey = remoteType === "MINIO" ? requireEnv(env, "accessKey") : "";
|
||||
const minioSecretKey = remoteType === "MINIO" ? requireEnv(env, "secretKey") : "";
|
||||
const minioBucketBase = remoteType === "MINIO" ? requireEnv(env, "bucketName") : "";
|
||||
const minioBucket = remoteType === "MINIO" ? `${minioBucketBase}-${dbSuffix}` : "";
|
||||
|
||||
const passphrase = "e2e-passphrase";
|
||||
|
||||
await using workDir = await TempDir.create(
|
||||
`livesync-cli-e2e-${remoteType.toLowerCase()}-${encrypt ? "enc1" : "enc0"}`
|
||||
);
|
||||
const vaultA = workDir.join("testvault_a");
|
||||
const vaultB = workDir.join("testvault_b");
|
||||
const settingsA = workDir.join("test-settings-a.json");
|
||||
const settingsB = workDir.join("test-settings-b.json");
|
||||
const pushSrc = workDir.join("push-source.txt");
|
||||
const pullDst = workDir.join("pull-destination.txt");
|
||||
const pushBinarySrc = workDir.join("push-source.bin");
|
||||
const pullBinaryDst = workDir.join("pull-destination.bin");
|
||||
await Deno.mkdir(vaultA, { recursive: true });
|
||||
await Deno.mkdir(vaultB, { recursive: true });
|
||||
|
||||
const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
|
||||
if (remoteType === "COUCHDB") {
|
||||
await startCouchdb(couchdbUri, couchdbUser, couchdbPassword, dbname);
|
||||
} else {
|
||||
await startMinio(minioEndpoint, minioAccessKey, minioSecretKey, minioBucket);
|
||||
}
|
||||
|
||||
try {
|
||||
await initSettingsFile(settingsA);
|
||||
await initSettingsFile(settingsB);
|
||||
await applyRemoteSyncSettings(settingsA, {
|
||||
remoteType,
|
||||
couchdbUri,
|
||||
couchdbUser,
|
||||
couchdbPassword,
|
||||
couchdbDbname: dbname,
|
||||
minioBucket,
|
||||
minioEndpoint,
|
||||
minioAccessKey,
|
||||
minioSecretKey,
|
||||
encrypt,
|
||||
passphrase,
|
||||
});
|
||||
await applyRemoteSyncSettings(settingsB, {
|
||||
remoteType,
|
||||
couchdbUri,
|
||||
couchdbUser,
|
||||
couchdbPassword,
|
||||
couchdbDbname: dbname,
|
||||
minioBucket,
|
||||
minioEndpoint,
|
||||
minioAccessKey,
|
||||
minioSecretKey,
|
||||
encrypt,
|
||||
passphrase,
|
||||
});
|
||||
|
||||
const syncBoth = async () => {
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
|
||||
};
|
||||
|
||||
const targetAOnly = "e2e/a-only-info.md";
|
||||
const targetSync = "e2e/sync-info.md";
|
||||
const targetSyncTwiceFirst = "e2e/sync-twice-first.md";
|
||||
const targetSyncTwiceSecond = "e2e/sync-twice-second.md";
|
||||
const targetPush = "e2e/pushed-from-a.md";
|
||||
const targetPut = "e2e/put-from-a.md";
|
||||
const targetPushBinary = "e2e/pushed-from-a.bin";
|
||||
const targetConflict = "e2e/conflict.md";
|
||||
|
||||
await runCliWithInputOrFail("alpha-from-a\n", vaultA, "--settings", settingsA, "put", targetAOnly);
|
||||
const infoAOnly = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetAOnly);
|
||||
assert(infoAOnly.includes(`"path": "${targetAOnly}"`));
|
||||
|
||||
await runCliWithInputOrFail("visible-after-sync\n", vaultA, "--settings", settingsA, "put", targetSync);
|
||||
await syncBoth();
|
||||
const infoBSync = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetSync);
|
||||
assert(infoBSync.includes(`"path": "${targetSync}"`));
|
||||
|
||||
await runCliWithInputOrFail(
|
||||
`first-sync-round-${dbSuffix}\n`,
|
||||
vaultA,
|
||||
"--settings",
|
||||
settingsA,
|
||||
"put",
|
||||
targetSyncTwiceFirst
|
||||
);
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
|
||||
const firstVisible = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetSyncTwiceFirst)
|
||||
).trimEnd();
|
||||
assert(firstVisible === `first-sync-round-${dbSuffix}`);
|
||||
|
||||
await runCliWithInputOrFail(
|
||||
`second-sync-round-${dbSuffix}\n`,
|
||||
vaultA,
|
||||
"--settings",
|
||||
settingsA,
|
||||
"put",
|
||||
targetSyncTwiceSecond
|
||||
);
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
|
||||
const secondVisible = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetSyncTwiceSecond)
|
||||
).trimEnd();
|
||||
assert(secondVisible === `second-sync-round-${dbSuffix}`);
|
||||
|
||||
await Deno.writeTextFile(pushSrc, `pushed-content-${dbSuffix}\n`);
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "push", pushSrc, targetPush);
|
||||
await runCliWithInputOrFail(`put-content-${dbSuffix}\n`, vaultA, "--settings", settingsA, "put", targetPut);
|
||||
await syncBoth();
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "pull", targetPush, pullDst);
|
||||
await assertFilesEqual(pushSrc, pullDst, "B pull result does not match pushed source");
|
||||
const catBPut = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetPut)
|
||||
).trimEnd();
|
||||
assert(catBPut === `put-content-${dbSuffix}`);
|
||||
|
||||
const binary = new Uint8Array(4096);
|
||||
binary.fill(0x61);
|
||||
await Deno.writeFile(pushBinarySrc, binary);
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "push", pushBinarySrc, targetPushBinary);
|
||||
await syncBoth();
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "pull", targetPushBinary, pullBinaryDst);
|
||||
await assertFilesEqual(pushBinarySrc, pullBinaryDst, "B pull result does not match pushed binary source");
|
||||
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "rm", targetPut);
|
||||
await syncBoth();
|
||||
const removed = await runCli(vaultB, "--settings", settingsB, "cat", targetPut);
|
||||
assert(removed.code !== 0, `B cat should fail after A removed the file\n${removed.combined}`);
|
||||
|
||||
await runCliWithInputOrFail("conflict-base\n", vaultA, "--settings", settingsA, "put", targetConflict);
|
||||
await syncBoth();
|
||||
await runCliWithInputOrFail(
|
||||
`conflict-from-a-${dbSuffix}\n`,
|
||||
vaultA,
|
||||
"--settings",
|
||||
settingsA,
|
||||
"put",
|
||||
targetConflict
|
||||
);
|
||||
await runCliWithInputOrFail(
|
||||
`conflict-from-b-${dbSuffix}\n`,
|
||||
vaultB,
|
||||
"--settings",
|
||||
settingsB,
|
||||
"put",
|
||||
targetConflict
|
||||
);
|
||||
|
||||
let infoAConflict = "";
|
||||
let infoBConflict = "";
|
||||
let conflictDetected = false;
|
||||
for (const side of ["a", "b", "a"] as const) {
|
||||
await runCliOrFail(
|
||||
side === "a" ? vaultA : vaultB,
|
||||
"--settings",
|
||||
side === "a" ? settingsA : settingsB,
|
||||
"sync"
|
||||
);
|
||||
infoAConflict = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetConflict);
|
||||
infoBConflict = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetConflict);
|
||||
if (
|
||||
jsonStringField(infoAConflict, "conflicts") !== "N/A" ||
|
||||
jsonStringField(infoBConflict, "conflicts") !== "N/A"
|
||||
) {
|
||||
conflictDetected = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
assert(conflictDetected, `conflict was expected\nA: ${infoAConflict}\nB: ${infoBConflict}`);
|
||||
|
||||
const lsAConflict =
|
||||
(await runCliOrFail(vaultA, "--settings", settingsA, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
|
||||
const lsBConflict =
|
||||
(await runCliOrFail(vaultB, "--settings", settingsB, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
|
||||
const revA = lsAConflict.split("\t")[3] ?? "";
|
||||
const revB = lsBConflict.split("\t")[3] ?? "";
|
||||
assert(
|
||||
revA.includes("*") || revB.includes("*"),
|
||||
`conflicted entry should be marked with '*'\nA: ${lsAConflict}\nB: ${lsBConflict}`
|
||||
);
|
||||
|
||||
const keepRevision = jsonStringField(infoAConflict, "revision");
|
||||
assert(keepRevision.length > 0, `could not extract revision\n${infoAConflict}`);
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "resolve", targetConflict, keepRevision);
|
||||
|
||||
let resolved = false;
|
||||
let infoAResolved = "";
|
||||
let infoBResolved = "";
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await syncBoth();
|
||||
infoAResolved = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetConflict);
|
||||
infoBResolved = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetConflict);
|
||||
if (
|
||||
jsonStringField(infoAResolved, "conflicts") === "N/A" &&
|
||||
jsonStringField(infoBResolved, "conflicts") === "N/A"
|
||||
) {
|
||||
resolved = true;
|
||||
break;
|
||||
}
|
||||
const retryRevision = jsonStringField(infoAResolved, "revision");
|
||||
if (retryRevision) {
|
||||
await runCli(vaultA, "--settings", settingsA, "resolve", targetConflict, retryRevision);
|
||||
}
|
||||
}
|
||||
assert(resolved, `conflicts should be resolved\nA: ${infoAResolved}\nB: ${infoBResolved}`);
|
||||
|
||||
const lsAResolved =
|
||||
(await runCliOrFail(vaultA, "--settings", settingsA, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
|
||||
const lsBResolved =
|
||||
(await runCliOrFail(vaultB, "--settings", settingsB, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
|
||||
assert(!(lsAResolved.split("\t")[3] ?? "").includes("*"));
|
||||
assert(!(lsBResolved.split("\t")[3] ?? "").includes("*"));
|
||||
|
||||
const catAResolved = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultA, "--settings", settingsA, "cat", targetConflict)
|
||||
).trimEnd();
|
||||
const catBResolved = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetConflict)
|
||||
).trimEnd();
|
||||
assert(catAResolved === catBResolved, `resolved content should match\nA: ${catAResolved}\nB: ${catBResolved}`);
|
||||
} finally {
|
||||
if (!keepDocker) {
|
||||
if (remoteType === "COUCHDB") {
|
||||
await stopCouchdb().catch(() => {});
|
||||
} else {
|
||||
await stopMinio().catch(() => {});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Deno.test("e2e: two vaults over CouchDB without encryption", async () => {
|
||||
await runScenario("COUCHDB", false);
|
||||
});
|
||||
|
||||
Deno.test("e2e: two vaults over CouchDB with encryption", async () => {
|
||||
await runScenario("COUCHDB", true);
|
||||
});
|
||||
20
src/apps/cli/testdeno/test-e2e-two-vaults-matrix.ts
Normal file
20
src/apps/cli/testdeno/test-e2e-two-vaults-matrix.ts
Normal file
@@ -0,0 +1,20 @@
|
||||
import { runScenario } from "./test-e2e-two-vaults-couchdb.ts";
|
||||
|
||||
type MatrixCase = {
|
||||
remoteType: "COUCHDB" | "MINIO";
|
||||
encrypt: boolean;
|
||||
label: string;
|
||||
};
|
||||
|
||||
const matrixCases: MatrixCase[] = [
|
||||
{ remoteType: "COUCHDB", encrypt: false, label: "COUCHDB-enc0" },
|
||||
{ remoteType: "COUCHDB", encrypt: true, label: "COUCHDB-enc1" },
|
||||
{ remoteType: "MINIO", encrypt: false, label: "MINIO-enc0" },
|
||||
{ remoteType: "MINIO", encrypt: true, label: "MINIO-enc1" },
|
||||
];
|
||||
|
||||
for (const tc of matrixCases) {
|
||||
Deno.test(`e2e matrix: ${tc.label}`, async () => {
|
||||
await runScenario(tc.remoteType, tc.encrypt);
|
||||
});
|
||||
}
|
||||
196
src/apps/cli/testdeno/test-mirror.ts
Normal file
196
src/apps/cli/testdeno/test-mirror.ts
Normal file
@@ -0,0 +1,196 @@
|
||||
/**
|
||||
* Deno port of test-mirror-linux.sh
|
||||
*
|
||||
* Tests the `mirror` command — bidirectional synchronisation between a local
|
||||
* storage directory (vault) and an in-process database.
|
||||
*
|
||||
* Covered cases (identical to the bash test):
|
||||
* 1. Storage-only file -> synced into DB (UPDATE DATABASE)
|
||||
* 2. DB-only file -> restored to storage (UPDATE STORAGE)
|
||||
* 3. DB-deleted file -> NOT restored to storage (UPDATE STORAGE skip)
|
||||
* 4. Both, storage newer -> DB updated (SYNC: STORAGE -> DB)
|
||||
* 5. Both, DB newer -> storage updated (SYNC: DB -> STORAGE)
|
||||
* 6. Compatibility mode -> omitted vault-path works (same DB + vault path)
|
||||
*
|
||||
* No external services are required.
|
||||
*
|
||||
* Run:
|
||||
* deno test -A test-mirror.ts
|
||||
*/
|
||||
|
||||
import { assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { runCliOrFail } from "./helpers/cli.ts";
|
||||
import { initSettingsFile, markSettingsConfigured } from "./helpers/settings.ts";
|
||||
|
||||
Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
|
||||
await using workDir = await TempDir.create("livesync-cli-mirror");
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Shared setup
|
||||
// -------------------------------------------------------------------
|
||||
const settingsFile = workDir.join("data.json");
|
||||
const vaultDir = workDir.join("vault");
|
||||
const dbDir = workDir.join("db");
|
||||
await Deno.mkdir(workDir.join("vault", "test"), { recursive: true });
|
||||
await Deno.mkdir(dbDir, { recursive: true });
|
||||
|
||||
await initSettingsFile(settingsFile);
|
||||
// isConfigured=true is required for canProceedScan in the mirror command.
|
||||
await markSettingsConfigured(settingsFile);
|
||||
|
||||
// Copy settings to the DB directory (separated-path mode)
|
||||
const dbSettings = workDir.join("db", "settings.json");
|
||||
await Deno.copyFile(settingsFile, dbSettings);
|
||||
|
||||
/** Run mirror in separated-path mode: DB dir ≠ vault dir. */
|
||||
const runMirror = () => runCliOrFail(dbDir, "--settings", dbSettings, "mirror", vaultDir);
|
||||
|
||||
/** Run mirror in compatibility mode: DB path = vault path. */
|
||||
const runMirrorCompat = () => runCliOrFail(vaultDir, "--settings", settingsFile, "mirror");
|
||||
|
||||
// Helper wrappers
|
||||
const dbRun = (...args: string[]) => runCliOrFail(dbDir, "--settings", dbSettings, ...args);
|
||||
const compatRun = (...args: string[]) => runCliOrFail(vaultDir, "--settings", settingsFile, ...args);
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Case 1: storage-only -> DB (UPDATE DATABASE)
|
||||
// -------------------------------------------------------------------
|
||||
await t.step("case 1: storage-only file is synced into DB", async () => {
|
||||
const storageFile = workDir.join("vault", "test", "storage-only.md");
|
||||
await Deno.writeTextFile(storageFile, "storage-only content\n");
|
||||
|
||||
await runMirror();
|
||||
|
||||
const resultFile = workDir.join("case1-pull.txt");
|
||||
await dbRun("pull", "test/storage-only.md", resultFile);
|
||||
|
||||
const storageContent = await Deno.readTextFile(storageFile);
|
||||
const pulledContent = await Deno.readTextFile(resultFile);
|
||||
assert(
|
||||
storageContent === pulledContent,
|
||||
`storage-only file NOT synced into DB\nexpected: ${storageContent}\ngot: ${pulledContent}`
|
||||
);
|
||||
console.log("[PASS] case 1: storage-only file was synced into DB");
|
||||
});
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Case 2: DB-only -> storage (UPDATE STORAGE)
|
||||
// -------------------------------------------------------------------
|
||||
await t.step("case 2: DB-only file is restored to storage", async () => {
|
||||
await dbRun(
|
||||
"push",
|
||||
// write inline via push (pipe not needed — push takes a file path)
|
||||
// create a temp file with content and push it
|
||||
await (async () => {
|
||||
const tmp = workDir.join("db-only-src.txt");
|
||||
await Deno.writeTextFile(tmp, "db-only content\n");
|
||||
return tmp;
|
||||
})(),
|
||||
"test/db-only.md"
|
||||
);
|
||||
|
||||
const storagePath = workDir.join("vault", "test", "db-only.md");
|
||||
assert(!(await exists(storagePath)), "db-only.md unexpectedly exists in storage before mirror");
|
||||
|
||||
await runMirror();
|
||||
|
||||
assert(await exists(storagePath), "DB-only file NOT restored to storage after mirror");
|
||||
const content = await Deno.readTextFile(storagePath);
|
||||
assert(content === "db-only content\n", `DB-only file restored but content mismatch: '${content}'`);
|
||||
console.log("[PASS] case 2: DB-only file was restored to storage");
|
||||
});
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Case 3: DB-deleted -> storage untouched
|
||||
// -------------------------------------------------------------------
|
||||
await t.step("case 3: DB-deleted entry is NOT restored to storage", async () => {
|
||||
const deletedSrc = workDir.join("deleted-src.txt");
|
||||
await Deno.writeTextFile(deletedSrc, "to-be-deleted\n");
|
||||
await dbRun("push", deletedSrc, "test/deleted.md");
|
||||
await dbRun("rm", "test/deleted.md");
|
||||
|
||||
await runMirror();
|
||||
|
||||
const storagePath = workDir.join("vault", "test", "deleted.md");
|
||||
assert(!(await exists(storagePath)), "deleted DB entry was incorrectly restored to storage");
|
||||
console.log("[PASS] case 3: deleted DB entry was NOT restored to storage");
|
||||
});
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Case 4: storage newer -> DB updated (SYNC: STORAGE -> DB)
|
||||
// -------------------------------------------------------------------
|
||||
await t.step("case 4: storage newer than DB -> DB is updated", async () => {
|
||||
// Seed DB with old content (mtime ~ now)
|
||||
const seedFile = workDir.join("case4-seed.txt");
|
||||
await Deno.writeTextFile(seedFile, "old content\n");
|
||||
await dbRun("push", seedFile, "test/sync-storage-newer.md");
|
||||
|
||||
// Write new content to storage with a timestamp 1 hour in the future
|
||||
const storageFile = workDir.join("vault", "test", "sync-storage-newer.md");
|
||||
await Deno.writeTextFile(storageFile, "new content\n");
|
||||
await Deno.utime(storageFile, new Date(), new Date(Date.now() + 3600_000));
|
||||
|
||||
await runMirror();
|
||||
|
||||
const resultFile = workDir.join("case4-pull.txt");
|
||||
await dbRun("pull", "test/sync-storage-newer.md", resultFile);
|
||||
const storageContent = await Deno.readTextFile(storageFile);
|
||||
const pulledContent = await Deno.readTextFile(resultFile);
|
||||
assert(
|
||||
storageContent === pulledContent,
|
||||
`DB NOT updated to match newer storage file\nexpected: ${storageContent}\ngot: ${pulledContent}`
|
||||
);
|
||||
console.log("[PASS] case 4: DB updated to match newer storage file");
|
||||
});
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Case 5: DB newer -> storage updated (SYNC: DB -> STORAGE)
|
||||
// -------------------------------------------------------------------
|
||||
await t.step("case 5: DB newer than storage -> storage is updated", async () => {
|
||||
// Write old content to storage with a timestamp 1 hour in the past
|
||||
const storageFile = workDir.join("vault", "test", "sync-db-newer.md");
|
||||
await Deno.writeTextFile(storageFile, "old storage content\n");
|
||||
await Deno.utime(storageFile, new Date(), new Date(Date.now() - 3600_000));
|
||||
|
||||
// Write new content to DB only (mtime ~ now, newer than the storage file)
|
||||
const dbNewFile = workDir.join("case5-db-new.txt");
|
||||
await Deno.writeTextFile(dbNewFile, "new db content\n");
|
||||
await dbRun("push", dbNewFile, "test/sync-db-newer.md");
|
||||
|
||||
await runMirror();
|
||||
|
||||
const content = await Deno.readTextFile(storageFile);
|
||||
assert(content === "new db content\n", `storage NOT updated to match newer DB entry (got: '${content}')`);
|
||||
console.log("[PASS] case 5: storage updated to match newer DB entry");
|
||||
});
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Case 6: compatibility mode (vault path = DB path)
|
||||
// -------------------------------------------------------------------
|
||||
await t.step("case 6: compatibility mode (omitted vault-path)", async () => {
|
||||
const compatFile = workDir.join("vault", "compat.md");
|
||||
await Deno.writeTextFile(compatFile, "compat-content\n");
|
||||
|
||||
await runMirrorCompat();
|
||||
|
||||
const resultFile = workDir.join("case6-pull.txt");
|
||||
await compatRun("pull", "compat.md", resultFile);
|
||||
const pulled = await Deno.readTextFile(resultFile);
|
||||
assert(pulled === "compat-content\n", `Compatibility mode failed to sync file into DB (got: '${pulled}')`);
|
||||
console.log("[PASS] case 6: compatibility mode works");
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Utility
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function exists(path: string): Promise<boolean> {
|
||||
try {
|
||||
await Deno.stat(path);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
40
src/apps/cli/testdeno/test-p2p-host.ts
Normal file
40
src/apps/cli/testdeno/test-p2p-host.ts
Normal file
@@ -0,0 +1,40 @@
|
||||
import { assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { initSettingsFile, applyP2pSettings } from "./helpers/settings.ts";
|
||||
import { startP2pRelay, stopP2pRelay, isLocalP2pRelay } from "./helpers/docker.ts";
|
||||
import { startCliInBackground } from "./helpers/backgroundCli.ts";
|
||||
|
||||
Deno.test("p2p-host: starts and becomes ready", async () => {
|
||||
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
|
||||
const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`;
|
||||
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
|
||||
const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests";
|
||||
const useInternalRelay = Deno.env.get("USE_INTERNAL_RELAY") !== "0";
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-host");
|
||||
const vaultDir = workDir.join("vault-host");
|
||||
const settingsFile = workDir.join("settings-host.json");
|
||||
await Deno.mkdir(vaultDir, { recursive: true });
|
||||
|
||||
let relayStarted = false;
|
||||
if (useInternalRelay && isLocalP2pRelay(relay)) {
|
||||
await startP2pRelay();
|
||||
relayStarted = true;
|
||||
}
|
||||
|
||||
try {
|
||||
await initSettingsFile(settingsFile);
|
||||
await applyP2pSettings(settingsFile, roomId, passphrase, appId, relay);
|
||||
const host = startCliInBackground(vaultDir, "--settings", settingsFile, "p2p-host");
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
assert(host.combined.includes("P2P host is running"));
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
} finally {
|
||||
if (relayStarted) {
|
||||
await stopP2pRelay().catch(() => {});
|
||||
}
|
||||
}
|
||||
});
|
||||
42
src/apps/cli/testdeno/test-p2p-peers-local-relay.ts
Normal file
42
src/apps/cli/testdeno/test-p2p-peers-local-relay.ts
Normal file
@@ -0,0 +1,42 @@
|
||||
import { assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { initSettingsFile, applyP2pSettings, applyP2pTestTweaks } from "./helpers/settings.ts";
|
||||
import { startCliInBackground } from "./helpers/backgroundCli.ts";
|
||||
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
|
||||
|
||||
Deno.test("p2p-peers: discovers host through local relay", async () => {
|
||||
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
|
||||
const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`;
|
||||
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
|
||||
const timeoutSeconds = Number(Deno.env.get("TIMEOUT_SECONDS") ?? "8");
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-peers-local-relay");
|
||||
const hostVault = workDir.join("vault-host");
|
||||
const hostSettings = workDir.join("settings-host.json");
|
||||
const clientVault = workDir.join("vault");
|
||||
const clientSettings = workDir.join("settings.json");
|
||||
await Deno.mkdir(hostVault, { recursive: true });
|
||||
await Deno.mkdir(clientVault, { recursive: true });
|
||||
|
||||
const relayStarted = await maybeStartLocalRelay(relay);
|
||||
try {
|
||||
await initSettingsFile(hostSettings);
|
||||
await initSettingsFile(clientSettings);
|
||||
await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
|
||||
await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
|
||||
await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase);
|
||||
await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase);
|
||||
|
||||
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const peer = await discoverPeer(clientVault, clientSettings, timeoutSeconds);
|
||||
assert(peer.id.length > 0);
|
||||
assert(peer.name.length > 0);
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
} finally {
|
||||
await stopLocalRelayIfStarted(relayStarted);
|
||||
}
|
||||
});
|
||||
59
src/apps/cli/testdeno/test-p2p-sync.ts
Normal file
59
src/apps/cli/testdeno/test-p2p-sync.ts
Normal file
@@ -0,0 +1,59 @@
|
||||
import { assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { initSettingsFile, applyP2pSettings, applyP2pTestTweaks } from "./helpers/settings.ts";
|
||||
import { startCliInBackground } from "./helpers/backgroundCli.ts";
|
||||
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
|
||||
import { runCli } from "./helpers/cli.ts";
|
||||
|
||||
Deno.test("p2p-sync: discovers peer and completes sync", async () => {
|
||||
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
|
||||
const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`;
|
||||
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
|
||||
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "12");
|
||||
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15");
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-sync");
|
||||
const hostVault = workDir.join("vault-host");
|
||||
const hostSettings = workDir.join("settings-host.json");
|
||||
const clientVault = workDir.join("vault-sync");
|
||||
const clientSettings = workDir.join("settings-sync.json");
|
||||
await Deno.mkdir(hostVault, { recursive: true });
|
||||
await Deno.mkdir(clientVault, { recursive: true });
|
||||
|
||||
const relayStarted = await maybeStartLocalRelay(relay);
|
||||
try {
|
||||
await initSettingsFile(hostSettings);
|
||||
await initSettingsFile(clientSettings);
|
||||
await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
|
||||
await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
|
||||
await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase);
|
||||
await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase);
|
||||
|
||||
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const peer = await discoverPeer(
|
||||
clientVault,
|
||||
clientSettings,
|
||||
peersTimeout,
|
||||
Deno.env.get("TARGET_PEER") ?? undefined
|
||||
);
|
||||
const syncResult = await runCli(
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"p2p-sync",
|
||||
peer.id,
|
||||
String(syncTimeout)
|
||||
);
|
||||
assert(
|
||||
syncResult.code === 0,
|
||||
`p2p-sync failed\nstdout: ${syncResult.stdout}\nstderr: ${syncResult.stderr}`
|
||||
);
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
} finally {
|
||||
await stopLocalRelayIfStarted(relayStarted);
|
||||
}
|
||||
});
|
||||
118
src/apps/cli/testdeno/test-p2p-three-nodes-conflict.ts
Normal file
118
src/apps/cli/testdeno/test-p2p-three-nodes-conflict.ts
Normal file
@@ -0,0 +1,118 @@
|
||||
import { assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { applyP2pSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||
import { startCliInBackground } from "./helpers/backgroundCli.ts";
|
||||
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
|
||||
import { jsonStringField, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts";
|
||||
|
||||
Deno.test("p2p: three nodes detect and resolve conflicts", async () => {
|
||||
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
|
||||
const roomId = `${Deno.env.get("ROOM_ID_PREFIX") ?? "p2p-room"}-${Date.now()}`;
|
||||
const passphrase = `${Deno.env.get("PASSPHRASE_PREFIX") ?? "p2p-pass"}-${Date.now()}`;
|
||||
const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests";
|
||||
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "10");
|
||||
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15");
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-3nodes");
|
||||
const vaultA = workDir.join("vault-a");
|
||||
const vaultB = workDir.join("vault-b");
|
||||
const vaultC = workDir.join("vault-c");
|
||||
const settingsA = workDir.join("settings-a.json");
|
||||
const settingsB = workDir.join("settings-b.json");
|
||||
const settingsC = workDir.join("settings-c.json");
|
||||
await Deno.mkdir(vaultA, { recursive: true });
|
||||
await Deno.mkdir(vaultB, { recursive: true });
|
||||
await Deno.mkdir(vaultC, { recursive: true });
|
||||
|
||||
const relayStarted = await maybeStartLocalRelay(relay);
|
||||
try {
|
||||
for (const settings of [settingsA, settingsB, settingsC]) {
|
||||
await initSettingsFile(settings);
|
||||
await applyP2pSettings(settings, roomId, passphrase, appId, relay);
|
||||
}
|
||||
|
||||
const host = startCliInBackground(vaultA, "--settings", settingsA, "p2p-host");
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const peerFromB = await discoverPeer(vaultB, settingsB, peersTimeout);
|
||||
const peerFromC = await discoverPeer(vaultC, settingsC, peersTimeout);
|
||||
const targetPath = "p2p/conflicted-from-two-clients.txt";
|
||||
|
||||
await runCliWithInputOrFail("from-client-b-v1\n", vaultB, "--settings", settingsB, "put", targetPath);
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout));
|
||||
await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout));
|
||||
|
||||
let visibleOnC = "";
|
||||
for (let i = 0; i < 5; i++) {
|
||||
try {
|
||||
visibleOnC = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultC, "--settings", settingsC, "cat", targetPath)
|
||||
).trimEnd();
|
||||
if (visibleOnC === "from-client-b-v1") break;
|
||||
} catch {
|
||||
// retry below
|
||||
}
|
||||
await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout));
|
||||
}
|
||||
assert(visibleOnC === "from-client-b-v1", `C should see file created by B, got: ${visibleOnC}`);
|
||||
|
||||
await runCliWithInputOrFail("from-client-b-v2\n", vaultB, "--settings", settingsB, "put", targetPath);
|
||||
await runCliWithInputOrFail("from-client-c-v2\n", vaultC, "--settings", settingsC, "put", targetPath);
|
||||
|
||||
const [syncB, syncC] = await Promise.all([
|
||||
runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout)),
|
||||
runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout)),
|
||||
]);
|
||||
void syncB;
|
||||
void syncC;
|
||||
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout));
|
||||
await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout));
|
||||
|
||||
const infoBBefore = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetPath);
|
||||
const conflictsBBefore = jsonStringField(infoBBefore, "conflicts");
|
||||
const keepRevB = jsonStringField(infoBBefore, "revision");
|
||||
assert(
|
||||
conflictsBBefore !== "N/A" && conflictsBBefore.length > 0,
|
||||
`expected conflicts on B\n${infoBBefore}`
|
||||
);
|
||||
assert(keepRevB.length > 0, `could not read revision on B\n${infoBBefore}`);
|
||||
|
||||
const infoCBefore = await runCliOrFail(vaultC, "--settings", settingsC, "info", targetPath);
|
||||
const conflictsCBefore = jsonStringField(infoCBefore, "conflicts");
|
||||
const keepRevC = jsonStringField(infoCBefore, "revision");
|
||||
assert(
|
||||
conflictsCBefore !== "N/A" && conflictsCBefore.length > 0,
|
||||
`expected conflicts on C\n${infoCBefore}`
|
||||
);
|
||||
assert(keepRevC.length > 0, `could not read revision on C\n${infoCBefore}`);
|
||||
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "resolve", targetPath, keepRevB);
|
||||
await runCliOrFail(vaultC, "--settings", settingsC, "resolve", targetPath, keepRevC);
|
||||
|
||||
const infoBAfter = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetPath);
|
||||
const infoCAfter = await runCliOrFail(vaultC, "--settings", settingsC, "info", targetPath);
|
||||
assert(jsonStringField(infoBAfter, "conflicts") === "N/A", `conflict still remains on B\n${infoBAfter}`);
|
||||
assert(jsonStringField(infoCAfter, "conflicts") === "N/A", `conflict still remains on C\n${infoCAfter}`);
|
||||
|
||||
const finalContentB = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetPath)
|
||||
).trimEnd();
|
||||
const finalContentC = sanitiseCatStdout(
|
||||
await runCliOrFail(vaultC, "--settings", settingsC, "cat", targetPath)
|
||||
).trimEnd();
|
||||
assert(
|
||||
finalContentB === "from-client-b-v2" || finalContentB === "from-client-c-v2",
|
||||
`unexpected final content on B: ${finalContentB}`
|
||||
);
|
||||
assert(
|
||||
finalContentC === "from-client-b-v2" || finalContentC === "from-client-c-v2",
|
||||
`unexpected final content on C: ${finalContentC}`
|
||||
);
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
} finally {
|
||||
await stopLocalRelayIfStarted(relayStarted);
|
||||
}
|
||||
});
|
||||
111
src/apps/cli/testdeno/test-p2p-upload-download-repro.ts
Normal file
111
src/apps/cli/testdeno/test-p2p-upload-download-repro.ts
Normal file
@@ -0,0 +1,111 @@
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
|
||||
import { startCliInBackground } from "./helpers/backgroundCli.ts";
|
||||
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
|
||||
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
|
||||
|
||||
async function writeFilledFile(path: string, size: number, byte: number): Promise<void> {
|
||||
const data = new Uint8Array(size);
|
||||
data.fill(byte);
|
||||
await Deno.writeFile(path, data);
|
||||
}
|
||||
|
||||
Deno.test("p2p: upload/download reproduction scenario", async () => {
|
||||
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
|
||||
const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests";
|
||||
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "20");
|
||||
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "240");
|
||||
const roomId = `p2p-room-${Date.now()}`;
|
||||
const passphrase = `p2p-pass-${Date.now()}`;
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-upload-download");
|
||||
const vaultHost = workDir.join("vault-host");
|
||||
const vaultUp = workDir.join("vault-up");
|
||||
const vaultDown = workDir.join("vault-down");
|
||||
const settingsHost = workDir.join("settings-host.json");
|
||||
const settingsUp = workDir.join("settings-up.json");
|
||||
const settingsDown = workDir.join("settings-down.json");
|
||||
for (const dir of [vaultHost, vaultUp, vaultDown]) {
|
||||
await Deno.mkdir(dir, { recursive: true });
|
||||
}
|
||||
|
||||
const relayStarted = await maybeStartLocalRelay(relay);
|
||||
try {
|
||||
for (const settings of [settingsHost, settingsUp, settingsDown]) {
|
||||
await initSettingsFile(settings);
|
||||
await applyP2pSettings(settings, roomId, passphrase, appId, relay, "~.*");
|
||||
}
|
||||
await applyP2pTestTweaks(settingsHost, "p2p-cli-host", passphrase);
|
||||
await applyP2pTestTweaks(settingsUp, `p2p-cli-upload-${Date.now()}`, passphrase);
|
||||
await applyP2pTestTweaks(settingsDown, `p2p-cli-download-${Date.now()}`, passphrase);
|
||||
|
||||
const host = startCliInBackground(vaultHost, "--settings", settingsHost, "p2p-host");
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const uploadPeer = await discoverPeer(vaultUp, settingsUp, peersTimeout);
|
||||
|
||||
const storeText = workDir.join("store-file.md");
|
||||
const diffA = workDir.join("test-diff-1.md");
|
||||
const diffB = workDir.join("test-diff-2.md");
|
||||
const diffC = workDir.join("test-diff-3.md");
|
||||
await Deno.writeTextFile(storeText, "Hello, World!\n");
|
||||
await Deno.writeTextFile(diffA, "Content A\n");
|
||||
await Deno.writeTextFile(diffB, "Content B\n");
|
||||
await Deno.writeTextFile(diffC, "Content C\n");
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", storeText, "p2p/store-file.md");
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffA, "p2p/test-diff-1.md");
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffB, "p2p/test-diff-2.md");
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffC, "p2p/test-diff-3.md");
|
||||
|
||||
const large100k = workDir.join("large-100k.txt");
|
||||
const large1m = workDir.join("large-1m.txt");
|
||||
const binary100k = workDir.join("binary-100k.bin");
|
||||
const binary5m = workDir.join("binary-5m.bin");
|
||||
await Deno.writeTextFile(large100k, "a".repeat(100000));
|
||||
await Deno.writeTextFile(large1m, "b".repeat(1000000));
|
||||
await writeFilledFile(binary100k, 100000, 0x5a);
|
||||
await writeFilledFile(binary5m, 5000000, 0x7c);
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", large100k, "p2p/large-100000.md");
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", large1m, "p2p/large-1000000.md");
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", binary100k, "p2p/binary-100000.bin");
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", binary5m, "p2p/binary-5000000.bin");
|
||||
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "p2p-sync", uploadPeer.id, String(syncTimeout));
|
||||
await runCliOrFail(vaultUp, "--settings", settingsUp, "p2p-sync", uploadPeer.id, String(syncTimeout));
|
||||
|
||||
const downloadPeer = await discoverPeer(vaultDown, settingsDown, peersTimeout);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "p2p-sync", downloadPeer.id, String(syncTimeout));
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "p2p-sync", downloadPeer.id, String(syncTimeout));
|
||||
|
||||
const downStoreText = workDir.join("down-store-file.md");
|
||||
const downDiffA = workDir.join("down-test-diff-1.md");
|
||||
const downDiffB = workDir.join("down-test-diff-2.md");
|
||||
const downDiffC = workDir.join("down-test-diff-3.md");
|
||||
const downLarge100k = workDir.join("down-large-100k.txt");
|
||||
const downLarge1m = workDir.join("down-large-1m.txt");
|
||||
const downBinary100k = workDir.join("down-binary-100k.bin");
|
||||
const downBinary5m = workDir.join("down-binary-5m.bin");
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/store-file.md", downStoreText);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-1.md", downDiffA);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-2.md", downDiffB);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-3.md", downDiffC);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/large-100000.md", downLarge100k);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/large-1000000.md", downLarge1m);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/binary-100000.bin", downBinary100k);
|
||||
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/binary-5000000.bin", downBinary5m);
|
||||
|
||||
await assertFilesEqual(storeText, downStoreText, "store-file mismatch");
|
||||
await assertFilesEqual(diffA, downDiffA, "test-diff-1 mismatch");
|
||||
await assertFilesEqual(diffB, downDiffB, "test-diff-2 mismatch");
|
||||
await assertFilesEqual(diffC, downDiffC, "test-diff-3 mismatch");
|
||||
await assertFilesEqual(large100k, downLarge100k, "large-100000 mismatch");
|
||||
await assertFilesEqual(large1m, downLarge1m, "large-1000000 mismatch");
|
||||
await assertFilesEqual(binary100k, downBinary100k, "binary-100000 mismatch");
|
||||
await assertFilesEqual(binary5m, downBinary5m, "binary-5000000 mismatch");
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
} finally {
|
||||
await stopLocalRelayIfStarted(relayStarted);
|
||||
}
|
||||
});
|
||||
64
src/apps/cli/testdeno/test-push-pull.ts
Normal file
64
src/apps/cli/testdeno/test-push-pull.ts
Normal file
@@ -0,0 +1,64 @@
|
||||
/**
|
||||
* Deno port of test-push-pull-linux.sh
|
||||
*
|
||||
* Requires CouchDB connection details either via environment variables or a
|
||||
* .test.env file. If neither is present the test logs a warning and the
|
||||
* CLI will likely fail at the push step.
|
||||
*
|
||||
* Run:
|
||||
* deno test -A test-push-pull.ts
|
||||
*
|
||||
* With explicit CouchDB:
|
||||
* COUCHDB_URI=http://127.0.0.1:5984 \
|
||||
* COUCHDB_USER=admin \
|
||||
* COUCHDB_PASSWORD=password \
|
||||
* COUCHDB_DBNAME=livesync-test \
|
||||
* deno test -A test-push-pull.ts
|
||||
*/
|
||||
|
||||
import { join } from "@std/path";
|
||||
import { assertEquals } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { runCliOrFail } from "./helpers/cli.ts";
|
||||
import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||
|
||||
const REMOTE_PATH = Deno.env.get("REMOTE_PATH") ?? "test/push-pull.txt";
|
||||
|
||||
Deno.test("push/pull roundtrip", async () => {
|
||||
await using workDir = await TempDir.create("livesync-cli-push-pull");
|
||||
|
||||
const settingsFile = workDir.join("data.json");
|
||||
const vaultDir = workDir.join("vault");
|
||||
await Deno.mkdir(join(vaultDir, "test"), { recursive: true });
|
||||
|
||||
await initSettingsFile(settingsFile);
|
||||
|
||||
const uri = Deno.env.get("COUCHDB_URI") ?? "";
|
||||
const user = Deno.env.get("COUCHDB_USER") ?? "";
|
||||
const password = Deno.env.get("COUCHDB_PASSWORD") ?? "";
|
||||
const dbname = Deno.env.get("COUCHDB_DBNAME") ?? "";
|
||||
|
||||
if (uri && user && password && dbname) {
|
||||
console.log("[INFO] applying CouchDB env vars to settings");
|
||||
await applyCouchdbSettings(settingsFile, uri, user, password, dbname);
|
||||
} else {
|
||||
console.warn(
|
||||
"[WARN] CouchDB env vars not fully set — push/pull may fail unless the generated settings already contain connection details"
|
||||
);
|
||||
}
|
||||
|
||||
const srcFile = workDir.join("push-source.txt");
|
||||
const pulledFile = workDir.join("pull-result.txt");
|
||||
const content = `push-pull-test ${new Date().toISOString()}\n`;
|
||||
await Deno.writeTextFile(srcFile, content);
|
||||
|
||||
console.log(`[INFO] push -> ${REMOTE_PATH}`);
|
||||
await runCliOrFail(vaultDir, "--settings", settingsFile, "push", srcFile, REMOTE_PATH);
|
||||
|
||||
console.log(`[INFO] pull <- ${REMOTE_PATH}`);
|
||||
await runCliOrFail(vaultDir, "--settings", settingsFile, "pull", REMOTE_PATH, pulledFile);
|
||||
|
||||
const pulled = await Deno.readTextFile(pulledFile);
|
||||
assertEquals(content, pulled, "push/pull roundtrip content mismatch");
|
||||
console.log("[PASS] push/pull roundtrip matched");
|
||||
});
|
||||
214
src/apps/cli/testdeno/test-setup-put-cat.ts
Normal file
214
src/apps/cli/testdeno/test-setup-put-cat.ts
Normal file
@@ -0,0 +1,214 @@
|
||||
/**
|
||||
* Deno port of test-setup-put-cat-linux.sh
|
||||
*
|
||||
* Tests all local-DB file operations that require no external remote:
|
||||
* setup /
|
||||
* push / cat / ls / info / rm / resolve / cat-rev / pull-rev
|
||||
*
|
||||
* Run (no external services needed):
|
||||
* deno test -A test-setup-put-cat.ts
|
||||
*/
|
||||
|
||||
import { join } from "@std/path";
|
||||
import { assertEquals, assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { runCli, runCliOrFail, runCliWithInput, sanitiseCatStdout } from "./helpers/cli.ts";
|
||||
import { generateSetupUriFromSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||
|
||||
const REMOTE_PATH = Deno.env.get("REMOTE_PATH") ?? "test/setup-put-cat.txt";
|
||||
const SETUP_PASSPHRASE = Deno.env.get("SETUP_PASSPHRASE") ?? "setup-passphrase";
|
||||
|
||||
Deno.test("CLI file operations: push / cat / ls / info / rm / resolve / cat-rev / pull-rev", async (t) => {
|
||||
await using workDir = await TempDir.create("livesync-cli-setup-put-cat");
|
||||
|
||||
const settingsFile = workDir.join("data.json");
|
||||
const vaultDir = workDir.join("vault");
|
||||
await Deno.mkdir(join(vaultDir, "test"), { recursive: true });
|
||||
|
||||
await initSettingsFile(settingsFile);
|
||||
|
||||
const setupUri = await generateSetupUriFromSettings(settingsFile, SETUP_PASSPHRASE);
|
||||
const setupResult = await runCliWithInput(
|
||||
`${SETUP_PASSPHRASE}\n`,
|
||||
vaultDir,
|
||||
"--settings",
|
||||
settingsFile,
|
||||
"setup",
|
||||
setupUri
|
||||
);
|
||||
assert(setupResult.code === 0, `setup command exited with ${setupResult.code}\n${setupResult.combined}`);
|
||||
assert(
|
||||
setupResult.combined.includes("[Command] setup ->"),
|
||||
`setup command did not execute expected code path\n${setupResult.combined}`
|
||||
);
|
||||
|
||||
const run = (...args: string[]) => runCliOrFail(vaultDir, "--settings", settingsFile, ...args);
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// push / cat roundtrip
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("push/cat roundtrip", async () => {
|
||||
const srcFile = workDir.join("put-source.txt");
|
||||
const content = `setup-put-cat-test ${new Date().toISOString()}\nline-2\n`;
|
||||
await Deno.writeTextFile(srcFile, content);
|
||||
|
||||
console.log(`[INFO] push -> ${REMOTE_PATH}`);
|
||||
await runCliWithInput(content, vaultDir, "--settings", settingsFile, "put", REMOTE_PATH);
|
||||
|
||||
console.log(`[INFO] cat <- ${REMOTE_PATH}`);
|
||||
const rawOutput = await run("cat", REMOTE_PATH);
|
||||
const catOutput = sanitiseCatStdout(rawOutput);
|
||||
|
||||
assertEquals(content, catOutput, "push/cat roundtrip content mismatch");
|
||||
console.log("[PASS] push/cat roundtrip matched");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// ls: single file
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("ls output format (single file)", async () => {
|
||||
const lsOutput = await run("ls", REMOTE_PATH);
|
||||
const line = lsOutput
|
||||
.trim()
|
||||
.split("\n")
|
||||
.find((l) => l.startsWith(REMOTE_PATH + "\t"));
|
||||
assert(line, `ls output did not include ${REMOTE_PATH}`);
|
||||
|
||||
const [lsPath, lsSize, lsMtime, lsRev] = line.split("\t");
|
||||
assertEquals(lsPath, REMOTE_PATH, "ls path column mismatch");
|
||||
assert(/^\d+$/.test(lsSize), `ls size not numeric: ${lsSize}`);
|
||||
assert(/^\d+$/.test(lsMtime), `ls mtime not numeric: ${lsMtime}`);
|
||||
assert(lsRev?.length > 0, "ls revision column is empty");
|
||||
console.log("[PASS] ls output format matched");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// ls: prefix filter and sort order
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("ls prefix filter and sort order", async () => {
|
||||
await runCliWithInput("file-a\n", vaultDir, "--settings", settingsFile, "put", "test/a-first.txt");
|
||||
await runCliWithInput("file-z\n", vaultDir, "--settings", settingsFile, "put", "test/z-last.txt");
|
||||
|
||||
const lsOut = await run("ls", "test/");
|
||||
const lines = lsOut.trim().split("\n").filter(Boolean);
|
||||
assert(lines.length >= 3, "ls prefix output expected at least 3 rows");
|
||||
|
||||
// Verify sorted ascending by path
|
||||
const paths = lines.map((l) => l.split("\t")[0]);
|
||||
for (let i = 1; i < paths.length; i++) {
|
||||
assert(paths[i - 1] <= paths[i], `ls output not sorted: ${paths[i - 1]} > ${paths[i]}`);
|
||||
}
|
||||
assert(
|
||||
lines.some((l) => l.startsWith("test/a-first.txt\t")),
|
||||
"ls prefix output missing test/a-first.txt"
|
||||
);
|
||||
assert(
|
||||
lines.some((l) => l.startsWith("test/z-last.txt\t")),
|
||||
"ls prefix output missing test/z-last.txt"
|
||||
);
|
||||
console.log("[PASS] ls prefix and sorting matched");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// ls: no-match prefix returns empty output
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("ls no-match prefix returns empty", async () => {
|
||||
const lsOut = await run("ls", "no-such-prefix/");
|
||||
assertEquals(lsOut.trim(), "", "ls no-match prefix should produce empty output");
|
||||
console.log("[PASS] ls no-match prefix matched");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// info: JSON output format
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("info output JSON format", async () => {
|
||||
const infoOut = await run("info", REMOTE_PATH);
|
||||
let data: Record<string, unknown>;
|
||||
try {
|
||||
data = JSON.parse(infoOut);
|
||||
} catch {
|
||||
throw new Error(`info output is not valid JSON:\n${infoOut}`);
|
||||
}
|
||||
assertEquals(data.path, REMOTE_PATH, "info .path mismatch");
|
||||
assertEquals(data.filename, REMOTE_PATH.split("/").at(-1), "info .filename mismatch");
|
||||
assert(typeof data.size === "number" && data.size >= 0, `info .size invalid: ${data.size}`);
|
||||
assert(typeof data.chunks === "number" && (data.chunks as number) >= 1, `info .chunks invalid: ${data.chunks}`);
|
||||
assertEquals(data.conflicts, "N/A", "info .conflicts should be N/A");
|
||||
console.log("[PASS] info output format matched");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// info: non-existent path exits non-zero
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("info non-existent path returns non-zero", async () => {
|
||||
const r = await runCli(vaultDir, "--settings", settingsFile, "info", "no-such-file.md");
|
||||
assert(r.code !== 0, "info on non-existent file should exit non-zero");
|
||||
console.log("[PASS] info non-existent path returns non-zero");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// rm: removes file from ls and makes cat fail
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("rm removes target from ls and cat", async () => {
|
||||
await run("rm", "test/z-last.txt");
|
||||
|
||||
const catResult = await runCli(vaultDir, "--settings", settingsFile, "cat", "test/z-last.txt");
|
||||
assert(catResult.code !== 0, "rm target should not be readable by cat");
|
||||
|
||||
const lsOut = await run("ls", "test/");
|
||||
assert(!lsOut.includes("test/z-last.txt\t"), "rm target should not appear in ls output");
|
||||
console.log("[PASS] rm removed target from visible entries");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// resolve: accepts current revision, rejects invalid revision
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("resolve: valid and invalid revisions", async () => {
|
||||
const lsLine = (await run("ls", "test/a-first.txt")).trim().split("\n")[0];
|
||||
assert(lsLine, "could not fetch revision for resolve test");
|
||||
const rev = lsLine.split("\t")[3];
|
||||
assert(rev?.length > 0, "revision was empty for resolve test");
|
||||
|
||||
await run("resolve", "test/a-first.txt", rev);
|
||||
console.log("[PASS] resolve accepted current revision");
|
||||
|
||||
const badR = await runCli(vaultDir, "--settings", settingsFile, "resolve", "test/a-first.txt", "9-no-such-rev");
|
||||
assert(badR.code !== 0, "resolve with non-existent revision should exit non-zero");
|
||||
console.log("[PASS] resolve non-existent revision returns non-zero");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// cat-rev / pull-rev: retrieve a past revision
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("cat-rev / pull-rev: retrieve past revision", async () => {
|
||||
const revPath = "test/revision-history.txt";
|
||||
await runCliWithInput("revision-v1\n", vaultDir, "--settings", settingsFile, "put", revPath);
|
||||
await runCliWithInput("revision-v2\n", vaultDir, "--settings", settingsFile, "put", revPath);
|
||||
await runCliWithInput("revision-v3\n", vaultDir, "--settings", settingsFile, "put", revPath);
|
||||
|
||||
const infoOut = await run("info", revPath);
|
||||
const infoData = JSON.parse(infoOut) as {
|
||||
revisions?: string[];
|
||||
};
|
||||
const revisions = Array.isArray(infoData.revisions) ? infoData.revisions : [];
|
||||
const pastRev = revisions.find((r): r is string => typeof r === "string" && r !== "N/A");
|
||||
assert(pastRev, "info output did not include any past revision");
|
||||
|
||||
const catRevOut = await run("cat-rev", revPath, pastRev);
|
||||
const catRevClean = sanitiseCatStdout(catRevOut);
|
||||
assert(
|
||||
catRevClean === "revision-v1\n" || catRevClean === "revision-v2\n",
|
||||
`cat-rev output did not match expected past revision:\n${catRevClean}`
|
||||
);
|
||||
console.log("[PASS] cat-rev matched one of the past revisions from info");
|
||||
|
||||
const pullRevFile = workDir.join("rev-pull-output.txt");
|
||||
await run("pull-rev", revPath, pullRevFile, pastRev);
|
||||
const pullRevContent = await Deno.readTextFile(pullRevFile);
|
||||
assert(
|
||||
pullRevContent === "revision-v1\n" || pullRevContent === "revision-v2\n",
|
||||
`pull-rev output did not match expected past revision:\n${pullRevContent}`
|
||||
);
|
||||
console.log("[PASS] pull-rev matched one of the past revisions from info");
|
||||
});
|
||||
});
|
||||
97
src/apps/cli/testdeno/test-sync-locked-remote.ts
Normal file
97
src/apps/cli/testdeno/test-sync-locked-remote.ts
Normal file
@@ -0,0 +1,97 @@
|
||||
/**
|
||||
* Deno port of test-sync-locked-remote-linux.sh
|
||||
*
|
||||
* Verifies CLI sync behaviour when the remote milestone document is unlocked
|
||||
* versus locked.
|
||||
*/
|
||||
|
||||
import { assert, assertStringIncludes } from "@std/assert";
|
||||
import { join } from "@std/path";
|
||||
import { loadEnvFile } from "./helpers/env.ts";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { runCli } from "./helpers/cli.ts";
|
||||
import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||
import { createCouchdbDatabase, startCouchdb, stopCouchdb, updateCouchdbDoc } from "./helpers/docker.ts";
|
||||
|
||||
const TEST_ENV = join(import.meta.dirname!, "..", ".test.env");
|
||||
const MILESTONE_DOC = "_local/obsydian_livesync_milestone";
|
||||
|
||||
function requireEnv(env: Record<string, string>, key: string): string {
|
||||
const value = env[key]?.trim();
|
||||
if (!value) {
|
||||
throw new Error(`Required env var is missing: ${key}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
Deno.test("sync: actionable error against locked remote DB", async () => {
|
||||
const env = await loadEnvFile(TEST_ENV);
|
||||
const couchdbUri = requireEnv(env, "hostname").replace(/\/$/, "");
|
||||
const couchdbUser = requireEnv(env, "username");
|
||||
const couchdbPassword = requireEnv(env, "password");
|
||||
const dbPrefix = requireEnv(env, "dbname");
|
||||
const dbname = `${dbPrefix}-locked-${Date.now()}-${Math.floor(Math.random() * 100000)}`;
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-locked-test");
|
||||
const vaultDir = workDir.join("vault");
|
||||
const settingsFile = workDir.join("settings.json");
|
||||
await Deno.mkdir(vaultDir, { recursive: true });
|
||||
|
||||
const shouldStartDocker = Deno.env.get("LIVESYNC_START_DOCKER") !== "0";
|
||||
const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
|
||||
|
||||
if (shouldStartDocker) {
|
||||
console.log(`[INFO] starting CouchDB and creating test database: ${dbname}`);
|
||||
await startCouchdb(couchdbUri, couchdbUser, couchdbPassword, dbname);
|
||||
} else {
|
||||
console.log(`[INFO] using existing CouchDB and creating test database: ${dbname}`);
|
||||
await createCouchdbDatabase(couchdbUri, couchdbUser, couchdbPassword, dbname);
|
||||
}
|
||||
|
||||
try {
|
||||
await initSettingsFile(settingsFile);
|
||||
await applyCouchdbSettings(settingsFile, couchdbUri, couchdbUser, couchdbPassword, dbname, true);
|
||||
|
||||
console.log("[CASE] initial sync to create milestone document");
|
||||
const initialSync = await runCli(vaultDir, "--settings", settingsFile, "sync");
|
||||
assert(
|
||||
initialSync.code === 0,
|
||||
`initial sync failed\nstdout: ${initialSync.stdout}\nstderr: ${initialSync.stderr}`
|
||||
);
|
||||
|
||||
const updateMilestone = async (locked: boolean) => {
|
||||
await updateCouchdbDoc(couchdbUri, couchdbUser, couchdbPassword, `${dbname}/${MILESTONE_DOC}`, (doc) => ({
|
||||
...doc,
|
||||
locked,
|
||||
accepted_nodes: [],
|
||||
}));
|
||||
};
|
||||
|
||||
console.log("[CASE] sync should succeed when remote is not locked");
|
||||
await updateMilestone(false);
|
||||
const unlockedSync = await runCli(vaultDir, "--settings", settingsFile, "sync");
|
||||
assert(
|
||||
unlockedSync.code === 0,
|
||||
`sync should succeed when remote is not locked\nstdout: ${unlockedSync.stdout}\nstderr: ${unlockedSync.stderr}`
|
||||
);
|
||||
assert(
|
||||
!unlockedSync.combined.includes("The remote database is locked"),
|
||||
`locked error should not appear when remote is not locked\n${unlockedSync.combined}`
|
||||
);
|
||||
console.log("[PASS] unlocked remote DB syncs successfully");
|
||||
|
||||
console.log("[CASE] sync should fail with actionable error when remote is locked");
|
||||
await updateMilestone(true);
|
||||
const lockedSync = await runCli(vaultDir, "--settings", settingsFile, "sync");
|
||||
assert(
|
||||
lockedSync.code !== 0,
|
||||
`sync should fail when remote is locked\nstdout: ${lockedSync.stdout}\nstderr: ${lockedSync.stderr}`
|
||||
);
|
||||
assertStringIncludes(lockedSync.combined, "The remote database is locked and this device is not yet accepted");
|
||||
console.log("[PASS] locked remote DB produces actionable CLI error");
|
||||
} finally {
|
||||
if (shouldStartDocker && !keepDocker) {
|
||||
await stopCouchdb().catch(() => {});
|
||||
}
|
||||
}
|
||||
});
|
||||
287
src/apps/cli/testdeno/test-sync-two-local-databases.ts
Normal file
287
src/apps/cli/testdeno/test-sync-two-local-databases.ts
Normal file
@@ -0,0 +1,287 @@
|
||||
/**
|
||||
* Deno port of test-sync-two-local-databases-linux.sh
|
||||
*
|
||||
* Tests two-vault synchronisation via CouchDB including conflict detection
|
||||
* and resolution.
|
||||
*
|
||||
* Requires CouchDB connection details. Provide them via environment variables
|
||||
* OR place a .test.env file at src/apps/cli/.test.env.
|
||||
*
|
||||
* By default, a CouchDB Docker container is started automatically
|
||||
* (LIVESYNC_START_DOCKER=1). Set LIVESYNC_START_DOCKER=0 to use an existing
|
||||
* CouchDB instance instead.
|
||||
*
|
||||
* Run:
|
||||
* deno test -A test-sync-two-local-databases.ts
|
||||
*
|
||||
* With an existing CouchDB:
|
||||
* COUCHDB_URI=http://127.0.0.1:5984 \
|
||||
* COUCHDB_USER=admin \
|
||||
* COUCHDB_PASSWORD=password \
|
||||
* COUCHDB_DBNAME=livesync-test \
|
||||
* LIVESYNC_START_DOCKER=0 \
|
||||
* deno test -A test-sync-two-local-databases.ts
|
||||
*/
|
||||
|
||||
import { join } from "@std/path";
|
||||
import { assertEquals, assert } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { CLI_DIR, runCliOrFail, jsonFieldIsNa } from "./helpers/cli.ts";
|
||||
import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||
import { startCouchdb, stopCouchdb } from "./helpers/docker.ts";
|
||||
import { loadEnvFile } from "./helpers/env.ts";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Load configuration
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function resolveConfig(): Promise<{
|
||||
uri: string;
|
||||
user: string;
|
||||
password: string;
|
||||
baseDbname: string;
|
||||
} | null> {
|
||||
let env: Record<string, string> = {};
|
||||
|
||||
// 1. Explicit environment variables take priority
|
||||
if (Deno.env.get("COUCHDB_URI")) {
|
||||
env = Object.fromEntries(Deno.env.toObject());
|
||||
} else {
|
||||
// 2. TEST_ENV_FILE env var
|
||||
const envFile = Deno.env.get("TEST_ENV_FILE") ?? join(CLI_DIR, ".test.env");
|
||||
try {
|
||||
env = await loadEnvFile(envFile);
|
||||
} catch {
|
||||
return null; // no config available — skip
|
||||
}
|
||||
}
|
||||
|
||||
const uri = (env["COUCHDB_URI"] ?? env["hostname"] ?? "").replace(/\/$/, "");
|
||||
const user = env["COUCHDB_USER"] ?? env["username"] ?? "";
|
||||
const password = env["COUCHDB_PASSWORD"] ?? env["password"] ?? "";
|
||||
const baseDbname = env["COUCHDB_DBNAME"] ?? env["dbname"] ?? "livesync-test";
|
||||
|
||||
if (!uri || !user || !password) return null;
|
||||
return { uri, user, password, baseDbname };
|
||||
}
|
||||
|
||||
const config = await resolveConfig();
|
||||
const START_DOCKER = Deno.env.get("LIVESYNC_START_DOCKER") !== "0";
|
||||
const KEEP_DOCKER = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
|
||||
const SYNC_RETRY = Number(Deno.env.get("LIVESYNC_SYNC_RETRY") ?? "8");
|
||||
|
||||
// Provide a sane default for flaky remote connectivity in Docker-on-WSL
|
||||
// environments. Users can override explicitly if needed.
|
||||
if (!Deno.env.has("LIVESYNC_CLI_RETRY")) {
|
||||
Deno.env.set("LIVESYNC_CLI_RETRY", "2");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test suite
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
Deno.test(
|
||||
{
|
||||
name: "sync two local databases: sync + conflict detection + resolution",
|
||||
ignore: config === null,
|
||||
},
|
||||
async (t) => {
|
||||
if (!config) return; // narrowing for TypeScript
|
||||
|
||||
const suffix = `${Date.now()}-${Math.floor(Math.random() * 65535)}`;
|
||||
const dbname = `${config.baseDbname}-${suffix}`;
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-two-db-test");
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Docker lifecycle
|
||||
// ------------------------------------------------------------------
|
||||
if (START_DOCKER) {
|
||||
await startCouchdb(config.uri, config.user, config.password, dbname);
|
||||
}
|
||||
|
||||
try {
|
||||
await runSuite(t, workDir, config, dbname);
|
||||
} finally {
|
||||
if (START_DOCKER && !KEEP_DOCKER) {
|
||||
await stopCouchdb().catch(() => {});
|
||||
}
|
||||
if (START_DOCKER && KEEP_DOCKER) {
|
||||
console.log("[INFO] LIVESYNC_DEBUG_KEEP_DOCKER=1, keeping couchdb-test container");
|
||||
}
|
||||
console.log(`[INFO] test database '${dbname}' is preserved for debugging.`);
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Suite implementation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function runSuite(
|
||||
t: Deno.TestContext,
|
||||
workDir: TempDir,
|
||||
config: { uri: string; user: string; password: string },
|
||||
dbname: string
|
||||
): Promise<void> {
|
||||
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
const runWithRetry = async <T>(label: string, fn: () => Promise<T>, retries = SYNC_RETRY): Promise<T> => {
|
||||
let lastErr: unknown;
|
||||
for (let i = 0; i <= retries; i++) {
|
||||
try {
|
||||
return await fn();
|
||||
} catch (err) {
|
||||
lastErr = err;
|
||||
if (i === retries) break;
|
||||
const delayMs = 500 * (i + 1);
|
||||
console.warn(`[WARN] ${label} failed, retrying (${i + 1}/${retries}) in ${delayMs}ms`);
|
||||
await sleep(delayMs);
|
||||
}
|
||||
}
|
||||
throw lastErr;
|
||||
};
|
||||
|
||||
const vaultA = workDir.join("vault-a");
|
||||
const vaultB = workDir.join("vault-b");
|
||||
const settingsA = workDir.join("a-settings.json");
|
||||
const settingsB = workDir.join("b-settings.json");
|
||||
await Deno.mkdir(vaultA, { recursive: true });
|
||||
await Deno.mkdir(vaultB, { recursive: true });
|
||||
|
||||
await initSettingsFile(settingsA);
|
||||
await initSettingsFile(settingsB);
|
||||
|
||||
const applySettings = async (f: string) =>
|
||||
applyCouchdbSettings(f, config.uri, config.user, config.password, dbname, /* liveSync */ true);
|
||||
await applySettings(settingsA);
|
||||
await applySettings(settingsB);
|
||||
|
||||
const runA = (...args: string[]) => runCliOrFail(vaultA, "--settings", settingsA, ...args);
|
||||
const runB = (...args: string[]) => runCliOrFail(vaultB, "--settings", settingsB, ...args);
|
||||
|
||||
const syncA = () => runWithRetry("syncA", () => runA("sync"));
|
||||
const syncB = () => runWithRetry("syncB", () => runB("sync"));
|
||||
const catA = (path: string) => runA("cat", path);
|
||||
const catB = (path: string) => runB("cat", path);
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Case 1: A creates file, B reads after sync
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("case 1: A creates file -> B can read after sync", async () => {
|
||||
const srcA = workDir.join("from-a-src.txt");
|
||||
await Deno.writeTextFile(srcA, "from-a\n");
|
||||
await runA("push", srcA, "shared/from-a.txt");
|
||||
await syncA();
|
||||
await syncB();
|
||||
const value = (await catB("shared/from-a.txt")).replace(/\r\n/g, "\n").trimEnd();
|
||||
assertEquals(value, "from-a", "B could not read file created on A");
|
||||
console.log("[PASS] case 1 passed");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Case 2: B creates file, A reads after sync
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("case 2: B creates file -> A can read after sync", async () => {
|
||||
const srcB = workDir.join("from-b-src.txt");
|
||||
await Deno.writeTextFile(srcB, "from-b\n");
|
||||
await runB("push", srcB, "shared/from-b.txt");
|
||||
await syncB();
|
||||
await syncA();
|
||||
const value = (await catA("shared/from-b.txt")).replace(/\r\n/g, "\n").trimEnd();
|
||||
assertEquals(value, "from-b", "A could not read file created on B");
|
||||
console.log("[PASS] case 2 passed");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Case 3: concurrent edits create a conflict
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("case 3: concurrent edits create conflict", async () => {
|
||||
const baseSrc = workDir.join("base-src.txt");
|
||||
await Deno.writeTextFile(baseSrc, "base\n");
|
||||
await runA("push", baseSrc, "shared/conflicted.txt");
|
||||
await syncA();
|
||||
await syncB();
|
||||
|
||||
const aEdit = workDir.join("edit-a.txt");
|
||||
const bEdit = workDir.join("edit-b.txt");
|
||||
await Deno.writeTextFile(aEdit, "edit-from-a\n");
|
||||
await Deno.writeTextFile(bEdit, "edit-from-b\n");
|
||||
await runA("push", aEdit, "shared/conflicted.txt");
|
||||
await runB("push", bEdit, "shared/conflicted.txt");
|
||||
|
||||
const infoFileA = workDir.join("info-a.json");
|
||||
const infoFileB = workDir.join("info-b.json");
|
||||
|
||||
let conflictDetected = false;
|
||||
for (const side of ["a", "b"] as const) {
|
||||
if (side === "a") await syncA();
|
||||
else await syncB();
|
||||
await Deno.writeTextFile(infoFileA, await runA("info", "shared/conflicted.txt"));
|
||||
await Deno.writeTextFile(infoFileB, await runB("info", "shared/conflicted.txt"));
|
||||
const da = JSON.parse(await Deno.readTextFile(infoFileA)) as Record<string, unknown>;
|
||||
const db = JSON.parse(await Deno.readTextFile(infoFileB)) as Record<string, unknown>;
|
||||
if (!jsonFieldIsNa(da, "conflicts") || !jsonFieldIsNa(db, "conflicts")) {
|
||||
conflictDetected = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
assert(conflictDetected, "expected conflict after concurrent edits, but both sides show N/A");
|
||||
console.log("[PASS] case 3 conflict detected");
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Case 4: resolve on A, verify B has no conflict after sync
|
||||
// ------------------------------------------------------------------
|
||||
await t.step("case 4: resolve on A propagates to B", async () => {
|
||||
const infoFileA = workDir.join("info-a-resolve.json");
|
||||
const infoFileB = workDir.join("info-b-resolve.json");
|
||||
|
||||
// Ensure A sees the conflict
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const raw = await runA("info", "shared/conflicted.txt");
|
||||
await Deno.writeTextFile(infoFileA, raw);
|
||||
const da = JSON.parse(raw) as Record<string, unknown>;
|
||||
if (!jsonFieldIsNa(da, "conflicts")) break;
|
||||
await syncB();
|
||||
await syncA();
|
||||
}
|
||||
|
||||
const rawA = await runA("info", "shared/conflicted.txt");
|
||||
await Deno.writeTextFile(infoFileA, rawA);
|
||||
const dataA = JSON.parse(rawA) as Record<string, unknown>;
|
||||
assert(!jsonFieldIsNa(dataA, "conflicts"), "A does not see conflict, cannot resolve from A only");
|
||||
|
||||
const keepRev = dataA["revision"] as string;
|
||||
assert(keepRev?.length > 0, "could not read revision from A info output");
|
||||
|
||||
await runA("resolve", "shared/conflicted.txt", keepRev);
|
||||
|
||||
let resolved = false;
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await syncA();
|
||||
await syncB();
|
||||
const rawA2 = await runA("info", "shared/conflicted.txt");
|
||||
const rawB2 = await runB("info", "shared/conflicted.txt");
|
||||
await Deno.writeTextFile(infoFileA, rawA2);
|
||||
await Deno.writeTextFile(infoFileB, rawB2);
|
||||
const da2 = JSON.parse(rawA2) as Record<string, unknown>;
|
||||
const db2 = JSON.parse(rawB2) as Record<string, unknown>;
|
||||
if (jsonFieldIsNa(da2, "conflicts") && jsonFieldIsNa(db2, "conflicts")) {
|
||||
resolved = true;
|
||||
break;
|
||||
}
|
||||
// If A still sees a conflict, resolve it again
|
||||
if (!jsonFieldIsNa(da2, "conflicts")) {
|
||||
const rev2 = da2["revision"] as string;
|
||||
if (rev2) await runA("resolve", "shared/conflicted.txt", rev2).catch(() => {});
|
||||
}
|
||||
}
|
||||
assert(resolved, "conflicts should be resolved on both A and B");
|
||||
|
||||
const contentA = (await catA("shared/conflicted.txt")).replace(/\r\n/g, "\n");
|
||||
const contentB = (await catB("shared/conflicted.txt")).replace(/\r\n/g, "\n");
|
||||
assertEquals(contentA, contentB, "resolved content mismatch between A and B");
|
||||
console.log("[PASS] case 4 passed");
|
||||
console.log("[PASS] all sync/resolve scenarios passed");
|
||||
});
|
||||
}
|
||||
292
src/apps/cli/testdeno/test_dev_deno.md
Normal file
292
src/apps/cli/testdeno/test_dev_deno.md
Normal file
@@ -0,0 +1,292 @@
|
||||
# CLI Deno Test Development Notes
|
||||
|
||||
This document provides an overview of the Deno-based compatibility tests under `src/apps/cli/testdeno/`.
|
||||
The existing bash tests under `src/apps/cli/test/` are preserved, while a Windows-friendly suite is maintained in parallel.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
- Keep existing bash tests intact.
|
||||
- Provide direct execution from Windows PowerShell.
|
||||
- Establish a TypeScript (Deno) foundation for core end-to-end and integration scenarios.
|
||||
|
||||
---
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
src/apps/cli/testdeno/
|
||||
deno.json
|
||||
CONTRIBUTING_TESTS.md
|
||||
helpers/
|
||||
backgroundCli.ts
|
||||
cli.ts
|
||||
docker.ts
|
||||
env.ts
|
||||
p2p.ts
|
||||
settings.ts
|
||||
temp.ts
|
||||
test-e2e-two-vaults-couchdb.ts
|
||||
test-push-pull.ts
|
||||
test-p2p-host.ts
|
||||
test-p2p-peers-local-relay.ts
|
||||
test-p2p-sync.ts
|
||||
test-p2p-three-nodes-conflict.ts
|
||||
test-p2p-upload-download-repro.ts
|
||||
test-e2e-two-vaults-matrix.ts
|
||||
test-setup-put-cat.ts
|
||||
test-mirror.ts
|
||||
test-sync-two-local-databases.ts
|
||||
test-sync-locked-remote.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key files
|
||||
|
||||
### `deno.json`
|
||||
|
||||
- Defines Deno tasks.
|
||||
- Defines import maps for `@std/assert` and `@std/path`.
|
||||
|
||||
Main tasks:
|
||||
|
||||
- `deno task test`
|
||||
- `deno task test:local`
|
||||
- `deno task test:push-pull`
|
||||
- `deno task test:setup-put-cat`
|
||||
- `deno task test:mirror`
|
||||
- `deno task test:sync-two-local`
|
||||
- `deno task test:sync-locked-remote`
|
||||
- `deno task test:p2p-host`
|
||||
- `deno task test:p2p-peers`
|
||||
- `deno task test:p2p-sync`
|
||||
- `deno task test:p2p-three-nodes`
|
||||
- `deno task test:p2p-upload-download`
|
||||
- `deno task test:e2e-couchdb`
|
||||
- `deno task test:e2e-matrix`
|
||||
|
||||
### `helpers/cli.ts`
|
||||
|
||||
- CLI execution wrappers.
|
||||
- `runCli`, `runCliOrFail`, `runCliWithInput`.
|
||||
- Output normalisation via `sanitiseCatStdout`.
|
||||
- Comparison utilities, including `assertFilesEqual`.
|
||||
|
||||
This file corresponds to `run_cli` and common assertions in `test-helpers.sh`.
|
||||
|
||||
### `helpers/settings.ts`
|
||||
|
||||
- Executes `init-settings --force`.
|
||||
- Marks `isConfigured = true`.
|
||||
- Applies CouchDB and P2P settings.
|
||||
- Applies remote synchronisation settings and P2P test tweaks.
|
||||
|
||||
This file corresponds to settings helpers in `test-helpers.sh`.
|
||||
|
||||
### `helpers/docker.ts`
|
||||
|
||||
- Starts, stops, and initialises CouchDB directly from Deno.
|
||||
- Configures CouchDB via `fetch + retry`.
|
||||
- Starts and stops the P2P relay through the same Docker runner.
|
||||
|
||||
Both CouchDB and P2P relay flows are bash-independent.
|
||||
|
||||
### `helpers/backgroundCli.ts`
|
||||
|
||||
- Starts long-running commands such as `p2p-host` in the background.
|
||||
- Waits for readiness logs and handles termination.
|
||||
|
||||
### `helpers/p2p.ts`
|
||||
|
||||
- Determines whether a local relay should be started.
|
||||
- Parses `p2p-peers` output.
|
||||
- Discovers peer IDs with a fallback based on advertisement logs.
|
||||
|
||||
### `helpers/env.ts`
|
||||
|
||||
- Loads `.test.env`.
|
||||
- Supports `KEY=value`, single-quoted values, and double-quoted values.
|
||||
|
||||
### `helpers/temp.ts`
|
||||
|
||||
- Provides `TempDir`.
|
||||
- Uses `await using` to auto-clean temporary directories.
|
||||
|
||||
---
|
||||
|
||||
## Implemented tests
|
||||
|
||||
### `test-push-pull.ts`
|
||||
|
||||
- Verifies push and pull round trips.
|
||||
- Uses environment variables or `.test.env` for CouchDB values.
|
||||
|
||||
### `test-setup-put-cat.ts`
|
||||
|
||||
- Verifies `setup` with full setup URI generation via `encodeSettingsToSetupURI`.
|
||||
- Verifies `push`, `cat`, `ls`, `info`, `rm`, `resolve`, `cat-rev`, and `pull-rev`.
|
||||
- Does not require an external remote.
|
||||
|
||||
### `test-mirror.ts`
|
||||
|
||||
- Verifies six core mirror scenarios.
|
||||
- Does not require an external remote.
|
||||
|
||||
### `test-sync-two-local-databases.ts`
|
||||
|
||||
- Verifies sync between two vaults and CouchDB.
|
||||
- Verifies conflict detection and resolve propagation.
|
||||
- Starts Docker CouchDB by default when `LIVESYNC_START_DOCKER != 0`.
|
||||
|
||||
### `test-sync-locked-remote.ts`
|
||||
|
||||
- Updates the CouchDB milestone `locked` flag.
|
||||
- Verifies sync success when unlocked.
|
||||
- Verifies actionable CLI error when locked.
|
||||
|
||||
### `test-p2p-host.ts`
|
||||
|
||||
- Verifies that `p2p-host` starts and emits readiness output.
|
||||
|
||||
### `test-p2p-peers-local-relay.ts`
|
||||
|
||||
- Verifies peer discovery through a local relay.
|
||||
|
||||
### `test-p2p-sync.ts`
|
||||
|
||||
- Verifies that `p2p-sync` completes after peer discovery.
|
||||
|
||||
### `test-p2p-three-nodes-conflict.ts`
|
||||
|
||||
- Uses one host and two clients.
|
||||
- Verifies conflict creation, detection via `info`, and resolution via `resolve`.
|
||||
|
||||
### `test-p2p-upload-download-repro.ts`
|
||||
|
||||
- Uses host, upload, and download nodes.
|
||||
- Verifies transfer of text files and binary files, including larger files.
|
||||
|
||||
### `test-e2e-two-vaults-couchdb.ts`
|
||||
|
||||
- Verifies two-vault end-to-end scenarios on CouchDB.
|
||||
- Runs both encryption-off and encryption-on cases.
|
||||
- Includes conflict marker checks in `ls` and resolve propagation checks.
|
||||
|
||||
### `test-e2e-two-vaults-matrix.ts`
|
||||
|
||||
- Verifies the matrix equivalent of the bash script.
|
||||
- Runs four combinations:
|
||||
- `COUCHDB-enc0`
|
||||
- `COUCHDB-enc1`
|
||||
- `MINIO-enc0`
|
||||
- `MINIO-enc1`
|
||||
|
||||
---
|
||||
|
||||
## Running tests (PowerShell)
|
||||
|
||||
From `src/apps/cli/testdeno`:
|
||||
|
||||
```powershell
|
||||
cd src/apps/cli/testdeno
|
||||
|
||||
# Local-only set
|
||||
deno task test:local
|
||||
|
||||
# Individual tests
|
||||
deno task test:setup-put-cat
|
||||
deno task test:mirror
|
||||
deno task test:push-pull
|
||||
deno task test:sync-locked-remote
|
||||
|
||||
# CouchDB-based tests
|
||||
deno task test:sync-two-local
|
||||
deno task test:e2e-couchdb
|
||||
|
||||
# P2P-based tests
|
||||
deno task test:p2p-host
|
||||
deno task test:p2p-peers
|
||||
deno task test:p2p-sync
|
||||
deno task test:p2p-three-nodes
|
||||
deno task test:p2p-upload-download
|
||||
deno task test:e2e-matrix
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment variables
|
||||
|
||||
### CouchDB
|
||||
|
||||
- `COUCHDB_URI`
|
||||
- `COUCHDB_USER`
|
||||
- `COUCHDB_PASSWORD`
|
||||
- `COUCHDB_DBNAME`
|
||||
|
||||
Equivalent keys in `src/apps/cli/.test.env`:
|
||||
|
||||
- `hostname`
|
||||
- `username`
|
||||
- `password`
|
||||
- `dbname`
|
||||
|
||||
### Behaviour switches
|
||||
|
||||
- `LIVESYNC_START_DOCKER=0`: use existing CouchDB.
|
||||
- `REMOTE_PATH`: override target path for selected tests.
|
||||
- `LIVESYNC_TEST_TEE=1`: stream CLI stdout and stderr during execution.
|
||||
- `LIVESYNC_DOCKER_TEE=1`: stream Docker stdout and stderr.
|
||||
- `LIVESYNC_CLI_RETRY=<n>`: retry transient network failures.
|
||||
- `LIVESYNC_DEBUG_KEEP_DOCKER=1`: keep `couchdb-test` after test completion.
|
||||
|
||||
### Docker command selection
|
||||
|
||||
`helpers/docker.ts` supports command selection via environment variables.
|
||||
|
||||
- `LIVESYNC_DOCKER_MODE=auto` (default)
|
||||
- Windows: tries `wsl docker` first, then `docker`.
|
||||
- Non-Windows: tries `docker` first, then `wsl docker`.
|
||||
- `LIVESYNC_DOCKER_MODE=native`: always uses `docker`.
|
||||
- `LIVESYNC_DOCKER_MODE=wsl`: always uses `wsl docker`.
|
||||
- `LIVESYNC_DOCKER_COMMAND="..."`: custom command, for example `wsl docker`.
|
||||
|
||||
`LIVESYNC_DOCKER_COMMAND` has priority over `LIVESYNC_DOCKER_MODE`.
|
||||
|
||||
PowerShell examples:
|
||||
|
||||
```powershell
|
||||
# Use Docker in WSL explicitly
|
||||
$env:LIVESYNC_DOCKER_MODE = "wsl"
|
||||
deno task test:sync-two-local
|
||||
|
||||
# Full custom command
|
||||
$env:LIVESYNC_DOCKER_COMMAND = "wsl docker"
|
||||
deno task test:sync-two-local
|
||||
```
|
||||
|
||||
### P2P
|
||||
|
||||
- `RELAY`
|
||||
- `ROOM_ID`
|
||||
- `PASSPHRASE`
|
||||
- `APP_ID`
|
||||
- `PEERS_TIMEOUT`
|
||||
- `SYNC_TIMEOUT`
|
||||
- `USE_INTERNAL_RELAY=0|1`
|
||||
- `TIMEOUT_SECONDS`
|
||||
|
||||
---
|
||||
|
||||
## Current limitations
|
||||
|
||||
- MinIO startup and matrix coverage are ported. Current limits are elsewhere, not setup URI generation.
|
||||
|
||||
---
|
||||
|
||||
## Maintenance policy
|
||||
|
||||
- Existing bash tests remain available.
|
||||
- Deno tests are expanded in parallel for cross-platform usage.
|
||||
- New scenarios should be added through reusable helpers in `helpers/`.
|
||||
Reference in New Issue
Block a user