mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-11-03 06:48:18 +00:00
Preserve CLI database continuity after reconstruction
This commit is contained in:
1 parent
0e458e5e20
commit
d50ee9a83e
21 files changed
+1577
-52
No files matched your search
@@ -38,6 +38,7 @@ Use these references for the area being changed, together with its relevant ADR.
|
|||||||
| Encryption and sharing | [Metadata encryption](design_docs/internal_metadata_encryption.md) and [time-bound Setup URIs](design_docs/time_bound_setup_uri.md) |
|
| Encryption and sharing | [Metadata encryption](design_docs/internal_metadata_encryption.md) and [time-bound Setup URIs](design_docs/time_bound_setup_uri.md) |
|
||||||
| P2P connection preparation | [TURN connection settings](design_docs/renewable_turn_credentials.md) |
|
| P2P connection preparation | [TURN connection settings](design_docs/renewable_turn_credentials.md) |
|
||||||
| Journal storage internals | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
|
| Journal storage internals | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
|
||||||
|
| CLI database selection and interrupted reconstruction | [CLI database continuity](design_docs/cli_database_continuity.md) |
|
||||||
|
|
||||||
The installed `@vrtmrz/livesync-commonlib` package is authoritative for shared runtime contracts. Use the exact dependency selected by [package.json](../package.json) and its lockfile when checking behaviour. The [Commonlib documentation](https://github.com/vrtmrz/livesync-commonlib/tree/main/docs) is maintained in that repository; this index does not replace its package contracts.
|
The installed `@vrtmrz/livesync-commonlib` package is authoritative for shared runtime contracts. Use the exact dependency selected by [package.json](../package.json) and its lockfile when checking behaviour. The [Commonlib documentation](https://github.com/vrtmrz/livesync-commonlib/tree/main/docs) is maintained in that repository; this index does not replace its package contracts.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
---
|
||||||
|
date: 2026-11-01
|
||||||
|
commonlib-version: "0.2.2"
|
||||||
|
self-hosted-livesync-version: "1.0.43"
|
||||||
|
status: accepted
|
||||||
|
---
|
||||||
|
|
||||||
|
# CLI database continuity
|
||||||
|
|
||||||
|
Publication note: the correction for #1304 is implemented and locally validated,
|
||||||
|
but has not been released. The reviewed baseline is Self-hosted LiveSync `0e458e5e` with
|
||||||
|
Commonlib `0.2.2`. The [CLI guide](../../src/apps/cli/README.md) describes the
|
||||||
|
command interface; the [test procedure](../../test/README.md#cli-fetch-database-continuity)
|
||||||
|
describes the Object Storage reproduction.
|
||||||
|
|
||||||
|
## Required behaviour
|
||||||
|
|
||||||
|
A successful `fetch` or `rebuild` must leave subsequent CLI commands using the
|
||||||
|
reconstructed local database. The Commonlib Rebuilder selects the host app ID
|
||||||
|
as the database suffix. The CLI saves that selection on success even without
|
||||||
|
`--write-settings`; the option continues to govern other lasting settings
|
||||||
|
changes. Temporary suspension values remain runtime state.
|
||||||
|
|
||||||
|
Before reconstruction can mutate a database, the CLI records the operation,
|
||||||
|
database directory, Vault path, and settings path. An interrupted operation
|
||||||
|
blocks ordinary mutation and synchronisation. Retrying the same reconstruction
|
||||||
|
with the same paths and `--force` is an explicit recovery action. It can repeat
|
||||||
|
the destructive operation, so users must preserve unique edits and establish
|
||||||
|
the authoritative copy before retrying.
|
||||||
|
|
||||||
|
Completion records the settings update before applying it. A later invocation
|
||||||
|
can finish a pending settings commit; if the settings file has independently
|
||||||
|
changed, it must stop for review rather than overwrite those changes. The
|
||||||
|
pending record is removed only after completion and settings persistence.
|
||||||
|
|
||||||
|
## Ownership of persistent state
|
||||||
|
|
||||||
|
The selected local database name identifies the dataset. Its runtime state is
|
||||||
|
further scoped by the canonical physical Vault path. This includes Journal
|
||||||
|
checkpoints, scanner state, replication-result queues, file-reflection
|
||||||
|
provenance, and storage-event snapshots. Paths identify local ownership; they
|
||||||
|
are not sent to the remote as a new document identity.
|
||||||
|
|
||||||
|
A local database reset invalidates every Vault scope associated with that
|
||||||
|
database, because each scope refers to the same discarded revision history.
|
||||||
|
Other local database suffixes retain their state. PouchDB database names and
|
||||||
|
Commonlib's reconstruction contract remain unchanged.
|
||||||
|
|
||||||
|
KV writes replace the file atomically. Invalid persisted data must fail
|
||||||
|
initialisation instead of becoming an empty checkpoint history. Closing a KV
|
||||||
|
handle must not overwrite the file from an old in-memory snapshot.
|
||||||
|
|
||||||
|
## Legacy state
|
||||||
|
|
||||||
|
The former shared `keyvalue-db.json` and Vault-local `.livesync-snapshot.json`
|
||||||
|
do not record their complete owner. A single surviving database directory does
|
||||||
|
not prove which physical Vault produced those records. Automatically importing
|
||||||
|
them can replay events against the wrong database; dropping them and immediately
|
||||||
|
scanning can misinterpret file absence before remote history is recovered.
|
||||||
|
|
||||||
|
Preserve unowned legacy state. When it contains work, stop ordinary mutating
|
||||||
|
commands and require explicit reconstruction from a verified source. Inspection
|
||||||
|
remains available. This deliberately requires more user action than automatic
|
||||||
|
migration, and must not silently delete the old records.
|
||||||
|
|
||||||
|
Check the selected scope's completion flag before reading legacy files. For an
|
||||||
|
unreviewed scope, inspect both legacy files after settings and scoped KV loading,
|
||||||
|
but before readiness starts scanning or event replay. Distinguish non-empty
|
||||||
|
unowned state, invalid JSON or shape, and read failures. Report affected paths
|
||||||
|
once per invocation on stderr without logging file contents. Read commands may
|
||||||
|
continue inspection; ordinary writes stop, and explicit reconstruction retains
|
||||||
|
the files. Once reconstruction has succeeded for that scope, legacy parse or
|
||||||
|
read failures must not block it again. The active scoped KV still fails
|
||||||
|
initialisation on corruption.
|
||||||
|
|
||||||
|
## Evidence and limits
|
||||||
|
|
||||||
|
The original Object Storage scenario uses encryption, path obfuscation, and a
|
||||||
|
second-generation file revision. Without settings writeback, A reopens an empty
|
||||||
|
database while sharing the fetched database's Journal history, creates an
|
||||||
|
independent root, and receives B's same-content resolution tombstone without
|
||||||
|
the original surviving revision. The original assertion fails because A's
|
||||||
|
physical file disappears. The explicit `--write-settings` control preserves
|
||||||
|
the file and the original revision. The remote and B retain the content in this
|
||||||
|
reproduction; it does not demonstrate irreversible loss.
|
||||||
|
|
||||||
|
Validation of the correction must check bytes and revision ancestry after
|
||||||
|
restart, interrupted reconstruction admission, pending settings completion,
|
||||||
|
legacy-state preservation, and reset isolation. A successful exit or an empty
|
||||||
|
queue alone does not establish preservation.
|
||||||
|
|
||||||
|
Local validation passed the encrypted Object Storage CLI regression and the
|
||||||
|
explicit-settings control on Node.js 26. Unit tests cover failed-operation
|
||||||
|
admission, saved running/completed markers, settings-edit preservation, scoped
|
||||||
|
stores, reset isolation, corrupt KV files, and legacy-state refusal. The marker
|
||||||
|
tests inject interruption states; they do not emulate a machine power failure.
|
||||||
|
|
||||||
|
The shared device-value JSON file retains its existing concurrency limitations.
|
||||||
|
This change does not establish concurrent CLI writers to one database directory
|
||||||
|
as a supported workflow. Native Obsidian and other transport behaviour require
|
||||||
|
their own runtime evidence.
|
||||||
@@ -155,10 +155,11 @@ preflight failure. If a required setup decision cannot be made in a headless
|
|||||||
CLI invocation, the command must fail with a clear remediation message rather
|
CLI invocation, the command must fail with a clear remediation message rather
|
||||||
than report success after cancellation.
|
than report success after cancellation.
|
||||||
|
|
||||||
`--write-settings` retains the Rebuilder's selected local database name and
|
The unreleased CLI correction for #1304 always retains the Rebuilder's selected
|
||||||
lasting settings changes for subsequent invocations. Without this option, the
|
local database suffix after successful `fetch` or `rebuild`. `--write-settings`
|
||||||
existing CLI policy leaves the settings file unchanged, so a later command can
|
still controls other lasting settings changes. Earlier versions required that
|
||||||
reopen the previously selected database.
|
option to retain the database selection; omitting it could reopen the previous
|
||||||
|
database on the next invocation. The examples retain the option for compatibility.
|
||||||
|
|
||||||
## State which follows the selected identity
|
## State which follows the selected identity
|
||||||
|
|
||||||
|
|||||||
Generated
+1
@@ -11765,6 +11765,7 @@
|
|||||||
"name": "self-hosted-livesync-cli",
|
"name": "self-hosted-livesync-cli",
|
||||||
"version": "1.0.43-cli",
|
"version": "1.0.43-cli",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@aws-crypto/sha256-js": "5.2.0",
|
||||||
"chokidar": "^4.0.0",
|
"chokidar": "^4.0.0",
|
||||||
"minimatch": "^10.2.5",
|
"minimatch": "^10.2.5",
|
||||||
"octagonal-wheels": "^0.1.55",
|
"octagonal-wheels": "^0.1.55",
|
||||||
|
|||||||
+36
-1
@@ -70,6 +70,7 @@ livesync-cli [database-path] [command] [args...]
|
|||||||
- `--force` / `-f`: Required by `rebuild` and `fetch` because they replace a database. It also retains its existing `init-settings` overwrite behaviour.
|
- `--force` / `-f`: Required by `rebuild` and `fetch` because they replace a database. It also retains its existing `init-settings` overwrite behaviour.
|
||||||
- `--write-settings`: Write setting migrations and other lasting changes after the command succeeds.
|
- `--write-settings`: Write setting migrations and other lasting changes after the command succeeds.
|
||||||
- `init-settings` writes its target file. `setup`, `remote-add`, `remote-rm`, `remote-set`, and `remote-activate` write their settings changes without this option.
|
- `init-settings` writes its target file. `setup`, `remote-add`, `remote-rm`, `remote-set`, and `remote-activate` write their settings changes without this option.
|
||||||
|
- `fetch` and `rebuild` always retain the selected local database suffix after success. Other lasting changes from these commands still require this option.
|
||||||
- All remaining commands leave the settings file unchanged by default.
|
- All remaining commands leave the settings file unchanged by default.
|
||||||
- Temporary values used to suspend synchronisation or select a remote for one command are never written.
|
- Temporary values used to suspend synchronisation or select a remote for one command are never written.
|
||||||
- `--compat-remote-admin-exit-zero`: Preserve the former zero exit code when `mark-resolved`, `lock-remote`, or `unlock-remote` returns a provider verification failure.
|
- `--compat-remote-admin-exit-zero`: Preserve the former zero exit code when `mark-resolved`, `lock-remote`, or `unlock-remote` returns a provider verification failure.
|
||||||
@@ -84,8 +85,9 @@ livesync-cli [database-path] [command] [args...]
|
|||||||
- If `vault-path` is omitted, it defaults to `database-path` (compatibility mode).
|
- If `vault-path` is omitted, it defaults to `database-path` (compatibility mode).
|
||||||
- Use this command to keep your local `.md` files in sync with the database.
|
- Use this command to keep your local `.md` files in sync with the database.
|
||||||
- `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`.
|
- `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`.
|
||||||
- This command may select a new local database suffix. Add `--write-settings` if later CLI commands should use the resulting database selection; otherwise the existing settings file remains unchanged.
|
- The selected local database suffix is saved automatically, so later commands use the reconstructed database.
|
||||||
- `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`.
|
- `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`.
|
||||||
|
- The selected local database suffix is saved automatically, including when `--write-settings` is omitted.
|
||||||
- `ls [prefix]`: List files currently stored in the local database.
|
- `ls [prefix]`: List files currently stored in the local database.
|
||||||
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
|
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
|
||||||
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
|
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
|
||||||
@@ -105,6 +107,39 @@ livesync-cli [database-path] [command] [args...]
|
|||||||
|
|
||||||
Remote-administration commands verify the resulting milestone state through the selected provider. The existing `[Verification]` lines remain suitable for scripts which inspect command output, while the default exit code now reflects whether that verification succeeded.
|
Remote-administration commands verify the resulting milestone state through the selected provider. The existing `[Verification]` lines remain suitable for scripts which inspect command output, while the default exit code now reflects whether that verification succeeded.
|
||||||
|
|
||||||
|
### Interrupted reconstruction and older runtime state
|
||||||
|
|
||||||
|
An interrupted `fetch` or `rebuild` leaves a record in
|
||||||
|
`<database-path>/.livesync/reconstruction.json`. Ordinary synchronisation and
|
||||||
|
mutating commands stop while reconstruction is incomplete. Preserve the Vault,
|
||||||
|
database directory, and settings, check the reported failure, then retry the
|
||||||
|
same command with `--force` and the same database, Vault, and settings paths.
|
||||||
|
`rebuild` replaces remote state as well: confirm that the Vault remains the
|
||||||
|
authoritative copy before retrying it. Do not remove the record merely to
|
||||||
|
restart `daemon`.
|
||||||
|
|
||||||
|
If reconstruction finished but its settings update was interrupted, the next
|
||||||
|
invocation completes that update before starting the core. If the settings
|
||||||
|
file was independently edited, the CLI stops for review. The pending record
|
||||||
|
can contain settings and credentials; protect it like the settings file.
|
||||||
|
|
||||||
|
Older CLI versions stored runtime state without recording its database and
|
||||||
|
Vault owner. If unowned state is present, the CLI preserves it and stops
|
||||||
|
ordinary mutating commands. Read commands remain available for inspection.
|
||||||
|
Back up unique edits and establish the authoritative copy before explicitly
|
||||||
|
using `fetch --force` or `rebuild --force`. The old runtime files remain
|
||||||
|
available for diagnosis; they are not automatically imported into a new scope.
|
||||||
|
|
||||||
|
This check runs before scanning, event replay, and synchronisation. Its message
|
||||||
|
identifies each affected file and distinguishes an unknown owner from a file
|
||||||
|
which cannot be read or parsed. Read commands display the warning and continue
|
||||||
|
inspection. After successful reconstruction for the selected database and Vault,
|
||||||
|
retained legacy files are no longer inspected, even if they are malformed.
|
||||||
|
Corruption in the currently selected runtime KV remains an initialisation error.
|
||||||
|
|
||||||
|
Use one CLI process at a time for each database directory. Separate runtime
|
||||||
|
scopes do not make concurrent writers to shared device settings supported.
|
||||||
|
|
||||||
### Examples
|
### Examples
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -78,12 +78,20 @@ describe("CLI database preparation", () => {
|
|||||||
await rm(directory, { recursive: true, force: true });
|
await rm(directory, { recursive: true, force: true });
|
||||||
});
|
});
|
||||||
|
|
||||||
async function start(command: CLICommand, runner: CliCommandRunner, exitCode = 1, force = false) {
|
async function start(
|
||||||
|
command: CLICommand,
|
||||||
|
runner: CliCommandRunner,
|
||||||
|
exitCode = 1,
|
||||||
|
force = false,
|
||||||
|
writeSettings = false
|
||||||
|
) {
|
||||||
|
process.exitCode = undefined;
|
||||||
process.argv = [
|
process.argv = [
|
||||||
"node",
|
"node",
|
||||||
"livesync-cli",
|
"livesync-cli",
|
||||||
directory,
|
directory,
|
||||||
...(force ? ["--force"] : []),
|
...(force ? ["--force"] : []),
|
||||||
|
...(writeSettings ? ["--write-settings"] : []),
|
||||||
"--vault",
|
"--vault",
|
||||||
vaultPath,
|
vaultPath,
|
||||||
"--settings",
|
"--settings",
|
||||||
@@ -386,6 +394,194 @@ describe("CLI database preparation", () => {
|
|||||||
expect(await readFile(join(vaultPath, "notes/local.md"), "utf-8")).toBe("local content");
|
expect(await readFile(join(vaultPath, "notes/local.md"), "utf-8")).toBe("local content");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it.each(["fetch", "rebuild"] as const)(
|
||||||
|
"persists the selected database after %s without other command settings",
|
||||||
|
async (command) => {
|
||||||
|
const before = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
||||||
|
await core.services.setting.applyPartial({
|
||||||
|
additionalSuffixOfDatabaseName: "headless-app",
|
||||||
|
customChunkSize: before.customChunkSize + 10,
|
||||||
|
});
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
|
||||||
|
await start(command, runner, 0, true);
|
||||||
|
|
||||||
|
const after = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
expect(after.additionalSuffixOfDatabaseName).toBe("headless-app");
|
||||||
|
expect(after.customChunkSize).toBe(before.customChunkSize);
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it.each(["suffix", "vault"] as const)("isolates persistent state by selected %s", async (selection) => {
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
await core.services.keyValueDB.kvDB.set("scope-probe", "original");
|
||||||
|
await core.services.config.setValue("scope-probe", "original");
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
if (selection === "suffix") {
|
||||||
|
const settings = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
settings.additionalSuffixOfDatabaseName = "other";
|
||||||
|
await writeFile(settingsPath, JSON.stringify(settings));
|
||||||
|
} else {
|
||||||
|
vaultPath = join(directory, "other-vault");
|
||||||
|
await mkdir(vaultPath);
|
||||||
|
}
|
||||||
|
const observed: unknown[] = [];
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
observed.push(await core.services.keyValueDB.kvDB.get("scope-probe"));
|
||||||
|
observed.push(await core.services.config.getValue("scope-probe"));
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
expect(observed).toEqual([undefined, null]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["sync", "daemon", "mirror", "put"] as const)(
|
||||||
|
"blocks %s after failed reconstruction before command preparation",
|
||||||
|
async (command) => {
|
||||||
|
await start("fetch", async () => false, 1, true);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const ready = vi.spyOn(ControlService.prototype, "onReady");
|
||||||
|
|
||||||
|
await start(command, runner);
|
||||||
|
|
||||||
|
expect(runner).not.toHaveBeenCalled();
|
||||||
|
expect(ready).not.toHaveBeenCalled();
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it("retains legacy queued work and blocks a scanner until explicit reconstruction succeeds", async () => {
|
||||||
|
const legacy = join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
const legacyContents = JSON.stringify({
|
||||||
|
"queued-files": ["notes/unique.md"],
|
||||||
|
fileStatusMap: { "notes/local.md": 1 },
|
||||||
|
});
|
||||||
|
await mkdir(join(directory, ".livesync/runtime"), { recursive: true });
|
||||||
|
await writeFile(legacy, legacyContents);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const scan = vi.spyOn(ServiceFileAccessCLI.prototype, "getFiles");
|
||||||
|
|
||||||
|
await start("mirror", runner);
|
||||||
|
expect(runner).not.toHaveBeenCalled();
|
||||||
|
expect(scan).not.toHaveBeenCalled();
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe(legacyContents);
|
||||||
|
|
||||||
|
await start("fetch", async () => true, 0, true);
|
||||||
|
const later = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("sync", later);
|
||||||
|
expect(later).toHaveBeenCalledOnce();
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe(legacyContents);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("permits normal commands after reconstruction while preserving malformed legacy JSON", async () => {
|
||||||
|
const legacy = join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
await mkdir(join(directory, ".livesync/runtime"), { recursive: true });
|
||||||
|
await writeFile(legacy, '{"unfinished":');
|
||||||
|
await start("fetch", async () => true, 0, true);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("sync", runner);
|
||||||
|
expect(runner).toHaveBeenCalledOnce();
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe('{"unfinished":');
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
["kv", '{"unfinished":', "cannot be parsed"],
|
||||||
|
["snapshot", "[", "cannot be parsed"],
|
||||||
|
["kv", '{"queued-files":["unique.md"]}', "owner is unknown"],
|
||||||
|
["snapshot", '[{"path":"unique.md"}]', "owner is unknown"],
|
||||||
|
])("diagnoses unowned %s (%s) before readiness and ignores it after recovery", async (kind, contents, reason) => {
|
||||||
|
const legacy =
|
||||||
|
kind === "kv"
|
||||||
|
? join(directory, ".livesync/runtime/keyvalue-db.json")
|
||||||
|
: join(vaultPath, ".livesync-snapshot.json");
|
||||||
|
await mkdir(join(directory, ".livesync/runtime"), { recursive: true });
|
||||||
|
await writeFile(legacy, contents);
|
||||||
|
const messages = () => standardIo.writeStderr.mock.calls.flat().join("");
|
||||||
|
const reader = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("ls", reader);
|
||||||
|
expect(reader).toHaveBeenCalledOnce();
|
||||||
|
expect(messages()).toContain(reason);
|
||||||
|
expect(messages()).toContain(legacy);
|
||||||
|
expect(messages()).toContain("inspection may continue");
|
||||||
|
expect(messages().split("[Legacy state]")).toHaveLength(2);
|
||||||
|
expect(standardIo.writeStdout.mock.calls.flat().join("")).not.toContain("[Legacy state]");
|
||||||
|
|
||||||
|
standardIo.writeStderr.mockClear();
|
||||||
|
const writer = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const ready = vi.spyOn(ControlService.prototype, "onReady");
|
||||||
|
await start("mirror", writer);
|
||||||
|
expect(writer).not.toHaveBeenCalled();
|
||||||
|
expect(ready).not.toHaveBeenCalled();
|
||||||
|
expect(messages()).toContain("stopped before scanning");
|
||||||
|
expect(messages()).toContain(reason);
|
||||||
|
expect(messages().split("[Legacy state]")).toHaveLength(2);
|
||||||
|
ready.mockRestore();
|
||||||
|
|
||||||
|
await start("fetch", async () => true, 0, true);
|
||||||
|
standardIo.writeStderr.mockClear();
|
||||||
|
const later = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("sync", later);
|
||||||
|
expect(later).toHaveBeenCalledOnce();
|
||||||
|
expect(messages()).not.toContain("[Legacy state]");
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe(contents);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("blocks read commands which request settings writes while reconstruction is interrupted", async () => {
|
||||||
|
await start("fetch", async () => false, 1, true);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const load = vi.spyOn(ControlService.prototype, "onLoad");
|
||||||
|
|
||||||
|
await start("ls", runner, 1, false, true);
|
||||||
|
|
||||||
|
expect(runner).not.toHaveBeenCalled();
|
||||||
|
expect(load).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("invalidates every Vault scope for the reset database and retains other database state", async () => {
|
||||||
|
const originalVault = vaultPath;
|
||||||
|
const otherVault = join(directory, "other-vault");
|
||||||
|
await mkdir(otherVault);
|
||||||
|
const select = async (suffix: string, selectedVault: string) => {
|
||||||
|
vaultPath = selectedVault;
|
||||||
|
const settings = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
settings.additionalSuffixOfDatabaseName = suffix;
|
||||||
|
await writeFile(settingsPath, JSON.stringify(settings));
|
||||||
|
};
|
||||||
|
for (const [suffix, selectedVault] of [
|
||||||
|
["", originalVault],
|
||||||
|
["", otherVault],
|
||||||
|
["other", originalVault],
|
||||||
|
]) {
|
||||||
|
await select(suffix, selectedVault);
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
await core.services.keyValueDB.kvDB.set("scope-probe", "stored");
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
await select("", originalVault);
|
||||||
|
let reset: boolean | undefined;
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
reset = await core.services.database.resetDatabase();
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
expect(reset).toBe(true);
|
||||||
|
|
||||||
|
const observed: unknown[] = [];
|
||||||
|
for (const [suffix, selectedVault] of [
|
||||||
|
["", originalVault],
|
||||||
|
["", otherVault],
|
||||||
|
["other", originalVault],
|
||||||
|
]) {
|
||||||
|
await select(suffix, selectedVault);
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
observed.push(await core.services.keyValueDB.kvDB.get("scope-probe"));
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
expect(observed).toEqual([undefined, undefined, "stored"]);
|
||||||
|
});
|
||||||
|
|
||||||
it("does not import vault files for standalone database commands", async () => {
|
it("does not import vault files for standalone database commands", async () => {
|
||||||
const storedPaths: string[] = [];
|
const storedPaths: string[] = [];
|
||||||
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
||||||
|
|||||||
+80
-16
@@ -41,6 +41,14 @@ import {
|
|||||||
reconcileDurableSettings,
|
reconcileDurableSettings,
|
||||||
settingsEqual,
|
settingsEqual,
|
||||||
} from "./settingsPersistence";
|
} from "./settingsPersistence";
|
||||||
|
import {
|
||||||
|
inspectLegacyState,
|
||||||
|
isMutatingCommand,
|
||||||
|
isReconstructionCommand,
|
||||||
|
readOptionalFile,
|
||||||
|
ReconstructionState,
|
||||||
|
writeFileAtomically,
|
||||||
|
} from "./runtimeState";
|
||||||
|
|
||||||
const SETTINGS_FILE = ".livesync/settings.json";
|
const SETTINGS_FILE = ".livesync/settings.json";
|
||||||
|
|
||||||
@@ -381,7 +389,7 @@ export async function main(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Resolve database path
|
// Resolve database path
|
||||||
const databasePath = path.resolve(options.databasePath!);
|
let databasePath = path.resolve(options.databasePath!);
|
||||||
// Check if database directory exists
|
// Check if database directory exists
|
||||||
try {
|
try {
|
||||||
const stat = await fs.stat(databasePath);
|
const stat = await fs.stat(databasePath);
|
||||||
@@ -389,18 +397,19 @@ export async function main(
|
|||||||
writeStderrLine(standardIo, `Error: ${databasePath} is not a directory`);
|
writeStderrLine(standardIo, `Error: ${databasePath} is not a directory`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
databasePath = await fs.realpath(databasePath);
|
||||||
} catch {
|
} catch {
|
||||||
writeStderrLine(standardIo, `Error: Database directory ${databasePath} does not exist`);
|
writeStderrLine(standardIo, `Error: Database directory ${databasePath} does not exist`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Resolve settings path
|
// Resolve settings path
|
||||||
const settingsPath = options.settingsPath
|
let settingsPath = options.settingsPath
|
||||||
? path.resolve(options.settingsPath)
|
? path.resolve(options.settingsPath)
|
||||||
: path.join(databasePath, SETTINGS_FILE);
|
: path.join(databasePath, SETTINGS_FILE);
|
||||||
// Resolve vault path: mirror positional argument takes priority,
|
// Resolve vault path: mirror positional argument takes priority,
|
||||||
// then --vault flag, otherwise fall back to databasePath.
|
// then --vault flag, otherwise fall back to databasePath.
|
||||||
const vaultPath =
|
let vaultPath =
|
||||||
options.command === "mirror" && options.commandArgs[0]
|
options.command === "mirror" && options.commandArgs[0]
|
||||||
? path.resolve(options.commandArgs[0])
|
? path.resolve(options.commandArgs[0])
|
||||||
: options.vaultPath
|
: options.vaultPath
|
||||||
@@ -414,11 +423,22 @@ export async function main(
|
|||||||
writeStderrLine(standardIo, `Error: Vault path ${vaultPath} is not a directory`);
|
writeStderrLine(standardIo, `Error: Vault path ${vaultPath} is not a directory`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
vaultPath = await fs.realpath(vaultPath);
|
||||||
} catch {
|
} catch {
|
||||||
writeStderrLine(standardIo, `Error: Vault directory ${vaultPath} does not exist`);
|
writeStderrLine(standardIo, `Error: Vault directory ${vaultPath} does not exist`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
settingsPath = await fs.realpath(settingsPath).catch(() => settingsPath);
|
||||||
|
const reconstruction = new ReconstructionState(databasePath, vaultPath, settingsPath);
|
||||||
|
try {
|
||||||
|
reconstruction.prepare(options.command, options.force ?? false, options.writeSettings ?? false);
|
||||||
|
} catch (error) {
|
||||||
|
writeStderrLine(standardIo, "[Error] Reconstruction recovery is required:", error);
|
||||||
|
process.exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
infoLog(`Self-hosted LiveSync CLI`);
|
infoLog(`Self-hosted LiveSync CLI`);
|
||||||
infoLog(`Database Path: ${databasePath}`);
|
infoLog(`Database Path: ${databasePath}`);
|
||||||
infoLog(`Vault Path: ${vaultPath}`);
|
infoLog(`Vault Path: ${vaultPath}`);
|
||||||
@@ -437,7 +457,7 @@ export async function main(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Create service context and hub
|
// Create service context and hub
|
||||||
const context = new NodeServiceContext(databasePath, standardIo);
|
const context = new NodeServiceContext(databasePath, standardIo, vaultPath);
|
||||||
const serviceHubInstance = new NodeServiceHub<NodeServiceContext>(databasePath, context);
|
const serviceHubInstance = new NodeServiceHub<NodeServiceContext>(databasePath, context);
|
||||||
serviceHubInstance.API.addLog.setHandler((message: unknown, level: LOG_LEVEL) => {
|
serviceHubInstance.API.addLog.setHandler((message: unknown, level: LOG_LEVEL) => {
|
||||||
let levelStr = "";
|
let levelStr = "";
|
||||||
@@ -479,7 +499,7 @@ export async function main(
|
|||||||
|
|
||||||
// Setup settings handlers
|
// Setup settings handlers
|
||||||
const settingService = serviceHubInstance.setting;
|
const settingService = serviceHubInstance.setting;
|
||||||
const originalSettingsText = await fs.readFile(settingsPath, "utf-8").catch(() => undefined);
|
const originalSettingsText = readOptionalFile(settingsPath);
|
||||||
let latestPreparedSettingsText: string | undefined;
|
let latestPreparedSettingsText: string | undefined;
|
||||||
let preparedSettingsRevision = 0;
|
let preparedSettingsRevision = 0;
|
||||||
let commandIsRunning = false;
|
let commandIsRunning = false;
|
||||||
@@ -600,21 +620,19 @@ export async function main(
|
|||||||
|
|
||||||
const writeSettingsAtomically = async (content: string | undefined): Promise<void> => {
|
const writeSettingsAtomically = async (content: string | undefined): Promise<void> => {
|
||||||
if (content === undefined || content === originalSettingsText) return;
|
if (content === undefined || content === originalSettingsText) return;
|
||||||
const temporaryPath = `${settingsPath}.${process.pid}.tmp`;
|
writeFileAtomically(settingsPath, content);
|
||||||
await fs.mkdir(path.dirname(settingsPath), { recursive: true });
|
|
||||||
try {
|
|
||||||
await fs.writeFile(temporaryPath, content, "utf-8");
|
|
||||||
await fs.rename(temporaryPath, settingsPath);
|
|
||||||
if (options.verbose) {
|
if (options.verbose) {
|
||||||
writeStderrLine(standardIo, `[Settings] Saved to ${settingsPath}`);
|
writeStderrLine(standardIo, `[Settings] Saved to ${settingsPath}`);
|
||||||
}
|
}
|
||||||
} catch (error) {
|
|
||||||
await fs.unlink(temporaryPath).catch(() => {});
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
};
|
};
|
||||||
|
|
||||||
// Start the core
|
// Start the core
|
||||||
|
let unloaded = false;
|
||||||
|
const unload = async () => {
|
||||||
|
if (unloaded) return;
|
||||||
|
unloaded = true;
|
||||||
|
await core.services.control.onUnload();
|
||||||
|
};
|
||||||
try {
|
try {
|
||||||
infoLog(`[Starting] Initializing LiveSync...`);
|
infoLog(`[Starting] Initializing LiveSync...`);
|
||||||
|
|
||||||
@@ -623,6 +641,31 @@ export async function main(
|
|||||||
writeStderrLine(standardIo, `[Error] Failed to initialize LiveSync`);
|
writeStderrLine(standardIo, `[Error] Failed to initialize LiveSync`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
if ((await core.services.keyValueDB.kvDB.get("cli-legacy-state-reviewed")) !== true) {
|
||||||
|
const legacyIssues = inspectLegacyState(databasePath, vaultPath);
|
||||||
|
if (legacyIssues.length > 0) {
|
||||||
|
const reasons = {
|
||||||
|
unowned: "database and Vault owner is unknown",
|
||||||
|
invalid: "state cannot be parsed in its expected format; owner and contents cannot be established",
|
||||||
|
unreadable: "state cannot be read; owner and contents cannot be established",
|
||||||
|
};
|
||||||
|
const details = legacyIssues.map((issue) => `${issue.path}: ${reasons[issue.reason]}`).join("; ");
|
||||||
|
const advice =
|
||||||
|
"Preserve these files, back up unique edits, and verify the authoritative source before choosing fetch --force or rebuild --force.";
|
||||||
|
const blocked =
|
||||||
|
!isReconstructionCommand(options.command) &&
|
||||||
|
(isMutatingCommand(options.command) || options.writeSettings);
|
||||||
|
if (blocked) {
|
||||||
|
throw new Error(
|
||||||
|
`[Legacy state] ${details}. Command '${options.command}' stopped before scanning, event replay, or synchronisation. ${advice}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const action = isReconstructionCommand(options.command)
|
||||||
|
? "Explicit reconstruction will proceed; the legacy files will be retained."
|
||||||
|
: "Read-only inspection may continue; the legacy files will be retained.";
|
||||||
|
writeStderrLine(standardIo, `[Legacy state] ${details}. ${action} ${advice}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
const settingsAfterLoadText = latestPreparedSettingsText
|
const settingsAfterLoadText = latestPreparedSettingsText
|
||||||
? preserveStoredSetting(latestPreparedSettingsText, originalSettingsText, "useIndexedDBAdapter")
|
? preserveStoredSetting(latestPreparedSettingsText, originalSettingsText, "useIndexedDBAdapter")
|
||||||
: originalSettingsText;
|
: originalSettingsText;
|
||||||
@@ -655,7 +698,7 @@ export async function main(
|
|||||||
try {
|
try {
|
||||||
readyResult = await core.services.control.onReady();
|
readyResult = await core.services.control.onReady();
|
||||||
} finally {
|
} finally {
|
||||||
if (!readyResult) await core.services.control.onUnload();
|
if (!readyResult) await unload();
|
||||||
}
|
}
|
||||||
if (!readyResult) {
|
if (!readyResult) {
|
||||||
throw new Error("Failed to initialise LiveSync.");
|
throw new Error("Failed to initialise LiveSync.");
|
||||||
@@ -751,6 +794,21 @@ export async function main(
|
|||||||
settingsTextToCommit = settingsAfterLoadText;
|
settingsTextToCommit = settingsAfterLoadText;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if (result && isReconstructionCommand(options.command)) {
|
||||||
|
if (!options.writeSettings) {
|
||||||
|
const durable =
|
||||||
|
originalSettingsText === undefined
|
||||||
|
? cloneSettings(durableSettingsBeforeSuspend)
|
||||||
|
: (JSON.parse(originalSettingsText) as ObsidianLiveSyncSettings);
|
||||||
|
durable.additionalSuffixOfDatabaseName =
|
||||||
|
core.services.setting.currentSettings().additionalSuffixOfDatabaseName;
|
||||||
|
settingsTextToCommit = JSON.stringify(durable, null, 2);
|
||||||
|
}
|
||||||
|
settingsTextToCommit ??= originalSettingsText;
|
||||||
|
if (settingsTextToCommit === undefined)
|
||||||
|
throw new Error("Reconstruction completed without database selection settings.");
|
||||||
|
await core.services.keyValueDB.kvDB.set("cli-legacy-state-reviewed", true);
|
||||||
|
}
|
||||||
|
|
||||||
if (!result) {
|
if (!result) {
|
||||||
writeStderrLine(standardIo, `[Error] Command '${options.command}' failed`);
|
writeStderrLine(standardIo, `[Error] Command '${options.command}' failed`);
|
||||||
@@ -764,10 +822,16 @@ export async function main(
|
|||||||
// Keep the process running
|
// Keep the process running
|
||||||
await new Promise(() => {});
|
await new Promise(() => {});
|
||||||
} else {
|
} else {
|
||||||
await core.services.control.onUnload();
|
await unload();
|
||||||
|
if (result && isReconstructionCommand(options.command)) {
|
||||||
|
reconstruction.complete(settingsTextToCommit!);
|
||||||
|
reconstruction.commit();
|
||||||
|
} else {
|
||||||
await writeSettingsAtomically(settingsTextToCommit);
|
await writeSettingsAtomically(settingsTextToCommit);
|
||||||
}
|
}
|
||||||
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
await unload().catch(() => {});
|
||||||
writeStderrLine(standardIo, `[Error] Failed to start:`, error);
|
writeStderrLine(standardIo, `[Error] Failed to start:`, error);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ import { watch as chokidarWatch, type FSWatcher } from "chokidar";
|
|||||||
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||||
import { fsPromises as fs, getNodeFileCreationTime, path, type Stats } from "@vrtmrz/livesync-commonlib/node";
|
import { fsPromises as fs, getNodeFileCreationTime, path, type Stats } from "@vrtmrz/livesync-commonlib/node";
|
||||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||||
|
import { writeFileAtomically } from "@/apps/cli/runtimeState";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* CLI-specific type guard adapter
|
* CLI-specific type guard adapter
|
||||||
@@ -44,18 +45,19 @@ class CLITypeGuardAdapter implements IStorageEventTypeGuardAdapter<NodeFile, Nod
|
|||||||
* CLI-specific persistence adapter (file-based snapshot)
|
* CLI-specific persistence adapter (file-based snapshot)
|
||||||
*/
|
*/
|
||||||
class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
|
class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
|
||||||
private snapshotPath: string;
|
private snapshotPath: () => string;
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
basePath: string,
|
basePath: string,
|
||||||
private reportDiagnostic: CliDiagnosticReporter = () => undefined
|
private reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||||
|
snapshotPath?: () => string
|
||||||
) {
|
) {
|
||||||
this.snapshotPath = path.join(basePath, ".livesync-snapshot.json");
|
this.snapshotPath = snapshotPath ?? (() => path.join(basePath, ".livesync-snapshot.json"));
|
||||||
}
|
}
|
||||||
|
|
||||||
async saveSnapshot(snapshot: (FileEventItem | FileEventItemSentinel)[]): Promise<void> {
|
async saveSnapshot(snapshot: (FileEventItem | FileEventItemSentinel)[]): Promise<void> {
|
||||||
try {
|
try {
|
||||||
await fs.writeFile(this.snapshotPath, JSON.stringify(snapshot, null, 2), "utf-8");
|
writeFileAtomically(this.snapshotPath(), JSON.stringify(snapshot, null, 2));
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.reportDiagnostic("Failed to save snapshot:", error);
|
this.reportDiagnostic("Failed to save snapshot:", error);
|
||||||
}
|
}
|
||||||
@@ -63,7 +65,7 @@ class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
|
|||||||
|
|
||||||
async loadSnapshot(): Promise<(FileEventItem | FileEventItemSentinel)[] | null> {
|
async loadSnapshot(): Promise<(FileEventItem | FileEventItemSentinel)[] | null> {
|
||||||
try {
|
try {
|
||||||
const content = await fs.readFile(this.snapshotPath, "utf-8");
|
const content = await fs.readFile(this.snapshotPath(), "utf-8");
|
||||||
return JSON.parse(content) as (FileEventItem | FileEventItemSentinel)[];
|
return JSON.parse(content) as (FileEventItem | FileEventItemSentinel)[];
|
||||||
} catch {
|
} catch {
|
||||||
return null;
|
return null;
|
||||||
@@ -221,10 +223,11 @@ export class CLIStorageEventManagerAdapter implements IStorageEventManagerAdapte
|
|||||||
basePath: string,
|
basePath: string,
|
||||||
ignoreRules?: IgnoreRules,
|
ignoreRules?: IgnoreRules,
|
||||||
watchEnabled: boolean = false,
|
watchEnabled: boolean = false,
|
||||||
reportDiagnostic: CliDiagnosticReporter = () => undefined
|
reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||||
|
snapshotPath?: () => string
|
||||||
) {
|
) {
|
||||||
this.typeGuard = new CLITypeGuardAdapter();
|
this.typeGuard = new CLITypeGuardAdapter();
|
||||||
this.persistence = new CLIPersistenceAdapter(basePath, reportDiagnostic);
|
this.persistence = new CLIPersistenceAdapter(basePath, reportDiagnostic, snapshotPath);
|
||||||
this.watch = new CLIWatchAdapter(basePath, ignoreRules, watchEnabled, reportDiagnostic);
|
this.watch = new CLIWatchAdapter(basePath, ignoreRules, watchEnabled, reportDiagnostic);
|
||||||
this.status = new CLIStatusAdapter();
|
this.status = new CLIStatusAdapter();
|
||||||
this.converter = new CLIConverterAdapter();
|
this.converter = new CLIConverterAdapter();
|
||||||
|
|||||||
@@ -1,6 +1,8 @@
|
|||||||
import { describe, expect, it, vi, beforeEach } from "vitest";
|
import { describe, expect, it, vi, beforeEach } from "vitest";
|
||||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||||
import type { NodeFile } from "@/apps/cli/adapters/NodeTypes";
|
import type { NodeFile } from "@/apps/cli/adapters/NodeTypes";
|
||||||
|
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import { runtimeScopeDirectory } from "../runtimeState";
|
||||||
|
|
||||||
// ── chokidar mock ──────────────────────────────────────────────────────────────
|
// ── chokidar mock ──────────────────────────────────────────────────────────────
|
||||||
// Must be hoisted before imports that pull in chokidar.
|
// Must be hoisted before imports that pull in chokidar.
|
||||||
@@ -44,6 +46,27 @@ describe("CLIStorageEventManagerAdapter", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("keeps pending snapshots with the selected database and Vault pair", async () => {
|
||||||
|
const directory = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-snapshot-"));
|
||||||
|
let database = "first";
|
||||||
|
let vault = "vault-a";
|
||||||
|
const adapter = new CLIStorageEventManagerAdapter(directory, undefined, false, undefined, () =>
|
||||||
|
path.join(runtimeScopeDirectory(directory, database, vault), "storage-events.json")
|
||||||
|
);
|
||||||
|
try {
|
||||||
|
await adapter.persistence.saveSnapshot([{ type: "SENTINEL_FLUSH" }]);
|
||||||
|
database = "second";
|
||||||
|
expect(await adapter.persistence.loadSnapshot()).toBeNull();
|
||||||
|
database = "first";
|
||||||
|
vault = "vault-b";
|
||||||
|
expect(await adapter.persistence.loadSnapshot()).toBeNull();
|
||||||
|
vault = "vault-a";
|
||||||
|
expect(await adapter.persistence.loadSnapshot()).toEqual([{ type: "SENTINEL_FLUSH" }]);
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(directory, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
it("beginWatch is no-op when watchEnabled=false", async () => {
|
it("beginWatch is no-op when watchEnabled=false", async () => {
|
||||||
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, false);
|
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, false);
|
||||||
const handlers = makeHandlers();
|
const handlers = makeHandlers();
|
||||||
|
|||||||
@@ -1,9 +1,15 @@
|
|||||||
import { StorageEventManagerBase, type StorageEventManagerBaseDependencies } from "@vrtmrz/livesync-commonlib/compat/managers/StorageEventManager";
|
import {
|
||||||
|
StorageEventManagerBase,
|
||||||
|
type StorageEventManagerBaseDependencies,
|
||||||
|
} from "@vrtmrz/livesync-commonlib/compat/managers/StorageEventManager";
|
||||||
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
|
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
|
||||||
import type { IMinimumLiveSyncCommands, LiveSyncBaseCore } from "@/LiveSyncBaseCore";
|
import type { IMinimumLiveSyncCommands, LiveSyncBaseCore } from "@/LiveSyncBaseCore";
|
||||||
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
||||||
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||||
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||||
|
import { runtimeScopeDirectory } from "@/apps/cli/runtimeState";
|
||||||
|
import type { NodeServiceContext } from "@/apps/cli/services/NodeServiceContext";
|
||||||
|
import { path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
||||||
// import type { IMinimumLiveSyncCommands } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
// import type { IMinimumLiveSyncCommands } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
||||||
|
|
||||||
export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEventManagerAdapter> {
|
export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEventManagerAdapter> {
|
||||||
@@ -16,12 +22,23 @@ export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEv
|
|||||||
ignoreRules?: IgnoreRules,
|
ignoreRules?: IgnoreRules,
|
||||||
watchEnabled?: boolean
|
watchEnabled?: boolean
|
||||||
) {
|
) {
|
||||||
const adapter = new CLIStorageEventManagerAdapter(basePath, ignoreRules, watchEnabled, (message, detail) => {
|
const context = core.services.context as NodeServiceContext;
|
||||||
|
const adapter = new CLIStorageEventManagerAdapter(
|
||||||
|
basePath,
|
||||||
|
ignoreRules,
|
||||||
|
watchEnabled,
|
||||||
|
(message, detail) => {
|
||||||
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
|
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
|
||||||
if (detail !== undefined) {
|
if (detail !== undefined) {
|
||||||
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
|
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
|
||||||
}
|
}
|
||||||
});
|
},
|
||||||
|
() =>
|
||||||
|
nodePath.join(
|
||||||
|
runtimeScopeDirectory(context.databasePath, dependencies.vaultService.getVaultName(), basePath),
|
||||||
|
"storage-events.json"
|
||||||
|
)
|
||||||
|
);
|
||||||
super(adapter, dependencies);
|
super(adapter, dependencies);
|
||||||
this.core = core;
|
this.core = core;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,7 +12,7 @@
|
|||||||
"buildRun": "npm run build && npm run cli --",
|
"buildRun": "npm run build && npm run cli --",
|
||||||
"build:docker": "docker build -f Dockerfile -t livesync-cli ../../..",
|
"build:docker": "docker build -f Dockerfile -t livesync-cli ../../..",
|
||||||
"check": "tsc -p tsconfig.json",
|
"check": "tsc -p tsconfig.json",
|
||||||
"test:unit": "cd ../../.. && npx vitest run --config vitest.config.unit.ts src/apps/cli/main.unit.spec.ts src/apps/cli/main.bootstrap.unit.spec.ts src/apps/cli/settingsPersistence.unit.spec.ts src/apps/cli/services/NodeLocalStorage.unit.spec.ts src/apps/cli/commands/utils.unit.spec.ts src/apps/cli/commands/runCommand.unit.spec.ts src/apps/cli/commands/daemonCommand.unit.spec.ts src/apps/cli/commands/p2p.unit.spec.ts src/apps/cli/deploy/install.unit.spec.ts src/apps/cli/adapters/NodeFileSystemAdapter.unit.spec.ts",
|
"test:unit": "cd ../../.. && npx vitest run --config vitest.config.unit.ts src/apps/cli/main.unit.spec.ts src/apps/cli/main.bootstrap.unit.spec.ts src/apps/cli/settingsPersistence.unit.spec.ts src/apps/cli/runtimeState.unit.spec.ts src/apps/cli/services/NodeKeyValueDBService.unit.spec.ts src/apps/cli/services/NodeLocalStorage.unit.spec.ts src/apps/cli/managers/CLIStorageEventManagerAdapter.unit.spec.ts src/apps/cli/commands/utils.unit.spec.ts src/apps/cli/commands/runCommand.unit.spec.ts src/apps/cli/commands/daemonCommand.unit.spec.ts src/apps/cli/commands/p2p.unit.spec.ts src/apps/cli/deploy/install.unit.spec.ts src/apps/cli/adapters/NodeFileSystemAdapter.unit.spec.ts",
|
||||||
"test:e2e:two-vaults": "bash test/test-e2e-two-vaults-with-docker-linux.sh",
|
"test:e2e:two-vaults": "bash test/test-e2e-two-vaults-with-docker-linux.sh",
|
||||||
"test:e2e:two-vaults:common": "bash test/test-e2e-two-vaults-common.sh",
|
"test:e2e:two-vaults:common": "bash test/test-e2e-two-vaults-common.sh",
|
||||||
"test:e2e:two-vaults:matrix": "bash test/test-e2e-two-vaults-matrix.sh",
|
"test:e2e:two-vaults:matrix": "bash test/test-e2e-two-vaults-matrix.sh",
|
||||||
@@ -37,6 +37,7 @@
|
|||||||
"test:e2e:coverage": "node ../../../test/e2e-cli/run-coverage.ts"
|
"test:e2e:coverage": "node ../../../test/e2e-cli/run-coverage.ts"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@aws-crypto/sha256-js": "5.2.0",
|
||||||
"chokidar": "^4.0.0",
|
"chokidar": "^4.0.0",
|
||||||
"minimatch": "^10.2.5",
|
"minimatch": "^10.2.5",
|
||||||
"octagonal-wheels": "^0.1.55",
|
"octagonal-wheels": "^0.1.55",
|
||||||
|
|||||||
@@ -0,0 +1,222 @@
|
|||||||
|
// This module belongs exclusively to the Node.js CLI host.
|
||||||
|
import { Sha256 } from "@aws-crypto/sha256-js";
|
||||||
|
import { fs, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import type { CLICommand } from "./commands/types";
|
||||||
|
|
||||||
|
function sha256Hex(value: string): string {
|
||||||
|
const hash = new Sha256();
|
||||||
|
hash.update(new TextEncoder().encode(value));
|
||||||
|
return Array.from(hash.digestSync(), (byte) => byte.toString(16).padStart(2, "0")).join("");
|
||||||
|
}
|
||||||
|
|
||||||
|
export function runtimeScopeDirectory(databasePath: string, databaseName: string, vaultPath: string): string {
|
||||||
|
const scope = sha256Hex(vaultPath);
|
||||||
|
return path.join(runtimeDatabaseDirectory(databasePath, databaseName), scope);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function runtimeDatabaseDirectory(databasePath: string, databaseName: string): string {
|
||||||
|
const database = sha256Hex(databaseName);
|
||||||
|
return path.join(databasePath, ".livesync", "runtime", "scopes", database);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function writeFileAtomically(filePath: string, content: string): void {
|
||||||
|
fs.mkdirSync(path.dirname(filePath), { recursive: true });
|
||||||
|
const temporaryPath = `${filePath}.${process.pid}.${crypto.randomUUID()}.tmp`;
|
||||||
|
try {
|
||||||
|
const descriptor = fs.openSync(temporaryPath, "wx", 0o600);
|
||||||
|
try {
|
||||||
|
fs.writeFileSync(descriptor, content, "utf-8");
|
||||||
|
fs.fsyncSync(descriptor);
|
||||||
|
} finally {
|
||||||
|
fs.closeSync(descriptor);
|
||||||
|
}
|
||||||
|
fs.renameSync(temporaryPath, filePath);
|
||||||
|
syncDirectory(path.dirname(filePath));
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(temporaryPath, { force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function syncDirectory(directory: string): void {
|
||||||
|
// Windows does not support opening a directory for fsync.
|
||||||
|
if (process.platform === "win32") return;
|
||||||
|
const descriptor = fs.openSync(directory, "r");
|
||||||
|
try {
|
||||||
|
fs.fsyncSync(descriptor);
|
||||||
|
} finally {
|
||||||
|
fs.closeSync(descriptor);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function readOptionalFile(filePath: string): string | undefined {
|
||||||
|
try {
|
||||||
|
return fs.readFileSync(filePath, "utf-8");
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as { code?: string }).code === "ENOENT") return undefined;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isReconstructionCommand(command: CLICommand): command is "fetch" | "rebuild" {
|
||||||
|
return command === "fetch" || command === "rebuild";
|
||||||
|
}
|
||||||
|
|
||||||
|
const READ_COMMANDS = new Set<CLICommand>([
|
||||||
|
"cat",
|
||||||
|
"cat-rev",
|
||||||
|
"ls",
|
||||||
|
"info",
|
||||||
|
"remote-ls",
|
||||||
|
"remote-export",
|
||||||
|
"remote-status",
|
||||||
|
"p2p-peers",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export function isMutatingCommand(command: CLICommand): boolean {
|
||||||
|
return !READ_COMMANDS.has(command);
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ReconstructionMarker {
|
||||||
|
version: 1;
|
||||||
|
command: "fetch" | "rebuild";
|
||||||
|
databasePath: string;
|
||||||
|
vaultPath: string;
|
||||||
|
settingsPath: string;
|
||||||
|
settingsBeforeHash: string;
|
||||||
|
phase: "running" | "completed";
|
||||||
|
settingsText?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hashSettings(content: string | undefined): string {
|
||||||
|
return sha256Hex(JSON.stringify(content ?? null));
|
||||||
|
}
|
||||||
|
|
||||||
|
export class ReconstructionState {
|
||||||
|
readonly markerPath: string;
|
||||||
|
private marker: ReconstructionMarker | undefined;
|
||||||
|
|
||||||
|
constructor(
|
||||||
|
private readonly databasePath: string,
|
||||||
|
private readonly vaultPath: string,
|
||||||
|
private readonly settingsPath: string
|
||||||
|
) {
|
||||||
|
this.markerPath = path.join(databasePath, ".livesync", "reconstruction.json");
|
||||||
|
}
|
||||||
|
|
||||||
|
prepare(command: CLICommand, force: boolean, writeSettings = false): void {
|
||||||
|
const contents = readOptionalFile(this.markerPath);
|
||||||
|
if (contents !== undefined) {
|
||||||
|
const parsed: unknown = JSON.parse(contents);
|
||||||
|
if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) {
|
||||||
|
throw new Error(`Invalid reconstruction state: ${this.markerPath}. Preserve it for recovery.`);
|
||||||
|
}
|
||||||
|
const marker = parsed as ReconstructionMarker;
|
||||||
|
if (
|
||||||
|
marker.version !== 1 ||
|
||||||
|
!isReconstructionCommand(marker.command) ||
|
||||||
|
(marker.phase !== "running" && marker.phase !== "completed") ||
|
||||||
|
typeof marker.settingsPath !== "string" ||
|
||||||
|
typeof marker.vaultPath !== "string" ||
|
||||||
|
typeof marker.settingsBeforeHash !== "string" ||
|
||||||
|
marker.databasePath !== this.databasePath ||
|
||||||
|
(marker.phase === "completed" && typeof marker.settingsText !== "string")
|
||||||
|
) {
|
||||||
|
throw new Error(`Invalid reconstruction state: ${this.markerPath}. Preserve it for recovery.`);
|
||||||
|
}
|
||||||
|
this.marker = marker;
|
||||||
|
if (marker.phase === "completed") {
|
||||||
|
this.commit();
|
||||||
|
} else if (
|
||||||
|
(isMutatingCommand(command) || writeSettings) &&
|
||||||
|
!(
|
||||||
|
command === marker.command &&
|
||||||
|
force &&
|
||||||
|
marker.vaultPath === this.vaultPath &&
|
||||||
|
marker.settingsPath === this.settingsPath
|
||||||
|
)
|
||||||
|
) {
|
||||||
|
throw new Error(
|
||||||
|
`An interrupted ${marker.command} requires recovery. Retry ${marker.command} --force with the same database, Vault, and settings paths. State: ${this.markerPath}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (isReconstructionCommand(command)) {
|
||||||
|
this.marker = {
|
||||||
|
version: 1,
|
||||||
|
command,
|
||||||
|
databasePath: this.databasePath,
|
||||||
|
vaultPath: this.vaultPath,
|
||||||
|
settingsPath: this.settingsPath,
|
||||||
|
phase: "running",
|
||||||
|
settingsBeforeHash: hashSettings(readOptionalFile(this.settingsPath)),
|
||||||
|
};
|
||||||
|
this.save();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
complete(settingsText: string): void {
|
||||||
|
if (!this.marker || this.marker.phase !== "running") throw new Error("Reconstruction was not started.");
|
||||||
|
this.marker = { ...this.marker, phase: "completed", settingsText };
|
||||||
|
this.save();
|
||||||
|
}
|
||||||
|
|
||||||
|
commit(): void {
|
||||||
|
const marker = this.marker;
|
||||||
|
if (!marker || marker.phase !== "completed" || marker.settingsText === undefined) {
|
||||||
|
throw new Error("Reconstruction has no completed settings update.");
|
||||||
|
}
|
||||||
|
const current = readOptionalFile(marker.settingsPath);
|
||||||
|
if (current !== marker.settingsText && hashSettings(current) !== marker.settingsBeforeHash) {
|
||||||
|
throw new Error(
|
||||||
|
`The settings file changed after reconstruction: ${marker.settingsPath}. Preserve the reconstruction state and review both settings versions before continuing.`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (current !== marker.settingsText) writeFileAtomically(marker.settingsPath, marker.settingsText);
|
||||||
|
this.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
clear(): void {
|
||||||
|
fs.rmSync(this.markerPath, { force: true });
|
||||||
|
syncDirectory(path.dirname(this.markerPath));
|
||||||
|
this.marker = undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
private save(): void {
|
||||||
|
writeFileAtomically(this.markerPath, JSON.stringify(this.marker, null, 2));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface LegacyStateIssue {
|
||||||
|
path: string;
|
||||||
|
reason: "unowned" | "invalid" | "unreadable";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function inspectLegacyState(databasePath: string, vaultPath: string): LegacyStateIssue[] {
|
||||||
|
const issues: LegacyStateIssue[] = [];
|
||||||
|
for (const [filePath, expectArray] of [
|
||||||
|
[path.join(databasePath, ".livesync", "runtime", "keyvalue-db.json"), false],
|
||||||
|
[path.join(vaultPath, ".livesync-snapshot.json"), true],
|
||||||
|
] as const) {
|
||||||
|
let contents: string | undefined;
|
||||||
|
try {
|
||||||
|
contents = readOptionalFile(filePath);
|
||||||
|
} catch {
|
||||||
|
issues.push({ path: filePath, reason: "unreadable" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (contents === undefined) continue;
|
||||||
|
let value: unknown;
|
||||||
|
try {
|
||||||
|
value = JSON.parse(contents);
|
||||||
|
} catch {
|
||||||
|
issues.push({ path: filePath, reason: "invalid" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (value === null || typeof value !== "object" || Array.isArray(value) !== expectArray) {
|
||||||
|
issues.push({ path: filePath, reason: "invalid" });
|
||||||
|
} else if (Object.keys(value).length > 0) {
|
||||||
|
issues.push({ path: filePath, reason: "unowned" });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return issues;
|
||||||
|
}
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
ReconstructionState,
|
||||||
|
inspectLegacyState,
|
||||||
|
readOptionalFile,
|
||||||
|
runtimeDatabaseDirectory,
|
||||||
|
runtimeScopeDirectory,
|
||||||
|
} from "./runtimeState";
|
||||||
|
|
||||||
|
describe("CLI runtime scope compatibility", () => {
|
||||||
|
it("retains SHA-256 directory names for database and Unicode Vault paths", () => {
|
||||||
|
expect(runtimeScopeDirectory("state", "headless-app", "/vault/日本語😀")).toBe(
|
||||||
|
path.join(
|
||||||
|
"state",
|
||||||
|
".livesync",
|
||||||
|
"runtime",
|
||||||
|
"scopes",
|
||||||
|
"511afda5613706862fa4add801b3f9b19e0db2acacc07b9218f2d01ff41ca61d",
|
||||||
|
"542d31d6918ad8bf915e204e41e479fdbd7251690f182fd802378323b4c6de23"
|
||||||
|
)
|
||||||
|
);
|
||||||
|
expect(path.basename(runtimeDatabaseDirectory("state", ""))).toBe(
|
||||||
|
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("CLI reconstruction state", () => {
|
||||||
|
let directory: string;
|
||||||
|
let settingsPath: string;
|
||||||
|
let state: ReconstructionState;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
directory = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-reconstruction-"));
|
||||||
|
settingsPath = path.join(directory, "settings.json");
|
||||||
|
fs.writeFileSync(settingsPath, JSON.stringify({ additionalSuffixOfDatabaseName: "original" }));
|
||||||
|
state = new ReconstructionState(directory, path.join(directory, "vault"), settingsPath);
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => fs.rmSync(directory, { recursive: true, force: true }));
|
||||||
|
|
||||||
|
it("retains the persisted settings fingerprint across hash implementations", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
expect(JSON.parse(fs.readFileSync(state.markerPath, "utf-8"))).toMatchObject({
|
||||||
|
settingsBeforeHash: "29e04e3dd1d10b352678a922ff5a03fbe529d3505e9cf46c2ecc860f44478ae8",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("persists progress before returning and permits only the matching forced retry", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
expect(JSON.parse(fs.readFileSync(state.markerPath, "utf-8"))).toMatchObject({
|
||||||
|
phase: "running",
|
||||||
|
command: "fetch",
|
||||||
|
});
|
||||||
|
if (process.platform !== "win32") expect(fs.statSync(state.markerPath).mode & 0o777).toBe(0o600);
|
||||||
|
expect(() => state.prepare("sync", false)).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("ls", false, true)).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("fetch", false)).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("rebuild", true)).toThrow("interrupted fetch");
|
||||||
|
expect(() =>
|
||||||
|
new ReconstructionState(directory, path.join(directory, "other"), settingsPath).prepare("fetch", true)
|
||||||
|
).toThrow("interrupted fetch");
|
||||||
|
expect(() =>
|
||||||
|
new ReconstructionState(
|
||||||
|
directory,
|
||||||
|
path.join(directory, "vault"),
|
||||||
|
path.join(directory, "other.json")
|
||||||
|
).prepare("fetch", true)
|
||||||
|
).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("ls", false)).not.toThrow();
|
||||||
|
expect(() => state.prepare("fetch", true)).not.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("recovers completed settings before permitting another command", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
const completed = JSON.stringify({ additionalSuffixOfDatabaseName: "headless-app", retained: true });
|
||||||
|
state.complete(completed);
|
||||||
|
|
||||||
|
new ReconstructionState(directory, path.join(directory, "vault"), settingsPath).prepare("sync", false);
|
||||||
|
|
||||||
|
expect(fs.readFileSync(settingsPath, "utf-8")).toBe(completed);
|
||||||
|
expect(readOptionalFile(state.markerPath)).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["normal completion", "restart"] as const)(
|
||||||
|
"preserves user settings changed during reconstruction at %s",
|
||||||
|
(phase) => {
|
||||||
|
state.prepare("rebuild", true);
|
||||||
|
state.complete(JSON.stringify({ additionalSuffixOfDatabaseName: "headless-app" }));
|
||||||
|
const edited = JSON.stringify({ additionalSuffixOfDatabaseName: "user-selection", unique: "edit" });
|
||||||
|
fs.writeFileSync(settingsPath, edited);
|
||||||
|
|
||||||
|
const commit =
|
||||||
|
phase === "restart"
|
||||||
|
? () =>
|
||||||
|
new ReconstructionState(directory, path.join(directory, "vault"), settingsPath).prepare(
|
||||||
|
"sync",
|
||||||
|
false
|
||||||
|
)
|
||||||
|
: () => state.commit();
|
||||||
|
|
||||||
|
expect(commit).toThrow("settings file changed");
|
||||||
|
expect(fs.readFileSync(settingsPath, "utf-8")).toBe(edited);
|
||||||
|
expect(JSON.parse(fs.readFileSync(state.markerPath, "utf-8"))).toMatchObject({ phase: "completed" });
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it("clears a completed marker after an interruption following the settings rename", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
const completed = JSON.stringify({ additionalSuffixOfDatabaseName: "headless-app" });
|
||||||
|
state.complete(completed);
|
||||||
|
fs.writeFileSync(settingsPath, completed);
|
||||||
|
state.prepare("ls", false);
|
||||||
|
expect(readOptionalFile(state.markerPath)).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["null", "[]", "{}", '{"version":2}', '{"version":1,"phase":"completed"}'])(
|
||||||
|
"refuses malformed state %s without clearing it",
|
||||||
|
(contents) => {
|
||||||
|
fs.mkdirSync(path.dirname(state.markerPath), { recursive: true });
|
||||||
|
fs.writeFileSync(state.markerPath, contents);
|
||||||
|
expect(() => state.prepare("sync", false)).toThrow("Invalid reconstruction state");
|
||||||
|
expect(fs.readFileSync(state.markerPath, "utf-8")).toBe(contents);
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it.each(["null", "[]", '"text"', "{", ""])("reports invalid legacy KV %s without changing it", (contents) => {
|
||||||
|
const legacy = path.join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
fs.mkdirSync(path.dirname(legacy), { recursive: true });
|
||||||
|
fs.writeFileSync(legacy, contents);
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([{ path: legacy, reason: "invalid" }]);
|
||||||
|
expect(fs.readFileSync(legacy, "utf-8")).toBe(contents);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports both unreadable KV and invalid snapshot instead of hiding the second problem", () => {
|
||||||
|
const legacy = path.join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
const snapshot = path.join(directory, ".livesync-snapshot.json");
|
||||||
|
fs.mkdirSync(legacy, { recursive: true });
|
||||||
|
fs.writeFileSync(snapshot, "{}");
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([
|
||||||
|
{ path: legacy, reason: "unreadable" },
|
||||||
|
{ path: snapshot, reason: "invalid" },
|
||||||
|
]);
|
||||||
|
expect(fs.statSync(legacy).isDirectory()).toBe(true);
|
||||||
|
expect(fs.readFileSync(snapshot, "utf-8")).toBe("{}");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves non-empty unowned legacy state and ignores empty legacy containers", () => {
|
||||||
|
const legacy = path.join(directory, ".livesync", "runtime", "keyvalue-db.json");
|
||||||
|
fs.mkdirSync(path.dirname(legacy), { recursive: true });
|
||||||
|
fs.writeFileSync(legacy, "{}");
|
||||||
|
fs.writeFileSync(path.join(directory, ".livesync-snapshot.json"), "[]");
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([]);
|
||||||
|
const queued = JSON.stringify({ "queued-files": ["unique-edit.md"] });
|
||||||
|
fs.writeFileSync(legacy, queued);
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([{ path: legacy, reason: "unowned" }]);
|
||||||
|
expect(fs.readFileSync(legacy, "utf-8")).toBe(queued);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -9,6 +9,7 @@ import type { IVaultService } from "@vrtmrz/livesync-commonlib/compat/services/b
|
|||||||
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
|
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
|
||||||
import { createInstanceLogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
|
import { createInstanceLogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
|
||||||
import { fs as nodeFs, path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
import { fs as nodeFs, path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import { readOptionalFile, writeFileAtomically } from "@/apps/cli/runtimeState";
|
||||||
|
|
||||||
const NODE_KV_TYPED_KEY = "__nodeKvType";
|
const NODE_KV_TYPED_KEY = "__nodeKvType";
|
||||||
const NODE_KV_VALUES_KEY = "values";
|
const NODE_KV_VALUES_KEY = "values";
|
||||||
@@ -103,12 +104,11 @@ class NodeFileKeyValueDatabase implements KeyValueDatabase {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private load() {
|
private load() {
|
||||||
try {
|
const contents = readOptionalFile(this.filePath);
|
||||||
const loaded = JSON.parse(nodeFs.readFileSync(this.filePath, "utf-8")) as Record<string, unknown>;
|
if (contents === undefined) return;
|
||||||
|
const loaded: unknown = JSON.parse(contents);
|
||||||
|
if (!isRecord(loaded) || Array.isArray(loaded)) throw new Error("Invalid Node key-value database");
|
||||||
this.data = new Map(Object.entries(loaded).map(([key, value]) => [key, deserializeFromNodeKV(value)]));
|
this.data = new Map(Object.entries(loaded).map(([key, value]) => [key, deserializeFromNodeKV(value)]));
|
||||||
} catch {
|
|
||||||
this.data = new Map();
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private flush() {
|
private flush() {
|
||||||
@@ -116,7 +116,7 @@ class NodeFileKeyValueDatabase implements KeyValueDatabase {
|
|||||||
const serializable = Object.fromEntries(
|
const serializable = Object.fromEntries(
|
||||||
[...this.data.entries()].map(([key, value]) => [key, serializeForNodeKV(value)])
|
[...this.data.entries()].map(([key, value]) => [key, serializeForNodeKV(value)])
|
||||||
);
|
);
|
||||||
nodeFs.writeFileSync(this.filePath, JSON.stringify(serializable, null, 2), "utf-8");
|
writeFileAtomically(this.filePath, JSON.stringify(serializable, null, 2));
|
||||||
}
|
}
|
||||||
|
|
||||||
async get<T>(key: IDBValidKey): Promise<T> {
|
async get<T>(key: IDBValidKey): Promise<T> {
|
||||||
@@ -163,9 +163,7 @@ class NodeFileKeyValueDatabase implements KeyValueDatabase {
|
|||||||
return filtered;
|
return filtered;
|
||||||
}
|
}
|
||||||
|
|
||||||
async close(): Promise<void> {
|
async close(): Promise<void> {}
|
||||||
this.flush();
|
|
||||||
}
|
|
||||||
|
|
||||||
async destroy(): Promise<void> {
|
async destroy(): Promise<void> {
|
||||||
this.data.clear();
|
this.data.clear();
|
||||||
@@ -185,7 +183,8 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
{
|
{
|
||||||
private _kvDB: KeyValueDatabase | undefined;
|
private _kvDB: KeyValueDatabase | undefined;
|
||||||
private _simpleStore: SimpleStore<unknown> | undefined;
|
private _simpleStore: SimpleStore<unknown> | undefined;
|
||||||
private filePath: string;
|
private filePath: string | (() => string);
|
||||||
|
private openedFilePath: string | undefined;
|
||||||
private _log = createInstanceLogFunction("NodeKeyValueDBService");
|
private _log = createInstanceLogFunction("NodeKeyValueDBService");
|
||||||
|
|
||||||
get simpleStore() {
|
get simpleStore() {
|
||||||
@@ -202,7 +201,7 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
return this._kvDB;
|
return this._kvDB;
|
||||||
}
|
}
|
||||||
|
|
||||||
constructor(context: T, dependencies: NodeKeyValueDBDependencies<T>, filePath: string) {
|
constructor(context: T, dependencies: NodeKeyValueDBDependencies<T>, filePath: string | (() => string)) {
|
||||||
super(context);
|
super(context);
|
||||||
this.filePath = filePath;
|
this.filePath = filePath;
|
||||||
|
|
||||||
@@ -215,7 +214,13 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
|
|
||||||
private async openKeyValueDB(): Promise<boolean> {
|
private async openKeyValueDB(): Promise<boolean> {
|
||||||
try {
|
try {
|
||||||
this._kvDB = new NodeFileKeyValueDatabase(this.filePath);
|
const filePath = typeof this.filePath === "string" ? this.filePath : this.filePath();
|
||||||
|
if (this._kvDB && this.openedFilePath === filePath) return true;
|
||||||
|
await this._kvDB?.close();
|
||||||
|
this._kvDB = undefined;
|
||||||
|
this.openedFilePath = undefined;
|
||||||
|
this._kvDB = new NodeFileKeyValueDatabase(filePath);
|
||||||
|
this.openedFilePath = filePath;
|
||||||
return true;
|
return true;
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log("Failed to open Node key-value database", LOG_LEVEL_NOTICE);
|
this._log("Failed to open Node key-value database", LOG_LEVEL_NOTICE);
|
||||||
@@ -226,8 +231,10 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
|
|
||||||
private async _everyOnResetDatabase(): Promise<boolean> {
|
private async _everyOnResetDatabase(): Promise<boolean> {
|
||||||
try {
|
try {
|
||||||
await this._kvDB?.del("queued-files");
|
if (!(await this.openKeyValueDB())) return false;
|
||||||
await this._kvDB?.destroy();
|
await this._kvDB?.destroy();
|
||||||
|
this._kvDB = undefined;
|
||||||
|
this.openedFilePath = undefined;
|
||||||
return await this.openKeyValueDB();
|
return await this.openKeyValueDB();
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log("Failed to reset Node key-value database", LOG_LEVEL_NOTICE);
|
this._log("Failed to reset Node key-value database", LOG_LEVEL_NOTICE);
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { describe, expect, it, vi } from "vitest";
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
import { createServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
|
import { createServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
|
||||||
import type { NodeKeyValueDBDependencies } from "./NodeKeyValueDBService";
|
import type { NodeKeyValueDBDependencies } from "./NodeKeyValueDBService";
|
||||||
import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
|
import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
|
||||||
@@ -45,3 +46,70 @@ describe("NodeKeyValueDBService.openSimpleStore", () => {
|
|||||||
await expect(store.get("key")).rejects.toThrow("KeyValueDB is not initialized yet");
|
await expect(store.get("key")).rejects.toThrow("KeyValueDB is not initialized yet");
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("NodeKeyValueDBService persistence", () => {
|
||||||
|
let directory: string;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
directory = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-node-kv-"));
|
||||||
|
});
|
||||||
|
afterEach(() => fs.rmSync(directory, { recursive: true, force: true }));
|
||||||
|
|
||||||
|
function compose(filePath: string | (() => string)) {
|
||||||
|
const handlers: Record<string, () => Promise<boolean>> = {};
|
||||||
|
const event = (name: string) => ({
|
||||||
|
addHandler: (handler: () => Promise<boolean>) => {
|
||||||
|
handlers[name] = handler;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const dependencies = {
|
||||||
|
appLifecycle: { onSettingLoaded: event("load") },
|
||||||
|
databaseEvents: {
|
||||||
|
onResetDatabase: event("reset"),
|
||||||
|
onDatabaseInitialisation: event("initialise"),
|
||||||
|
onUnloadDatabase: event("unload"),
|
||||||
|
onCloseDatabase: event("close"),
|
||||||
|
},
|
||||||
|
vault: {},
|
||||||
|
} as unknown as NodeKeyValueDBDependencies;
|
||||||
|
return { service: new NodeKeyValueDBService(createServiceContext(), dependencies, filePath), handlers };
|
||||||
|
}
|
||||||
|
|
||||||
|
it("retargets existing SimpleStore handles after database selection and resets only the selected file", async () => {
|
||||||
|
let selected = path.join(directory, "first.json");
|
||||||
|
const { service, handlers } = compose(() => selected);
|
||||||
|
const store = service.openSimpleStore("journal");
|
||||||
|
expect(await handlers.load()).toBe(true);
|
||||||
|
await store.set("checkpoint", new Set(["first"]));
|
||||||
|
selected = path.join(directory, "second.json");
|
||||||
|
expect(await handlers.initialise()).toBe(true);
|
||||||
|
expect(await store.get("checkpoint")).toBeUndefined();
|
||||||
|
await store.set("checkpoint", "second");
|
||||||
|
expect(await handlers.reset()).toBe(true);
|
||||||
|
expect(await store.get("checkpoint")).toBeUndefined();
|
||||||
|
selected = path.join(directory, "first.json");
|
||||||
|
expect(await handlers.initialise()).toBe(true);
|
||||||
|
expect(await store.get("checkpoint")).toEqual(new Set(["first"]));
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not overwrite newer data when a stale handle closes", async () => {
|
||||||
|
const filePath = path.join(directory, "state.json");
|
||||||
|
const first = compose(filePath);
|
||||||
|
await first.handlers.load();
|
||||||
|
await first.service.kvDB.set("queued-files", ["original.md"]);
|
||||||
|
const second = compose(filePath);
|
||||||
|
await second.handlers.load();
|
||||||
|
await first.service.kvDB.set("queued-files", ["unique-edit.md"]);
|
||||||
|
await second.handlers.close();
|
||||||
|
expect(JSON.parse(fs.readFileSync(filePath, "utf-8"))["queued-files"]).toEqual(["unique-edit.md"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["{", "null", "[]"])("refuses corrupt state %s without replacing it", async (contents) => {
|
||||||
|
const filePath = path.join(directory, "state.json");
|
||||||
|
fs.writeFileSync(filePath, contents);
|
||||||
|
const { service, handlers } = compose(filePath);
|
||||||
|
expect(await handlers.load()).toBe(false);
|
||||||
|
expect(() => service.kvDB).toThrow("not initialized");
|
||||||
|
expect(fs.readFileSync(filePath, "utf-8")).toBe(contents);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -6,7 +6,8 @@ import { ServiceContext, type StandardIo } from "@vrtmrz/livesync-commonlib/cont
|
|||||||
export class NodeServiceContext extends ServiceContext {
|
export class NodeServiceContext extends ServiceContext {
|
||||||
constructor(
|
constructor(
|
||||||
readonly databasePath: string,
|
readonly databasePath: string,
|
||||||
readonly standardIo: StandardIo
|
readonly standardIo: StandardIo,
|
||||||
|
readonly runtimeVaultPath: string = databasePath
|
||||||
) {
|
) {
|
||||||
super({ events: eventHub, translate: translateLiveSyncMessage });
|
super({ events: eventHub, translate: translateLiveSyncMessage });
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -24,11 +24,14 @@ import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
|
|||||||
import { createNodeLocalStorage } from "./NodeLocalStorage";
|
import { createNodeLocalStorage } from "./NodeLocalStorage";
|
||||||
import { DatabaseService } from "@vrtmrz/livesync-commonlib/compat/services/base/DatabaseService";
|
import { DatabaseService } from "@vrtmrz/livesync-commonlib/compat/services/base/DatabaseService";
|
||||||
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
import { path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
import { fs, path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
||||||
import type { KeyValueDBService } from "@vrtmrz/livesync-commonlib/compat/services/base/KeyValueDBService";
|
import type { KeyValueDBService } from "@vrtmrz/livesync-commonlib/compat/services/base/KeyValueDBService";
|
||||||
import { PouchDB } from "@/apps/cli/lib/pouchdb-node";
|
import { PouchDB } from "@/apps/cli/lib/pouchdb-node";
|
||||||
import { NodeServiceContext } from "./NodeServiceContext";
|
import { NodeServiceContext } from "./NodeServiceContext";
|
||||||
import { setLang } from "@/common/translation";
|
import { setLang } from "@/common/translation";
|
||||||
|
import { runtimeDatabaseDirectory, runtimeScopeDirectory } from "@/apps/cli/runtimeState";
|
||||||
|
import { createDeletableValueStore } from "@vrtmrz/value-store";
|
||||||
|
import type { SettingServiceDependencies } from "@vrtmrz/livesync-commonlib/compat/services/base/SettingService";
|
||||||
|
|
||||||
export { NodeServiceContext } from "./NodeServiceContext";
|
export { NodeServiceContext } from "./NodeServiceContext";
|
||||||
|
|
||||||
@@ -86,17 +89,31 @@ class NodeUIService<T extends ServiceContext> extends UIService<T> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
class NodeSettingService<T extends NodeServiceContext> extends InjectableSettingService<T> {
|
||||||
|
declare readonly config: InjectableSettingService<T>["config"];
|
||||||
|
|
||||||
|
constructor(context: T, dependencies: SettingServiceDependencies) {
|
||||||
|
super(context, dependencies);
|
||||||
|
const databaseConfig = this.config;
|
||||||
|
const vaultScope = nodePath.basename(runtimeScopeDirectory(context.databasePath, "", context.runtimeVaultPath));
|
||||||
|
this.config = createDeletableValueStore({
|
||||||
|
get: (key) => databaseConfig.getValue(`${key}-${vaultScope}`),
|
||||||
|
set: (key, value) => databaseConfig.setValue(`${key}-${vaultScope}`, value),
|
||||||
|
delete: (key) => databaseConfig.delete(`${key}-${vaultScope}`),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServiceHub<T> {
|
export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServiceHub<T> {
|
||||||
constructor(basePath: string, context: T) {
|
constructor(basePath: string, context: T) {
|
||||||
const runtimeDir = nodePath.join(basePath, ".livesync", "runtime");
|
const runtimeDir = nodePath.join(basePath, ".livesync", "runtime");
|
||||||
const localStoragePath = nodePath.join(runtimeDir, "local-storage.json");
|
const localStoragePath = nodePath.join(runtimeDir, "local-storage.json");
|
||||||
const keyValueDBPath = nodePath.join(runtimeDir, "keyvalue-db.json");
|
|
||||||
|
|
||||||
const API = new HeadlessAPIService<T>(context);
|
const API = new HeadlessAPIService<T>(context);
|
||||||
const conflict = new InjectableConflictService(context);
|
const conflict = new InjectableConflictService(context);
|
||||||
const fileProcessing = new InjectableFileProcessingService(context);
|
const fileProcessing = new InjectableFileProcessingService(context);
|
||||||
|
|
||||||
const setting = new InjectableSettingService<T>(context, {
|
const setting = new NodeSettingService<T>(context, {
|
||||||
APIService: API,
|
APIService: API,
|
||||||
onDisplayLanguageChanged: setLang,
|
onDisplayLanguageChanged: setLang,
|
||||||
deviceValues: createNodeLocalStorage(localStoragePath),
|
deviceValues: createNodeLocalStorage(localStoragePath),
|
||||||
@@ -118,8 +135,13 @@ export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServ
|
|||||||
settingService: setting,
|
settingService: setting,
|
||||||
APIService: API,
|
APIService: API,
|
||||||
});
|
});
|
||||||
|
const scopeDirectory = () => runtimeScopeDirectory(basePath, vault.getVaultName(), context.runtimeVaultPath);
|
||||||
const test = new InjectableTestService(context);
|
const test = new InjectableTestService(context);
|
||||||
const databaseEvents = new InjectableDatabaseEventService(context);
|
const databaseEvents = new InjectableDatabaseEventService(context);
|
||||||
|
databaseEvents.onResetDatabase.addHandler(async () => {
|
||||||
|
fs.rmSync(runtimeDatabaseDirectory(basePath, vault.getVaultName()), { force: true, recursive: true });
|
||||||
|
return true;
|
||||||
|
});
|
||||||
const path = new PathServiceCompat(context, {
|
const path = new PathServiceCompat(context, {
|
||||||
settingService: setting,
|
settingService: setting,
|
||||||
});
|
});
|
||||||
@@ -156,7 +178,7 @@ export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServ
|
|||||||
databaseEvents,
|
databaseEvents,
|
||||||
vault,
|
vault,
|
||||||
},
|
},
|
||||||
keyValueDBPath
|
() => nodePath.join(scopeDirectory(), "keyvalue-db.json")
|
||||||
);
|
);
|
||||||
|
|
||||||
const control = new ControlService(context, {
|
const control = new ControlService(context, {
|
||||||
|
|||||||
@@ -77,6 +77,32 @@ npm run test:e2e:docker:mirror --workspace self-hosted-livesync-cli
|
|||||||
|
|
||||||
The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Node.js and Bash suites.
|
The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Node.js and Bash suites.
|
||||||
|
|
||||||
|
## CLI Fetch database continuity
|
||||||
|
|
||||||
|
After building the CLI, run `node test/e2e-cli/run-task.ts test:fetch-database-continuity`
|
||||||
|
with Node.js 26 and a disposable S3-compatible service. Set
|
||||||
|
`LIVESYNC_ISSUE1304_MINIO_ENDPOINT`, `LIVESYNC_ISSUE1304_MINIO_ACCESS_KEY`,
|
||||||
|
`LIVESYNC_ISSUE1304_MINIO_SECRET_KEY`, and `LIVESYNC_ISSUE1304_MINIO_BUCKET`.
|
||||||
|
The bucket must be fresh and empty for every run. Set
|
||||||
|
`LIVESYNC_ISSUE1304_EVIDENCE_DIR` to an evidence directory outside the worktree.
|
||||||
|
This test writes remote data; never point it at a production bucket.
|
||||||
|
|
||||||
|
The test seeds an encrypted, path-obfuscated file with a second-generation
|
||||||
|
revision, imports one Setup URI on two clients, fetches on A without
|
||||||
|
`--write-settings`, and restarts A while B remains online. It checks the saved
|
||||||
|
database suffix, file bytes, original revision, and absence of a duplicate root.
|
||||||
|
Set `LIVESYNC_ISSUE1304_POSITIVE_CONTROL=1` and
|
||||||
|
`LIVESYNC_ISSUE1304_POSITIVE_CONTROL_BUCKET` to a second fresh bucket to exercise
|
||||||
|
the explicit `--write-settings` variant as well.
|
||||||
|
|
||||||
|
For historical reproduction against the unmodified implementation, set
|
||||||
|
`LIVESYNC_ISSUE1304_VERIFY_LEGACY_FAILURE=1`. This preserves the original
|
||||||
|
same-content conflict-resolution sequence and its file-preservation assertion;
|
||||||
|
the affected implementation fails because A's file disappears. Leave this
|
||||||
|
variable unset when validating the correction. The test saves redacted process
|
||||||
|
logs, file hashes, and database evidence. It does not cover Obsidian runtime or
|
||||||
|
CouchDB behaviour.
|
||||||
|
|
||||||
## Multiple-device conflict regression tests
|
## Multiple-device conflict regression tests
|
||||||
|
|
||||||
### Preparation and execution
|
### Preparation and execution
|
||||||
|
|||||||
@@ -47,6 +47,11 @@
|
|||||||
"test-settings-writeback.ts"
|
"test-settings-writeback.ts"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
"test:fetch-database-continuity": {
|
||||||
|
"test": [
|
||||||
|
"test-fetch-database-continuity.ts"
|
||||||
|
]
|
||||||
|
},
|
||||||
"test:remote-administration-exit-codes": {
|
"test:remote-administration-exit-codes": {
|
||||||
"test": [
|
"test": [
|
||||||
"test-remote-administration-exit-codes.ts"
|
"test-remote-administration-exit-codes.ts"
|
||||||
|
|||||||
@@ -0,0 +1,568 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { cp, mkdir, readFile, readdir, writeFile } from "node:fs/promises";
|
||||||
|
import { join, resolve } from "node:path";
|
||||||
|
import { test } from "node:test";
|
||||||
|
import { CLI_DIR, CLI_DIST } from "./helpers/cli.ts";
|
||||||
|
import { applyRemoteSyncSettings, generateSetupUriFromSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||||
|
import { TempDir } from "./helpers/temp.ts";
|
||||||
|
import { spawnProcess, type RunningProcess } from "./helpers/process.ts";
|
||||||
|
|
||||||
|
type RemoteSettings = {
|
||||||
|
endpoint: string;
|
||||||
|
accessKey: string;
|
||||||
|
secretKey: string;
|
||||||
|
bucket: string;
|
||||||
|
evidenceDir: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type CliResult = { code: number; stdout: string; stderr: string; combined: string };
|
||||||
|
type ScenarioMode = "reproduction" | "write-settings-control";
|
||||||
|
|
||||||
|
const SETUP_PASSPHRASE = "issue-1304-setup-passphrase";
|
||||||
|
const RESOLUTION_MARKER = "Resolving [same]: Conflicted revision has been deleted";
|
||||||
|
const INITIAL_REPLICATION_COMPLETE = "[Daemon] Initial replication complete";
|
||||||
|
const INITIAL_REPLICATION_FAILED = "[Daemon] Initial replication failed";
|
||||||
|
const VERIFY_LEGACY_FAILURE = process.env["LIVESYNC_ISSUE1304_VERIFY_LEGACY_FAILURE"] === "1";
|
||||||
|
|
||||||
|
function requiredEnv(name: string): string {
|
||||||
|
const value = process.env[name]?.trim();
|
||||||
|
if (!value) throw new Error(`Required environment variable is missing: ${name}`);
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function remoteSettings(bucketOverride?: string): RemoteSettings {
|
||||||
|
return {
|
||||||
|
endpoint: requiredEnv("LIVESYNC_ISSUE1304_MINIO_ENDPOINT").replace(/\/$/, ""),
|
||||||
|
accessKey: requiredEnv("LIVESYNC_ISSUE1304_MINIO_ACCESS_KEY"),
|
||||||
|
secretKey: requiredEnv("LIVESYNC_ISSUE1304_MINIO_SECRET_KEY"),
|
||||||
|
bucket: bucketOverride ?? requiredEnv("LIVESYNC_ISSUE1304_MINIO_BUCKET"),
|
||||||
|
evidenceDir: resolve(requiredEnv("LIVESYNC_ISSUE1304_EVIDENCE_DIR")),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function redactText(value: string, secrets: readonly string[]): string {
|
||||||
|
let redacted = value;
|
||||||
|
for (const secret of secrets) {
|
||||||
|
if (!secret) continue;
|
||||||
|
redacted = redacted.split(secret).join("[redacted]");
|
||||||
|
const encoded = encodeURIComponent(secret);
|
||||||
|
if (encoded !== secret) redacted = redacted.split(encoded).join("[redacted]");
|
||||||
|
}
|
||||||
|
return redacted;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function streamText(stream: ReadableStream<Uint8Array>, onText: (text: string) => void): Promise<void> {
|
||||||
|
const reader = stream.getReader();
|
||||||
|
const decoder = new TextDecoder();
|
||||||
|
try {
|
||||||
|
while (true) {
|
||||||
|
const { done, value } = await reader.read();
|
||||||
|
if (done) break;
|
||||||
|
if (value) onText(decoder.decode(value, { stream: true }));
|
||||||
|
}
|
||||||
|
const tail = decoder.decode();
|
||||||
|
if (tail) onText(tail);
|
||||||
|
} finally {
|
||||||
|
reader.releaseLock();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
class CapturedCliProcess {
|
||||||
|
readonly child: RunningProcess;
|
||||||
|
#stdout = "";
|
||||||
|
#stderr = "";
|
||||||
|
#secrets = [
|
||||||
|
process.env["LIVESYNC_ISSUE1304_MINIO_ENDPOINT"] ?? "",
|
||||||
|
process.env["LIVESYNC_ISSUE1304_MINIO_ACCESS_KEY"] ?? "",
|
||||||
|
process.env["LIVESYNC_ISSUE1304_MINIO_SECRET_KEY"] ?? "",
|
||||||
|
SETUP_PASSPHRASE,
|
||||||
|
"issue-1304-content-passphrase",
|
||||||
|
];
|
||||||
|
#streamsDone: Promise<void>;
|
||||||
|
|
||||||
|
constructor(args: string[]) {
|
||||||
|
this.child = spawnProcess(process.execPath, {
|
||||||
|
args: [CLI_DIST, ...args],
|
||||||
|
cwd: CLI_DIR,
|
||||||
|
stdin: "null",
|
||||||
|
stdout: "piped",
|
||||||
|
stderr: "piped",
|
||||||
|
});
|
||||||
|
this.#streamsDone = Promise.all([
|
||||||
|
streamText(this.child.stdout, (text) => (this.#stdout += text)),
|
||||||
|
streamText(this.child.stderr, (text) => (this.#stderr += text)),
|
||||||
|
]).then(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
get combined(): string {
|
||||||
|
return this.#stdout + this.#stderr;
|
||||||
|
}
|
||||||
|
|
||||||
|
async waitUntil(predicate: (output: string) => boolean, description: string, timeoutMs = 60_000): Promise<void> {
|
||||||
|
const deadline = Date.now() + timeoutMs;
|
||||||
|
while (Date.now() < deadline) {
|
||||||
|
if (predicate(this.combined)) return;
|
||||||
|
const tick = Math.min(100, deadline - Date.now());
|
||||||
|
const event = await Promise.race([
|
||||||
|
this.child.status.then((status) => ({ kind: "exit" as const, status })),
|
||||||
|
new Promise<{ kind: "tick" }>((resolveTick) => setTimeout(() => resolveTick({ kind: "tick" }), tick)),
|
||||||
|
]);
|
||||||
|
if (event.kind === "exit") {
|
||||||
|
await this.#streamsDone;
|
||||||
|
if (predicate(this.combined)) return;
|
||||||
|
throw new Error(
|
||||||
|
`${description}; process exited with ${event.status.code}\n${redactText(this.combined, this.#secrets).slice(-5000)}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw new Error(`Timed out waiting for ${description}\n${redactText(this.combined, this.#secrets).slice(-5000)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async stop(): Promise<number> {
|
||||||
|
try {
|
||||||
|
this.child.kill("SIGTERM");
|
||||||
|
} catch {
|
||||||
|
// A completed process needs no further signal.
|
||||||
|
}
|
||||||
|
const status = await this.child.status;
|
||||||
|
await this.#streamsDone;
|
||||||
|
return status.code;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runCliQuietly(args: string[], input?: string): Promise<CliResult> {
|
||||||
|
const child = spawnProcess(process.execPath, {
|
||||||
|
args: [CLI_DIST, ...args],
|
||||||
|
cwd: CLI_DIR,
|
||||||
|
stdin: input === undefined ? "null" : "piped",
|
||||||
|
stdout: "piped",
|
||||||
|
stderr: "piped",
|
||||||
|
});
|
||||||
|
let stdout = "";
|
||||||
|
let stderr = "";
|
||||||
|
const streamsDone = Promise.all([
|
||||||
|
streamText(child.stdout, (text) => (stdout += text)),
|
||||||
|
streamText(child.stderr, (text) => (stderr += text)),
|
||||||
|
]);
|
||||||
|
if (input !== undefined) {
|
||||||
|
const writer = child.stdin.getWriter();
|
||||||
|
await writer.write(new TextEncoder().encode(input));
|
||||||
|
await writer.close();
|
||||||
|
}
|
||||||
|
const [status] = await Promise.all([child.status, streamsDone]);
|
||||||
|
return { code: status.code, stdout, stderr, combined: stdout + stderr };
|
||||||
|
}
|
||||||
|
|
||||||
|
function countOccurrences(value: string, needle: string): number {
|
||||||
|
return value.split(needle).length - 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
function safeCommand(
|
||||||
|
label: string,
|
||||||
|
result: CliResult,
|
||||||
|
commandLog: Array<{ label: string; code: number; output: string }>,
|
||||||
|
secrets: readonly string[]
|
||||||
|
): void {
|
||||||
|
const output = redactText(result.combined, secrets);
|
||||||
|
commandLog.push({ label, code: result.code, output });
|
||||||
|
assert.equal(result.code, 0, `${label} failed with exit code ${result.code}\n${output.slice(-5000)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fileState(filePath: string): Promise<{ exists: boolean; bytes?: number; sha256?: string }> {
|
||||||
|
try {
|
||||||
|
const contents = await readFile(filePath);
|
||||||
|
return {
|
||||||
|
exists: true,
|
||||||
|
bytes: contents.byteLength,
|
||||||
|
sha256: createHash("sha256").update(contents).digest("hex"),
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return { exists: false };
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function collectDatabaseTree(root: string, relative = ""): Promise<Array<{ path: string; bytes: number; sha256: string }>> {
|
||||||
|
const result: Array<{ path: string; bytes: number; sha256: string }> = [];
|
||||||
|
let entries;
|
||||||
|
try {
|
||||||
|
entries = await readdir(join(root, relative), { withFileTypes: true });
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return result;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
entries.sort((left, right) => left.name.localeCompare(right.name));
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (!relative && entry.name === ".livesync") continue;
|
||||||
|
const child = relative ? join(relative, entry.name) : entry.name;
|
||||||
|
const absolute = join(root, child);
|
||||||
|
if (entry.isDirectory()) {
|
||||||
|
result.push(...(await collectDatabaseTree(root, child)));
|
||||||
|
} else if (entry.isFile()) {
|
||||||
|
const contents = await readFile(absolute);
|
||||||
|
result.push({
|
||||||
|
path: child,
|
||||||
|
bytes: contents.byteLength,
|
||||||
|
sha256: createHash("sha256").update(contents).digest("hex"),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function copyDatabaseTree(source: string, destination: string): Promise<void> {
|
||||||
|
await mkdir(destination, { recursive: true });
|
||||||
|
let entries;
|
||||||
|
try {
|
||||||
|
entries = await readdir(source, { withFileTypes: true });
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (entry.name === ".livesync") continue;
|
||||||
|
await cp(join(source, entry.name), join(destination, entry.name), { recursive: true, errorOnExist: false });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function sanitisedRuntimeState(root: string, secrets: readonly string[]): Promise<Record<string, string>> {
|
||||||
|
const runtimeRoot = join(root, ".livesync", "runtime");
|
||||||
|
const files: Record<string, string> = {};
|
||||||
|
let entries;
|
||||||
|
try {
|
||||||
|
entries = await readdir(runtimeRoot, { withFileTypes: true });
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return files;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (!entry.isFile() || !entry.name.endsWith(".json")) continue;
|
||||||
|
const text = await readFile(join(runtimeRoot, entry.name), "utf8");
|
||||||
|
try {
|
||||||
|
files[entry.name] = JSON.stringify(JSON.parse(text), (_key, value: unknown) => {
|
||||||
|
return typeof value === "string" ? redactText(value, secrets) : value;
|
||||||
|
}, 2);
|
||||||
|
} catch {
|
||||||
|
files[entry.name] = redactText(text, secrets);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return files;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function settingsSummary(settingsPath: string): Promise<Record<string, unknown> | null> {
|
||||||
|
try {
|
||||||
|
const settings = JSON.parse(await readFile(settingsPath, "utf8")) as Record<string, unknown>;
|
||||||
|
return {
|
||||||
|
additionalSuffixOfDatabaseName: settings.additionalSuffixOfDatabaseName ?? null,
|
||||||
|
remoteType: settings.remoteType ?? null,
|
||||||
|
liveSync: settings.liveSync ?? null,
|
||||||
|
syncOnStart: settings.syncOnStart ?? null,
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return null;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runScenario(mode: ScenarioMode, bucket: string): Promise<void> {
|
||||||
|
const remote = remoteSettings(bucket);
|
||||||
|
const evidenceRoot = remote.evidenceDir;
|
||||||
|
const evidenceName = `${mode}-${Date.now()}-${process.pid}`;
|
||||||
|
const evidenceDir = join(evidenceRoot, evidenceName);
|
||||||
|
const workDir = await TempDir.create(`livesync-cli-issue1304-${mode}`);
|
||||||
|
const scenarioId = `${Date.now()}-${process.pid}`;
|
||||||
|
const suffix = `issue1304-${scenarioId}`;
|
||||||
|
const targetPath = `notes/issue1304-${scenarioId}/continuity.md`;
|
||||||
|
const initialContent = `issue-1304 initial ${scenarioId}\n`;
|
||||||
|
const finalContent = `issue-1304 final ${scenarioId}\n`;
|
||||||
|
const finalBytes = new TextEncoder().encode(finalContent);
|
||||||
|
const secrets = [remote.endpoint, remote.accessKey, remote.secretKey, SETUP_PASSPHRASE, "issue-1304-content-passphrase"];
|
||||||
|
const commandLog: Array<{ label: string; code: number; output: string }> = [];
|
||||||
|
const processes: Record<string, CapturedCliProcess | undefined> = {};
|
||||||
|
const settingsTimeline: Record<string, unknown> = {};
|
||||||
|
const postResolutionInfo: Record<string, Record<string, unknown> | null> = {};
|
||||||
|
let observedFinalFileA: { exists: boolean; bytes?: number; sha256?: string } | undefined;
|
||||||
|
let observedFinalContentA: string | undefined;
|
||||||
|
let setupUri: string | undefined;
|
||||||
|
let failure: unknown;
|
||||||
|
|
||||||
|
const databaseSeed = workDir.join("database-seed");
|
||||||
|
const databaseA = workDir.join("database-a");
|
||||||
|
const databaseB = workDir.join("database-b");
|
||||||
|
const vaultSeed = workDir.join("vault-seed");
|
||||||
|
const vaultA = workDir.join("vault-a");
|
||||||
|
const vaultB = workDir.join("vault-b");
|
||||||
|
const settingsSeed = workDir.join("settings-seed.json");
|
||||||
|
const settingsA = workDir.join("settings-a.json");
|
||||||
|
const settingsB = workDir.join("settings-b.json");
|
||||||
|
const finalFileA = join(vaultA, targetPath);
|
||||||
|
const finalFileB = join(vaultB, targetPath);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await Promise.all([
|
||||||
|
mkdir(databaseSeed, { recursive: true }),
|
||||||
|
mkdir(databaseA, { recursive: true }),
|
||||||
|
mkdir(databaseB, { recursive: true }),
|
||||||
|
mkdir(vaultSeed, { recursive: true }),
|
||||||
|
mkdir(vaultA, { recursive: true }),
|
||||||
|
mkdir(vaultB, { recursive: true }),
|
||||||
|
]);
|
||||||
|
await initSettingsFile(settingsSeed);
|
||||||
|
await applyRemoteSyncSettings(settingsSeed, {
|
||||||
|
remoteType: "MINIO",
|
||||||
|
minioBucket: remote.bucket,
|
||||||
|
minioEndpoint: remote.endpoint,
|
||||||
|
minioAccessKey: remote.accessKey,
|
||||||
|
minioSecretKey: remote.secretKey,
|
||||||
|
encrypt: true,
|
||||||
|
passphrase: "issue-1304-content-passphrase",
|
||||||
|
usePathObfuscation: true,
|
||||||
|
});
|
||||||
|
const seedSettings = JSON.parse(await readFile(settingsSeed, "utf8")) as Record<string, unknown>;
|
||||||
|
seedSettings.additionalSuffixOfDatabaseName = suffix;
|
||||||
|
await writeFile(settingsSeed, JSON.stringify(seedSettings, null, 2));
|
||||||
|
settingsTimeline.setupSuffix = suffix;
|
||||||
|
setupUri = await generateSetupUriFromSettings(settingsSeed, SETUP_PASSPHRASE, true);
|
||||||
|
secrets.push(setupUri);
|
||||||
|
|
||||||
|
const seedRun = async (label: string, args: string[], input?: string) => {
|
||||||
|
const result = await runCliQuietly(args, input);
|
||||||
|
safeCommand(label, result, commandLog, secrets);
|
||||||
|
return result;
|
||||||
|
};
|
||||||
|
|
||||||
|
await seedRun("seed-put-initial", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "put", targetPath], initialContent);
|
||||||
|
await seedRun("seed-sync-initial", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "sync"]);
|
||||||
|
await seedRun("seed-put-final", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "put", targetPath], finalContent);
|
||||||
|
await seedRun("seed-sync-final", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "sync"]);
|
||||||
|
const seedInfoResult = await seedRun("seed-info", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "info", targetPath]);
|
||||||
|
const seedInfo = JSON.parse(seedInfoResult.stdout) as { revision?: string; revisions?: string[] };
|
||||||
|
const seedRevisionGeneration = Number(seedInfo.revision?.split("-")[0] ?? 0);
|
||||||
|
assert(
|
||||||
|
seedRevisionGeneration >= 2 && (seedInfo.revisions?.length ?? 0) >= 1,
|
||||||
|
`Expected a remote seed revision with generation >= 2; got ${JSON.stringify({ revision: seedInfo.revision, revisions: seedInfo.revisions })}`
|
||||||
|
);
|
||||||
|
settingsTimeline.seedRevision = seedInfo.revision;
|
||||||
|
|
||||||
|
await initSettingsFile(settingsA);
|
||||||
|
await initSettingsFile(settingsB);
|
||||||
|
const setupA = await runCliQuietly(
|
||||||
|
[databaseA, "--vault", vaultA, "--settings", settingsA, "setup", setupUri],
|
||||||
|
`${SETUP_PASSPHRASE}\n`
|
||||||
|
);
|
||||||
|
safeCommand("setup-client-a", setupA, commandLog, secrets);
|
||||||
|
const setupB = await runCliQuietly(
|
||||||
|
[databaseB, "--vault", vaultB, "--settings", settingsB, "setup", setupUri],
|
||||||
|
`${SETUP_PASSPHRASE}\n`
|
||||||
|
);
|
||||||
|
safeCommand("setup-client-b", setupB, commandLog, secrets);
|
||||||
|
|
||||||
|
const bFetchArgs = [databaseB, "--vault", vaultB, "--settings", settingsB, "--force", "fetch", "--write-settings"];
|
||||||
|
const bFetch = await runCliQuietly(bFetchArgs);
|
||||||
|
safeCommand("fetch-client-b", bFetch, commandLog, secrets);
|
||||||
|
assert.equal(await readFile(finalFileB, "utf8"), finalContent, "Client B did not fetch the final file bytes");
|
||||||
|
settingsTimeline.clientBSuffixAfterFetch = (await settingsSummary(settingsB))?.additionalSuffixOfDatabaseName;
|
||||||
|
assert.equal(settingsTimeline.clientBSuffixAfterFetch, "headless-app");
|
||||||
|
|
||||||
|
// Client B remains live so it can resolve the equal-content root after A
|
||||||
|
// scans its old database and uploads a new root.
|
||||||
|
processes.clientB = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseB,
|
||||||
|
"--vault",
|
||||||
|
vaultB,
|
||||||
|
"--settings",
|
||||||
|
settingsB,
|
||||||
|
"--interval",
|
||||||
|
"1",
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientB.waitUntil((output) => output.includes(INITIAL_REPLICATION_COMPLETE), "Client B initial replication");
|
||||||
|
|
||||||
|
settingsTimeline.clientASuffixBeforeFetch = (await settingsSummary(settingsA))?.additionalSuffixOfDatabaseName;
|
||||||
|
assert.equal(settingsTimeline.clientASuffixBeforeFetch, suffix);
|
||||||
|
const aFetchArgs = [databaseA, "--vault", vaultA, "--settings", settingsA, "--force", "fetch"];
|
||||||
|
if (mode === "write-settings-control") aFetchArgs.push("--write-settings");
|
||||||
|
const aFetch = await runCliQuietly(aFetchArgs);
|
||||||
|
safeCommand("fetch-client-a", aFetch, commandLog, secrets);
|
||||||
|
assert.equal(await readFile(finalFileA, "utf8"), finalContent, "Client A did not fetch the final file bytes");
|
||||||
|
settingsTimeline.clientASuffixAfterFetch = (await settingsSummary(settingsA))?.additionalSuffixOfDatabaseName;
|
||||||
|
assert.equal(
|
||||||
|
settingsTimeline.clientASuffixAfterFetch,
|
||||||
|
mode === "reproduction" && VERIFY_LEGACY_FAILURE ? suffix : "headless-app",
|
||||||
|
"Client A settings suffix did not match the selected fetch mode"
|
||||||
|
);
|
||||||
|
|
||||||
|
// This startup selects A's persisted suffix. In the reproduction mode it
|
||||||
|
// scans the old, empty database while the fetched bytes are in the vault.
|
||||||
|
processes.clientAStartup = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseA,
|
||||||
|
"--vault",
|
||||||
|
vaultA,
|
||||||
|
"--settings",
|
||||||
|
settingsA,
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientAStartup.waitUntil(
|
||||||
|
(output) => output.includes(INITIAL_REPLICATION_COMPLETE) || output.includes(INITIAL_REPLICATION_FAILED),
|
||||||
|
"Client A startup replication result",
|
||||||
|
45_000
|
||||||
|
);
|
||||||
|
await processes.clientAStartup.stop();
|
||||||
|
|
||||||
|
const markResolved = await runCliQuietly([databaseA, "--vault", vaultA, "--settings", settingsA, "mark-resolved"]);
|
||||||
|
safeCommand("mark-resolved-client-a", markResolved, commandLog, secrets);
|
||||||
|
|
||||||
|
if (mode === "reproduction" && VERIFY_LEGACY_FAILURE) {
|
||||||
|
processes.clientA = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseA,
|
||||||
|
"--vault",
|
||||||
|
vaultA,
|
||||||
|
"--settings",
|
||||||
|
settingsA,
|
||||||
|
"--interval",
|
||||||
|
"1",
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientA.waitUntil((output) => output.includes(INITIAL_REPLICATION_COMPLETE), "Client A interval daemon initial replication");
|
||||||
|
|
||||||
|
const processingMarker = `Processing ${targetPath}`;
|
||||||
|
const aProcessingBeforeResolution = countOccurrences(processes.clientA.combined, processingMarker);
|
||||||
|
await processes.clientB.waitUntil((output) => output.includes(RESOLUTION_MARKER), "Client B automatic same-content resolution");
|
||||||
|
await processes.clientA.waitUntil(
|
||||||
|
(output) => countOccurrences(output, processingMarker) > aProcessingBeforeResolution,
|
||||||
|
"Client A to reflect the post-resolution revision",
|
||||||
|
45_000
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
processes.clientA = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseA,
|
||||||
|
"--vault",
|
||||||
|
vaultA,
|
||||||
|
"--settings",
|
||||||
|
settingsA,
|
||||||
|
"--interval",
|
||||||
|
"1",
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientA.waitUntil((output) => output.includes(INITIAL_REPLICATION_COMPLETE), "Client A control daemon initial replication");
|
||||||
|
await new Promise((resolveDelay) => setTimeout(resolveDelay, 2_500));
|
||||||
|
assert.equal(await readFile(finalFileA, "utf8"), finalContent, "Client A control lost the fetched file");
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
failure = error;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const processCapture of Object.values(processes)) {
|
||||||
|
if (!processCapture) continue;
|
||||||
|
try {
|
||||||
|
await processCapture.stop();
|
||||||
|
} catch (error) {
|
||||||
|
if (failure === undefined) failure = error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await mkdir(evidenceDir, { recursive: true });
|
||||||
|
for (const [name, database, vault, settingsPath] of [
|
||||||
|
["clientA", databaseA, vaultA, settingsA],
|
||||||
|
["clientB", databaseB, vaultB, settingsB],
|
||||||
|
] as const) {
|
||||||
|
const result = await runCliQuietly([database, "--vault", vault, "--settings", settingsPath, "info", targetPath]);
|
||||||
|
const output = redactText(result.combined, secrets);
|
||||||
|
commandLog.push({ label: `post-resolution-info-${name}`, code: result.code, output });
|
||||||
|
if (result.code === 0) {
|
||||||
|
try {
|
||||||
|
postResolutionInfo[name] = JSON.parse(result.stdout) as Record<string, unknown>;
|
||||||
|
} catch {
|
||||||
|
postResolutionInfo[name] = null;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
postResolutionInfo[name] = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
observedFinalFileA = await fileState(finalFileA);
|
||||||
|
if (observedFinalFileA.exists) observedFinalContentA = await readFile(finalFileA, "utf8");
|
||||||
|
const databaseTrees: Record<string, Array<{ path: string; bytes: number; sha256: string }>> = {};
|
||||||
|
const runtimeState: Record<string, Record<string, string>> = {};
|
||||||
|
const databasePaths = { seed: databaseSeed, clientA: databaseA, clientB: databaseB };
|
||||||
|
for (const [name, path] of Object.entries(databasePaths)) {
|
||||||
|
databaseTrees[name] = await collectDatabaseTree(path);
|
||||||
|
await copyDatabaseTree(path, join(evidenceDir, `database-${name}`));
|
||||||
|
runtimeState[name] = await sanitisedRuntimeState(path, secrets);
|
||||||
|
if (Object.keys(runtimeState[name]).length > 0) {
|
||||||
|
const runtimeDir = join(evidenceDir, `runtime-${name}`);
|
||||||
|
await mkdir(runtimeDir, { recursive: true });
|
||||||
|
for (const [fileName, contents] of Object.entries(runtimeState[name])) {
|
||||||
|
await writeFile(join(runtimeDir, fileName), contents);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const logs = Object.fromEntries(
|
||||||
|
Object.entries(processes).map(([name, processCapture]) => [
|
||||||
|
name,
|
||||||
|
processCapture ? redactText(processCapture.combined, secrets) : "",
|
||||||
|
])
|
||||||
|
);
|
||||||
|
const fileStates = {
|
||||||
|
clientA: await fileState(finalFileA),
|
||||||
|
clientB: await fileState(finalFileB),
|
||||||
|
};
|
||||||
|
const evidence = {
|
||||||
|
scenario: mode,
|
||||||
|
targetPath,
|
||||||
|
expectedContentBytes: finalBytes.byteLength,
|
||||||
|
expectedContentSha256: createHash("sha256").update(finalBytes).digest("hex"),
|
||||||
|
settingsTimeline,
|
||||||
|
settings: {
|
||||||
|
seed: await settingsSummary(settingsSeed),
|
||||||
|
clientA: await settingsSummary(settingsA),
|
||||||
|
clientB: await settingsSummary(settingsB),
|
||||||
|
},
|
||||||
|
fileStates,
|
||||||
|
postResolutionInfo,
|
||||||
|
databaseTrees,
|
||||||
|
runtimeState,
|
||||||
|
commands: commandLog,
|
||||||
|
processLogs: logs,
|
||||||
|
remote: { endpoint: "[configured]", bucket: remote.bucket },
|
||||||
|
failure: failure instanceof Error ? redactText(failure.stack ?? failure.message, secrets) : String(failure ?? ""),
|
||||||
|
};
|
||||||
|
await writeFile(join(evidenceDir, "evidence.json"), JSON.stringify(evidence, null, 2));
|
||||||
|
} catch (error) {
|
||||||
|
if (failure === undefined) failure = error;
|
||||||
|
else if (failure instanceof Error) {
|
||||||
|
failure = new Error(`${failure.message}\nEvidence capture failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
await workDir[Symbol.asyncDispose]();
|
||||||
|
if (failure !== undefined) throw failure;
|
||||||
|
assert.equal(observedFinalFileA?.exists, true, "Client A file is absent after the resolved journal was applied");
|
||||||
|
assert.equal(observedFinalContentA, finalContent, "Client A lost the fetched file after remote conflict resolution");
|
||||||
|
if (!VERIFY_LEGACY_FAILURE) {
|
||||||
|
const infoA = postResolutionInfo.clientA;
|
||||||
|
const infoB = postResolutionInfo.clientB;
|
||||||
|
assert(infoA && infoB, "Could not inspect both local databases after the remote resolution");
|
||||||
|
assert.equal(infoA.conflicts, "N/A", "Client A still has a duplicate root revision after the resolution");
|
||||||
|
assert.equal(infoA.revision, infoB.revision, "Client A did not apply the revision selected by Client B after the resolved journal");
|
||||||
|
assert.equal(infoA.revision, settingsTimeline.seedRevision, "Daemon startup replaced the fetched revision ancestry");
|
||||||
|
assert.equal(processes.clientB?.combined.includes(RESOLUTION_MARKER), false, "Client A created an unnecessary competing root");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
test("CLI fetch: files remain after same-content duplicate root resolution", async () => {
|
||||||
|
const remote = remoteSettings();
|
||||||
|
await runScenario("reproduction", remote.bucket);
|
||||||
|
});
|
||||||
|
|
||||||
|
if (process.env["LIVESYNC_ISSUE1304_POSITIVE_CONTROL"] === "1") {
|
||||||
|
test("CLI fetch: --write-settings keeps the selected database across daemon startup", async () => {
|
||||||
|
const controlBucket = requiredEnv("LIVESYNC_ISSUE1304_POSITIVE_CONTROL_BUCKET");
|
||||||
|
await runScenario("write-settings-control", controlBucket);
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -12,6 +12,9 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
|
- CLI `fetch` and `rebuild` retain their selected local database for subsequent commands even without `--write-settings`, preventing the empty-database restart and file deletion reported in #1304.
|
||||||
|
- Runtime state is separated by database and Vault. Interrupted reconstruction blocks ordinary synchronisation until recovery, and a pending settings commit can be completed on restart.
|
||||||
|
- Older runtime state with no recorded owner is preserved and requires explicit reconstruction from a verified source before ordinary writes resume. Back up unique edits before choosing Fetch or Rebuild.
|
||||||
- The CLI mirror now starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. It checks logical Vault paths using `/` separators, correctly excluding internal database files and nested hidden directories even when the host uses Windows path separators.
|
- The CLI mirror now starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. It checks logical Vault paths using `/` separators, correctly excluding internal database files and nested hidden directories even when the host uses Windows path separators.
|
||||||
|
|
||||||
## 1.0.43
|
## 1.0.43
|
||||||
|
|||||||
Reference in new issue
Block a user