mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-11-02 22:38:18 +00:00
Merge pull request #1306 from vrtmrz/fix/cli-database-continuity
Preserve CLI database selection and runtime state after reconstruction
This commit is contained in:
24 files changed
+1746
-63
No files matched your search
@@ -38,6 +38,7 @@ Use these references for the area being changed, together with its relevant ADR.
|
|||||||
| Encryption and sharing | [Metadata encryption](design_docs/internal_metadata_encryption.md) and [time-bound Setup URIs](design_docs/time_bound_setup_uri.md) |
|
| Encryption and sharing | [Metadata encryption](design_docs/internal_metadata_encryption.md) and [time-bound Setup URIs](design_docs/time_bound_setup_uri.md) |
|
||||||
| P2P connection preparation | [TURN connection settings](design_docs/renewable_turn_credentials.md) |
|
| P2P connection preparation | [TURN connection settings](design_docs/renewable_turn_credentials.md) |
|
||||||
| Journal storage internals | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
|
| Journal storage internals | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
|
||||||
|
| CLI database selection and interrupted reconstruction | [CLI database continuity](design_docs/cli_database_continuity.md) |
|
||||||
|
|
||||||
The installed `@vrtmrz/livesync-commonlib` package is authoritative for shared runtime contracts. Use the exact dependency selected by [package.json](../package.json) and its lockfile when checking behaviour. The [Commonlib documentation](https://github.com/vrtmrz/livesync-commonlib/tree/main/docs) is maintained in that repository; this index does not replace its package contracts.
|
The installed `@vrtmrz/livesync-commonlib` package is authoritative for shared runtime contracts. Use the exact dependency selected by [package.json](../package.json) and its lockfile when checking behaviour. The [Commonlib documentation](https://github.com/vrtmrz/livesync-commonlib/tree/main/docs) is maintained in that repository; this index does not replace its package contracts.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
---
|
||||||
|
date: 2026-11-01
|
||||||
|
commonlib-version: "0.2.2"
|
||||||
|
self-hosted-livesync-version: "1.0.43"
|
||||||
|
status: accepted
|
||||||
|
---
|
||||||
|
|
||||||
|
# CLI database continuity
|
||||||
|
|
||||||
|
Publication note: the correction for #1304 is implemented and locally validated,
|
||||||
|
but has not been released. The reviewed baseline is Self-hosted LiveSync `0e458e5e` with
|
||||||
|
Commonlib `0.2.2`. The [CLI guide](../../src/apps/cli/README.md) describes the
|
||||||
|
command interface; the [test procedure](../../test/README.md#cli-fetch-database-continuity)
|
||||||
|
describes the Object Storage reproduction.
|
||||||
|
|
||||||
|
## Required behaviour
|
||||||
|
|
||||||
|
A successful `fetch` or `rebuild` must leave subsequent CLI commands using the
|
||||||
|
reconstructed local database. The Commonlib Rebuilder selects the host app ID
|
||||||
|
as the database suffix. The CLI saves that selection on success even without
|
||||||
|
`--write-settings`; the option continues to govern other lasting settings
|
||||||
|
changes. Temporary suspension values remain runtime state.
|
||||||
|
|
||||||
|
Before reconstruction can mutate a database, the CLI records the operation,
|
||||||
|
database directory, Vault path, and settings path. An interrupted operation
|
||||||
|
blocks ordinary mutation and synchronisation. Retrying the same reconstruction
|
||||||
|
with the same paths and `--force` is an explicit recovery action. It can repeat
|
||||||
|
the destructive operation, so users must preserve unique edits and establish
|
||||||
|
the authoritative copy before retrying.
|
||||||
|
|
||||||
|
Completion records the settings update before applying it. A later invocation
|
||||||
|
can finish a pending settings commit; if the settings file has independently
|
||||||
|
changed, it must stop for review rather than overwrite those changes. The
|
||||||
|
pending record is removed only after completion and settings persistence.
|
||||||
|
|
||||||
|
## Ownership of persistent state
|
||||||
|
|
||||||
|
The selected local database name identifies the dataset. Its runtime state is
|
||||||
|
further scoped by the canonical physical Vault path. This includes Journal
|
||||||
|
checkpoints, scanner state, replication-result queues, file-reflection
|
||||||
|
provenance, and storage-event snapshots. Paths identify local ownership; they
|
||||||
|
are not sent to the remote as a new document identity.
|
||||||
|
|
||||||
|
A local database reset invalidates every Vault scope associated with that
|
||||||
|
database, because each scope refers to the same discarded revision history.
|
||||||
|
Other local database suffixes retain their state. PouchDB database names and
|
||||||
|
Commonlib's reconstruction contract remain unchanged.
|
||||||
|
|
||||||
|
KV writes replace the file atomically. Invalid persisted data must fail
|
||||||
|
initialisation instead of becoming an empty checkpoint history. Closing a KV
|
||||||
|
handle must not overwrite the file from an old in-memory snapshot.
|
||||||
|
|
||||||
|
## Legacy state
|
||||||
|
|
||||||
|
The former shared `keyvalue-db.json` and Vault-local `.livesync-snapshot.json`
|
||||||
|
do not record their complete owner. A single surviving database directory does
|
||||||
|
not prove which physical Vault produced those records. Automatically importing
|
||||||
|
them can replay events against the wrong database; dropping them and immediately
|
||||||
|
scanning can misinterpret file absence before remote history is recovered.
|
||||||
|
|
||||||
|
Preserve unowned legacy state. When it contains work, stop ordinary mutating
|
||||||
|
commands and require explicit reconstruction from a verified source. Inspection
|
||||||
|
remains available. This deliberately requires more user action than automatic
|
||||||
|
migration, and must not silently delete the old records.
|
||||||
|
|
||||||
|
Check the selected scope's completion flag before reading legacy files. For an
|
||||||
|
unreviewed scope, inspect both legacy files after settings and scoped KV loading,
|
||||||
|
but before readiness starts scanning or event replay. Distinguish non-empty
|
||||||
|
unowned state, invalid JSON or shape, and read failures. Report affected paths
|
||||||
|
once per invocation on stderr without logging file contents. Read commands may
|
||||||
|
continue inspection; ordinary writes stop, and explicit reconstruction retains
|
||||||
|
the files. Once reconstruction has succeeded for that scope, legacy parse or
|
||||||
|
read failures must not block it again. The active scoped KV still fails
|
||||||
|
initialisation on corruption.
|
||||||
|
|
||||||
|
## Evidence and limits
|
||||||
|
|
||||||
|
The original Object Storage scenario uses encryption, path obfuscation, and a
|
||||||
|
second-generation file revision. Without settings writeback, A reopens an empty
|
||||||
|
database while sharing the fetched database's Journal history, creates an
|
||||||
|
independent root, and receives B's same-content resolution tombstone without
|
||||||
|
the original surviving revision. The original assertion fails because A's
|
||||||
|
physical file disappears. The explicit `--write-settings` control preserves
|
||||||
|
the file and the original revision. The remote and B retain the content in this
|
||||||
|
reproduction; it does not demonstrate irreversible loss.
|
||||||
|
|
||||||
|
Validation of the correction must check bytes and revision ancestry after
|
||||||
|
restart, interrupted reconstruction admission, pending settings completion,
|
||||||
|
legacy-state preservation, and reset isolation. A successful exit or an empty
|
||||||
|
queue alone does not establish preservation.
|
||||||
|
|
||||||
|
Local validation passed the encrypted Object Storage CLI regression and the
|
||||||
|
explicit-settings control on Node.js 26. Unit tests cover failed-operation
|
||||||
|
admission, saved running/completed markers, settings-edit preservation, scoped
|
||||||
|
stores, reset isolation, corrupt KV files, and legacy-state refusal. The marker
|
||||||
|
tests inject interruption states; they do not emulate a machine power failure.
|
||||||
|
|
||||||
|
The shared device-value JSON file retains its existing concurrency limitations.
|
||||||
|
This change does not establish concurrent CLI writers to one database directory
|
||||||
|
as a supported workflow. Native Obsidian and other transport behaviour require
|
||||||
|
their own runtime evidence.
|
||||||
@@ -155,10 +155,11 @@ preflight failure. If a required setup decision cannot be made in a headless
|
|||||||
CLI invocation, the command must fail with a clear remediation message rather
|
CLI invocation, the command must fail with a clear remediation message rather
|
||||||
than report success after cancellation.
|
than report success after cancellation.
|
||||||
|
|
||||||
`--write-settings` retains the Rebuilder's selected local database name and
|
The unreleased CLI correction for #1304 always retains the Rebuilder's selected
|
||||||
lasting settings changes for subsequent invocations. Without this option, the
|
local database suffix after successful `fetch` or `rebuild`. `--write-settings`
|
||||||
existing CLI policy leaves the settings file unchanged, so a later command can
|
still controls other lasting settings changes. Earlier versions required that
|
||||||
reopen the previously selected database.
|
option to retain the database selection; omitting it could reopen the previous
|
||||||
|
database on the next invocation. The examples retain the option for compatibility.
|
||||||
|
|
||||||
## State which follows the selected identity
|
## State which follows the selected identity
|
||||||
|
|
||||||
|
|||||||
Generated
+1
@@ -11765,6 +11765,7 @@
|
|||||||
"name": "self-hosted-livesync-cli",
|
"name": "self-hosted-livesync-cli",
|
||||||
"version": "1.0.43-cli",
|
"version": "1.0.43-cli",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@aws-crypto/sha256-js": "5.2.0",
|
||||||
"chokidar": "^4.0.0",
|
"chokidar": "^4.0.0",
|
||||||
"minimatch": "^10.2.5",
|
"minimatch": "^10.2.5",
|
||||||
"octagonal-wheels": "^0.1.55",
|
"octagonal-wheels": "^0.1.55",
|
||||||
|
|||||||
+36
-2
@@ -70,6 +70,7 @@ livesync-cli [database-path] [command] [args...]
|
|||||||
- `--force` / `-f`: Required by `rebuild` and `fetch` because they replace a database. It also retains its existing `init-settings` overwrite behaviour.
|
- `--force` / `-f`: Required by `rebuild` and `fetch` because they replace a database. It also retains its existing `init-settings` overwrite behaviour.
|
||||||
- `--write-settings`: Write setting migrations and other lasting changes after the command succeeds.
|
- `--write-settings`: Write setting migrations and other lasting changes after the command succeeds.
|
||||||
- `init-settings` writes its target file. `setup`, `remote-add`, `remote-rm`, `remote-set`, and `remote-activate` write their settings changes without this option.
|
- `init-settings` writes its target file. `setup`, `remote-add`, `remote-rm`, `remote-set`, and `remote-activate` write their settings changes without this option.
|
||||||
|
- `fetch` and `rebuild` always retain the selected local database suffix after success. Other lasting changes from these commands still require this option.
|
||||||
- All remaining commands leave the settings file unchanged by default.
|
- All remaining commands leave the settings file unchanged by default.
|
||||||
- Temporary values used to suspend synchronisation or select a remote for one command are never written.
|
- Temporary values used to suspend synchronisation or select a remote for one command are never written.
|
||||||
- `--compat-remote-admin-exit-zero`: Preserve the former zero exit code when `mark-resolved`, `lock-remote`, or `unlock-remote` returns a provider verification failure.
|
- `--compat-remote-admin-exit-zero`: Preserve the former zero exit code when `mark-resolved`, `lock-remote`, or `unlock-remote` returns a provider verification failure.
|
||||||
@@ -84,10 +85,10 @@ livesync-cli [database-path] [command] [args...]
|
|||||||
- If `vault-path` is omitted, it defaults to `database-path` (compatibility mode).
|
- If `vault-path` is omitted, it defaults to `database-path` (compatibility mode).
|
||||||
- Use this command to keep your local `.md` files in sync with the database.
|
- Use this command to keep your local `.md` files in sync with the database.
|
||||||
- `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`.
|
- `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`.
|
||||||
- This command may select a new local database suffix. Add `--write-settings` if later CLI commands should use the resulting database selection; otherwise the existing settings file remains unchanged.
|
- The selected local database suffix is saved automatically, so later commands use the reconstructed database.
|
||||||
- `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`.
|
- `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`.
|
||||||
|
- The selected local database suffix is saved automatically, including when `--write-settings` is omitted.
|
||||||
- Differing local files can remain and be preserved as conflict branches alongside the fetched content. For recovery from an intact remote, preserve the existing Vault separately and Fetch into an empty Vault; `--force` does not force file overwriting.
|
- Differing local files can remain and be preserved as conflict branches alongside the fetched content. For recovery from an intact remote, preserve the existing Vault separately and Fetch into an empty Vault; `--force` does not force file overwriting.
|
||||||
- This command may select a new local database suffix. Add `--write-settings` if later CLI commands should use the resulting database selection; otherwise the existing settings file remains unchanged.
|
|
||||||
- `ls [prefix]`: List files currently stored in the local database.
|
- `ls [prefix]`: List files currently stored in the local database.
|
||||||
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
|
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
|
||||||
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
|
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
|
||||||
@@ -107,6 +108,39 @@ livesync-cli [database-path] [command] [args...]
|
|||||||
|
|
||||||
Remote-administration commands verify the resulting milestone state through the selected provider. The existing `[Verification]` lines remain suitable for scripts which inspect command output, while the default exit code now reflects whether that verification succeeded.
|
Remote-administration commands verify the resulting milestone state through the selected provider. The existing `[Verification]` lines remain suitable for scripts which inspect command output, while the default exit code now reflects whether that verification succeeded.
|
||||||
|
|
||||||
|
### Interrupted reconstruction and older runtime state
|
||||||
|
|
||||||
|
An interrupted `fetch` or `rebuild` leaves a record in
|
||||||
|
`<database-path>/.livesync/reconstruction.json`. Ordinary synchronisation and
|
||||||
|
mutating commands stop while reconstruction is incomplete. Preserve the Vault,
|
||||||
|
database directory, and settings, check the reported failure, then retry the
|
||||||
|
same command with `--force` and the same database, Vault, and settings paths.
|
||||||
|
`rebuild` replaces remote state as well: confirm that the Vault remains the
|
||||||
|
authoritative copy before retrying it. Do not remove the record merely to
|
||||||
|
restart `daemon`.
|
||||||
|
|
||||||
|
If reconstruction finished but its settings update was interrupted, the next
|
||||||
|
invocation completes that update before starting the core. If the settings
|
||||||
|
file was independently edited, the CLI stops for review. The pending record
|
||||||
|
can contain settings and credentials; protect it like the settings file.
|
||||||
|
|
||||||
|
Older CLI versions stored runtime state without recording its database and
|
||||||
|
Vault owner. If unowned state is present, the CLI preserves it and stops
|
||||||
|
ordinary mutating commands. Read commands remain available for inspection.
|
||||||
|
Back up unique edits and establish the authoritative copy before explicitly
|
||||||
|
using `fetch --force` or `rebuild --force`. The old runtime files remain
|
||||||
|
available for diagnosis; they are not automatically imported into a new scope.
|
||||||
|
|
||||||
|
This check runs before scanning, event replay, and synchronisation. Its message
|
||||||
|
identifies each affected file and distinguishes an unknown owner from a file
|
||||||
|
which cannot be read or parsed. Read commands display the warning and continue
|
||||||
|
inspection. After successful reconstruction for the selected database and Vault,
|
||||||
|
retained legacy files are no longer inspected, even if they are malformed.
|
||||||
|
Corruption in the currently selected runtime KV remains an initialisation error.
|
||||||
|
|
||||||
|
Use one CLI process at a time for each database directory. Separate runtime
|
||||||
|
scopes do not make concurrent writers to shared device settings supported.
|
||||||
|
|
||||||
### Examples
|
### Examples
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -78,12 +78,20 @@ describe("CLI database preparation", () => {
|
|||||||
await rm(directory, { recursive: true, force: true });
|
await rm(directory, { recursive: true, force: true });
|
||||||
});
|
});
|
||||||
|
|
||||||
async function start(command: CLICommand, runner: CliCommandRunner, exitCode = 1, force = false) {
|
async function start(
|
||||||
|
command: CLICommand,
|
||||||
|
runner: CliCommandRunner,
|
||||||
|
exitCode = 1,
|
||||||
|
force = false,
|
||||||
|
writeSettings = false
|
||||||
|
) {
|
||||||
|
process.exitCode = undefined;
|
||||||
process.argv = [
|
process.argv = [
|
||||||
"node",
|
"node",
|
||||||
"livesync-cli",
|
"livesync-cli",
|
||||||
directory,
|
directory,
|
||||||
...(force ? ["--force"] : []),
|
...(force ? ["--force"] : []),
|
||||||
|
...(writeSettings ? ["--write-settings"] : []),
|
||||||
"--vault",
|
"--vault",
|
||||||
vaultPath,
|
vaultPath,
|
||||||
"--settings",
|
"--settings",
|
||||||
@@ -386,6 +394,194 @@ describe("CLI database preparation", () => {
|
|||||||
expect(await readFile(join(vaultPath, "notes/local.md"), "utf-8")).toBe("local content");
|
expect(await readFile(join(vaultPath, "notes/local.md"), "utf-8")).toBe("local content");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it.each(["fetch", "rebuild"] as const)(
|
||||||
|
"persists the selected database after %s without other command settings",
|
||||||
|
async (command) => {
|
||||||
|
const before = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
||||||
|
await core.services.setting.applyPartial({
|
||||||
|
additionalSuffixOfDatabaseName: "headless-app",
|
||||||
|
customChunkSize: before.customChunkSize + 10,
|
||||||
|
});
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
|
||||||
|
await start(command, runner, 0, true);
|
||||||
|
|
||||||
|
const after = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
expect(after.additionalSuffixOfDatabaseName).toBe("headless-app");
|
||||||
|
expect(after.customChunkSize).toBe(before.customChunkSize);
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it.each(["suffix", "vault"] as const)("isolates persistent state by selected %s", async (selection) => {
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
await core.services.keyValueDB.kvDB.set("scope-probe", "original");
|
||||||
|
await core.services.config.setValue("scope-probe", "original");
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
if (selection === "suffix") {
|
||||||
|
const settings = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
settings.additionalSuffixOfDatabaseName = "other";
|
||||||
|
await writeFile(settingsPath, JSON.stringify(settings));
|
||||||
|
} else {
|
||||||
|
vaultPath = join(directory, "other-vault");
|
||||||
|
await mkdir(vaultPath);
|
||||||
|
}
|
||||||
|
const observed: unknown[] = [];
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
observed.push(await core.services.keyValueDB.kvDB.get("scope-probe"));
|
||||||
|
observed.push(await core.services.config.getValue("scope-probe"));
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
expect(observed).toEqual([undefined, null]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["sync", "daemon", "mirror", "put"] as const)(
|
||||||
|
"blocks %s after failed reconstruction before command preparation",
|
||||||
|
async (command) => {
|
||||||
|
await start("fetch", async () => false, 1, true);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const ready = vi.spyOn(ControlService.prototype, "onReady");
|
||||||
|
|
||||||
|
await start(command, runner);
|
||||||
|
|
||||||
|
expect(runner).not.toHaveBeenCalled();
|
||||||
|
expect(ready).not.toHaveBeenCalled();
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it("retains legacy queued work and blocks a scanner until explicit reconstruction succeeds", async () => {
|
||||||
|
const legacy = join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
const legacyContents = JSON.stringify({
|
||||||
|
"queued-files": ["notes/unique.md"],
|
||||||
|
fileStatusMap: { "notes/local.md": 1 },
|
||||||
|
});
|
||||||
|
await mkdir(join(directory, ".livesync/runtime"), { recursive: true });
|
||||||
|
await writeFile(legacy, legacyContents);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const scan = vi.spyOn(ServiceFileAccessCLI.prototype, "getFiles");
|
||||||
|
|
||||||
|
await start("mirror", runner);
|
||||||
|
expect(runner).not.toHaveBeenCalled();
|
||||||
|
expect(scan).not.toHaveBeenCalled();
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe(legacyContents);
|
||||||
|
|
||||||
|
await start("fetch", async () => true, 0, true);
|
||||||
|
const later = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("sync", later);
|
||||||
|
expect(later).toHaveBeenCalledOnce();
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe(legacyContents);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("permits normal commands after reconstruction while preserving malformed legacy JSON", async () => {
|
||||||
|
const legacy = join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
await mkdir(join(directory, ".livesync/runtime"), { recursive: true });
|
||||||
|
await writeFile(legacy, '{"unfinished":');
|
||||||
|
await start("fetch", async () => true, 0, true);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("sync", runner);
|
||||||
|
expect(runner).toHaveBeenCalledOnce();
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe('{"unfinished":');
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
["kv", '{"unfinished":', "cannot be parsed"],
|
||||||
|
["snapshot", "[", "cannot be parsed"],
|
||||||
|
["kv", '{"queued-files":["unique.md"]}', "owner is unknown"],
|
||||||
|
["snapshot", '[{"path":"unique.md"}]', "owner is unknown"],
|
||||||
|
])("diagnoses unowned %s (%s) before readiness and ignores it after recovery", async (kind, contents, reason) => {
|
||||||
|
const legacy =
|
||||||
|
kind === "kv"
|
||||||
|
? join(directory, ".livesync/runtime/keyvalue-db.json")
|
||||||
|
: join(vaultPath, ".livesync-snapshot.json");
|
||||||
|
await mkdir(join(directory, ".livesync/runtime"), { recursive: true });
|
||||||
|
await writeFile(legacy, contents);
|
||||||
|
const messages = () => standardIo.writeStderr.mock.calls.flat().join("");
|
||||||
|
const reader = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("ls", reader);
|
||||||
|
expect(reader).toHaveBeenCalledOnce();
|
||||||
|
expect(messages()).toContain(reason);
|
||||||
|
expect(messages()).toContain(legacy);
|
||||||
|
expect(messages()).toContain("inspection may continue");
|
||||||
|
expect(messages().split("[Legacy state]")).toHaveLength(2);
|
||||||
|
expect(standardIo.writeStdout.mock.calls.flat().join("")).not.toContain("[Legacy state]");
|
||||||
|
|
||||||
|
standardIo.writeStderr.mockClear();
|
||||||
|
const writer = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const ready = vi.spyOn(ControlService.prototype, "onReady");
|
||||||
|
await start("mirror", writer);
|
||||||
|
expect(writer).not.toHaveBeenCalled();
|
||||||
|
expect(ready).not.toHaveBeenCalled();
|
||||||
|
expect(messages()).toContain("stopped before scanning");
|
||||||
|
expect(messages()).toContain(reason);
|
||||||
|
expect(messages().split("[Legacy state]")).toHaveLength(2);
|
||||||
|
ready.mockRestore();
|
||||||
|
|
||||||
|
await start("fetch", async () => true, 0, true);
|
||||||
|
standardIo.writeStderr.mockClear();
|
||||||
|
const later = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
await start("sync", later);
|
||||||
|
expect(later).toHaveBeenCalledOnce();
|
||||||
|
expect(messages()).not.toContain("[Legacy state]");
|
||||||
|
expect(await readFile(legacy, "utf-8")).toBe(contents);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("blocks read commands which request settings writes while reconstruction is interrupted", async () => {
|
||||||
|
await start("fetch", async () => false, 1, true);
|
||||||
|
const runner = vi.fn<CliCommandRunner>(async () => false);
|
||||||
|
const load = vi.spyOn(ControlService.prototype, "onLoad");
|
||||||
|
|
||||||
|
await start("ls", runner, 1, false, true);
|
||||||
|
|
||||||
|
expect(runner).not.toHaveBeenCalled();
|
||||||
|
expect(load).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("invalidates every Vault scope for the reset database and retains other database state", async () => {
|
||||||
|
const originalVault = vaultPath;
|
||||||
|
const otherVault = join(directory, "other-vault");
|
||||||
|
await mkdir(otherVault);
|
||||||
|
const select = async (suffix: string, selectedVault: string) => {
|
||||||
|
vaultPath = selectedVault;
|
||||||
|
const settings = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||||
|
settings.additionalSuffixOfDatabaseName = suffix;
|
||||||
|
await writeFile(settingsPath, JSON.stringify(settings));
|
||||||
|
};
|
||||||
|
for (const [suffix, selectedVault] of [
|
||||||
|
["", originalVault],
|
||||||
|
["", otherVault],
|
||||||
|
["other", originalVault],
|
||||||
|
]) {
|
||||||
|
await select(suffix, selectedVault);
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
await core.services.keyValueDB.kvDB.set("scope-probe", "stored");
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
await select("", originalVault);
|
||||||
|
let reset: boolean | undefined;
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
reset = await core.services.database.resetDatabase();
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
expect(reset).toBe(true);
|
||||||
|
|
||||||
|
const observed: unknown[] = [];
|
||||||
|
for (const [suffix, selectedVault] of [
|
||||||
|
["", originalVault],
|
||||||
|
["", otherVault],
|
||||||
|
["other", originalVault],
|
||||||
|
]) {
|
||||||
|
await select(suffix, selectedVault);
|
||||||
|
await start("ls", async (_options, { core }) => {
|
||||||
|
observed.push(await core.services.keyValueDB.kvDB.get("scope-probe"));
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
expect(observed).toEqual([undefined, undefined, "stored"]);
|
||||||
|
});
|
||||||
|
|
||||||
it("does not import vault files for standalone database commands", async () => {
|
it("does not import vault files for standalone database commands", async () => {
|
||||||
const storedPaths: string[] = [];
|
const storedPaths: string[] = [];
|
||||||
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
||||||
|
|||||||
+83
-19
@@ -41,6 +41,14 @@ import {
|
|||||||
reconcileDurableSettings,
|
reconcileDurableSettings,
|
||||||
settingsEqual,
|
settingsEqual,
|
||||||
} from "./settingsPersistence";
|
} from "./settingsPersistence";
|
||||||
|
import {
|
||||||
|
inspectLegacyState,
|
||||||
|
isMutatingCommand,
|
||||||
|
isReconstructionCommand,
|
||||||
|
readOptionalFile,
|
||||||
|
ReconstructionState,
|
||||||
|
writeFileAtomically,
|
||||||
|
} from "./runtimeState";
|
||||||
|
|
||||||
const SETTINGS_FILE = ".livesync/settings.json";
|
const SETTINGS_FILE = ".livesync/settings.json";
|
||||||
|
|
||||||
@@ -381,7 +389,7 @@ export async function main(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Resolve database path
|
// Resolve database path
|
||||||
const databasePath = path.resolve(options.databasePath!);
|
let databasePath = path.resolve(options.databasePath!);
|
||||||
// Check if database directory exists
|
// Check if database directory exists
|
||||||
try {
|
try {
|
||||||
const stat = await fs.stat(databasePath);
|
const stat = await fs.stat(databasePath);
|
||||||
@@ -389,18 +397,19 @@ export async function main(
|
|||||||
writeStderrLine(standardIo, `Error: ${databasePath} is not a directory`);
|
writeStderrLine(standardIo, `Error: ${databasePath} is not a directory`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
databasePath = await fs.realpath(databasePath);
|
||||||
} catch {
|
} catch {
|
||||||
writeStderrLine(standardIo, `Error: Database directory ${databasePath} does not exist`);
|
writeStderrLine(standardIo, `Error: Database directory ${databasePath} does not exist`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Resolve settings path
|
// Resolve settings path
|
||||||
const settingsPath = options.settingsPath
|
let settingsPath = options.settingsPath
|
||||||
? path.resolve(options.settingsPath)
|
? path.resolve(options.settingsPath)
|
||||||
: path.join(databasePath, SETTINGS_FILE);
|
: path.join(databasePath, SETTINGS_FILE);
|
||||||
// Resolve vault path: mirror positional argument takes priority,
|
// Resolve vault path: mirror positional argument takes priority,
|
||||||
// then --vault flag, otherwise fall back to databasePath.
|
// then --vault flag, otherwise fall back to databasePath.
|
||||||
const vaultPath =
|
let vaultPath =
|
||||||
options.command === "mirror" && options.commandArgs[0]
|
options.command === "mirror" && options.commandArgs[0]
|
||||||
? path.resolve(options.commandArgs[0])
|
? path.resolve(options.commandArgs[0])
|
||||||
: options.vaultPath
|
: options.vaultPath
|
||||||
@@ -414,11 +423,22 @@ export async function main(
|
|||||||
writeStderrLine(standardIo, `Error: Vault path ${vaultPath} is not a directory`);
|
writeStderrLine(standardIo, `Error: Vault path ${vaultPath} is not a directory`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
vaultPath = await fs.realpath(vaultPath);
|
||||||
} catch {
|
} catch {
|
||||||
writeStderrLine(standardIo, `Error: Vault directory ${vaultPath} does not exist`);
|
writeStderrLine(standardIo, `Error: Vault directory ${vaultPath} does not exist`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
settingsPath = await fs.realpath(settingsPath).catch(() => settingsPath);
|
||||||
|
const reconstruction = new ReconstructionState(databasePath, vaultPath, settingsPath);
|
||||||
|
try {
|
||||||
|
reconstruction.prepare(options.command, options.force ?? false, options.writeSettings ?? false);
|
||||||
|
} catch (error) {
|
||||||
|
writeStderrLine(standardIo, "[Error] Reconstruction recovery is required:", error);
|
||||||
|
process.exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
infoLog(`Self-hosted LiveSync CLI`);
|
infoLog(`Self-hosted LiveSync CLI`);
|
||||||
infoLog(`Database Path: ${databasePath}`);
|
infoLog(`Database Path: ${databasePath}`);
|
||||||
infoLog(`Vault Path: ${vaultPath}`);
|
infoLog(`Vault Path: ${vaultPath}`);
|
||||||
@@ -437,7 +457,7 @@ export async function main(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Create service context and hub
|
// Create service context and hub
|
||||||
const context = new NodeServiceContext(databasePath, standardIo);
|
const context = new NodeServiceContext(databasePath, standardIo, vaultPath);
|
||||||
const serviceHubInstance = new NodeServiceHub<NodeServiceContext>(databasePath, context);
|
const serviceHubInstance = new NodeServiceHub<NodeServiceContext>(databasePath, context);
|
||||||
serviceHubInstance.API.addLog.setHandler((message: unknown, level: LOG_LEVEL) => {
|
serviceHubInstance.API.addLog.setHandler((message: unknown, level: LOG_LEVEL) => {
|
||||||
let levelStr = "";
|
let levelStr = "";
|
||||||
@@ -479,7 +499,7 @@ export async function main(
|
|||||||
|
|
||||||
// Setup settings handlers
|
// Setup settings handlers
|
||||||
const settingService = serviceHubInstance.setting;
|
const settingService = serviceHubInstance.setting;
|
||||||
const originalSettingsText = await fs.readFile(settingsPath, "utf-8").catch(() => undefined);
|
const originalSettingsText = readOptionalFile(settingsPath);
|
||||||
let latestPreparedSettingsText: string | undefined;
|
let latestPreparedSettingsText: string | undefined;
|
||||||
let preparedSettingsRevision = 0;
|
let preparedSettingsRevision = 0;
|
||||||
let commandIsRunning = false;
|
let commandIsRunning = false;
|
||||||
@@ -600,21 +620,19 @@ export async function main(
|
|||||||
|
|
||||||
const writeSettingsAtomically = async (content: string | undefined): Promise<void> => {
|
const writeSettingsAtomically = async (content: string | undefined): Promise<void> => {
|
||||||
if (content === undefined || content === originalSettingsText) return;
|
if (content === undefined || content === originalSettingsText) return;
|
||||||
const temporaryPath = `${settingsPath}.${process.pid}.tmp`;
|
writeFileAtomically(settingsPath, content);
|
||||||
await fs.mkdir(path.dirname(settingsPath), { recursive: true });
|
if (options.verbose) {
|
||||||
try {
|
writeStderrLine(standardIo, `[Settings] Saved to ${settingsPath}`);
|
||||||
await fs.writeFile(temporaryPath, content, "utf-8");
|
|
||||||
await fs.rename(temporaryPath, settingsPath);
|
|
||||||
if (options.verbose) {
|
|
||||||
writeStderrLine(standardIo, `[Settings] Saved to ${settingsPath}`);
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
await fs.unlink(temporaryPath).catch(() => {});
|
|
||||||
throw error;
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
// Start the core
|
// Start the core
|
||||||
|
let unloaded = false;
|
||||||
|
const unload = async () => {
|
||||||
|
if (unloaded) return;
|
||||||
|
unloaded = true;
|
||||||
|
await core.services.control.onUnload();
|
||||||
|
};
|
||||||
try {
|
try {
|
||||||
infoLog(`[Starting] Initializing LiveSync...`);
|
infoLog(`[Starting] Initializing LiveSync...`);
|
||||||
|
|
||||||
@@ -623,6 +641,31 @@ export async function main(
|
|||||||
writeStderrLine(standardIo, `[Error] Failed to initialize LiveSync`);
|
writeStderrLine(standardIo, `[Error] Failed to initialize LiveSync`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
if ((await core.services.keyValueDB.kvDB.get("cli-legacy-state-reviewed")) !== true) {
|
||||||
|
const legacyIssues = inspectLegacyState(databasePath, vaultPath);
|
||||||
|
if (legacyIssues.length > 0) {
|
||||||
|
const reasons = {
|
||||||
|
unowned: "database and Vault owner is unknown",
|
||||||
|
invalid: "state cannot be parsed in its expected format; owner and contents cannot be established",
|
||||||
|
unreadable: "state cannot be read; owner and contents cannot be established",
|
||||||
|
};
|
||||||
|
const details = legacyIssues.map((issue) => `${issue.path}: ${reasons[issue.reason]}`).join("; ");
|
||||||
|
const advice =
|
||||||
|
"Preserve these files, back up unique edits, and verify the authoritative source before choosing fetch --force or rebuild --force.";
|
||||||
|
const blocked =
|
||||||
|
!isReconstructionCommand(options.command) &&
|
||||||
|
(isMutatingCommand(options.command) || options.writeSettings);
|
||||||
|
if (blocked) {
|
||||||
|
throw new Error(
|
||||||
|
`[Legacy state] ${details}. Command '${options.command}' stopped before scanning, event replay, or synchronisation. ${advice}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const action = isReconstructionCommand(options.command)
|
||||||
|
? "Explicit reconstruction will proceed; the legacy files will be retained."
|
||||||
|
: "Read-only inspection may continue; the legacy files will be retained.";
|
||||||
|
writeStderrLine(standardIo, `[Legacy state] ${details}. ${action} ${advice}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
const settingsAfterLoadText = latestPreparedSettingsText
|
const settingsAfterLoadText = latestPreparedSettingsText
|
||||||
? preserveStoredSetting(latestPreparedSettingsText, originalSettingsText, "useIndexedDBAdapter")
|
? preserveStoredSetting(latestPreparedSettingsText, originalSettingsText, "useIndexedDBAdapter")
|
||||||
: originalSettingsText;
|
: originalSettingsText;
|
||||||
@@ -655,7 +698,7 @@ export async function main(
|
|||||||
try {
|
try {
|
||||||
readyResult = await core.services.control.onReady();
|
readyResult = await core.services.control.onReady();
|
||||||
} finally {
|
} finally {
|
||||||
if (!readyResult) await core.services.control.onUnload();
|
if (!readyResult) await unload();
|
||||||
}
|
}
|
||||||
if (!readyResult) {
|
if (!readyResult) {
|
||||||
throw new Error("Failed to initialise LiveSync.");
|
throw new Error("Failed to initialise LiveSync.");
|
||||||
@@ -751,6 +794,21 @@ export async function main(
|
|||||||
settingsTextToCommit = settingsAfterLoadText;
|
settingsTextToCommit = settingsAfterLoadText;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if (result && isReconstructionCommand(options.command)) {
|
||||||
|
if (!options.writeSettings) {
|
||||||
|
const durable =
|
||||||
|
originalSettingsText === undefined
|
||||||
|
? cloneSettings(durableSettingsBeforeSuspend)
|
||||||
|
: (JSON.parse(originalSettingsText) as ObsidianLiveSyncSettings);
|
||||||
|
durable.additionalSuffixOfDatabaseName =
|
||||||
|
core.services.setting.currentSettings().additionalSuffixOfDatabaseName;
|
||||||
|
settingsTextToCommit = JSON.stringify(durable, null, 2);
|
||||||
|
}
|
||||||
|
settingsTextToCommit ??= originalSettingsText;
|
||||||
|
if (settingsTextToCommit === undefined)
|
||||||
|
throw new Error("Reconstruction completed without database selection settings.");
|
||||||
|
await core.services.keyValueDB.kvDB.set("cli-legacy-state-reviewed", true);
|
||||||
|
}
|
||||||
|
|
||||||
if (!result) {
|
if (!result) {
|
||||||
writeStderrLine(standardIo, `[Error] Command '${options.command}' failed`);
|
writeStderrLine(standardIo, `[Error] Command '${options.command}' failed`);
|
||||||
@@ -764,10 +822,16 @@ export async function main(
|
|||||||
// Keep the process running
|
// Keep the process running
|
||||||
await new Promise(() => {});
|
await new Promise(() => {});
|
||||||
} else {
|
} else {
|
||||||
await core.services.control.onUnload();
|
await unload();
|
||||||
await writeSettingsAtomically(settingsTextToCommit);
|
if (result && isReconstructionCommand(options.command)) {
|
||||||
|
reconstruction.complete(settingsTextToCommit!);
|
||||||
|
reconstruction.commit();
|
||||||
|
} else {
|
||||||
|
await writeSettingsAtomically(settingsTextToCommit);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
await unload().catch(() => {});
|
||||||
writeStderrLine(standardIo, `[Error] Failed to start:`, error);
|
writeStderrLine(standardIo, `[Error] Failed to start:`, error);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ import { watch as chokidarWatch, type FSWatcher } from "chokidar";
|
|||||||
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||||
import { fsPromises as fs, getNodeFileCreationTime, path, type Stats } from "@vrtmrz/livesync-commonlib/node";
|
import { fsPromises as fs, getNodeFileCreationTime, path, type Stats } from "@vrtmrz/livesync-commonlib/node";
|
||||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||||
|
import { writeFileAtomically } from "@/apps/cli/runtimeState";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* CLI-specific type guard adapter
|
* CLI-specific type guard adapter
|
||||||
@@ -44,18 +45,19 @@ class CLITypeGuardAdapter implements IStorageEventTypeGuardAdapter<NodeFile, Nod
|
|||||||
* CLI-specific persistence adapter (file-based snapshot)
|
* CLI-specific persistence adapter (file-based snapshot)
|
||||||
*/
|
*/
|
||||||
class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
|
class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
|
||||||
private snapshotPath: string;
|
private snapshotPath: () => string;
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
basePath: string,
|
basePath: string,
|
||||||
private reportDiagnostic: CliDiagnosticReporter = () => undefined
|
private reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||||
|
snapshotPath?: () => string
|
||||||
) {
|
) {
|
||||||
this.snapshotPath = path.join(basePath, ".livesync-snapshot.json");
|
this.snapshotPath = snapshotPath ?? (() => path.join(basePath, ".livesync-snapshot.json"));
|
||||||
}
|
}
|
||||||
|
|
||||||
async saveSnapshot(snapshot: (FileEventItem | FileEventItemSentinel)[]): Promise<void> {
|
async saveSnapshot(snapshot: (FileEventItem | FileEventItemSentinel)[]): Promise<void> {
|
||||||
try {
|
try {
|
||||||
await fs.writeFile(this.snapshotPath, JSON.stringify(snapshot, null, 2), "utf-8");
|
writeFileAtomically(this.snapshotPath(), JSON.stringify(snapshot, null, 2));
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.reportDiagnostic("Failed to save snapshot:", error);
|
this.reportDiagnostic("Failed to save snapshot:", error);
|
||||||
}
|
}
|
||||||
@@ -63,7 +65,7 @@ class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
|
|||||||
|
|
||||||
async loadSnapshot(): Promise<(FileEventItem | FileEventItemSentinel)[] | null> {
|
async loadSnapshot(): Promise<(FileEventItem | FileEventItemSentinel)[] | null> {
|
||||||
try {
|
try {
|
||||||
const content = await fs.readFile(this.snapshotPath, "utf-8");
|
const content = await fs.readFile(this.snapshotPath(), "utf-8");
|
||||||
return JSON.parse(content) as (FileEventItem | FileEventItemSentinel)[];
|
return JSON.parse(content) as (FileEventItem | FileEventItemSentinel)[];
|
||||||
} catch {
|
} catch {
|
||||||
return null;
|
return null;
|
||||||
@@ -221,10 +223,11 @@ export class CLIStorageEventManagerAdapter implements IStorageEventManagerAdapte
|
|||||||
basePath: string,
|
basePath: string,
|
||||||
ignoreRules?: IgnoreRules,
|
ignoreRules?: IgnoreRules,
|
||||||
watchEnabled: boolean = false,
|
watchEnabled: boolean = false,
|
||||||
reportDiagnostic: CliDiagnosticReporter = () => undefined
|
reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||||
|
snapshotPath?: () => string
|
||||||
) {
|
) {
|
||||||
this.typeGuard = new CLITypeGuardAdapter();
|
this.typeGuard = new CLITypeGuardAdapter();
|
||||||
this.persistence = new CLIPersistenceAdapter(basePath, reportDiagnostic);
|
this.persistence = new CLIPersistenceAdapter(basePath, reportDiagnostic, snapshotPath);
|
||||||
this.watch = new CLIWatchAdapter(basePath, ignoreRules, watchEnabled, reportDiagnostic);
|
this.watch = new CLIWatchAdapter(basePath, ignoreRules, watchEnabled, reportDiagnostic);
|
||||||
this.status = new CLIStatusAdapter();
|
this.status = new CLIStatusAdapter();
|
||||||
this.converter = new CLIConverterAdapter();
|
this.converter = new CLIConverterAdapter();
|
||||||
|
|||||||
@@ -1,6 +1,8 @@
|
|||||||
import { describe, expect, it, vi, beforeEach } from "vitest";
|
import { describe, expect, it, vi, beforeEach } from "vitest";
|
||||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||||
import type { NodeFile } from "@/apps/cli/adapters/NodeTypes";
|
import type { NodeFile } from "@/apps/cli/adapters/NodeTypes";
|
||||||
|
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import { runtimeScopeDirectory } from "../runtimeState";
|
||||||
|
|
||||||
// ── chokidar mock ──────────────────────────────────────────────────────────────
|
// ── chokidar mock ──────────────────────────────────────────────────────────────
|
||||||
// Must be hoisted before imports that pull in chokidar.
|
// Must be hoisted before imports that pull in chokidar.
|
||||||
@@ -44,6 +46,27 @@ describe("CLIStorageEventManagerAdapter", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("keeps pending snapshots with the selected database and Vault pair", async () => {
|
||||||
|
const directory = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-snapshot-"));
|
||||||
|
let database = "first";
|
||||||
|
let vault = "vault-a";
|
||||||
|
const adapter = new CLIStorageEventManagerAdapter(directory, undefined, false, undefined, () =>
|
||||||
|
path.join(runtimeScopeDirectory(directory, database, vault), "storage-events.json")
|
||||||
|
);
|
||||||
|
try {
|
||||||
|
await adapter.persistence.saveSnapshot([{ type: "SENTINEL_FLUSH" }]);
|
||||||
|
database = "second";
|
||||||
|
expect(await adapter.persistence.loadSnapshot()).toBeNull();
|
||||||
|
database = "first";
|
||||||
|
vault = "vault-b";
|
||||||
|
expect(await adapter.persistence.loadSnapshot()).toBeNull();
|
||||||
|
vault = "vault-a";
|
||||||
|
expect(await adapter.persistence.loadSnapshot()).toEqual([{ type: "SENTINEL_FLUSH" }]);
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(directory, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
it("beginWatch is no-op when watchEnabled=false", async () => {
|
it("beginWatch is no-op when watchEnabled=false", async () => {
|
||||||
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, false);
|
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, false);
|
||||||
const handlers = makeHandlers();
|
const handlers = makeHandlers();
|
||||||
|
|||||||
@@ -1,9 +1,15 @@
|
|||||||
import { StorageEventManagerBase, type StorageEventManagerBaseDependencies } from "@vrtmrz/livesync-commonlib/compat/managers/StorageEventManager";
|
import {
|
||||||
|
StorageEventManagerBase,
|
||||||
|
type StorageEventManagerBaseDependencies,
|
||||||
|
} from "@vrtmrz/livesync-commonlib/compat/managers/StorageEventManager";
|
||||||
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
|
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
|
||||||
import type { IMinimumLiveSyncCommands, LiveSyncBaseCore } from "@/LiveSyncBaseCore";
|
import type { IMinimumLiveSyncCommands, LiveSyncBaseCore } from "@/LiveSyncBaseCore";
|
||||||
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
||||||
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||||
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||||
|
import { runtimeScopeDirectory } from "@/apps/cli/runtimeState";
|
||||||
|
import type { NodeServiceContext } from "@/apps/cli/services/NodeServiceContext";
|
||||||
|
import { path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
||||||
// import type { IMinimumLiveSyncCommands } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
// import type { IMinimumLiveSyncCommands } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
||||||
|
|
||||||
export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEventManagerAdapter> {
|
export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEventManagerAdapter> {
|
||||||
@@ -16,12 +22,23 @@ export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEv
|
|||||||
ignoreRules?: IgnoreRules,
|
ignoreRules?: IgnoreRules,
|
||||||
watchEnabled?: boolean
|
watchEnabled?: boolean
|
||||||
) {
|
) {
|
||||||
const adapter = new CLIStorageEventManagerAdapter(basePath, ignoreRules, watchEnabled, (message, detail) => {
|
const context = core.services.context as NodeServiceContext;
|
||||||
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
|
const adapter = new CLIStorageEventManagerAdapter(
|
||||||
if (detail !== undefined) {
|
basePath,
|
||||||
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
|
ignoreRules,
|
||||||
}
|
watchEnabled,
|
||||||
});
|
(message, detail) => {
|
||||||
|
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
|
||||||
|
if (detail !== undefined) {
|
||||||
|
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
() =>
|
||||||
|
nodePath.join(
|
||||||
|
runtimeScopeDirectory(context.databasePath, dependencies.vaultService.getVaultName(), basePath),
|
||||||
|
"storage-events.json"
|
||||||
|
)
|
||||||
|
);
|
||||||
super(adapter, dependencies);
|
super(adapter, dependencies);
|
||||||
this.core = core;
|
this.core = core;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,7 +12,7 @@
|
|||||||
"buildRun": "npm run build && npm run cli --",
|
"buildRun": "npm run build && npm run cli --",
|
||||||
"build:docker": "docker build -f Dockerfile -t livesync-cli ../../..",
|
"build:docker": "docker build -f Dockerfile -t livesync-cli ../../..",
|
||||||
"check": "tsc -p tsconfig.json",
|
"check": "tsc -p tsconfig.json",
|
||||||
"test:unit": "cd ../../.. && npx vitest run --config vitest.config.unit.ts src/apps/cli/main.unit.spec.ts src/apps/cli/main.bootstrap.unit.spec.ts src/apps/cli/settingsPersistence.unit.spec.ts src/apps/cli/services/NodeLocalStorage.unit.spec.ts src/apps/cli/commands/utils.unit.spec.ts src/apps/cli/commands/runCommand.unit.spec.ts src/apps/cli/commands/daemonCommand.unit.spec.ts src/apps/cli/commands/p2p.unit.spec.ts src/apps/cli/deploy/install.unit.spec.ts src/apps/cli/adapters/NodeFileSystemAdapter.unit.spec.ts",
|
"test:unit": "cd ../../.. && npx vitest run --config vitest.config.unit.ts src/apps/cli/main.unit.spec.ts src/apps/cli/main.bootstrap.unit.spec.ts src/apps/cli/settingsPersistence.unit.spec.ts src/apps/cli/runtimeState.unit.spec.ts src/apps/cli/services/NodeKeyValueDBService.unit.spec.ts src/apps/cli/services/NodeLocalStorage.unit.spec.ts src/apps/cli/managers/CLIStorageEventManagerAdapter.unit.spec.ts src/apps/cli/commands/utils.unit.spec.ts src/apps/cli/commands/runCommand.unit.spec.ts src/apps/cli/commands/daemonCommand.unit.spec.ts src/apps/cli/commands/p2p.unit.spec.ts src/apps/cli/deploy/install.unit.spec.ts src/apps/cli/adapters/NodeFileSystemAdapter.unit.spec.ts",
|
||||||
"test:e2e:two-vaults": "bash test/test-e2e-two-vaults-with-docker-linux.sh",
|
"test:e2e:two-vaults": "bash test/test-e2e-two-vaults-with-docker-linux.sh",
|
||||||
"test:e2e:two-vaults:common": "bash test/test-e2e-two-vaults-common.sh",
|
"test:e2e:two-vaults:common": "bash test/test-e2e-two-vaults-common.sh",
|
||||||
"test:e2e:two-vaults:matrix": "bash test/test-e2e-two-vaults-matrix.sh",
|
"test:e2e:two-vaults:matrix": "bash test/test-e2e-two-vaults-matrix.sh",
|
||||||
@@ -37,6 +37,7 @@
|
|||||||
"test:e2e:coverage": "node ../../../test/e2e-cli/run-coverage.ts"
|
"test:e2e:coverage": "node ../../../test/e2e-cli/run-coverage.ts"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@aws-crypto/sha256-js": "5.2.0",
|
||||||
"chokidar": "^4.0.0",
|
"chokidar": "^4.0.0",
|
||||||
"minimatch": "^10.2.5",
|
"minimatch": "^10.2.5",
|
||||||
"octagonal-wheels": "^0.1.55",
|
"octagonal-wheels": "^0.1.55",
|
||||||
|
|||||||
@@ -0,0 +1,222 @@
|
|||||||
|
// This module belongs exclusively to the Node.js CLI host.
|
||||||
|
import { Sha256 } from "@aws-crypto/sha256-js";
|
||||||
|
import { fs, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import type { CLICommand } from "./commands/types";
|
||||||
|
|
||||||
|
function sha256Hex(value: string): string {
|
||||||
|
const hash = new Sha256();
|
||||||
|
hash.update(new TextEncoder().encode(value));
|
||||||
|
return Array.from(hash.digestSync(), (byte) => byte.toString(16).padStart(2, "0")).join("");
|
||||||
|
}
|
||||||
|
|
||||||
|
export function runtimeScopeDirectory(databasePath: string, databaseName: string, vaultPath: string): string {
|
||||||
|
const scope = sha256Hex(vaultPath);
|
||||||
|
return path.join(runtimeDatabaseDirectory(databasePath, databaseName), scope);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function runtimeDatabaseDirectory(databasePath: string, databaseName: string): string {
|
||||||
|
const database = sha256Hex(databaseName);
|
||||||
|
return path.join(databasePath, ".livesync", "runtime", "scopes", database);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function writeFileAtomically(filePath: string, content: string): void {
|
||||||
|
fs.mkdirSync(path.dirname(filePath), { recursive: true });
|
||||||
|
const temporaryPath = `${filePath}.${process.pid}.${crypto.randomUUID()}.tmp`;
|
||||||
|
try {
|
||||||
|
const descriptor = fs.openSync(temporaryPath, "wx", 0o600);
|
||||||
|
try {
|
||||||
|
fs.writeFileSync(descriptor, content, "utf-8");
|
||||||
|
fs.fsyncSync(descriptor);
|
||||||
|
} finally {
|
||||||
|
fs.closeSync(descriptor);
|
||||||
|
}
|
||||||
|
fs.renameSync(temporaryPath, filePath);
|
||||||
|
syncDirectory(path.dirname(filePath));
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(temporaryPath, { force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function syncDirectory(directory: string): void {
|
||||||
|
// Windows does not support opening a directory for fsync.
|
||||||
|
if (process.platform === "win32") return;
|
||||||
|
const descriptor = fs.openSync(directory, "r");
|
||||||
|
try {
|
||||||
|
fs.fsyncSync(descriptor);
|
||||||
|
} finally {
|
||||||
|
fs.closeSync(descriptor);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function readOptionalFile(filePath: string): string | undefined {
|
||||||
|
try {
|
||||||
|
return fs.readFileSync(filePath, "utf-8");
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as { code?: string }).code === "ENOENT") return undefined;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isReconstructionCommand(command: CLICommand): command is "fetch" | "rebuild" {
|
||||||
|
return command === "fetch" || command === "rebuild";
|
||||||
|
}
|
||||||
|
|
||||||
|
const READ_COMMANDS = new Set<CLICommand>([
|
||||||
|
"cat",
|
||||||
|
"cat-rev",
|
||||||
|
"ls",
|
||||||
|
"info",
|
||||||
|
"remote-ls",
|
||||||
|
"remote-export",
|
||||||
|
"remote-status",
|
||||||
|
"p2p-peers",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export function isMutatingCommand(command: CLICommand): boolean {
|
||||||
|
return !READ_COMMANDS.has(command);
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ReconstructionMarker {
|
||||||
|
version: 1;
|
||||||
|
command: "fetch" | "rebuild";
|
||||||
|
databasePath: string;
|
||||||
|
vaultPath: string;
|
||||||
|
settingsPath: string;
|
||||||
|
settingsBeforeHash: string;
|
||||||
|
phase: "running" | "completed";
|
||||||
|
settingsText?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hashSettings(content: string | undefined): string {
|
||||||
|
return sha256Hex(JSON.stringify(content ?? null));
|
||||||
|
}
|
||||||
|
|
||||||
|
export class ReconstructionState {
|
||||||
|
readonly markerPath: string;
|
||||||
|
private marker: ReconstructionMarker | undefined;
|
||||||
|
|
||||||
|
constructor(
|
||||||
|
private readonly databasePath: string,
|
||||||
|
private readonly vaultPath: string,
|
||||||
|
private readonly settingsPath: string
|
||||||
|
) {
|
||||||
|
this.markerPath = path.join(databasePath, ".livesync", "reconstruction.json");
|
||||||
|
}
|
||||||
|
|
||||||
|
prepare(command: CLICommand, force: boolean, writeSettings = false): void {
|
||||||
|
const contents = readOptionalFile(this.markerPath);
|
||||||
|
if (contents !== undefined) {
|
||||||
|
const parsed: unknown = JSON.parse(contents);
|
||||||
|
if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) {
|
||||||
|
throw new Error(`Invalid reconstruction state: ${this.markerPath}. Preserve it for recovery.`);
|
||||||
|
}
|
||||||
|
const marker = parsed as ReconstructionMarker;
|
||||||
|
if (
|
||||||
|
marker.version !== 1 ||
|
||||||
|
!isReconstructionCommand(marker.command) ||
|
||||||
|
(marker.phase !== "running" && marker.phase !== "completed") ||
|
||||||
|
typeof marker.settingsPath !== "string" ||
|
||||||
|
typeof marker.vaultPath !== "string" ||
|
||||||
|
typeof marker.settingsBeforeHash !== "string" ||
|
||||||
|
marker.databasePath !== this.databasePath ||
|
||||||
|
(marker.phase === "completed" && typeof marker.settingsText !== "string")
|
||||||
|
) {
|
||||||
|
throw new Error(`Invalid reconstruction state: ${this.markerPath}. Preserve it for recovery.`);
|
||||||
|
}
|
||||||
|
this.marker = marker;
|
||||||
|
if (marker.phase === "completed") {
|
||||||
|
this.commit();
|
||||||
|
} else if (
|
||||||
|
(isMutatingCommand(command) || writeSettings) &&
|
||||||
|
!(
|
||||||
|
command === marker.command &&
|
||||||
|
force &&
|
||||||
|
marker.vaultPath === this.vaultPath &&
|
||||||
|
marker.settingsPath === this.settingsPath
|
||||||
|
)
|
||||||
|
) {
|
||||||
|
throw new Error(
|
||||||
|
`An interrupted ${marker.command} requires recovery. Retry ${marker.command} --force with the same database, Vault, and settings paths. State: ${this.markerPath}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (isReconstructionCommand(command)) {
|
||||||
|
this.marker = {
|
||||||
|
version: 1,
|
||||||
|
command,
|
||||||
|
databasePath: this.databasePath,
|
||||||
|
vaultPath: this.vaultPath,
|
||||||
|
settingsPath: this.settingsPath,
|
||||||
|
phase: "running",
|
||||||
|
settingsBeforeHash: hashSettings(readOptionalFile(this.settingsPath)),
|
||||||
|
};
|
||||||
|
this.save();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
complete(settingsText: string): void {
|
||||||
|
if (!this.marker || this.marker.phase !== "running") throw new Error("Reconstruction was not started.");
|
||||||
|
this.marker = { ...this.marker, phase: "completed", settingsText };
|
||||||
|
this.save();
|
||||||
|
}
|
||||||
|
|
||||||
|
commit(): void {
|
||||||
|
const marker = this.marker;
|
||||||
|
if (!marker || marker.phase !== "completed" || marker.settingsText === undefined) {
|
||||||
|
throw new Error("Reconstruction has no completed settings update.");
|
||||||
|
}
|
||||||
|
const current = readOptionalFile(marker.settingsPath);
|
||||||
|
if (current !== marker.settingsText && hashSettings(current) !== marker.settingsBeforeHash) {
|
||||||
|
throw new Error(
|
||||||
|
`The settings file changed after reconstruction: ${marker.settingsPath}. Preserve the reconstruction state and review both settings versions before continuing.`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (current !== marker.settingsText) writeFileAtomically(marker.settingsPath, marker.settingsText);
|
||||||
|
this.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
clear(): void {
|
||||||
|
fs.rmSync(this.markerPath, { force: true });
|
||||||
|
syncDirectory(path.dirname(this.markerPath));
|
||||||
|
this.marker = undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
private save(): void {
|
||||||
|
writeFileAtomically(this.markerPath, JSON.stringify(this.marker, null, 2));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface LegacyStateIssue {
|
||||||
|
path: string;
|
||||||
|
reason: "unowned" | "invalid" | "unreadable";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function inspectLegacyState(databasePath: string, vaultPath: string): LegacyStateIssue[] {
|
||||||
|
const issues: LegacyStateIssue[] = [];
|
||||||
|
for (const [filePath, expectArray] of [
|
||||||
|
[path.join(databasePath, ".livesync", "runtime", "keyvalue-db.json"), false],
|
||||||
|
[path.join(vaultPath, ".livesync-snapshot.json"), true],
|
||||||
|
] as const) {
|
||||||
|
let contents: string | undefined;
|
||||||
|
try {
|
||||||
|
contents = readOptionalFile(filePath);
|
||||||
|
} catch {
|
||||||
|
issues.push({ path: filePath, reason: "unreadable" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (contents === undefined) continue;
|
||||||
|
let value: unknown;
|
||||||
|
try {
|
||||||
|
value = JSON.parse(contents);
|
||||||
|
} catch {
|
||||||
|
issues.push({ path: filePath, reason: "invalid" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (value === null || typeof value !== "object" || Array.isArray(value) !== expectArray) {
|
||||||
|
issues.push({ path: filePath, reason: "invalid" });
|
||||||
|
} else if (Object.keys(value).length > 0) {
|
||||||
|
issues.push({ path: filePath, reason: "unowned" });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return issues;
|
||||||
|
}
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
ReconstructionState,
|
||||||
|
inspectLegacyState,
|
||||||
|
readOptionalFile,
|
||||||
|
runtimeDatabaseDirectory,
|
||||||
|
runtimeScopeDirectory,
|
||||||
|
} from "./runtimeState";
|
||||||
|
|
||||||
|
describe("CLI runtime scope compatibility", () => {
|
||||||
|
it("retains SHA-256 directory names for database and Unicode Vault paths", () => {
|
||||||
|
expect(runtimeScopeDirectory("state", "headless-app", "/vault/日本語😀")).toBe(
|
||||||
|
path.join(
|
||||||
|
"state",
|
||||||
|
".livesync",
|
||||||
|
"runtime",
|
||||||
|
"scopes",
|
||||||
|
"511afda5613706862fa4add801b3f9b19e0db2acacc07b9218f2d01ff41ca61d",
|
||||||
|
"542d31d6918ad8bf915e204e41e479fdbd7251690f182fd802378323b4c6de23"
|
||||||
|
)
|
||||||
|
);
|
||||||
|
expect(path.basename(runtimeDatabaseDirectory("state", ""))).toBe(
|
||||||
|
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("CLI reconstruction state", () => {
|
||||||
|
let directory: string;
|
||||||
|
let settingsPath: string;
|
||||||
|
let state: ReconstructionState;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
directory = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-reconstruction-"));
|
||||||
|
settingsPath = path.join(directory, "settings.json");
|
||||||
|
fs.writeFileSync(settingsPath, JSON.stringify({ additionalSuffixOfDatabaseName: "original" }));
|
||||||
|
state = new ReconstructionState(directory, path.join(directory, "vault"), settingsPath);
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => fs.rmSync(directory, { recursive: true, force: true }));
|
||||||
|
|
||||||
|
it("retains the persisted settings fingerprint across hash implementations", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
expect(JSON.parse(fs.readFileSync(state.markerPath, "utf-8"))).toMatchObject({
|
||||||
|
settingsBeforeHash: "29e04e3dd1d10b352678a922ff5a03fbe529d3505e9cf46c2ecc860f44478ae8",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("persists progress before returning and permits only the matching forced retry", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
expect(JSON.parse(fs.readFileSync(state.markerPath, "utf-8"))).toMatchObject({
|
||||||
|
phase: "running",
|
||||||
|
command: "fetch",
|
||||||
|
});
|
||||||
|
if (process.platform !== "win32") expect(fs.statSync(state.markerPath).mode & 0o777).toBe(0o600);
|
||||||
|
expect(() => state.prepare("sync", false)).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("ls", false, true)).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("fetch", false)).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("rebuild", true)).toThrow("interrupted fetch");
|
||||||
|
expect(() =>
|
||||||
|
new ReconstructionState(directory, path.join(directory, "other"), settingsPath).prepare("fetch", true)
|
||||||
|
).toThrow("interrupted fetch");
|
||||||
|
expect(() =>
|
||||||
|
new ReconstructionState(
|
||||||
|
directory,
|
||||||
|
path.join(directory, "vault"),
|
||||||
|
path.join(directory, "other.json")
|
||||||
|
).prepare("fetch", true)
|
||||||
|
).toThrow("interrupted fetch");
|
||||||
|
expect(() => state.prepare("ls", false)).not.toThrow();
|
||||||
|
expect(() => state.prepare("fetch", true)).not.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("recovers completed settings before permitting another command", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
const completed = JSON.stringify({ additionalSuffixOfDatabaseName: "headless-app", retained: true });
|
||||||
|
state.complete(completed);
|
||||||
|
|
||||||
|
new ReconstructionState(directory, path.join(directory, "vault"), settingsPath).prepare("sync", false);
|
||||||
|
|
||||||
|
expect(fs.readFileSync(settingsPath, "utf-8")).toBe(completed);
|
||||||
|
expect(readOptionalFile(state.markerPath)).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["normal completion", "restart"] as const)(
|
||||||
|
"preserves user settings changed during reconstruction at %s",
|
||||||
|
(phase) => {
|
||||||
|
state.prepare("rebuild", true);
|
||||||
|
state.complete(JSON.stringify({ additionalSuffixOfDatabaseName: "headless-app" }));
|
||||||
|
const edited = JSON.stringify({ additionalSuffixOfDatabaseName: "user-selection", unique: "edit" });
|
||||||
|
fs.writeFileSync(settingsPath, edited);
|
||||||
|
|
||||||
|
const commit =
|
||||||
|
phase === "restart"
|
||||||
|
? () =>
|
||||||
|
new ReconstructionState(directory, path.join(directory, "vault"), settingsPath).prepare(
|
||||||
|
"sync",
|
||||||
|
false
|
||||||
|
)
|
||||||
|
: () => state.commit();
|
||||||
|
|
||||||
|
expect(commit).toThrow("settings file changed");
|
||||||
|
expect(fs.readFileSync(settingsPath, "utf-8")).toBe(edited);
|
||||||
|
expect(JSON.parse(fs.readFileSync(state.markerPath, "utf-8"))).toMatchObject({ phase: "completed" });
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it("clears a completed marker after an interruption following the settings rename", () => {
|
||||||
|
state.prepare("fetch", true);
|
||||||
|
const completed = JSON.stringify({ additionalSuffixOfDatabaseName: "headless-app" });
|
||||||
|
state.complete(completed);
|
||||||
|
fs.writeFileSync(settingsPath, completed);
|
||||||
|
state.prepare("ls", false);
|
||||||
|
expect(readOptionalFile(state.markerPath)).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["null", "[]", "{}", '{"version":2}', '{"version":1,"phase":"completed"}'])(
|
||||||
|
"refuses malformed state %s without clearing it",
|
||||||
|
(contents) => {
|
||||||
|
fs.mkdirSync(path.dirname(state.markerPath), { recursive: true });
|
||||||
|
fs.writeFileSync(state.markerPath, contents);
|
||||||
|
expect(() => state.prepare("sync", false)).toThrow("Invalid reconstruction state");
|
||||||
|
expect(fs.readFileSync(state.markerPath, "utf-8")).toBe(contents);
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it.each(["null", "[]", '"text"', "{", ""])("reports invalid legacy KV %s without changing it", (contents) => {
|
||||||
|
const legacy = path.join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
fs.mkdirSync(path.dirname(legacy), { recursive: true });
|
||||||
|
fs.writeFileSync(legacy, contents);
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([{ path: legacy, reason: "invalid" }]);
|
||||||
|
expect(fs.readFileSync(legacy, "utf-8")).toBe(contents);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports both unreadable KV and invalid snapshot instead of hiding the second problem", () => {
|
||||||
|
const legacy = path.join(directory, ".livesync/runtime/keyvalue-db.json");
|
||||||
|
const snapshot = path.join(directory, ".livesync-snapshot.json");
|
||||||
|
fs.mkdirSync(legacy, { recursive: true });
|
||||||
|
fs.writeFileSync(snapshot, "{}");
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([
|
||||||
|
{ path: legacy, reason: "unreadable" },
|
||||||
|
{ path: snapshot, reason: "invalid" },
|
||||||
|
]);
|
||||||
|
expect(fs.statSync(legacy).isDirectory()).toBe(true);
|
||||||
|
expect(fs.readFileSync(snapshot, "utf-8")).toBe("{}");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves non-empty unowned legacy state and ignores empty legacy containers", () => {
|
||||||
|
const legacy = path.join(directory, ".livesync", "runtime", "keyvalue-db.json");
|
||||||
|
fs.mkdirSync(path.dirname(legacy), { recursive: true });
|
||||||
|
fs.writeFileSync(legacy, "{}");
|
||||||
|
fs.writeFileSync(path.join(directory, ".livesync-snapshot.json"), "[]");
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([]);
|
||||||
|
const queued = JSON.stringify({ "queued-files": ["unique-edit.md"] });
|
||||||
|
fs.writeFileSync(legacy, queued);
|
||||||
|
expect(inspectLegacyState(directory, directory)).toEqual([{ path: legacy, reason: "unowned" }]);
|
||||||
|
expect(fs.readFileSync(legacy, "utf-8")).toBe(queued);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -9,6 +9,7 @@ import type { IVaultService } from "@vrtmrz/livesync-commonlib/compat/services/b
|
|||||||
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
|
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
|
||||||
import { createInstanceLogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
|
import { createInstanceLogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
|
||||||
import { fs as nodeFs, path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
import { fs as nodeFs, path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
||||||
|
import { readOptionalFile, writeFileAtomically } from "@/apps/cli/runtimeState";
|
||||||
|
|
||||||
const NODE_KV_TYPED_KEY = "__nodeKvType";
|
const NODE_KV_TYPED_KEY = "__nodeKvType";
|
||||||
const NODE_KV_VALUES_KEY = "values";
|
const NODE_KV_VALUES_KEY = "values";
|
||||||
@@ -103,12 +104,11 @@ class NodeFileKeyValueDatabase implements KeyValueDatabase {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private load() {
|
private load() {
|
||||||
try {
|
const contents = readOptionalFile(this.filePath);
|
||||||
const loaded = JSON.parse(nodeFs.readFileSync(this.filePath, "utf-8")) as Record<string, unknown>;
|
if (contents === undefined) return;
|
||||||
this.data = new Map(Object.entries(loaded).map(([key, value]) => [key, deserializeFromNodeKV(value)]));
|
const loaded: unknown = JSON.parse(contents);
|
||||||
} catch {
|
if (!isRecord(loaded) || Array.isArray(loaded)) throw new Error("Invalid Node key-value database");
|
||||||
this.data = new Map();
|
this.data = new Map(Object.entries(loaded).map(([key, value]) => [key, deserializeFromNodeKV(value)]));
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private flush() {
|
private flush() {
|
||||||
@@ -116,7 +116,7 @@ class NodeFileKeyValueDatabase implements KeyValueDatabase {
|
|||||||
const serializable = Object.fromEntries(
|
const serializable = Object.fromEntries(
|
||||||
[...this.data.entries()].map(([key, value]) => [key, serializeForNodeKV(value)])
|
[...this.data.entries()].map(([key, value]) => [key, serializeForNodeKV(value)])
|
||||||
);
|
);
|
||||||
nodeFs.writeFileSync(this.filePath, JSON.stringify(serializable, null, 2), "utf-8");
|
writeFileAtomically(this.filePath, JSON.stringify(serializable, null, 2));
|
||||||
}
|
}
|
||||||
|
|
||||||
async get<T>(key: IDBValidKey): Promise<T> {
|
async get<T>(key: IDBValidKey): Promise<T> {
|
||||||
@@ -163,9 +163,7 @@ class NodeFileKeyValueDatabase implements KeyValueDatabase {
|
|||||||
return filtered;
|
return filtered;
|
||||||
}
|
}
|
||||||
|
|
||||||
async close(): Promise<void> {
|
async close(): Promise<void> {}
|
||||||
this.flush();
|
|
||||||
}
|
|
||||||
|
|
||||||
async destroy(): Promise<void> {
|
async destroy(): Promise<void> {
|
||||||
this.data.clear();
|
this.data.clear();
|
||||||
@@ -185,7 +183,8 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
{
|
{
|
||||||
private _kvDB: KeyValueDatabase | undefined;
|
private _kvDB: KeyValueDatabase | undefined;
|
||||||
private _simpleStore: SimpleStore<unknown> | undefined;
|
private _simpleStore: SimpleStore<unknown> | undefined;
|
||||||
private filePath: string;
|
private filePath: string | (() => string);
|
||||||
|
private openedFilePath: string | undefined;
|
||||||
private _log = createInstanceLogFunction("NodeKeyValueDBService");
|
private _log = createInstanceLogFunction("NodeKeyValueDBService");
|
||||||
|
|
||||||
get simpleStore() {
|
get simpleStore() {
|
||||||
@@ -202,7 +201,7 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
return this._kvDB;
|
return this._kvDB;
|
||||||
}
|
}
|
||||||
|
|
||||||
constructor(context: T, dependencies: NodeKeyValueDBDependencies<T>, filePath: string) {
|
constructor(context: T, dependencies: NodeKeyValueDBDependencies<T>, filePath: string | (() => string)) {
|
||||||
super(context);
|
super(context);
|
||||||
this.filePath = filePath;
|
this.filePath = filePath;
|
||||||
|
|
||||||
@@ -215,7 +214,13 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
|
|
||||||
private async openKeyValueDB(): Promise<boolean> {
|
private async openKeyValueDB(): Promise<boolean> {
|
||||||
try {
|
try {
|
||||||
this._kvDB = new NodeFileKeyValueDatabase(this.filePath);
|
const filePath = typeof this.filePath === "string" ? this.filePath : this.filePath();
|
||||||
|
if (this._kvDB && this.openedFilePath === filePath) return true;
|
||||||
|
await this._kvDB?.close();
|
||||||
|
this._kvDB = undefined;
|
||||||
|
this.openedFilePath = undefined;
|
||||||
|
this._kvDB = new NodeFileKeyValueDatabase(filePath);
|
||||||
|
this.openedFilePath = filePath;
|
||||||
return true;
|
return true;
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log("Failed to open Node key-value database", LOG_LEVEL_NOTICE);
|
this._log("Failed to open Node key-value database", LOG_LEVEL_NOTICE);
|
||||||
@@ -226,8 +231,10 @@ export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
|
|||||||
|
|
||||||
private async _everyOnResetDatabase(): Promise<boolean> {
|
private async _everyOnResetDatabase(): Promise<boolean> {
|
||||||
try {
|
try {
|
||||||
await this._kvDB?.del("queued-files");
|
if (!(await this.openKeyValueDB())) return false;
|
||||||
await this._kvDB?.destroy();
|
await this._kvDB?.destroy();
|
||||||
|
this._kvDB = undefined;
|
||||||
|
this.openedFilePath = undefined;
|
||||||
return await this.openKeyValueDB();
|
return await this.openKeyValueDB();
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log("Failed to reset Node key-value database", LOG_LEVEL_NOTICE);
|
this._log("Failed to reset Node key-value database", LOG_LEVEL_NOTICE);
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { describe, expect, it, vi } from "vitest";
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||||
import { createServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
|
import { createServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
|
||||||
import type { NodeKeyValueDBDependencies } from "./NodeKeyValueDBService";
|
import type { NodeKeyValueDBDependencies } from "./NodeKeyValueDBService";
|
||||||
import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
|
import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
|
||||||
@@ -45,3 +46,70 @@ describe("NodeKeyValueDBService.openSimpleStore", () => {
|
|||||||
await expect(store.get("key")).rejects.toThrow("KeyValueDB is not initialized yet");
|
await expect(store.get("key")).rejects.toThrow("KeyValueDB is not initialized yet");
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("NodeKeyValueDBService persistence", () => {
|
||||||
|
let directory: string;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
directory = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-node-kv-"));
|
||||||
|
});
|
||||||
|
afterEach(() => fs.rmSync(directory, { recursive: true, force: true }));
|
||||||
|
|
||||||
|
function compose(filePath: string | (() => string)) {
|
||||||
|
const handlers: Record<string, () => Promise<boolean>> = {};
|
||||||
|
const event = (name: string) => ({
|
||||||
|
addHandler: (handler: () => Promise<boolean>) => {
|
||||||
|
handlers[name] = handler;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const dependencies = {
|
||||||
|
appLifecycle: { onSettingLoaded: event("load") },
|
||||||
|
databaseEvents: {
|
||||||
|
onResetDatabase: event("reset"),
|
||||||
|
onDatabaseInitialisation: event("initialise"),
|
||||||
|
onUnloadDatabase: event("unload"),
|
||||||
|
onCloseDatabase: event("close"),
|
||||||
|
},
|
||||||
|
vault: {},
|
||||||
|
} as unknown as NodeKeyValueDBDependencies;
|
||||||
|
return { service: new NodeKeyValueDBService(createServiceContext(), dependencies, filePath), handlers };
|
||||||
|
}
|
||||||
|
|
||||||
|
it("retargets existing SimpleStore handles after database selection and resets only the selected file", async () => {
|
||||||
|
let selected = path.join(directory, "first.json");
|
||||||
|
const { service, handlers } = compose(() => selected);
|
||||||
|
const store = service.openSimpleStore("journal");
|
||||||
|
expect(await handlers.load()).toBe(true);
|
||||||
|
await store.set("checkpoint", new Set(["first"]));
|
||||||
|
selected = path.join(directory, "second.json");
|
||||||
|
expect(await handlers.initialise()).toBe(true);
|
||||||
|
expect(await store.get("checkpoint")).toBeUndefined();
|
||||||
|
await store.set("checkpoint", "second");
|
||||||
|
expect(await handlers.reset()).toBe(true);
|
||||||
|
expect(await store.get("checkpoint")).toBeUndefined();
|
||||||
|
selected = path.join(directory, "first.json");
|
||||||
|
expect(await handlers.initialise()).toBe(true);
|
||||||
|
expect(await store.get("checkpoint")).toEqual(new Set(["first"]));
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not overwrite newer data when a stale handle closes", async () => {
|
||||||
|
const filePath = path.join(directory, "state.json");
|
||||||
|
const first = compose(filePath);
|
||||||
|
await first.handlers.load();
|
||||||
|
await first.service.kvDB.set("queued-files", ["original.md"]);
|
||||||
|
const second = compose(filePath);
|
||||||
|
await second.handlers.load();
|
||||||
|
await first.service.kvDB.set("queued-files", ["unique-edit.md"]);
|
||||||
|
await second.handlers.close();
|
||||||
|
expect(JSON.parse(fs.readFileSync(filePath, "utf-8"))["queued-files"]).toEqual(["unique-edit.md"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["{", "null", "[]"])("refuses corrupt state %s without replacing it", async (contents) => {
|
||||||
|
const filePath = path.join(directory, "state.json");
|
||||||
|
fs.writeFileSync(filePath, contents);
|
||||||
|
const { service, handlers } = compose(filePath);
|
||||||
|
expect(await handlers.load()).toBe(false);
|
||||||
|
expect(() => service.kvDB).toThrow("not initialized");
|
||||||
|
expect(fs.readFileSync(filePath, "utf-8")).toBe(contents);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -6,7 +6,8 @@ import { ServiceContext, type StandardIo } from "@vrtmrz/livesync-commonlib/cont
|
|||||||
export class NodeServiceContext extends ServiceContext {
|
export class NodeServiceContext extends ServiceContext {
|
||||||
constructor(
|
constructor(
|
||||||
readonly databasePath: string,
|
readonly databasePath: string,
|
||||||
readonly standardIo: StandardIo
|
readonly standardIo: StandardIo,
|
||||||
|
readonly runtimeVaultPath: string = databasePath
|
||||||
) {
|
) {
|
||||||
super({ events: eventHub, translate: translateLiveSyncMessage });
|
super({ events: eventHub, translate: translateLiveSyncMessage });
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -24,11 +24,14 @@ import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
|
|||||||
import { createNodeLocalStorage } from "./NodeLocalStorage";
|
import { createNodeLocalStorage } from "./NodeLocalStorage";
|
||||||
import { DatabaseService } from "@vrtmrz/livesync-commonlib/compat/services/base/DatabaseService";
|
import { DatabaseService } from "@vrtmrz/livesync-commonlib/compat/services/base/DatabaseService";
|
||||||
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
import { path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
import { fs, path as nodePath } from "@vrtmrz/livesync-commonlib/node";
|
||||||
import type { KeyValueDBService } from "@vrtmrz/livesync-commonlib/compat/services/base/KeyValueDBService";
|
import type { KeyValueDBService } from "@vrtmrz/livesync-commonlib/compat/services/base/KeyValueDBService";
|
||||||
import { PouchDB } from "@/apps/cli/lib/pouchdb-node";
|
import { PouchDB } from "@/apps/cli/lib/pouchdb-node";
|
||||||
import { NodeServiceContext } from "./NodeServiceContext";
|
import { NodeServiceContext } from "./NodeServiceContext";
|
||||||
import { setLang } from "@/common/translation";
|
import { setLang } from "@/common/translation";
|
||||||
|
import { runtimeDatabaseDirectory, runtimeScopeDirectory } from "@/apps/cli/runtimeState";
|
||||||
|
import { createDeletableValueStore } from "@vrtmrz/value-store";
|
||||||
|
import type { SettingServiceDependencies } from "@vrtmrz/livesync-commonlib/compat/services/base/SettingService";
|
||||||
|
|
||||||
export { NodeServiceContext } from "./NodeServiceContext";
|
export { NodeServiceContext } from "./NodeServiceContext";
|
||||||
|
|
||||||
@@ -86,17 +89,31 @@ class NodeUIService<T extends ServiceContext> extends UIService<T> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
class NodeSettingService<T extends NodeServiceContext> extends InjectableSettingService<T> {
|
||||||
|
declare readonly config: InjectableSettingService<T>["config"];
|
||||||
|
|
||||||
|
constructor(context: T, dependencies: SettingServiceDependencies) {
|
||||||
|
super(context, dependencies);
|
||||||
|
const databaseConfig = this.config;
|
||||||
|
const vaultScope = nodePath.basename(runtimeScopeDirectory(context.databasePath, "", context.runtimeVaultPath));
|
||||||
|
this.config = createDeletableValueStore({
|
||||||
|
get: (key) => databaseConfig.getValue(`${key}-${vaultScope}`),
|
||||||
|
set: (key, value) => databaseConfig.setValue(`${key}-${vaultScope}`, value),
|
||||||
|
delete: (key) => databaseConfig.delete(`${key}-${vaultScope}`),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServiceHub<T> {
|
export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServiceHub<T> {
|
||||||
constructor(basePath: string, context: T) {
|
constructor(basePath: string, context: T) {
|
||||||
const runtimeDir = nodePath.join(basePath, ".livesync", "runtime");
|
const runtimeDir = nodePath.join(basePath, ".livesync", "runtime");
|
||||||
const localStoragePath = nodePath.join(runtimeDir, "local-storage.json");
|
const localStoragePath = nodePath.join(runtimeDir, "local-storage.json");
|
||||||
const keyValueDBPath = nodePath.join(runtimeDir, "keyvalue-db.json");
|
|
||||||
|
|
||||||
const API = new HeadlessAPIService<T>(context);
|
const API = new HeadlessAPIService<T>(context);
|
||||||
const conflict = new InjectableConflictService(context);
|
const conflict = new InjectableConflictService(context);
|
||||||
const fileProcessing = new InjectableFileProcessingService(context);
|
const fileProcessing = new InjectableFileProcessingService(context);
|
||||||
|
|
||||||
const setting = new InjectableSettingService<T>(context, {
|
const setting = new NodeSettingService<T>(context, {
|
||||||
APIService: API,
|
APIService: API,
|
||||||
onDisplayLanguageChanged: setLang,
|
onDisplayLanguageChanged: setLang,
|
||||||
deviceValues: createNodeLocalStorage(localStoragePath),
|
deviceValues: createNodeLocalStorage(localStoragePath),
|
||||||
@@ -118,8 +135,13 @@ export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServ
|
|||||||
settingService: setting,
|
settingService: setting,
|
||||||
APIService: API,
|
APIService: API,
|
||||||
});
|
});
|
||||||
|
const scopeDirectory = () => runtimeScopeDirectory(basePath, vault.getVaultName(), context.runtimeVaultPath);
|
||||||
const test = new InjectableTestService(context);
|
const test = new InjectableTestService(context);
|
||||||
const databaseEvents = new InjectableDatabaseEventService(context);
|
const databaseEvents = new InjectableDatabaseEventService(context);
|
||||||
|
databaseEvents.onResetDatabase.addHandler(async () => {
|
||||||
|
fs.rmSync(runtimeDatabaseDirectory(basePath, vault.getVaultName()), { force: true, recursive: true });
|
||||||
|
return true;
|
||||||
|
});
|
||||||
const path = new PathServiceCompat(context, {
|
const path = new PathServiceCompat(context, {
|
||||||
settingService: setting,
|
settingService: setting,
|
||||||
});
|
});
|
||||||
@@ -156,7 +178,7 @@ export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServ
|
|||||||
databaseEvents,
|
databaseEvents,
|
||||||
vault,
|
vault,
|
||||||
},
|
},
|
||||||
keyValueDBPath
|
() => nodePath.join(scopeDirectory(), "keyvalue-db.json")
|
||||||
);
|
);
|
||||||
|
|
||||||
const control = new ControlService(context, {
|
const control = new ControlService(context, {
|
||||||
|
|||||||
@@ -77,6 +77,48 @@ npm run test:e2e:docker:mirror --workspace self-hosted-livesync-cli
|
|||||||
|
|
||||||
The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Node.js and Bash suites.
|
The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Node.js and Bash suites.
|
||||||
|
|
||||||
|
## CLI Fetch database continuity
|
||||||
|
|
||||||
|
The standard CLI CI suite and direct task runner execute
|
||||||
|
`test:fetch-database-continuity` against a dedicated RustFS container. Each
|
||||||
|
scenario creates its own bucket. The test binds a uniquely named container to a
|
||||||
|
dynamic loopback port, then stops only that container. Redacted evidence is
|
||||||
|
written under the operating system's temporary directory and is not uploaded as
|
||||||
|
an artefact.
|
||||||
|
|
||||||
|
For a local run, build the CLI and run
|
||||||
|
`node test/e2e-cli/run-task.ts test:fetch-database-continuity` with Node.js 26
|
||||||
|
and Docker available. The managed container uses `accessKey` and `secretKey`
|
||||||
|
from the CLI `.test.env` file when present, and defaults to `minioadmin` for
|
||||||
|
both. No endpoint, bucket, or evidence-directory variables are needed. Set
|
||||||
|
`LIVESYNC_ISSUE1304_EVIDENCE_DIR` to choose another evidence directory outside
|
||||||
|
the worktree.
|
||||||
|
|
||||||
|
To use an external S3-compatible fixture without Docker, set all four
|
||||||
|
`LIVESYNC_ISSUE1304_MINIO_ENDPOINT`, `LIVESYNC_ISSUE1304_MINIO_ACCESS_KEY`,
|
||||||
|
`LIVESYNC_ISSUE1304_MINIO_SECRET_KEY`, and `LIVESYNC_ISSUE1304_MINIO_BUCKET`.
|
||||||
|
The bucket must be fresh and empty for every run. Partial external fixture
|
||||||
|
configuration is rejected. This test writes remote data; never point it at a
|
||||||
|
production bucket.
|
||||||
|
|
||||||
|
The test seeds an encrypted, path-obfuscated file with a second-generation
|
||||||
|
revision, imports one Setup URI on two clients, fetches on A without
|
||||||
|
`--write-settings`, and restarts A while B remains online. It checks the saved
|
||||||
|
database suffix, file bytes, original revision, and absence of a duplicate root.
|
||||||
|
Managed runs execute the explicit `--write-settings` control automatically in
|
||||||
|
a separate scenario bucket. For an external fixture, set
|
||||||
|
`LIVESYNC_ISSUE1304_POSITIVE_CONTROL=1` and
|
||||||
|
`LIVESYNC_ISSUE1304_POSITIVE_CONTROL_BUCKET` to a second fresh bucket to run
|
||||||
|
that control.
|
||||||
|
|
||||||
|
For historical reproduction against the unmodified implementation, set
|
||||||
|
`LIVESYNC_ISSUE1304_VERIFY_LEGACY_FAILURE=1`. This preserves the original
|
||||||
|
same-content conflict-resolution sequence and its file-preservation assertion;
|
||||||
|
the affected implementation fails because A's file disappears. Leave this
|
||||||
|
variable unset when validating the correction. The test saves redacted process
|
||||||
|
logs, file hashes, and database evidence. It does not cover Obsidian runtime or
|
||||||
|
CouchDB behaviour.
|
||||||
|
|
||||||
## Object Storage Chunk recovery regression tests
|
## Object Storage Chunk recovery regression tests
|
||||||
|
|
||||||
After building the CLI, run the focused recovery test with Node.js 26 or later:
|
After building the CLI, run the focused recovery test with Node.js 26 or later:
|
||||||
|
|||||||
@@ -15,5 +15,6 @@
|
|||||||
"test:e2e-matrix:couchdb-enc1",
|
"test:e2e-matrix:couchdb-enc1",
|
||||||
"test:e2e-matrix:minio-enc0",
|
"test:e2e-matrix:minio-enc0",
|
||||||
"test:e2e-matrix:minio-enc1",
|
"test:e2e-matrix:minio-enc1",
|
||||||
|
"test:fetch-database-continuity",
|
||||||
"test:journal-chunk-recovery"
|
"test:journal-chunk-recovery"
|
||||||
]
|
]
|
||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { randomUUID } from "node:crypto";
|
||||||
import { writeSync } from "node:fs";
|
import { writeSync } from "node:fs";
|
||||||
import { spawnProcess } from "./process.ts";
|
import { spawnProcess } from "./process.ts";
|
||||||
/**
|
/**
|
||||||
@@ -577,6 +578,59 @@ export async function startMinio(
|
|||||||
await waitForMinioBucket(minioEndpoint, accessKey, secretKey, bucket);
|
await waitForMinioBucket(minioEndpoint, accessKey, secretKey, bucket);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type IsolatedMinioFixture = {
|
||||||
|
endpoint: string;
|
||||||
|
accessKey: string;
|
||||||
|
secretKey: string;
|
||||||
|
stop: () => Promise<void>;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Start an S3 fixture owned by this caller without touching the shared test container. */
|
||||||
|
export async function startIsolatedMinio(accessKey: string, secretKey: string): Promise<IsolatedMinioFixture> {
|
||||||
|
const container = `livesync-cli-continuity-${randomUUID()}`;
|
||||||
|
console.log(`[INFO] starting isolated S3 test container: ${container}`);
|
||||||
|
let tracked = false;
|
||||||
|
const stop = async () => {
|
||||||
|
await stopAndRemoveContainer(container);
|
||||||
|
if (tracked) untrackContainer(container);
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
await dockerOrFail(
|
||||||
|
"run",
|
||||||
|
"-d",
|
||||||
|
"--name",
|
||||||
|
container,
|
||||||
|
"-p",
|
||||||
|
"127.0.0.1::9000",
|
||||||
|
"-e",
|
||||||
|
`RUSTFS_ACCESS_KEY=${accessKey}`,
|
||||||
|
"-e",
|
||||||
|
`RUSTFS_SECRET_KEY=${secretKey}`,
|
||||||
|
"-e",
|
||||||
|
"RUSTFS_CONSOLE_ENABLE=true",
|
||||||
|
"-e",
|
||||||
|
"RUSTFS_CORS_ALLOWED_ORIGINS=*",
|
||||||
|
S3_IMAGE,
|
||||||
|
"/data"
|
||||||
|
);
|
||||||
|
trackContainer(container);
|
||||||
|
tracked = true;
|
||||||
|
|
||||||
|
const mapping = await dockerOrFail("port", container, "9000/tcp");
|
||||||
|
const port = Number(mapping.trim().match(/:(\d+)$/)?.[1]);
|
||||||
|
if (!Number.isInteger(port) || port < 1 || port > 65535) {
|
||||||
|
throw new Error(`Could not determine the isolated S3 fixture port from: ${mapping.trim()}`);
|
||||||
|
}
|
||||||
|
const endpoint = `http://127.0.0.1:${port}`;
|
||||||
|
await fetchRetry(endpoint, { method: "GET" }, 30, 1000, [403, 404]);
|
||||||
|
return { endpoint, accessKey, secretKey, stop };
|
||||||
|
} catch (error) {
|
||||||
|
await stop();
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// P2P relay (strfry)
|
// P2P relay (strfry)
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
{
|
{
|
||||||
"test:helpers": {
|
"test:helpers": {
|
||||||
"test": [
|
"test": [
|
||||||
"helpers/process.test.ts"
|
"helpers/process.test.ts",
|
||||||
|
"test-ci-task-contract.ts"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"test": {
|
"test": {
|
||||||
@@ -47,6 +48,11 @@
|
|||||||
"test-settings-writeback.ts"
|
"test-settings-writeback.ts"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
"test:fetch-database-continuity": {
|
||||||
|
"test": [
|
||||||
|
"test-fetch-database-continuity.ts"
|
||||||
|
]
|
||||||
|
},
|
||||||
"test:remote-administration-exit-codes": {
|
"test:remote-administration-exit-codes": {
|
||||||
"test": [
|
"test": [
|
||||||
"test-remote-administration-exit-codes.ts"
|
"test-remote-administration-exit-codes.ts"
|
||||||
|
|||||||
@@ -0,0 +1,18 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { readFile } from "node:fs/promises";
|
||||||
|
import { test } from "node:test";
|
||||||
|
|
||||||
|
test("fetch database continuity is exercised by standard CLI CI", async () => {
|
||||||
|
const ciTasks = JSON.parse(await readFile(new URL("./ci-tasks.json", import.meta.url), "utf8")) as string[];
|
||||||
|
const tasks = JSON.parse(await readFile(new URL("./tasks.json", import.meta.url), "utf8")) as Record<
|
||||||
|
string,
|
||||||
|
{ test?: string[] }
|
||||||
|
>;
|
||||||
|
const workflow = await readFile(new URL("../../.github/workflows/cli-node-tests.yml", import.meta.url), "utf8");
|
||||||
|
|
||||||
|
assert(ciTasks.includes("test:helpers"), "the standard CI suite must run the task registration contract");
|
||||||
|
assert(tasks["test:helpers"]?.test?.includes("test-ci-task-contract.ts"));
|
||||||
|
assert(ciTasks.includes("test:fetch-database-continuity"), "the continuity task must be in the standard CI suite");
|
||||||
|
assert.deepEqual(tasks["test:fetch-database-continuity"]?.test, ["test-fetch-database-continuity.ts"]);
|
||||||
|
assert.match(workflow, /test\/e2e-cli\/ci-tasks\.json/, "the automatic CI matrix must read the authoritative task list");
|
||||||
|
});
|
||||||
@@ -0,0 +1,637 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { createHash, randomUUID } from "node:crypto";
|
||||||
|
import { cp, mkdir, readFile, readdir, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join, resolve } from "node:path";
|
||||||
|
import { after, before, test } from "node:test";
|
||||||
|
import { CreateBucketCommand, S3Client } from "@aws-sdk/client-s3";
|
||||||
|
import { CLI_DIR, CLI_DIST } from "./helpers/cli.ts";
|
||||||
|
import { startIsolatedMinio, type IsolatedMinioFixture } from "./helpers/docker.ts";
|
||||||
|
import { applyRemoteSyncSettings, generateSetupUriFromSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||||
|
import { TempDir } from "./helpers/temp.ts";
|
||||||
|
import { spawnProcess, type RunningProcess } from "./helpers/process.ts";
|
||||||
|
|
||||||
|
type RemoteSettings = {
|
||||||
|
endpoint: string;
|
||||||
|
accessKey: string;
|
||||||
|
secretKey: string;
|
||||||
|
bucket: string;
|
||||||
|
evidenceDir: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type CliResult = { code: number; stdout: string; stderr: string; combined: string };
|
||||||
|
type ScenarioMode = "reproduction" | "write-settings-control";
|
||||||
|
|
||||||
|
const SETUP_PASSPHRASE = "issue-1304-setup-passphrase";
|
||||||
|
const RESOLUTION_MARKER = "Resolving [same]: Conflicted revision has been deleted";
|
||||||
|
const INITIAL_REPLICATION_COMPLETE = "[Daemon] Initial replication complete";
|
||||||
|
const INITIAL_REPLICATION_FAILED = "[Daemon] Initial replication failed";
|
||||||
|
const VERIFY_LEGACY_FAILURE = process.env["LIVESYNC_ISSUE1304_VERIFY_LEGACY_FAILURE"] === "1";
|
||||||
|
const EXPLICIT_FIXTURE_VARIABLES = [
|
||||||
|
"LIVESYNC_ISSUE1304_MINIO_ENDPOINT",
|
||||||
|
"LIVESYNC_ISSUE1304_MINIO_ACCESS_KEY",
|
||||||
|
"LIVESYNC_ISSUE1304_MINIO_SECRET_KEY",
|
||||||
|
"LIVESYNC_ISSUE1304_MINIO_BUCKET",
|
||||||
|
];
|
||||||
|
const EXPLICIT_FIXTURE = EXPLICIT_FIXTURE_VARIABLES.some((name) => process.env[name]?.trim());
|
||||||
|
|
||||||
|
if (EXPLICIT_FIXTURE && EXPLICIT_FIXTURE_VARIABLES.some((name) => !process.env[name]?.trim())) {
|
||||||
|
throw new Error(`External fixture mode requires all of: ${EXPLICIT_FIXTURE_VARIABLES.join(", ")}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
let managedFixture: IsolatedMinioFixture | undefined;
|
||||||
|
if (!EXPLICIT_FIXTURE) {
|
||||||
|
before(async () => {
|
||||||
|
managedFixture = await startIsolatedMinio(
|
||||||
|
process.env["accessKey"]?.trim() ?? "minioadmin",
|
||||||
|
process.env["secretKey"]?.trim() ?? "minioadmin"
|
||||||
|
);
|
||||||
|
});
|
||||||
|
after(async () => {
|
||||||
|
await managedFixture?.stop();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function requiredEnv(...names: string[]): string {
|
||||||
|
for (const name of names) {
|
||||||
|
const value = process.env[name]?.trim();
|
||||||
|
if (value) return value;
|
||||||
|
}
|
||||||
|
throw new Error(`Required environment variable is missing: ${names.join(" or ")}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function evidenceDirectory(): string {
|
||||||
|
return resolve(
|
||||||
|
process.env["LIVESYNC_ISSUE1304_EVIDENCE_DIR"]?.trim() || join(tmpdir(), "livesync-cli-issue1304-evidence")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function remoteSettings(): RemoteSettings {
|
||||||
|
return {
|
||||||
|
endpoint: requiredEnv("LIVESYNC_ISSUE1304_MINIO_ENDPOINT").replace(/\/$/, ""),
|
||||||
|
accessKey: requiredEnv("LIVESYNC_ISSUE1304_MINIO_ACCESS_KEY"),
|
||||||
|
secretKey: requiredEnv("LIVESYNC_ISSUE1304_MINIO_SECRET_KEY"),
|
||||||
|
bucket: requiredEnv("LIVESYNC_ISSUE1304_MINIO_BUCKET"),
|
||||||
|
evidenceDir: evidenceDirectory(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function createManagedBucket(remote: Pick<RemoteSettings, "endpoint" | "accessKey" | "secretKey">, mode: ScenarioMode): Promise<string> {
|
||||||
|
const shortMode = mode === "reproduction" ? "repro" : "control";
|
||||||
|
const bucket = `livesync-ci-1304-${shortMode}-${randomUUID().replaceAll("-", "")}`;
|
||||||
|
const client = new S3Client({
|
||||||
|
endpoint: remote.endpoint,
|
||||||
|
region: "us-east-1",
|
||||||
|
forcePathStyle: true,
|
||||||
|
credentials: { accessKeyId: remote.accessKey, secretAccessKey: remote.secretKey },
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
await client.send(new CreateBucketCommand({ Bucket: bucket }));
|
||||||
|
} finally {
|
||||||
|
client.destroy();
|
||||||
|
}
|
||||||
|
return bucket;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function scenarioRemoteSettings(mode: ScenarioMode): Promise<RemoteSettings> {
|
||||||
|
if (EXPLICIT_FIXTURE) {
|
||||||
|
const remote = remoteSettings();
|
||||||
|
if (mode === "write-settings-control") remote.bucket = requiredEnv("LIVESYNC_ISSUE1304_POSITIVE_CONTROL_BUCKET");
|
||||||
|
return remote;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!managedFixture) throw new Error("The managed S3 fixture was not started");
|
||||||
|
const remote = {
|
||||||
|
endpoint: managedFixture.endpoint,
|
||||||
|
accessKey: managedFixture.accessKey,
|
||||||
|
secretKey: managedFixture.secretKey,
|
||||||
|
};
|
||||||
|
return {
|
||||||
|
...remote,
|
||||||
|
bucket: await createManagedBucket(remote, mode),
|
||||||
|
evidenceDir: evidenceDirectory(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function redactText(value: string, secrets: readonly string[]): string {
|
||||||
|
let redacted = value;
|
||||||
|
for (const secret of secrets) {
|
||||||
|
if (!secret) continue;
|
||||||
|
redacted = redacted.split(secret).join("[redacted]");
|
||||||
|
const encoded = encodeURIComponent(secret);
|
||||||
|
if (encoded !== secret) redacted = redacted.split(encoded).join("[redacted]");
|
||||||
|
}
|
||||||
|
return redacted;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function streamText(stream: ReadableStream<Uint8Array>, onText: (text: string) => void): Promise<void> {
|
||||||
|
const reader = stream.getReader();
|
||||||
|
const decoder = new TextDecoder();
|
||||||
|
try {
|
||||||
|
while (true) {
|
||||||
|
const { done, value } = await reader.read();
|
||||||
|
if (done) break;
|
||||||
|
if (value) onText(decoder.decode(value, { stream: true }));
|
||||||
|
}
|
||||||
|
const tail = decoder.decode();
|
||||||
|
if (tail) onText(tail);
|
||||||
|
} finally {
|
||||||
|
reader.releaseLock();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
class CapturedCliProcess {
|
||||||
|
readonly child: RunningProcess;
|
||||||
|
#stdout = "";
|
||||||
|
#stderr = "";
|
||||||
|
#secrets = [
|
||||||
|
process.env["LIVESYNC_ISSUE1304_MINIO_ENDPOINT"] ?? "",
|
||||||
|
process.env["LIVESYNC_ISSUE1304_MINIO_ACCESS_KEY"] ?? "",
|
||||||
|
process.env["LIVESYNC_ISSUE1304_MINIO_SECRET_KEY"] ?? "",
|
||||||
|
SETUP_PASSPHRASE,
|
||||||
|
"issue-1304-content-passphrase",
|
||||||
|
];
|
||||||
|
#streamsDone: Promise<void>;
|
||||||
|
|
||||||
|
constructor(args: string[]) {
|
||||||
|
this.child = spawnProcess(process.execPath, {
|
||||||
|
args: [CLI_DIST, ...args],
|
||||||
|
cwd: CLI_DIR,
|
||||||
|
stdin: "null",
|
||||||
|
stdout: "piped",
|
||||||
|
stderr: "piped",
|
||||||
|
});
|
||||||
|
this.#streamsDone = Promise.all([
|
||||||
|
streamText(this.child.stdout, (text) => (this.#stdout += text)),
|
||||||
|
streamText(this.child.stderr, (text) => (this.#stderr += text)),
|
||||||
|
]).then(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
get combined(): string {
|
||||||
|
return this.#stdout + this.#stderr;
|
||||||
|
}
|
||||||
|
|
||||||
|
async waitUntil(predicate: (output: string) => boolean, description: string, timeoutMs = 60_000): Promise<void> {
|
||||||
|
const deadline = Date.now() + timeoutMs;
|
||||||
|
while (Date.now() < deadline) {
|
||||||
|
if (predicate(this.combined)) return;
|
||||||
|
const tick = Math.min(100, deadline - Date.now());
|
||||||
|
const event = await Promise.race([
|
||||||
|
this.child.status.then((status) => ({ kind: "exit" as const, status })),
|
||||||
|
new Promise<{ kind: "tick" }>((resolveTick) => setTimeout(() => resolveTick({ kind: "tick" }), tick)),
|
||||||
|
]);
|
||||||
|
if (event.kind === "exit") {
|
||||||
|
await this.#streamsDone;
|
||||||
|
if (predicate(this.combined)) return;
|
||||||
|
throw new Error(
|
||||||
|
`${description}; process exited with ${event.status.code}\n${redactText(this.combined, this.#secrets).slice(-5000)}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw new Error(`Timed out waiting for ${description}\n${redactText(this.combined, this.#secrets).slice(-5000)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async stop(): Promise<number> {
|
||||||
|
try {
|
||||||
|
this.child.kill("SIGTERM");
|
||||||
|
} catch {
|
||||||
|
// A completed process needs no further signal.
|
||||||
|
}
|
||||||
|
const status = await this.child.status;
|
||||||
|
await this.#streamsDone;
|
||||||
|
return status.code;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runCliQuietly(args: string[], input?: string): Promise<CliResult> {
|
||||||
|
const child = spawnProcess(process.execPath, {
|
||||||
|
args: [CLI_DIST, ...args],
|
||||||
|
cwd: CLI_DIR,
|
||||||
|
stdin: input === undefined ? "null" : "piped",
|
||||||
|
stdout: "piped",
|
||||||
|
stderr: "piped",
|
||||||
|
});
|
||||||
|
let stdout = "";
|
||||||
|
let stderr = "";
|
||||||
|
const streamsDone = Promise.all([
|
||||||
|
streamText(child.stdout, (text) => (stdout += text)),
|
||||||
|
streamText(child.stderr, (text) => (stderr += text)),
|
||||||
|
]);
|
||||||
|
if (input !== undefined) {
|
||||||
|
const writer = child.stdin.getWriter();
|
||||||
|
await writer.write(new TextEncoder().encode(input));
|
||||||
|
await writer.close();
|
||||||
|
}
|
||||||
|
const [status] = await Promise.all([child.status, streamsDone]);
|
||||||
|
return { code: status.code, stdout, stderr, combined: stdout + stderr };
|
||||||
|
}
|
||||||
|
|
||||||
|
function countOccurrences(value: string, needle: string): number {
|
||||||
|
return value.split(needle).length - 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
function safeCommand(
|
||||||
|
label: string,
|
||||||
|
result: CliResult,
|
||||||
|
commandLog: Array<{ label: string; code: number; output: string }>,
|
||||||
|
secrets: readonly string[]
|
||||||
|
): void {
|
||||||
|
const output = redactText(result.combined, secrets);
|
||||||
|
commandLog.push({ label, code: result.code, output });
|
||||||
|
assert.equal(result.code, 0, `${label} failed with exit code ${result.code}\n${output.slice(-5000)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fileState(filePath: string): Promise<{ exists: boolean; bytes?: number; sha256?: string }> {
|
||||||
|
try {
|
||||||
|
const contents = await readFile(filePath);
|
||||||
|
return {
|
||||||
|
exists: true,
|
||||||
|
bytes: contents.byteLength,
|
||||||
|
sha256: createHash("sha256").update(contents).digest("hex"),
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return { exists: false };
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function collectDatabaseTree(root: string, relative = ""): Promise<Array<{ path: string; bytes: number; sha256: string }>> {
|
||||||
|
const result: Array<{ path: string; bytes: number; sha256: string }> = [];
|
||||||
|
let entries;
|
||||||
|
try {
|
||||||
|
entries = await readdir(join(root, relative), { withFileTypes: true });
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return result;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
entries.sort((left, right) => left.name.localeCompare(right.name));
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (!relative && entry.name === ".livesync") continue;
|
||||||
|
const child = relative ? join(relative, entry.name) : entry.name;
|
||||||
|
const absolute = join(root, child);
|
||||||
|
if (entry.isDirectory()) {
|
||||||
|
result.push(...(await collectDatabaseTree(root, child)));
|
||||||
|
} else if (entry.isFile()) {
|
||||||
|
const contents = await readFile(absolute);
|
||||||
|
result.push({
|
||||||
|
path: child,
|
||||||
|
bytes: contents.byteLength,
|
||||||
|
sha256: createHash("sha256").update(contents).digest("hex"),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function copyDatabaseTree(source: string, destination: string): Promise<void> {
|
||||||
|
await mkdir(destination, { recursive: true });
|
||||||
|
let entries;
|
||||||
|
try {
|
||||||
|
entries = await readdir(source, { withFileTypes: true });
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (entry.name === ".livesync") continue;
|
||||||
|
await cp(join(source, entry.name), join(destination, entry.name), { recursive: true, errorOnExist: false });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function sanitisedRuntimeState(root: string, secrets: readonly string[]): Promise<Record<string, string>> {
|
||||||
|
const runtimeRoot = join(root, ".livesync", "runtime");
|
||||||
|
const files: Record<string, string> = {};
|
||||||
|
let entries;
|
||||||
|
try {
|
||||||
|
entries = await readdir(runtimeRoot, { withFileTypes: true });
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return files;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (!entry.isFile() || !entry.name.endsWith(".json")) continue;
|
||||||
|
const text = await readFile(join(runtimeRoot, entry.name), "utf8");
|
||||||
|
try {
|
||||||
|
files[entry.name] = JSON.stringify(JSON.parse(text), (_key, value: unknown) => {
|
||||||
|
return typeof value === "string" ? redactText(value, secrets) : value;
|
||||||
|
}, 2);
|
||||||
|
} catch {
|
||||||
|
files[entry.name] = redactText(text, secrets);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return files;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function settingsSummary(settingsPath: string): Promise<Record<string, unknown> | null> {
|
||||||
|
try {
|
||||||
|
const settings = JSON.parse(await readFile(settingsPath, "utf8")) as Record<string, unknown>;
|
||||||
|
return {
|
||||||
|
additionalSuffixOfDatabaseName: settings.additionalSuffixOfDatabaseName ?? null,
|
||||||
|
remoteType: settings.remoteType ?? null,
|
||||||
|
liveSync: settings.liveSync ?? null,
|
||||||
|
syncOnStart: settings.syncOnStart ?? null,
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "ENOENT") return null;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runScenario(mode: ScenarioMode, remote: RemoteSettings): Promise<void> {
|
||||||
|
const evidenceRoot = remote.evidenceDir;
|
||||||
|
const evidenceName = `${mode}-${Date.now()}-${process.pid}`;
|
||||||
|
const evidenceDir = join(evidenceRoot, evidenceName);
|
||||||
|
const workDir = await TempDir.create(`livesync-cli-issue1304-${mode}`);
|
||||||
|
const scenarioId = `${Date.now()}-${process.pid}`;
|
||||||
|
const suffix = `issue1304-${scenarioId}`;
|
||||||
|
const targetPath = `notes/issue1304-${scenarioId}/continuity.md`;
|
||||||
|
const initialContent = `issue-1304 initial ${scenarioId}\n`;
|
||||||
|
const finalContent = `issue-1304 final ${scenarioId}\n`;
|
||||||
|
const finalBytes = new TextEncoder().encode(finalContent);
|
||||||
|
const secrets = [remote.endpoint, remote.accessKey, remote.secretKey, SETUP_PASSPHRASE, "issue-1304-content-passphrase"];
|
||||||
|
const commandLog: Array<{ label: string; code: number; output: string }> = [];
|
||||||
|
const processes: Record<string, CapturedCliProcess | undefined> = {};
|
||||||
|
const settingsTimeline: Record<string, unknown> = {};
|
||||||
|
const postResolutionInfo: Record<string, Record<string, unknown> | null> = {};
|
||||||
|
let observedFinalFileA: { exists: boolean; bytes?: number; sha256?: string } | undefined;
|
||||||
|
let observedFinalContentA: string | undefined;
|
||||||
|
let setupUri: string | undefined;
|
||||||
|
let failure: unknown;
|
||||||
|
|
||||||
|
const databaseSeed = workDir.join("database-seed");
|
||||||
|
const databaseA = workDir.join("database-a");
|
||||||
|
const databaseB = workDir.join("database-b");
|
||||||
|
const vaultSeed = workDir.join("vault-seed");
|
||||||
|
const vaultA = workDir.join("vault-a");
|
||||||
|
const vaultB = workDir.join("vault-b");
|
||||||
|
const settingsSeed = workDir.join("settings-seed.json");
|
||||||
|
const settingsA = workDir.join("settings-a.json");
|
||||||
|
const settingsB = workDir.join("settings-b.json");
|
||||||
|
const finalFileA = join(vaultA, targetPath);
|
||||||
|
const finalFileB = join(vaultB, targetPath);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await Promise.all([
|
||||||
|
mkdir(databaseSeed, { recursive: true }),
|
||||||
|
mkdir(databaseA, { recursive: true }),
|
||||||
|
mkdir(databaseB, { recursive: true }),
|
||||||
|
mkdir(vaultSeed, { recursive: true }),
|
||||||
|
mkdir(vaultA, { recursive: true }),
|
||||||
|
mkdir(vaultB, { recursive: true }),
|
||||||
|
]);
|
||||||
|
await initSettingsFile(settingsSeed);
|
||||||
|
await applyRemoteSyncSettings(settingsSeed, {
|
||||||
|
remoteType: "MINIO",
|
||||||
|
minioBucket: remote.bucket,
|
||||||
|
minioEndpoint: remote.endpoint,
|
||||||
|
minioAccessKey: remote.accessKey,
|
||||||
|
minioSecretKey: remote.secretKey,
|
||||||
|
encrypt: true,
|
||||||
|
passphrase: "issue-1304-content-passphrase",
|
||||||
|
usePathObfuscation: true,
|
||||||
|
});
|
||||||
|
const seedSettings = JSON.parse(await readFile(settingsSeed, "utf8")) as Record<string, unknown>;
|
||||||
|
seedSettings.additionalSuffixOfDatabaseName = suffix;
|
||||||
|
await writeFile(settingsSeed, JSON.stringify(seedSettings, null, 2));
|
||||||
|
settingsTimeline.setupSuffix = suffix;
|
||||||
|
setupUri = await generateSetupUriFromSettings(settingsSeed, SETUP_PASSPHRASE, true);
|
||||||
|
secrets.push(setupUri);
|
||||||
|
|
||||||
|
const seedRun = async (label: string, args: string[], input?: string) => {
|
||||||
|
const result = await runCliQuietly(args, input);
|
||||||
|
safeCommand(label, result, commandLog, secrets);
|
||||||
|
return result;
|
||||||
|
};
|
||||||
|
|
||||||
|
await seedRun("seed-put-initial", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "put", targetPath], initialContent);
|
||||||
|
await seedRun("seed-sync-initial", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "sync"]);
|
||||||
|
await seedRun("seed-put-final", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "put", targetPath], finalContent);
|
||||||
|
await seedRun("seed-sync-final", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "sync"]);
|
||||||
|
const seedInfoResult = await seedRun("seed-info", [databaseSeed, "--vault", vaultSeed, "--settings", settingsSeed, "info", targetPath]);
|
||||||
|
const seedInfo = JSON.parse(seedInfoResult.stdout) as { revision?: string; revisions?: string[] };
|
||||||
|
const seedRevisionGeneration = Number(seedInfo.revision?.split("-")[0] ?? 0);
|
||||||
|
assert(
|
||||||
|
seedRevisionGeneration >= 2 && (seedInfo.revisions?.length ?? 0) >= 1,
|
||||||
|
`Expected a remote seed revision with generation >= 2; got ${JSON.stringify({ revision: seedInfo.revision, revisions: seedInfo.revisions })}`
|
||||||
|
);
|
||||||
|
settingsTimeline.seedRevision = seedInfo.revision;
|
||||||
|
|
||||||
|
await initSettingsFile(settingsA);
|
||||||
|
await initSettingsFile(settingsB);
|
||||||
|
const setupA = await runCliQuietly(
|
||||||
|
[databaseA, "--vault", vaultA, "--settings", settingsA, "setup", setupUri],
|
||||||
|
`${SETUP_PASSPHRASE}\n`
|
||||||
|
);
|
||||||
|
safeCommand("setup-client-a", setupA, commandLog, secrets);
|
||||||
|
const setupB = await runCliQuietly(
|
||||||
|
[databaseB, "--vault", vaultB, "--settings", settingsB, "setup", setupUri],
|
||||||
|
`${SETUP_PASSPHRASE}\n`
|
||||||
|
);
|
||||||
|
safeCommand("setup-client-b", setupB, commandLog, secrets);
|
||||||
|
|
||||||
|
const bFetchArgs = [databaseB, "--vault", vaultB, "--settings", settingsB, "--force", "fetch", "--write-settings"];
|
||||||
|
const bFetch = await runCliQuietly(bFetchArgs);
|
||||||
|
safeCommand("fetch-client-b", bFetch, commandLog, secrets);
|
||||||
|
assert.equal(await readFile(finalFileB, "utf8"), finalContent, "Client B did not fetch the final file bytes");
|
||||||
|
settingsTimeline.clientBSuffixAfterFetch = (await settingsSummary(settingsB))?.additionalSuffixOfDatabaseName;
|
||||||
|
assert.equal(settingsTimeline.clientBSuffixAfterFetch, "headless-app");
|
||||||
|
|
||||||
|
// Client B remains live so it can resolve the equal-content root after A
|
||||||
|
// scans its old database and uploads a new root.
|
||||||
|
processes.clientB = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseB,
|
||||||
|
"--vault",
|
||||||
|
vaultB,
|
||||||
|
"--settings",
|
||||||
|
settingsB,
|
||||||
|
"--interval",
|
||||||
|
"1",
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientB.waitUntil((output) => output.includes(INITIAL_REPLICATION_COMPLETE), "Client B initial replication");
|
||||||
|
|
||||||
|
settingsTimeline.clientASuffixBeforeFetch = (await settingsSummary(settingsA))?.additionalSuffixOfDatabaseName;
|
||||||
|
assert.equal(settingsTimeline.clientASuffixBeforeFetch, suffix);
|
||||||
|
const aFetchArgs = [databaseA, "--vault", vaultA, "--settings", settingsA, "--force", "fetch"];
|
||||||
|
if (mode === "write-settings-control") aFetchArgs.push("--write-settings");
|
||||||
|
const aFetch = await runCliQuietly(aFetchArgs);
|
||||||
|
safeCommand("fetch-client-a", aFetch, commandLog, secrets);
|
||||||
|
assert.equal(await readFile(finalFileA, "utf8"), finalContent, "Client A did not fetch the final file bytes");
|
||||||
|
settingsTimeline.clientASuffixAfterFetch = (await settingsSummary(settingsA))?.additionalSuffixOfDatabaseName;
|
||||||
|
assert.equal(
|
||||||
|
settingsTimeline.clientASuffixAfterFetch,
|
||||||
|
mode === "reproduction" && VERIFY_LEGACY_FAILURE ? suffix : "headless-app",
|
||||||
|
"Client A settings suffix did not match the selected fetch mode"
|
||||||
|
);
|
||||||
|
|
||||||
|
// This startup selects A's persisted suffix. In the reproduction mode it
|
||||||
|
// scans the old, empty database while the fetched bytes are in the vault.
|
||||||
|
processes.clientAStartup = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseA,
|
||||||
|
"--vault",
|
||||||
|
vaultA,
|
||||||
|
"--settings",
|
||||||
|
settingsA,
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientAStartup.waitUntil(
|
||||||
|
(output) => output.includes(INITIAL_REPLICATION_COMPLETE) || output.includes(INITIAL_REPLICATION_FAILED),
|
||||||
|
"Client A startup replication result",
|
||||||
|
45_000
|
||||||
|
);
|
||||||
|
await processes.clientAStartup.stop();
|
||||||
|
|
||||||
|
const markResolved = await runCliQuietly([databaseA, "--vault", vaultA, "--settings", settingsA, "mark-resolved"]);
|
||||||
|
safeCommand("mark-resolved-client-a", markResolved, commandLog, secrets);
|
||||||
|
|
||||||
|
if (mode === "reproduction" && VERIFY_LEGACY_FAILURE) {
|
||||||
|
processes.clientA = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseA,
|
||||||
|
"--vault",
|
||||||
|
vaultA,
|
||||||
|
"--settings",
|
||||||
|
settingsA,
|
||||||
|
"--interval",
|
||||||
|
"1",
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientA.waitUntil((output) => output.includes(INITIAL_REPLICATION_COMPLETE), "Client A interval daemon initial replication");
|
||||||
|
|
||||||
|
const processingMarker = `Processing ${targetPath}`;
|
||||||
|
const aProcessingBeforeResolution = countOccurrences(processes.clientA.combined, processingMarker);
|
||||||
|
await processes.clientB.waitUntil((output) => output.includes(RESOLUTION_MARKER), "Client B automatic same-content resolution");
|
||||||
|
await processes.clientA.waitUntil(
|
||||||
|
(output) => countOccurrences(output, processingMarker) > aProcessingBeforeResolution,
|
||||||
|
"Client A to reflect the post-resolution revision",
|
||||||
|
45_000
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
processes.clientA = new CapturedCliProcess([
|
||||||
|
"-v",
|
||||||
|
databaseA,
|
||||||
|
"--vault",
|
||||||
|
vaultA,
|
||||||
|
"--settings",
|
||||||
|
settingsA,
|
||||||
|
"--interval",
|
||||||
|
"1",
|
||||||
|
"daemon",
|
||||||
|
]);
|
||||||
|
await processes.clientA.waitUntil((output) => output.includes(INITIAL_REPLICATION_COMPLETE), "Client A control daemon initial replication");
|
||||||
|
await new Promise((resolveDelay) => setTimeout(resolveDelay, 2_500));
|
||||||
|
assert.equal(await readFile(finalFileA, "utf8"), finalContent, "Client A control lost the fetched file");
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
failure = error;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const processCapture of Object.values(processes)) {
|
||||||
|
if (!processCapture) continue;
|
||||||
|
try {
|
||||||
|
await processCapture.stop();
|
||||||
|
} catch (error) {
|
||||||
|
if (failure === undefined) failure = error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await mkdir(evidenceDir, { recursive: true });
|
||||||
|
for (const [name, database, vault, settingsPath] of [
|
||||||
|
["clientA", databaseA, vaultA, settingsA],
|
||||||
|
["clientB", databaseB, vaultB, settingsB],
|
||||||
|
] as const) {
|
||||||
|
const result = await runCliQuietly([database, "--vault", vault, "--settings", settingsPath, "info", targetPath]);
|
||||||
|
const output = redactText(result.combined, secrets);
|
||||||
|
commandLog.push({ label: `post-resolution-info-${name}`, code: result.code, output });
|
||||||
|
if (result.code === 0) {
|
||||||
|
try {
|
||||||
|
postResolutionInfo[name] = JSON.parse(result.stdout) as Record<string, unknown>;
|
||||||
|
} catch {
|
||||||
|
postResolutionInfo[name] = null;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
postResolutionInfo[name] = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
observedFinalFileA = await fileState(finalFileA);
|
||||||
|
if (observedFinalFileA.exists) observedFinalContentA = await readFile(finalFileA, "utf8");
|
||||||
|
const databaseTrees: Record<string, Array<{ path: string; bytes: number; sha256: string }>> = {};
|
||||||
|
const runtimeState: Record<string, Record<string, string>> = {};
|
||||||
|
const databasePaths = { seed: databaseSeed, clientA: databaseA, clientB: databaseB };
|
||||||
|
for (const [name, path] of Object.entries(databasePaths)) {
|
||||||
|
databaseTrees[name] = await collectDatabaseTree(path);
|
||||||
|
await copyDatabaseTree(path, join(evidenceDir, `database-${name}`));
|
||||||
|
runtimeState[name] = await sanitisedRuntimeState(path, secrets);
|
||||||
|
if (Object.keys(runtimeState[name]).length > 0) {
|
||||||
|
const runtimeDir = join(evidenceDir, `runtime-${name}`);
|
||||||
|
await mkdir(runtimeDir, { recursive: true });
|
||||||
|
for (const [fileName, contents] of Object.entries(runtimeState[name])) {
|
||||||
|
await writeFile(join(runtimeDir, fileName), contents);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const logs = Object.fromEntries(
|
||||||
|
Object.entries(processes).map(([name, processCapture]) => [
|
||||||
|
name,
|
||||||
|
processCapture ? redactText(processCapture.combined, secrets) : "",
|
||||||
|
])
|
||||||
|
);
|
||||||
|
const fileStates = {
|
||||||
|
clientA: await fileState(finalFileA),
|
||||||
|
clientB: await fileState(finalFileB),
|
||||||
|
};
|
||||||
|
const evidence = {
|
||||||
|
scenario: mode,
|
||||||
|
targetPath,
|
||||||
|
expectedContentBytes: finalBytes.byteLength,
|
||||||
|
expectedContentSha256: createHash("sha256").update(finalBytes).digest("hex"),
|
||||||
|
settingsTimeline,
|
||||||
|
settings: {
|
||||||
|
seed: await settingsSummary(settingsSeed),
|
||||||
|
clientA: await settingsSummary(settingsA),
|
||||||
|
clientB: await settingsSummary(settingsB),
|
||||||
|
},
|
||||||
|
fileStates,
|
||||||
|
postResolutionInfo,
|
||||||
|
databaseTrees,
|
||||||
|
runtimeState,
|
||||||
|
commands: commandLog,
|
||||||
|
processLogs: logs,
|
||||||
|
remote: { endpoint: "[configured]", bucket: remote.bucket },
|
||||||
|
failure: failure instanceof Error ? redactText(failure.stack ?? failure.message, secrets) : String(failure ?? ""),
|
||||||
|
};
|
||||||
|
await writeFile(join(evidenceDir, "evidence.json"), JSON.stringify(evidence, null, 2));
|
||||||
|
} catch (error) {
|
||||||
|
if (failure === undefined) failure = error;
|
||||||
|
else if (failure instanceof Error) {
|
||||||
|
failure = new Error(`${failure.message}\nEvidence capture failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
await workDir[Symbol.asyncDispose]();
|
||||||
|
if (failure !== undefined) throw failure;
|
||||||
|
assert.equal(observedFinalFileA?.exists, true, "Client A file is absent after the resolved journal was applied");
|
||||||
|
assert.equal(observedFinalContentA, finalContent, "Client A lost the fetched file after remote conflict resolution");
|
||||||
|
if (!VERIFY_LEGACY_FAILURE) {
|
||||||
|
const infoA = postResolutionInfo.clientA;
|
||||||
|
const infoB = postResolutionInfo.clientB;
|
||||||
|
assert(infoA && infoB, "Could not inspect both local databases after the remote resolution");
|
||||||
|
assert.equal(infoA.conflicts, "N/A", "Client A still has a duplicate root revision after the resolution");
|
||||||
|
assert.equal(infoA.revision, infoB.revision, "Client A did not apply the revision selected by Client B after the resolved journal");
|
||||||
|
assert.equal(infoA.revision, settingsTimeline.seedRevision, "Daemon startup replaced the fetched revision ancestry");
|
||||||
|
assert.equal(processes.clientB?.combined.includes(RESOLUTION_MARKER), false, "Client A created an unnecessary competing root");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
test("CLI fetch: files remain after same-content duplicate root resolution", async () => {
|
||||||
|
await runScenario("reproduction", await scenarioRemoteSettings("reproduction"));
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!EXPLICIT_FIXTURE || process.env["LIVESYNC_ISSUE1304_POSITIVE_CONTROL"] === "1") {
|
||||||
|
test("CLI fetch: --write-settings keeps the selected database across daemon startup", async () => {
|
||||||
|
await runScenario("write-settings-control", await scenarioRemoteSettings("write-settings-control"));
|
||||||
|
});
|
||||||
|
}
|
||||||
+4
-1
@@ -12,13 +12,16 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
|
- CLI `fetch` and `rebuild` retain their selected local database for subsequent commands even without `--write-settings`, preventing the empty-database restart and file deletion reported in #1304.
|
||||||
|
- Runtime state is separated by database and Vault. Interrupted reconstruction blocks ordinary synchronisation until recovery, and a pending settings commit can be completed on restart.
|
||||||
|
- Older runtime state with no recorded owner is preserved and requires explicit reconstruction from a verified source before ordinary writes resume. Back up unique edits before choosing Fetch or Rebuild.
|
||||||
|
|
||||||
- Files containing a raw unit separator (`U+001F`) retain their complete contents when received through S3-compatible Object Storage. (#1303)
|
- Files containing a raw unit separator (`U+001F`) retain their complete contents when received through S3-compatible Object Storage. (#1303)
|
||||||
- The Journal receiver in Commonlib 0.2.3 preserves the complete Chunk payload, including batches written by earlier versions, without changing the stored format.
|
- The Journal receiver in Commonlib 0.2.3 preserves the complete Chunk payload, including batches written by earlier versions, without changing the stored format.
|
||||||
- Updating prevents further truncation during reception but does not repair existing damaged copies. Update every participating client, preserve an intact recovery source, and follow the [recovery procedure](docs/recovery.md#recover-files-truncated-during-object-storage-reception). Rebuild and Fetch leave Hidden File Sync and Customisation Sync disabled until they are deliberately initialised again.
|
- Updating prevents further truncation during reception but does not repair existing damaged copies. Update every participating client, preserve an intact recovery source, and follow the [recovery procedure](docs/recovery.md#recover-files-truncated-during-object-storage-reception). Rebuild and Fetch leave Hidden File Sync and Customisation Sync disabled until they are deliberately initialised again.
|
||||||
|
|
||||||
- Files waiting for encrypted Chunks which arrive after their Metadata receive the decrypted contents once the local database is ready.
|
- Files waiting for encrypted Chunks which arrive after their Metadata receive the decrypted contents once the local database is ready.
|
||||||
- Commonlib 0.2.3 restarts the database change feed after initialisation installs the decryption transforms. This corrects delivery of later Chunks; it does not replay earlier events. ([Commonlib #191](https://github.com/vrtmrz/livesync-commonlib/pull/191))
|
- Commonlib 0.2.3 restarts the database change feed after initialisation installs the decryption transforms. This corrects delivery of later Chunks; it does not replay earlier events. ([Commonlib #191](https://github.com/vrtmrz/livesync-commonlib/pull/191))
|
||||||
|
|
||||||
- The CLI mirror now starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. It checks logical Vault paths using `/` separators, correctly excluding internal database files and nested hidden directories even when the host uses Windows path separators.
|
- The CLI mirror now starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. It checks logical Vault paths using `/` separators, correctly excluding internal database files and nested hidden directories even when the host uses Windows path separators.
|
||||||
|
|
||||||
## 1.0.43
|
## 1.0.43
|
||||||
|
|||||||
Reference in new issue
Block a user