Merge completed fixes before shared runtime integration

This commit is contained in:
vorotamoroz committed 2026-11-01 11:50:54 +00:00
commit 9222852834
9 files changed
+233 -34

No files matched your search

+1 -1
View File
@@ -22,7 +22,7 @@ Do not switch transport, enable P2P, or run Garbage Collection as a substitute f
This procedure addresses [issue #1303](https://github.com/vrtmrz/obsidian-livesync/issues/1303): an older Journal receiver can truncate a Chunk at a raw unit separator (`U+001F`) within its data. This can damage a received text file, including a plug-in's `main.js`, while the original batch remains intact in the bucket. This procedure addresses [issue #1303](https://github.com/vrtmrz/obsidian-livesync/issues/1303): an older Journal receiver can truncate a Chunk at a raw unit separator (`U+001F`) within its data. This can damage a received text file, including a plug-in's `main.js`, while the original batch remains intact in the bucket.
The receiver correction is currently unreleased. Use a build containing that correction on every participating client before recovery; versions 1.0.42 and 1.0.43 still contain the defect. Updating alone does not repair Chunks already stored locally. Resetting only the Journal received history is also insufficient because ordinary reception skips existing Chunk IDs. The receiver correction is available in Commonlib 0.2.3 and is included in the unreleased Self-hosted LiveSync build. Use a build containing that correction on every participating client before recovery; versions 1.0.42 and 1.0.43 still contain the defect. Updating alone does not repair Chunks already stored locally. Resetting only the Journal received history is also insufficient because ordinary reception skips existing Chunk IDs.
### Preserve the intact source ### Preserve the intact source
+1 -1
View File
@@ -87,8 +87,8 @@ livesync-cli [database-path] [command] [args...]
- `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`. - `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`.
- The selected local database suffix is saved automatically, so later commands use the reconstructed database. - The selected local database suffix is saved automatically, so later commands use the reconstructed database.
- `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`. - `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`.
- Differing local files can remain and be preserved as conflict branches alongside the fetched content. For recovery from an intact remote, preserve the existing Vault separately and Fetch into an empty Vault; `--force` does not force file overwriting.
- The selected local database suffix is saved automatically, including when `--write-settings` is omitted. - The selected local database suffix is saved automatically, including when `--write-settings` is omitted.
- Differing local files can remain and be preserved as conflict branches alongside the fetched content. For recovery from an intact remote, preserve the existing Vault separately and Fetch into an empty Vault; `--force` does not force file overwriting.
- `ls [prefix]`: List files currently stored in the local database. - `ls [prefix]`: List files currently stored in the local database.
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`. - `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`. - `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
@@ -16,6 +16,43 @@ declare module "obsidian" {
} }
} }
const cacheReuseDirectives = new Set(["max-age", "max-stale", "min-fresh", "only-if-cached"]);
function splitHeaderDirectives(value: string): string[] {
const directives: string[] = [];
let start = 0;
let inQuotes = false;
let escaped = false;
for (let index = 0; index < value.length; index++) {
const character = value[index];
if (inQuotes && escaped) {
escaped = false;
continue;
}
if (inQuotes && character === "\\") {
escaped = true;
continue;
}
if (character === '"') {
inQuotes = !inQuotes;
} else if (character === "," && !inQuotes) {
const directive = value.slice(start, index).trim();
if (directive) directives.push(directive);
start = index + 1;
}
}
const lastDirective = value.slice(start).trim();
if (lastDirective) directives.push(lastDirective);
return directives;
}
function getDirectiveName(directive: string): string {
const equalsIndex = directive.indexOf("=");
return (equalsIndex === -1 ? directive : directive.slice(0, equalsIndex)).trim().toLowerCase();
}
export class ObsidianAPIService extends InjectableAPIService<ObsidianServiceContext> { export class ObsidianAPIService extends InjectableAPIService<ObsidianServiceContext> {
_customHandler: ObsHttpHandler | undefined; _customHandler: ObsHttpHandler | undefined;
_confirmInstance: Confirm; _confirmInstance: Confirm;
@@ -182,6 +219,30 @@ export class ObsidianAPIService extends InjectableAPIService<ObsidianServiceCont
delete transformedHeaders["Host"]; delete transformedHeaders["Host"];
delete transformedHeaders["content-length"]; delete transformedHeaders["content-length"];
delete transformedHeaders["Content-Length"]; delete transformedHeaders["Content-Length"];
const cacheMode = opts?.cache ?? (req instanceof Request ? req.cache : undefined);
if (cacheMode === "no-store" && ["GET", "HEAD"].includes(method.toUpperCase())) {
// requestUrl has no cache option; no-store alone can reuse an existing cached response.
const cacheControlKeys = Object.keys(transformedHeaders).filter(
(key) => key.toLowerCase() === "cache-control"
);
const cacheControlDirectives = cacheControlKeys
.flatMap((key) => splitHeaderDirectives(transformedHeaders[key]))
.filter((directive) => {
const name = getDirectiveName(directive);
return !cacheReuseDirectives.has(name) && name !== "no-cache" && name !== "no-store";
});
const cacheControlKey = cacheControlKeys[0] ?? "cache-control";
for (const key of cacheControlKeys) delete transformedHeaders[key];
transformedHeaders[cacheControlKey] = [...cacheControlDirectives, "no-cache", "no-store"].join(", ");
const pragmaKeys = Object.keys(transformedHeaders).filter((key) => key.toLowerCase() === "pragma");
const pragmaDirectives = pragmaKeys
.flatMap((key) => splitHeaderDirectives(transformedHeaders[key]))
.filter((directive) => getDirectiveName(directive) !== "no-cache");
const pragmaKey = pragmaKeys[0] ?? "pragma";
for (const key of pragmaKeys) delete transformedHeaders[key];
transformedHeaders[pragmaKey] = [...pragmaDirectives, "no-cache"].join(", ");
}
const contentType = const contentType =
transformedHeaders["content-type"] ?? transformedHeaders["Content-Type"] ?? "application/json"; transformedHeaders["content-type"] ?? transformedHeaders["Content-Type"] ?? "application/json";
const requestParam: RequestUrlParam = { const requestParam: RequestUrlParam = {
@@ -1,6 +1,14 @@
import { beforeEach, describe, expect, it, vi } from "vitest"; import { beforeEach, describe, expect, it, vi } from "vitest";
import type { RequestUrlParam } from "@/deps";
const mocks = vi.hoisted(() => ({ const mocks = vi.hoisted(() => ({
requestUrl: vi.fn<
(request: RequestUrlParam) => Promise<{
status: number;
headers: Record<string, string>;
arrayBuffer: ArrayBuffer;
}>
>(),
platform: { platform: {
isMobile: false, isMobile: false,
}, },
@@ -8,12 +16,12 @@ const mocks = vi.hoisted(() => ({
vi.mock("@/deps.ts", () => ({ vi.mock("@/deps.ts", () => ({
Platform: mocks.platform, Platform: mocks.platform,
requestUrl: vi.fn(), requestUrl: mocks.requestUrl,
})); }));
vi.mock("@/deps", () => ({ vi.mock("@/deps", () => ({
Platform: mocks.platform, Platform: mocks.platform,
requestUrl: vi.fn(), requestUrl: mocks.requestUrl,
})); }));
vi.mock("@/modules/essentialObsidian/APILib/ObsHttpHandler", () => ({ vi.mock("@/modules/essentialObsidian/APILib/ObsHttpHandler", () => ({
@@ -36,6 +44,8 @@ function createService(workspace: Record<string, unknown>, isMobile = false): Ob
beforeEach(() => { beforeEach(() => {
mocks.platform.isMobile = false; mocks.platform.isMobile = false;
vi.clearAllMocks(); vi.clearAllMocks();
mocks.requestUrl.mockReset();
mocks.requestUrl.mockResolvedValue({ status: 200, headers: {}, arrayBuffer: new ArrayBuffer(0) });
}); });
describe("ObsidianAPIService.showWindowOnRight", () => { describe("ObsidianAPIService.showWindowOnRight", () => {
@@ -65,3 +75,115 @@ describe("ObsidianAPIService.showWindowOnRight", () => {
expect(workspace.revealLeaf).toHaveBeenCalledWith(rightLeaf); expect(workspace.revealLeaf).toHaveBeenCalledWith(rightLeaf);
}); });
}); });
describe("ObsidianAPIService.nativeFetch cache control", () => {
const url = "https://couchdb.example.test/vault/f%3Ametadata";
function sentHeaders(): Headers {
expect(mocks.requestUrl).toHaveBeenCalledOnce();
return new Headers(mocks.requestUrl.mock.calls[0][0].headers);
}
it.each(["GET", "HEAD", "get", "head"])("bypasses cached responses for an explicit no-store %s", async (method) => {
const headers = {
Authorization: "Basic fixture",
"Cache-Control": "public, max-age=3600",
"cAcHe-CoNtRoL": "max-stale",
Pragma: "cache",
};
await createService({}).nativeFetch(url, { method, cache: "no-store", headers });
expect(sentHeaders().get("cache-control")).toBe("public, no-cache, no-store");
expect(sentHeaders().get("pragma")).toBe("cache, no-cache");
expect(sentHeaders().get("authorization")).toBe("Basic fixture");
expect(headers["Cache-Control"]).toBe("public, max-age=3600");
expect(mocks.requestUrl.mock.calls[0][0]).toMatchObject({ url, method });
});
it("preserves non-conflicting cache directives when enforcing no-store", async () => {
const headers = {
"Cache-Control": 'max-age=3600, no-transform, custom="a,b"',
"cAcHe-CoNtRoL": 'max-stale=30, min-fresh=5, only-if-cached, custom-two="max-age=1, only-if-cached"',
Pragma: 'legacy="a,b", no-cache',
"X-Custom-Header": "preserved",
};
const originalHeaders = { ...headers };
await createService({}).nativeFetch(url, { cache: "no-store", headers });
const sent = mocks.requestUrl.mock.calls[0][0].headers ?? {};
expect(sent["Cache-Control"]).toBe(
'no-transform, custom="a,b", custom-two="max-age=1, only-if-cached", no-cache, no-store'
);
expect(sent["cAcHe-CoNtRoL"]).toBeUndefined();
expect(sent.Pragma).toBe('legacy="a,b", no-cache');
expect(sent["X-Custom-Header"]).toBe("preserved");
expect(headers).toEqual(originalHeaders);
});
it("uses the Request cache mode when the options do not override it", async () => {
const request = new Request(url, {
method: "HEAD",
cache: "no-store",
headers: { "X-Fixture": "request" },
});
const headers = new Headers({ "X-Other": "options" });
await createService({}).nativeFetch(request, { headers });
expect(sentHeaders().get("cache-control")).toBe("no-cache, no-store");
expect(sentHeaders().get("pragma")).toBe("no-cache");
expect(sentHeaders().get("x-fixture")).toBe("request");
expect(sentHeaders().get("x-other")).toBe("options");
expect(headers.has("cache-control")).toBe(false);
expect(request.headers.has("cache-control")).toBe(false);
});
it("lets the options request no-store for an otherwise cacheable Request", async () => {
await createService({}).nativeFetch(new Request(url, { cache: "force-cache" }), { cache: "no-store" });
expect(sentHeaders().get("cache-control")).toBe("no-cache, no-store");
});
it("lets an explicit default cache mode override a no-store Request", async () => {
await createService({}).nativeFetch(new Request(url, { cache: "no-store" }), { cache: "default" });
expect(sentHeaders().has("cache-control")).toBe(false);
expect(sentHeaders().has("pragma")).toBe(false);
});
it.each([undefined, "default", "no-cache", "reload", "force-cache"] as const)(
"preserves caller headers for cache mode %s",
async (cache) => {
const headers = { "Cache-Control": "max-age=60", Pragma: "fixture" };
await createService({}).nativeFetch(url, { cache, headers });
expect(mocks.requestUrl.mock.calls[0][0].headers).toEqual(headers);
}
);
it("does not add cache directives to an ordinary request", async () => {
await createService({}).nativeFetch(url);
expect(sentHeaders().has("cache-control")).toBe(false);
expect(sentHeaders().has("pragma")).toBe(false);
});
it.each(["POST", "PUT", "DELETE"])("preserves a no-store %s request", async (method) => {
const headers = { "Cache-Control": "no-transform" };
await createService({}).nativeFetch(url, { method, cache: "no-store", headers });
expect(mocks.requestUrl.mock.calls[0][0]).toMatchObject({ url, method, headers });
expect(sentHeaders().has("pragma")).toBe(false);
});
it("uses the effective method from the options", async () => {
await createService({}).nativeFetch(new Request(url, { cache: "no-store" }), { method: "POST" });
expect(sentHeaders().has("cache-control")).toBe(false);
expect(sentHeaders().has("pragma")).toBe(false);
});
});
+15 -15
View File
@@ -77,21 +77,6 @@ npm run test:e2e:docker:mirror --workspace self-hosted-livesync-cli
The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Node.js and Bash suites. The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Node.js and Bash suites.
## Object Storage Chunk recovery regression tests
After building the CLI, run the focused recovery test with Node.js 26 or later:
```bash
NODE_OPTIONS=--max-old-space-size=3072 npm run build --workspace self-hosted-livesync-cli
NODE_OPTIONS=--max-old-space-size=3072 MINIO_ENDPOINT=http://127.0.0.1:9000 MINIO_ACCESS_KEY=minioadmin MINIO_SECRET_KEY=minioadmin MINIO_BUCKET_NAME=livesync-test npm run test:e2e:task --workspace self-hosted-livesync-cli -- test:journal-chunk-recovery
```
The test manages the disposable `minio-test` RustFS container on ports 9000 and 9001. Run it separately from other tests using that container or those ports. It creates a distinct bucket and temporary Vaults for each case and removes its test container afterwards.
Both the unencrypted and encrypted cases rebuild from a complete file containing raw unit separators (`U+001F`), Fetch into another Vault, and compare the exact bytes. A fresh CLI process reads the persisted content. The test then damages the receiving Vault file and verifies that Fetch retains that newer local file and preserves both contents as independent conflict branches. Each revision is read explicitly so the assertions do not depend on which branch PouchDB selects as the winner. After moving the damaged file to a separate backup, another Fetch into the empty Vault restores the remote contents. It retains `--write-settings` to exercise explicit settings writeback; database selection is now saved automatically, and the continuity test below covers omission of that option. This test is included in the standard CLI CI task list.
This establishes the CLI and shared Rebuild/Fetch behaviour. It does not exercise Obsidian's Hidden File Sync or Customisation Sync initialisation, nor inject a damaged Chunk under an existing content-addressed ID.
## CLI Fetch database continuity ## CLI Fetch database continuity
The standard CLI CI suite and direct task runner execute The standard CLI CI suite and direct task runner execute
@@ -134,6 +119,21 @@ variable unset when validating the correction. The test saves redacted process
logs, file hashes, and database evidence. It does not cover Obsidian runtime or logs, file hashes, and database evidence. It does not cover Obsidian runtime or
CouchDB behaviour. CouchDB behaviour.
## Object Storage Chunk recovery regression tests
After building the CLI, run the focused recovery test with Node.js 26 or later:
```bash
NODE_OPTIONS=--max-old-space-size=3072 npm run build --workspace self-hosted-livesync-cli
NODE_OPTIONS=--max-old-space-size=3072 MINIO_ENDPOINT=http://127.0.0.1:9000 MINIO_ACCESS_KEY=minioadmin MINIO_SECRET_KEY=minioadmin MINIO_BUCKET_NAME=livesync-test npm run test:e2e:task --workspace self-hosted-livesync-cli -- test:journal-chunk-recovery
```
The test manages the disposable `minio-test` RustFS container on ports 9000 and 9001. Run it separately from other tests using that container or those ports. It creates a distinct bucket and temporary Vaults for each case and removes its test container afterwards.
Both the unencrypted and encrypted cases rebuild from a complete file containing raw unit separators (`U+001F`), Fetch into another Vault, and compare the exact bytes. A fresh CLI process reads the persisted content. The test then damages the receiving Vault file and verifies that Fetch retains that newer local file and preserves both contents as independent conflict branches. Each revision is read explicitly so the assertions do not depend on which branch PouchDB selects as the winner. After moving the damaged file to a separate backup, another Fetch into the empty Vault restores the remote contents. It retains `--write-settings` to exercise explicit settings writeback; database selection is now saved automatically, and the continuity test below covers omission of that option. This test is included in the standard CLI CI task list.
This establishes the CLI and shared Rebuild/Fetch behaviour. It does not exercise Obsidian's Hidden File Sync or Customisation Sync initialisation, nor inject a damaged Chunk under an existing content-addressed ID.
## Multiple-device conflict regression tests ## Multiple-device conflict regression tests
### Preparation and execution ### Preparation and execution
+2 -2
View File
@@ -15,6 +15,6 @@
"test:e2e-matrix:couchdb-enc1", "test:e2e-matrix:couchdb-enc1",
"test:e2e-matrix:minio-enc0", "test:e2e-matrix:minio-enc0",
"test:e2e-matrix:minio-enc1", "test:e2e-matrix:minio-enc1",
"test:journal-chunk-recovery", "test:fetch-database-continuity",
"test:fetch-database-continuity" "test:journal-chunk-recovery"
] ]
+2
View File
@@ -174,6 +174,8 @@ The same workflow checks the two remote-activity status boundaries. It first hol
`test:e2e:obsidian:couchdb-http-cache` checks replication after rebuilding an encrypted CouchDB remote while an earlier HTTP response remains cached. It enables internal Metadata encryption with Hidden File Sync and Customisation Sync, verifies that the Rebuild replaces the Security Seed and ciphertext while retaining the document revision, and requires the replication pre-check and subsequent sends to succeed without clearing the browser cache. It uses the CouchDB fixture variables, an isolated Vault and profile, and a unique remote database. The raw command uses the current built plug-in by default; `E2E_OBSIDIAN_ARTIFACT_ROOT` can select an exact earlier build for regression comparison. `test:e2e:obsidian:couchdb-http-cache` checks replication after rebuilding an encrypted CouchDB remote while an earlier HTTP response remains cached. It enables internal Metadata encryption with Hidden File Sync and Customisation Sync, verifies that the Rebuild replaces the Security Seed and ciphertext while retaining the document revision, and requires the replication pre-check and subsequent sends to succeed without clearing the browser cache. It uses the CouchDB fixture variables, an isolated Vault and profile, and a unique remote database. The raw command uses the current built plug-in by default; `E2E_OBSIDIAN_ARTIFACT_ROOT` can select an exact earlier build for regression comparison.
Run `npm run test:e2e:obsidian:focused -- couchdb-http-cache --request-api` to warm Obsidian's Internal API cache and perform the same workflow with **Use Internal API** enabled. Both modes require an explicit `no-store` read to return the rebuilt ciphertext. The default mode also retains the stale browser-cache assertion; the Internal API mode permits reads during Rebuild to refresh that cache.
`npm run test:e2e:obsidian:focused -- chunk-fetch-retry` checks delayed Chunk availability through a real CouchDB service and Obsidian. It creates a Metadata-only remote fixture, starts ordinary one-shot replication with `readChunksOnline`, and inserts the missing Chunk only after a real fetch has returned an empty result. A pass-through observer records the replicator's call times and results without substituting responses or adding waits. The fixture sets the existing minimum request interval to 500 ms to keep the real retry status observable even if finite completion expedites the final probe. The actual status bar must show zero initial requests (`🛄`) and one retry (`🔁`), and both counts must return to zero after delivery ends. On-demand replication excludes Chunk documents from the ordinary pull, so the delayed Chunk must arrive through the observed fetch and produce the exact Vault content. `npm run test:e2e:obsidian:focused -- chunk-fetch-retry` checks delayed Chunk availability through a real CouchDB service and Obsidian. It creates a Metadata-only remote fixture, starts ordinary one-shot replication with `readChunksOnline`, and inserts the missing Chunk only after a real fetch has returned an empty result. A pass-through observer records the replicator's call times and results without substituting responses or adding waits. The fixture sets the existing minimum request interval to 500 ms to keep the real retry status observable even if finite completion expedites the final probe. The actual status bar must show zero initial requests (`🛄`) and one retry (`🔁`), and both counts must return to zero after delivery ends. On-demand replication excludes Chunk documents from the ordinary pull, so the delayed Chunk must arrive through the observed fetch and produce the exact Vault content.
If finite replication was already inactive when the initial lookup began, the scenario requires a retry at least two seconds later and no physical request slot occupied during backoff. If finite replication ends during the initial lookup or the following backoff, the retry must instead be a post-completion final probe before the two-second delay would expire. This distinction is determined from the observed finite-count transitions, not an assumed ordering between replication and HTTP completion. If finite replication was already inactive when the initial lookup began, the scenario requires a retry at least two seconds later and no physical request slot occupied during backoff. If finite replication ends during the initial lookup or the following backoff, the retry must instead be a post-completion final probe before the two-second delay would expire. This distinction is determined from the observed finite-count transitions, not an assumed ordering between replication and HTTP completion.
+18 -10
View File
@@ -28,6 +28,8 @@ import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000"; process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
const useRequestAPI = process.argv.includes("--request-api");
const customPath = ".obsidian/snippets/http-cache.css"; const customPath = ".obsidian/snippets/http-cache.css";
const nextCustomPath = ".obsidian/snippets/http-cache-new.css"; const nextCustomPath = ".obsidian/snippets/http-cache-new.css";
const settings = { const settings = {
@@ -45,7 +47,7 @@ const settings = {
syncInternalFilesInterval: 0, syncInternalFilesInterval: 0,
syncInternalFilesTargetPatterns: "^\\.http-cache(?:/|$)", syncInternalFilesTargetPatterns: "^\\.http-cache(?:/|$)",
watchInternalFileChanges: false, watchInternalFileChanges: false,
useRequestAPI: false, useRequestAPI,
autoAcceptCompatibleTweak: false, autoAcceptCompatibleTweak: false,
doctorProcessedVersion: DoctorRegulation.version, doctorProcessedVersion: DoctorRegulation.version,
deviceAndVaultName: "http-cache-source", deviceAndVaultName: "http-cache-source",
@@ -74,10 +76,11 @@ async function main(): Promise<void> {
.storeCustomizationFiles(path,core.services.setting.getDeviceAndVaultName()); .storeCustomizationFiles(path,core.services.setting.getDeviceAndVaultName());
return JSON.stringify(true);`); return JSON.stringify(true);`);
}; };
const browserRead = async (id: string, cache: RequestCache): Promise<CouchDbDocument> => { const readCachedDocument = async (id: string, cache: RequestCache): Promise<CouchDbDocument> => {
const url = `${couchDb.uri}/${encodeURIComponent(dbName)}/${encodeURIComponent(id)}`; const url = `${couchDb.uri}/${encodeURIComponent(dbName)}/${encodeURIComponent(id)}`;
const authorization = `Basic ${Buffer.from(`${couchDb.username}:${couchDb.password}`).toString("base64")}`; const authorization = `Basic ${Buffer.from(`${couchDb.username}:${couchDb.password}`).toString("base64")}`;
return await evaluate(`const response=await fetch(${JSON.stringify(url)},{ const fetchFunction = useRequestAPI ? "core.services.API.nativeFetch.bind(core.services.API)" : "fetch";
return await evaluate(`const response=await ${fetchFunction}(${JSON.stringify(url)},{
headers:{authorization:${JSON.stringify(authorization)}},credentials:'include',cache:${JSON.stringify(cache)}}); headers:{authorization:${JSON.stringify(authorization)}},credentials:'include',cache:${JSON.stringify(cache)}});
if(!response.ok)throw new Error('Fixture read failed: '+response.status); if(!response.ok)throw new Error('Fixture read failed: '+response.status);
return JSON.stringify(await response.json());`); return JSON.stringify(await response.json());`);
@@ -110,7 +113,7 @@ async function main(): Promise<void> {
assert.ok(original?.path, "The encrypted ordinary Metadata fixture is missing."); assert.ok(original?.path, "The encrypted ordinary Metadata fixture is missing.");
const initialParameters = await fetchCouchDbDocument(couchDb, dbName, DOCID_SYNC_PARAMETERS); const initialParameters = await fetchCouchDbDocument(couchDb, dbName, DOCID_SYNC_PARAMETERS);
// Populate the HTTP cache as an earlier plug-in version would have done. // Populate the HTTP cache as an earlier plug-in version would have done.
assert.deepEqual(await browserRead(original._id, "reload"), original); assert.deepEqual(await readCachedDocument(original._id, "reload"), original);
await evaluate(`await core.services.setting.applyPartial({encryptInternalMetadata:true},true); await evaluate(`await core.services.setting.applyPartial({encryptInternalMetadata:true},true);
await core.services.control.applySettings(); await core.services.control.applySettings();
@@ -128,11 +131,13 @@ async function main(): Promise<void> {
assert.notEqual(rebuiltParameters.pbkdf2salt, initialParameters.pbkdf2salt, "The Seed did not change."); assert.notEqual(rebuiltParameters.pbkdf2salt, initialParameters.pbkdf2salt, "The Seed did not change.");
assert.equal(rebuilt._rev, original._rev, "The document revision was not retained."); assert.equal(rebuilt._rev, original._rev, "The document revision was not retained.");
assert.notEqual(rebuilt.path, original.path, "The remote ciphertext did not change."); assert.notEqual(rebuilt.path, original.path, "The remote ciphertext did not change.");
assert.equal( if (!useRequestAPI) {
(await browserRead(original._id, "default")).path, assert.equal(
original.path, (await readCachedDocument(original._id, "default")).path,
"The stale browser HTTP cache fixture is no longer present." original.path,
); "The stale browser HTTP cache fixture is no longer present."
);
}
await evaluate(`await core.services.setting.applyPartial({usePluginSync:true,syncInternalFiles:true},true); await evaluate(`await core.services.setting.applyPartial({usePluginSync:true,syncInternalFiles:true},true);
await core.services.control.applySettings();return JSON.stringify(true);`); await core.services.control.applySettings();return JSON.stringify(true);`);
@@ -143,9 +148,12 @@ async function main(): Promise<void> {
if(opened)await opened.close();return JSON.stringify({admitted:!!opened}); if(opened)await opened.close();return JSON.stringify({admitted:!!opened});
}catch(error){return JSON.stringify({admitted:false,error:error.name});}`); }catch(error){return JSON.stringify({admitted:false,error:error.name});}`);
assert.ok(admission.admitted, `Replication admission failed: ${JSON.stringify(admission)}`); assert.ok(admission.admitted, `Replication admission failed: ${JSON.stringify(admission)}`);
assert.equal((await readCachedDocument(original._id, "no-store")).path, rebuilt.path);
await pushLocalChanges(cliBinary, session.cliEnv); await pushLocalChanges(cliBinary, session.cliEnv);
await pushLocalChanges(cliBinary, session.cliEnv); await pushLocalChanges(cliBinary, session.cliEnv);
console.log("Rebuild and subsequent replication succeeded with stale ciphertext still in the HTTP cache."); console.log(
`Rebuild and subsequent replication succeeded after warming the ${useRequestAPI ? "Internal API" : "browser"} HTTP cache.`
);
} finally { } finally {
await session?.app.stop(); await session?.app.stop();
await vault.dispose(); await vault.dispose();
+9 -3
View File
@@ -12,14 +12,20 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
## Unreleased ## Unreleased
- Files containing a raw unit separator (`U+001F`) retain their complete contents when received through S3-compatible Object Storage. (#1303) - We can continue CouchDB synchronisation after rebuilding an encrypted remote when **Use Internal API** is enabled. (#1234)
- The Journal receiver preserves the complete Chunk payload, including batches written by earlier versions, without changing the stored format. - CouchDB read requests now carry cache-control headers to avoid reusing responses encrypted with the previous Security Seed.
- Updating prevents further truncation during reception but does not repair existing damaged copies. Update every participating client, preserve an intact recovery source, and follow the [recovery procedure](docs/recovery.md#recover-files-truncated-during-object-storage-reception). Rebuild and Fetch leave Hidden File Sync and Customisation Sync disabled until they are deliberately initialised again.
- CLI `fetch` and `rebuild` retain their selected local database for subsequent commands even without `--write-settings`, preventing the empty-database restart and file deletion reported in #1304. - CLI `fetch` and `rebuild` retain their selected local database for subsequent commands even without `--write-settings`, preventing the empty-database restart and file deletion reported in #1304.
- Runtime state is separated by database and Vault. Interrupted reconstruction blocks ordinary synchronisation until recovery, and a pending settings commit can be completed on restart. - Runtime state is separated by database and Vault. Interrupted reconstruction blocks ordinary synchronisation until recovery, and a pending settings commit can be completed on restart.
- Older runtime state with no recorded owner is preserved and requires explicit reconstruction from a verified source before ordinary writes resume. Back up unique edits before choosing Fetch or Rebuild. - Older runtime state with no recorded owner is preserved and requires explicit reconstruction from a verified source before ordinary writes resume. Back up unique edits before choosing Fetch or Rebuild.
- Files containing a raw unit separator (`U+001F`) retain their complete contents when received through S3-compatible Object Storage. (#1303)
- The Journal receiver in Commonlib 0.2.3 preserves the complete Chunk payload, including batches written by earlier versions, without changing the stored format.
- Updating prevents further truncation during reception but does not repair existing damaged copies. Update every participating client, preserve an intact recovery source, and follow the [recovery procedure](docs/recovery.md#recover-files-truncated-during-object-storage-reception). Rebuild and Fetch leave Hidden File Sync and Customisation Sync disabled until they are deliberately initialised again.
- Files waiting for encrypted Chunks which arrive after their Metadata receive the decrypted contents once the local database is ready.
- Commonlib 0.2.3 restarts the database change feed after initialisation installs the decryption transforms. This corrects delivery of later Chunks; it does not replay earlier events. ([Commonlib #191](https://github.com/vrtmrz/livesync-commonlib/pull/191))
- The CLI mirror now starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. It checks logical Vault paths using `/` separators, correctly excluding internal database files and nested hidden directories even when the host uses Windows path separators. - The CLI mirror now starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. It checks logical Vault paths using `/` separators, correctly excluding internal database files and nested hidden directories even when the host uses Windows path separators.
- Hidden File Sync now stores newer local files during `Merge` initialisation by comparing local and database modification times in the correct direction. - Hidden File Sync now stores newer local files during `Merge` initialisation by comparing local and database modification times in the correct direction.