Compare commits

...
Author SHA1 Message Date
vorotamorozandGitHub 76560e3bf2 Merge pull request #1083 from calvinbui/fix/qr-aggregator-special-characters
Fix special characters in multi-part settings QR codes
2026-08-08 21:19:22 +09:00
vorotamoroz 1e190d042c Merge main into multi-part settings QR fix 2026-08-08 12:13:31 +00:00
vorotamorozandGitHub 40215032dd Merge pull request #1088 from vrtmrz/fix/fast-fetch-page-timeout
Complete bounded Fast Fetch pages on CouchDB 3.2
2026-08-08 20:39:19 +09:00
vorotamoroz fd9a9175dd Use Commonlib 0.1.8 for reliable Fast Fetch pagination 2026-08-08 11:22:34 +00:00
vorotamorozandGitHub 1dfdb72fbd Merge pull request #1085 from vrtmrz/fix/fast-fetch-bounded-pages
Complete bounded Fast Fetch and stop repeated cancelled Fetch
2026-08-08 17:02:05 +09:00
Calvin Bui cf5181bb28 Fix special characters in aggregated QR settings 2026-08-07 21:48:14 +10:00
6 changed files with 52 additions and 21 deletions
+2 -1
View File
@@ -33,7 +33,8 @@
const id = params.get('id');
const total = parseInt(params.get('n') || '0');
const index = parseInt(params.get('i') || '-1');
const data = params.get('d');
// Keep the chunk percent-encoded so URI delimiters remain part of the settings payload.
const data = hash.match(/(?:^|&)d=([^&]*)/)?.[1];
const app = document.getElementById('app');
@@ -42,10 +42,20 @@ probe. CouchDB's API documentation says that `limit=0` has the same effect as
no result rows and leave the complete count in `pending`. Fast Fetch therefore
uses an explicit one-row normal probe and includes no document bodies.
A continuous feed with a heartbeat remains open at the current tail. It closes
with a `{ "last_seq": ... }` line only after its finite `limit` has been met.
Consequently, a limit larger than the workload already known to be available
can leave initialisation waiting for future writes.
Finite continuous-feed completion differs across supported CouchDB releases.
CouchDB 3.5.0 was observed to close a heartbeat-enabled feed with a
`{ "last_seq": ... }` line when its finite `limit` is met. CouchDB 3.2 instead
continues to wait for database updates after the limit has been consumed. With
a heartbeat configured, each wait emits another heartbeat and the page can
remain open indefinitely, even after every requested row has arrived.
On CouchDB 3.2, an explicit `timeout` without a heartbeat has different
semantics from a total request deadline. Shard-result waits may emit blank
keep-alive lines and continue processing. Once the currently available changes
have been exhausted and the feed is waiting for another database update, the
timeout stops that wait and returns the feed-level `last_seq`. The timeout can
therefore terminate a finite page without limiting the duration of an active
page transfer.
The CouchDB sequence token is opaque and must be handled using CouchDB's
sequence semantics, without parsing, ordering, or comparison. On clustered
@@ -73,8 +83,16 @@ request starts again from that same cursor.
The number currently available is `results.length + pending`. If it is zero,
Fast Fetch is caught up and completes without opening another stream. Otherwise,
the next continuous request uses the smaller of that count and 10,000 as its
finite `limit`. This prevents a heartbeat-enabled request from waiting for
future changes merely to fill an oversized page.
finite `limit`.
Each finite page omits `heartbeat` and sets `timeout=1000`. This lets CouchDB
3.2 return the page's terminator one second after it exhausts the currently
available changes, rather than keeping the request open for future writes. The
client immediately reconnects from that terminator while another normal probe
reports available work. This bounded cycle also preserves the intent of the
earlier iOS and iPadOS heartbeat workaround: Fast Fetch no longer depends on a
silent continuous request eventually closing at CouchDB's default 60-second
timeout.
The probe and page are separate HTTP requests, not a transactional snapshot.
New writes, replica selection, or administrative changes may alter the rows
@@ -252,6 +270,8 @@ writer. Verify that:
`pending` is zero;
- every probe uses `limit=1`, excludes document bodies, and is repeated from the
previous page's opaque terminator;
- each bounded page omits `heartbeat`, uses `timeout=1000`, and can complete
under CouchDB 3.2 after its current rows have been delivered;
- deletion and document-less rows consume a page slot;
- a row count cannot complete a page without its `last_seq` terminator;
- a page terminator cannot advance the checkpoint before its batch is durable;
@@ -286,11 +306,14 @@ existing setup sequence and cleanup.
Commonlib's CouchDB integration test remains responsible for the real HTTP
changes feed, opaque sequence tokens, deletion rows, and local batch
persistence. It should use a two-shard database, include a data set large enough
to cross a local batch boundary, and confirm that the final checkpoint can be
passed back to CouchDB as `since` with no result rows or pending changes. The
test must not compare that token's representation with a separately requested
target or changes-row token.
persistence. It should use the maintained CI CouchDB release, a two-shard
database, and a data set large enough to cross a local batch boundary, and
confirm that the final checkpoint can be passed back to CouchDB as `since` with
no result rows or pending changes. The test must not compare that token's
representation with a separately requested target or changes-row token. The
focused regression test covers CouchDB 3.2's page-tail behaviour; compatibility
with a real CouchDB 3.2 server can be confirmed manually without expanding the
permanent CI matrix.
LiveSync's real Obsidian Setup URI workflow remains responsible for the actual
Fast Fetch selection, E2EE passphrase, Vault reflection, ordinary file round
@@ -310,6 +333,9 @@ This follows [Real Obsidian E2E](2026_06_real_obsidian_e2e.md).
or local persistence failures which cannot repair themselves.
- Progress totals remain approximate and may grow when a later probe observes
new work, without affecting correctness.
- A completed page can spend up to one second waiting for its terminator before
Fast Fetch probes and reconnects. Active page transfer is not constrained to
one second.
- The implementation requires coordinated changes in Commonlib and LiveSync.
Commonlib remains the authoritative package for streaming and rebuilder
behaviour; LiveSync consumes an immutable Commonlib release and owns its setup
+4 -4
View File
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.7",
"@vrtmrz/livesync-commonlib": "0.1.8",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4775,9 +4775,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.7",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.7.tgz",
"integrity": "sha512-xwRXuPqYmPbWlSaVrQqeVacZz4X5km3fgoUee6Odl8/C3dNx+pFndxoHvXIDo3HmbMKcz/JA09JTKvfxVqBwQQ==",
"version": "0.1.8",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.8.tgz",
"integrity": "sha512-Kn1AF41h2Dog37ThU7KgLcKxItCCerLEBWg1eSGAUoTk3TyPBYynvtmVFwWhy6LCePcuwB/+x7EQNTL3Sgzkyg==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
+1 -1
View File
@@ -177,7 +177,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.7",
"@vrtmrz/livesync-commonlib": "0.1.8",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -30,12 +30,15 @@ Deno.test({
const aggregator = await browser.newPage();
const assertNoAggregatorFailures = observePageFailures(aggregator);
const assertNoAggregatorNetworkFailures = observeNetworkFailures(aggregator);
await aggregator.goto(new URL("aggregator.html#id=pages-smoke&n=2&i=0&d=first-", server.baseUrl).href);
await aggregator.goto(new URL("aggregator.html#id=pages-smoke&n=2&i=0&d=before%2", server.baseUrl).href);
await aggregator.getByText("1 / 2 Loaded", { exact: true }).waitFor();
await aggregator.goto(new URL("aggregator.html#id=pages-smoke&n=2&i=1&d=second", server.baseUrl).href);
await aggregator.goto(
new URL("aggregator.html#id=pages-smoke&n=2&i=1&d=3after%26amp%2Bplus%25percent", server.baseUrl)
.href
);
assertEquals(
await aggregator.getByRole("link", { name: "Open Obsidian to complete setup" }).getAttribute("href"),
"obsidian://setuplivesync?settingsQR=first-second"
"obsidian://setuplivesync?settingsQR=before%23after%26amp%2Bplus%25percent"
);
assertNoAggregatorFailures();
assertNoAggregatorNetworkFailures();
+2 -1
View File
@@ -16,7 +16,8 @@ Earlier releases remain available in the 0.25 release history and the legacy rel
#### Fixed
- Fast Setup now sizes each finite CouchDB changes page from a one-row status probe, counts the returned result together with `pending`, and resumes from the page's opaque `last_seq` without comparing token representations. Heartbeat-enabled feeds no longer wait for future writes after the currently available rows have been persisted (#1065).
- Multi-part settings QR codes now preserve special characters in passwords, passphrases, and other settings (PR #1083). Thank you to @calvinbui for the improvement!
- Fast Setup now sizes each finite CouchDB changes page from a one-row status probe, counts the returned result together with `pending`, and resumes from the page's opaque `last_seq` without comparing token representations. Each page uses a one-second idle timeout instead of a heartbeat, allowing CouchDB 3.2 to return its terminator after the currently available rows have been persisted.
- Cancelling remote selection during a scheduled Fetch now removes the Fetch flag before restarting with file and database reflection paused, preventing the same selection dialogue from reopening on every start-up.
## 1.0.7