Compare commits

..
Author SHA1 Message Date
vorotamoroz fd9a9175dd Use Commonlib 0.1.8 for reliable Fast Fetch pagination 2026-08-08 11:22:34 +00:00
9 changed files with 54 additions and 33 deletions
@@ -42,10 +42,20 @@ probe. CouchDB's API documentation says that `limit=0` has the same effect as
no result rows and leave the complete count in `pending`. Fast Fetch therefore
uses an explicit one-row normal probe and includes no document bodies.
A continuous feed with a heartbeat remains open at the current tail. It closes
with a `{ "last_seq": ... }` line only after its finite `limit` has been met.
Consequently, a limit larger than the workload already known to be available
can leave initialisation waiting for future writes.
Finite continuous-feed completion differs across supported CouchDB releases.
CouchDB 3.5.0 was observed to close a heartbeat-enabled feed with a
`{ "last_seq": ... }` line when its finite `limit` is met. CouchDB 3.2 instead
continues to wait for database updates after the limit has been consumed. With
a heartbeat configured, each wait emits another heartbeat and the page can
remain open indefinitely, even after every requested row has arrived.
On CouchDB 3.2, an explicit `timeout` without a heartbeat has different
semantics from a total request deadline. Shard-result waits may emit blank
keep-alive lines and continue processing. Once the currently available changes
have been exhausted and the feed is waiting for another database update, the
timeout stops that wait and returns the feed-level `last_seq`. The timeout can
therefore terminate a finite page without limiting the duration of an active
page transfer.
The CouchDB sequence token is opaque and must be handled using CouchDB's
sequence semantics, without parsing, ordering, or comparison. On clustered
@@ -73,8 +83,16 @@ request starts again from that same cursor.
The number currently available is `results.length + pending`. If it is zero,
Fast Fetch is caught up and completes without opening another stream. Otherwise,
the next continuous request uses the smaller of that count and 10,000 as its
finite `limit`. This prevents a heartbeat-enabled request from waiting for
future changes merely to fill an oversized page.
finite `limit`.
Each finite page omits `heartbeat` and sets `timeout=1000`. This lets CouchDB
3.2 return the page's terminator one second after it exhausts the currently
available changes, rather than keeping the request open for future writes. The
client immediately reconnects from that terminator while another normal probe
reports available work. This bounded cycle also preserves the intent of the
earlier iOS and iPadOS heartbeat workaround: Fast Fetch no longer depends on a
silent continuous request eventually closing at CouchDB's default 60-second
timeout.
The probe and page are separate HTTP requests, not a transactional snapshot.
New writes, replica selection, or administrative changes may alter the rows
@@ -252,6 +270,8 @@ writer. Verify that:
`pending` is zero;
- every probe uses `limit=1`, excludes document bodies, and is repeated from the
previous page's opaque terminator;
- each bounded page omits `heartbeat`, uses `timeout=1000`, and can complete
under CouchDB 3.2 after its current rows have been delivered;
- deletion and document-less rows consume a page slot;
- a row count cannot complete a page without its `last_seq` terminator;
- a page terminator cannot advance the checkpoint before its batch is durable;
@@ -286,11 +306,14 @@ existing setup sequence and cleanup.
Commonlib's CouchDB integration test remains responsible for the real HTTP
changes feed, opaque sequence tokens, deletion rows, and local batch
persistence. It should use a two-shard database, include a data set large enough
to cross a local batch boundary, and confirm that the final checkpoint can be
passed back to CouchDB as `since` with no result rows or pending changes. The
test must not compare that token's representation with a separately requested
target or changes-row token.
persistence. It should use the maintained CI CouchDB release, a two-shard
database, and a data set large enough to cross a local batch boundary, and
confirm that the final checkpoint can be passed back to CouchDB as `since` with
no result rows or pending changes. The test must not compare that token's
representation with a separately requested target or changes-row token. The
focused regression test covers CouchDB 3.2's page-tail behaviour; compatibility
with a real CouchDB 3.2 server can be confirmed manually without expanding the
permanent CI matrix.
LiveSync's real Obsidian Setup URI workflow remains responsible for the actual
Fast Fetch selection, E2EE passphrase, Vault reflection, ordinary file round
@@ -310,6 +333,9 @@ This follows [Real Obsidian E2E](2026_06_real_obsidian_e2e.md).
or local persistence failures which cannot repair themselves.
- Progress totals remain approximate and may grow when a later probe observes
new work, without affecting correctness.
- A completed page can spend up to one second waiting for its terminator before
Fast Fetch probes and reconnects. Active page transfer is not constrained to
one second.
- The implementation requires coordinated changes in Commonlib and LiveSync.
Commonlib remains the authoritative package for streaming and rebuilder
behaviour; LiveSync consumes an immutable Commonlib release and owns its setup
+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "1.0.8",
"version": "1.0.7",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",
+9 -9
View File
@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "1.0.8",
"version": "1.0.7",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "1.0.8",
"version": "1.0.7",
"license": "MIT",
"workspaces": [
"src/apps/cli",
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.7",
"@vrtmrz/livesync-commonlib": "0.1.8",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4775,9 +4775,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.7",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.7.tgz",
"integrity": "sha512-xwRXuPqYmPbWlSaVrQqeVacZz4X5km3fgoUee6Odl8/C3dNx+pFndxoHvXIDo3HmbMKcz/JA09JTKvfxVqBwQQ==",
"version": "0.1.8",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.8.tgz",
"integrity": "sha512-Kn1AF41h2Dog37ThU7KgLcKxItCCerLEBWg1eSGAUoTk3TyPBYynvtmVFwWhy6LCePcuwB/+x7EQNTL3Sgzkyg==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
@@ -15924,7 +15924,7 @@
},
"src/apps/cli": {
"name": "self-hosted-livesync-cli",
"version": "1.0.8-cli",
"version": "1.0.7-cli",
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
@@ -15949,7 +15949,7 @@
},
"src/apps/webapp": {
"name": "livesync-webapp",
"version": "1.0.8-webapp",
"version": "1.0.7-webapp",
"dependencies": {
"octagonal-wheels": "^0.1.52"
},
@@ -15961,7 +15961,7 @@
}
},
"src/apps/webpeer": {
"version": "1.0.8-webpeer",
"version": "1.0.7-webpeer",
"dependencies": {
"octagonal-wheels": "^0.1.52"
},
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "1.0.8",
"version": "1.0.7",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -177,7 +177,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.7",
"@vrtmrz/livesync-commonlib": "0.1.8",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "self-hosted-livesync-cli",
"private": true,
"version": "1.0.8-cli",
"version": "1.0.7-cli",
"main": "dist/index.cjs",
"type": "module",
"scripts": {
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "livesync-webapp",
"private": true,
"version": "1.0.8-webapp",
"version": "1.0.7-webapp",
"type": "module",
"description": "Browser-based Self-hosted LiveSync using FileSystem API",
"scripts": {
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "webpeer",
"private": true,
"version": "1.0.8-webpeer",
"version": "1.0.7-webpeer",
"type": "module",
"scripts": {
"dev": "vite",
+1 -5
View File
@@ -12,15 +12,11 @@ Earlier releases remain available in the 0.25 release history and the legacy rel
## Unreleased
## 1.0.8
8th August, 2026
### Setup and compatibility
#### Fixed
- Fast Setup now sizes each finite CouchDB changes page from a one-row status probe, counts the returned result together with `pending`, and resumes from the page's opaque `last_seq` without comparing token representations. Heartbeat-enabled feeds no longer wait for future writes after the currently available rows have been persisted (#1065).
- Fast Setup now sizes each finite CouchDB changes page from a one-row status probe, counts the returned result together with `pending`, and resumes from the page's opaque `last_seq` without comparing token representations. Each page uses a one-second idle timeout instead of a heartbeat, allowing CouchDB 3.2 to return its terminator after the currently available rows have been persisted.
- Cancelling remote selection during a scheduled Fetch now removes the Fetch flag before restarting with file and database reflection paused, preventing the same selection dialogue from reopening on every start-up.
## 1.0.7
+1 -2
View File
@@ -19,6 +19,5 @@
"1.0.4": "1.7.2",
"1.0.5": "1.7.2",
"1.0.6": "1.7.2",
"1.0.7": "1.7.2",
"1.0.8": "1.7.2"
"1.0.7": "1.7.2"
}