mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-10-27 03:23:20 +00:00
Expose explicit LiveSync batching and honour save failures
Show the batching choice for the current synchronisation mode while preserving both mode preferences. Observe local save failures before file-open work and report failed asynchronous unload preparation. Record the accepted recovery and batching contract, update the primary settings documentation, and add two-device runtime coverage for batching, deletions, saved settings, and interrupted local processing. Requires Commonlib commit 72bdc66af5d7338d8f2f69ebe83c3866def93360. Update the exact registry dependency when its package version is published. Validation: full repository check; latest packed dependency type checks, production build, 1,373 unit tests (one skipped), and six focused runtime recovery cases. The earlier implementation passed the full 23-case runtime comparison and five additional interruption conditions.
This commit is contained in:
vorotamoroz committed
1 parent
73a14d4e4b
commit
68fa04af3c
9 files changed
+1512
-15
No files matched your search
@@ -0,0 +1,356 @@
|
||||
---
|
||||
date: 2026-10-24
|
||||
commonlib-version: "0.2.0"
|
||||
self-hosted-livesync-version: "1.0.40"
|
||||
status: accepted-unreleased
|
||||
---
|
||||
|
||||
# Architectural Decision Record: Local File Save Completion and Batching
|
||||
|
||||
## Status
|
||||
|
||||
Accepted on 2026-10-25. The package versions above identify the investigated
|
||||
baseline, rather than a released implementation of this decision.
|
||||
|
||||
## Context
|
||||
|
||||
[Issue #1207](https://github.com/vrtmrz/obsidian-livesync/issues/1207) includes
|
||||
conflicts around automated note creation, editing, and renaming. Batching local
|
||||
file updates can reduce the intermediate revisions which another device receives.
|
||||
It also extends the period during which the Vault contains an edit which the
|
||||
local database has not saved. Recovery and incoming-file protection must remain
|
||||
correct throughout that period.
|
||||
|
||||
Two-device Obsidian E2E tests using Self-hosted LiveSync 1.0.40 and Commonlib
|
||||
0.2.0 found that a test-only removal of the LiveSync mode restriction reduced
|
||||
six consecutive edits from six additional revisions to one, in both directions.
|
||||
Incoming identical content produced no extra conflict, while genuinely different
|
||||
edits or a deletion preserved the necessary branches. A synthetic
|
||||
create–edit–rename sequence still delivered intermediate content. These tests
|
||||
used Vault operations, without Templater or Modal Forms themselves.
|
||||
|
||||
Restart tests exposed a separate recovery gap:
|
||||
|
||||
1. The event manager removes an event from its waiting map when its timer is
|
||||
released. It also removes a buffered event before dispatching it. Its saved
|
||||
snapshot covers buffered and waiting events, but omits work in progress.
|
||||
2. `waitForIdle()` waits for the waiting gates to open, rather than for the file
|
||||
handlers to finish. `ServiceFileAccessBase` reports that pending events have
|
||||
been committed after this gate-only wait. Some handler failures also settle
|
||||
without propagating a failure to that caller.
|
||||
3. If the process stops before the database write, the file can retain the new
|
||||
content while the database retains the old content and the saved event
|
||||
snapshot is empty.
|
||||
4. At start-up, the Offline Scanner's `EVEN` branch tries to record missing
|
||||
provenance, then reports completion. It ignores that helper's Boolean result.
|
||||
Different content can consequently remain unsaved when both modification
|
||||
times fall in the same two-second comparison bucket.
|
||||
|
||||
The last condition was reproduced by pausing the target operation immediately
|
||||
before database processing, then restarting the same isolated profile. A
|
||||
different timestamp bucket recovered the edit; the same bucket did not. The
|
||||
changed-condition success demonstrates the boundary, rather than a fix. This
|
||||
does not establish the cause of the reporter's historical conflicts.
|
||||
|
||||
The runtime evidence covers Linux aarch64, Obsidian 1.12.7, and CouchDB 3.5.0.
|
||||
Windows, mobile lifecycles, and the reporter's full automation remain outside
|
||||
that evidence. The inspected Commonlib source is commit
|
||||
`aa54282937c20d3db7f43778d240feeb1586cced`.
|
||||
|
||||
## Decision
|
||||
|
||||
Make local file processing recoverable until its result is established, use
|
||||
content and provenance to reconcile ambiguous timestamps, and enable batching
|
||||
in LiveSync mode only through an explicit choice after those contracts pass
|
||||
verification. Apply the recovery contract with batching enabled or disabled.
|
||||
|
||||
The content-reconciliation change targets selected ordinary files in the
|
||||
Offline Scanner's `NEWER_WINS` / `sync-newer` action. Keep the existing action
|
||||
matrix, explicit `DB_APPLY` policy, selection and size limits, conflict guards,
|
||||
and recovery suspensions. Hidden and internal files retain their owning
|
||||
feature's recovery contract; an ordinary file scan cannot recover those paths
|
||||
on its behalf.
|
||||
|
||||
### 1. Distinguish local persistence from replication
|
||||
|
||||
The following boundaries have different meanings:
|
||||
|
||||
| Boundary | Meaning |
|
||||
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Batch wait released | The file operation may start. Its content can still be absent from the local database. |
|
||||
| Local processing completed | The intended change is represented durably by the local database and its required content, or has been proved already represented or safely superseded. |
|
||||
| Replication completed | A finite replication reports its own outcome. A local save alone does not prove remote receipt or reflection into another Vault. |
|
||||
|
||||
Ordinary file saving by Obsidian remains independent of database saving. Pending
|
||||
and processing indicators must include work in progress and unresolved failures,
|
||||
so an empty timer map cannot imply that local processing is complete.
|
||||
|
||||
### 2. Keep a recoverable record through processing
|
||||
|
||||
The event manager owns each operation from queue admission through completion.
|
||||
Its existing persistent snapshot must represent buffered, waiting, and
|
||||
in-progress work in an order which preserves deletion and rename dependencies.
|
||||
Moving work between those states must not create a persistent gap.
|
||||
|
||||
An event can leave the recoverable set only after its effect is durable, current
|
||||
state proves that the effect is already represented, or a durably recorded
|
||||
successor safely subsumes it. Finishing an older operation must not remove a
|
||||
newer operation for the same path. Cancellation of a batch timer alone is not
|
||||
evidence that its content is safe to forget.
|
||||
|
||||
Snapshot writes must preserve mutation order. An older asynchronous write must
|
||||
not replace a newer snapshot, and a failed snapshot write must remain visible
|
||||
to the lifecycle caller. Reuse the existing device-local snapshot store; a
|
||||
second journal and a transaction spanning the file system and database are not
|
||||
part of this proposal.
|
||||
|
||||
The first implementation should retain the existing event snapshot format,
|
||||
including event keys and rename intent, and add pending ownership within the
|
||||
event manager. The persistence adapter remains responsible for storing the
|
||||
ordered snapshots. Handler outcomes must distinguish a failed attempt from a
|
||||
policy-approved skip before the manager can acknowledge either safely.
|
||||
|
||||
A crash after the database write but before snapshot removal can replay an
|
||||
operation. Replay must check current storage and revision provenance, retain
|
||||
the existing protection against stale restored events, and avoid duplicate
|
||||
revisions or deletion of a newer same-path file. It must be safe to interrupt
|
||||
replay again. Existing saved snapshots remain readable.
|
||||
|
||||
Boolean failure, an exception, and an unavailable comparison are not successful
|
||||
completion. Keep unresolved work recoverable and report the failure through the
|
||||
existing service and diagnostic boundaries. Explicit exclusions or a verified
|
||||
superseding change may finish work without a new revision; a generic handler
|
||||
failure may not. Recovery may continue with independent paths, while retaining
|
||||
the failed path and its dependencies. This proposal adds no unbounded retry loop.
|
||||
|
||||
The recoverable snapshot is an operation record, not a backup of every transient
|
||||
editor buffer. File content already saved by the host remains available for
|
||||
start-up reconciliation if the process stops before the first snapshot write.
|
||||
Crash guarantees cannot include bytes or events which the host never persisted
|
||||
or delivered, or repair arbitrary storage failure.
|
||||
|
||||
### 3. Give flush a finite, truthful completion boundary
|
||||
|
||||
An explicit commit of pending file events captures the work accepted before
|
||||
that request, releases its batch delays, and waits for the actual processing
|
||||
results, including buffered and already-running work. Later edits cannot extend
|
||||
that boundary indefinitely. A newer coalesced operation may satisfy earlier work
|
||||
only when its successful result covers that work.
|
||||
|
||||
The internal ordering barrier before a delete or rename waits only for its
|
||||
predecessors. It must not wait for itself or later events in the same dispatch
|
||||
loop. Replacing every gate wait with a wait for global idleness would risk a
|
||||
dependency cycle or starvation.
|
||||
|
||||
Manual synchronisation and replication readiness use the completed local
|
||||
boundary. Settings application must observe its result before reporting success
|
||||
or resuming work which depends on it. Persisting a settings file is a separate
|
||||
operation: `saveSettingData()` returning is not a file-processing completion
|
||||
signal, and making every settings-saved notification synchronous is unnecessary.
|
||||
|
||||
When a host permits orderly shutdown, stop admitting watcher work before
|
||||
draining accepted operations or preserving their pending records. Keep their
|
||||
storage available until that work settles. Suspension, forced termination, and
|
||||
Obsidian's synchronous unload callback cannot guarantee that asynchronous work
|
||||
finishes; correctness must also follow from the persistent record and start-up
|
||||
reconciliation. Remote availability must not determine whether local pending
|
||||
work survives shutdown.
|
||||
|
||||
### 4. Reconcile content when timestamps are inconclusive
|
||||
|
||||
Retain the existing timestamp tolerance for platforms with coarse modification
|
||||
times. Timestamp equality, including exactly equal raw timestamps, does not
|
||||
prove content equality. The Offline Scanner must establish one of the following
|
||||
outcomes through the ordinary non-forced file handler and its provenance rules:
|
||||
|
||||
| Evidence | Required outcome |
|
||||
| ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Vault content already matches an eligible current revision | Record or retain valid provenance as appropriate; create no duplicate revision. |
|
||||
| Vault content matches its recorded revision or saved content evidence, while the current database revision has advanced | Apply the database state through the existing protected reflection path. Do not resurrect the unchanged older content as an edit. |
|
||||
| Vault content differs from its recorded revision | Preserve the edit on that recorded branch. Keep independent concurrent branches available for conflict resolution. |
|
||||
| Vault content has no provable origin and matches no current live leaf | Preserve it according to the existing independent-branch rules. Do not attach it to an arbitrary database winner. |
|
||||
| Required content or provenance cannot be read, or state changes during comparison | Retain unresolved work and report or re-evaluate it. Do not mark it reconciled merely because the timestamp comparison is `EVEN`. |
|
||||
|
||||
Revalidate the relevant file and database state before acting on a comparison.
|
||||
Reuse the existing content and provenance checks; avoid a separate conflict
|
||||
algorithm in the scanner. Metadata continues to reference Chunks containing the
|
||||
file content.
|
||||
|
||||
`tryRecordUntrackedFileRevision()` answers whether it could safely record
|
||||
provenance. Its `false` also covers an already-populated record, conflicting
|
||||
history, unavailable content, and a concurrent change. It is not an 'edited
|
||||
file' predicate. A fallback which treats every `false` as permission for a
|
||||
forced save would violate the required outcomes.
|
||||
|
||||
Propagate the file handler's actual outcome into the pair result, including for
|
||||
storage-to-database writes. Preserve the scanner's `completed`, `skipped`, and
|
||||
`failed` distinctions and its existing `continueOnFileFailure` policy. Ordinary
|
||||
Obsidian start-up can report `completed-with-file-failures` and allow unaffected
|
||||
files to proceed; an unresolved pair must not become `completed` to achieve
|
||||
that result. Keep the stricter callers' existing completion requirements.
|
||||
|
||||
This deliberately extends the current equal-time scanner contract. Existing
|
||||
provenance tests expect no database write during that scan, including for an
|
||||
edited file. Preserve their unchanged-file and later incoming-protection
|
||||
scenarios; change the edited-file expectation to require preservation during
|
||||
the scan itself. Test both identical timestamps and distinct raw timestamps
|
||||
within one comparison bucket.
|
||||
|
||||
### 5. Make batch timing and activation explicit
|
||||
|
||||
Batching is a bounded delay before local database processing. The quiet-period
|
||||
timer follows the latest eligible content update; the maximum intentional delay
|
||||
starts with the first uncommitted update and is not reset by later edits or time
|
||||
spent awaiting a processing slot. Once due, work must become eligible to run.
|
||||
The maximum delay is not a guarantee of database completion, network delivery,
|
||||
or another device's reflection time. Waiting timers should not consume the
|
||||
capacity needed to process unrelated ready files.
|
||||
|
||||
For ordinary local content updates, queue the changed path and operation intent
|
||||
without reading the file body merely to wait. Acquire the current content when
|
||||
the operation becomes due, after coalescing eligible events. Reuse that content
|
||||
snapshot within the operation for comparison, provenance evidence, and Chunk
|
||||
creation. This retains the current ordinary-save boundary; it does not require
|
||||
reading and hashing the file for every keystroke or queued notification.
|
||||
|
||||
Content reuse within one operation is distinct from keeping a body cache across
|
||||
edits. A cached body would need a proven invalidation and freshness contract;
|
||||
mtime and size alone cannot establish that it is current. A saved provenance
|
||||
hash records evidence about earlier content, rather than replacing acquisition
|
||||
of the current bytes. Restart recovery, an incoming update, or a comparison
|
||||
which needs revalidation can still require additional reads. Preserve those
|
||||
checks rather than making a global promise of one read per path.
|
||||
|
||||
Batching therefore aims to reduce repeated content acquisition, comparison,
|
||||
Chunk processing, and local database updates, as well as intermediate revisions
|
||||
and subsequent replication and reflection work. Event filtering, bookkeeping,
|
||||
and pending-operation persistence still occur while waiting. The revision
|
||||
reduction measured by the existing E2E does not quantify file reads, CPU cost,
|
||||
physical disk I/O, or network traffic; measure those costs separately. Host or
|
||||
operating-system caching also means that a read API call is not a physical disk
|
||||
read count.
|
||||
|
||||
Manual flush and required deletion or rename ordering can end a batch early.
|
||||
Ordinary incoming replication continues, using the existing protection for
|
||||
unsaved local content. It must not require all unrelated local batches to flush
|
||||
for every incoming document.
|
||||
|
||||
Add an independent `batchSaveInLiveSync` Boolean, defaulting to `false`
|
||||
for both existing settings and new Vaults. In LiveSync mode, this value selects
|
||||
batching; other modes continue to use `batchSave`. Both choices use the existing
|
||||
minimum and maximum delay settings. The interface presents the choice for the
|
||||
current mode. An ordinary mode switch preserves each mode's choice.
|
||||
|
||||
This adds one persisted setting to distinguish a new explicit choice from old
|
||||
`liveSync: true, batchSave: true` configurations, where batching was ignored.
|
||||
Neither upgrade nor import of that legacy pair may activate batching. An older
|
||||
client continues to apply its existing LiveSync restriction. Include the new
|
||||
property in Commonlib's setting policy and import/export tests; do not infer
|
||||
consent from the presence of the old `batchSave` property.
|
||||
|
||||
The setting and scheduling changes follow the recovery work. Recovery in the
|
||||
currently supported modes must pass verification before the new mode is enabled.
|
||||
|
||||
### 6. Keep batching separate from automation completion
|
||||
|
||||
The intended benefit is fewer short-lived intermediate revisions. Batching
|
||||
cannot discover when a third-party template has finished, particularly while a
|
||||
dialogue is open or another device also runs automation. A maximum delay or a
|
||||
rename boundary can legitimately publish intermediate content.
|
||||
|
||||
Coalescing `CREATE` followed by `CHANGED` for the same still-existing file is a
|
||||
useful follow-up, but it needs its own ordering and interruption tests. Preserve
|
||||
the existing cross-path rename rule: store the destination before deleting the
|
||||
proven source branch. Do not retarget or discard pending source operations
|
||||
without proving the effect is already covered.
|
||||
|
||||
A guarantee that a multi-step template appears remotely only after completion
|
||||
would need an explicit producer completion contract and its failure behaviour.
|
||||
That is a separate design. Enabling batching alone must not be described as
|
||||
eliminating genuine concurrent-edit conflicts or as a confirmed fix for #1207.
|
||||
|
||||
## Ownership and implementation order
|
||||
|
||||
1. **Commonlib recovery:** event ownership, ordered snapshot persistence,
|
||||
successful completion and failure propagation, finite flush boundaries, and
|
||||
same-time content reconciliation. Add focused failing regressions before
|
||||
changing production code. Reuse the existing event manager, file handler,
|
||||
scanner, and provenance store.
|
||||
2. **Host integration:** keep the Obsidian persistence adapter and lifecycle
|
||||
wiring aligned with those contracts. Verify callers of
|
||||
`commitPendingFileEvents`, including settings application and database
|
||||
preparation, and preserve their failure outcomes. Validate the packed
|
||||
Commonlib artefact and update the consumer's exact dependency.
|
||||
3. **LiveSync mode batching:** add the explicit setting and its primary-language
|
||||
presentation only after the recovery regressions pass. Establish the stated
|
||||
timing semantics and repeat the two-device comparisons with the actual
|
||||
package, including start-up with batching already enabled.
|
||||
4. **Further coalescing:** assess same-path creation/editing and any proposed
|
||||
rename optimisation separately, against the original automation problem.
|
||||
|
||||
Keep these changes independently reviewable. Shared behaviour belongs in
|
||||
Commonlib; the plug-in must not acquire a source mirror or patch installed
|
||||
package files. Snapshot format changes, a second persistent store, or a new
|
||||
automation transaction API require a further design decision.
|
||||
|
||||
## Verification required
|
||||
|
||||
Use controlled promises at the manager and persistence boundaries for focused
|
||||
regressions, and real PouchDB tests for content and ancestry. The acceptance
|
||||
matrix is:
|
||||
|
||||
| Scenario | Required observation |
|
||||
| -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Stop after dequeue, timer release, or during a database write | Pending work remains recoverable; restart eventually represents the saved Vault edit. |
|
||||
| Database write succeeds before snapshot cleanup; restart twice | Replay creates no duplicate revision and preserves later edits. |
|
||||
| Old and new events share a path; snapshot writes overlap | Completing old work cannot remove new work or regress the durable snapshot. |
|
||||
| Handler returns false, throws, or snapshot persistence fails | Flush reports failure; pending work is retained; unrelated recoverable work can continue. |
|
||||
| Buffered or in-progress work exists with no waiting timer | Flush still waits for actual completion. |
|
||||
| A flush overlaps new typing or a rename/delete barrier | The captured work completes without waiting for later work or deadlocking. |
|
||||
| Equal timestamps: unchanged, edited, untracked, pruned provenance, unavailable Chunks, or concurrent recheck changes | Preserve correct ancestry and bytes, avoid duplicate revisions, and distinguish unresolved comparisons from successful reconciliation. |
|
||||
| Settings save followed by stop; ordinary unload; interruption during replay | No operation is forgotten because a settings callback or shutdown released its timer. |
|
||||
| Sustained edits and more paths than available processing slots | Observe the intentional-delay deadline and progress of unrelated ready files. |
|
||||
| Old settings, mode changes, explicit opt-in, and settings import | Existing LiveSync mode users retain immediate saving until they choose batching. |
|
||||
| Incoming identical content, an edit, a deletion, or delete/recreate while waiting | Preserve genuine branches without extra conflicts or loss of local work. |
|
||||
|
||||
Keep the existing E2E comparison between batching disabled, an ignored legacy
|
||||
setting, and explicitly enabled batching. Repeat both directions, manual
|
||||
synchronisation, saved-snapshot restart, and controlled in-progress restart with
|
||||
the fix. Record the original failure separately from repaired results; a pass
|
||||
with a different timestamp bucket or a test-only bundle is not verification of
|
||||
the production fix. Run platform-specific lifecycle checks before extending the
|
||||
evidence to Windows or mobile.
|
||||
|
||||
## Alternatives and consequences
|
||||
|
||||
- **Remove only the LiveSync guard:** leaves recovery gaps and can activate a
|
||||
previously ignored setting. The explicit choice follows correctness work.
|
||||
- **Wait only at unload:** cannot cover abrupt termination and hosts which do
|
||||
not await asynchronous unload. Continuous recoverability is still required.
|
||||
- **Repair only the scanner:** improves surviving-file recovery, but does not
|
||||
establish event ordering, deletion intent, or truthful flush completion.
|
||||
- **Retain only in-progress events:** closes a persistence gap, but still needs
|
||||
success/failure handling, replay checks, and recovery before the first snapshot.
|
||||
- **Remove timestamp rounding or force the Vault copy into the current winner:**
|
||||
fails for exactly equal timestamps and can destroy branch intent. Content and
|
||||
provenance determine the safe action.
|
||||
|
||||
Keeping work recoverable adds snapshot bookkeeping and can make local flush
|
||||
wait longer because it now waits for real processing. Same-time reconciliation
|
||||
can require additional content reads. Batching trades prompt remote visibility
|
||||
for fewer database updates and may increase the window for genuine concurrent
|
||||
edits. Measure these costs without weakening the content-preservation contract.
|
||||
|
||||
## References
|
||||
|
||||
- [Conflict resolution and revision provenance](../specs_conflict_resolution.md)
|
||||
- [Batch database update settings](../settings.md#batch-database-update)
|
||||
- [Common library package boundary](2026_07_common_library_package_boundary.md)
|
||||
- [Fast Fetch persistence and completion](2026_08_fast_fetch_persistence_and_completion.md)
|
||||
- [Two-device Batch save reproduction](../../test/e2e-obsidian/scripts/issue1207-livesync-batch.ts)
|
||||
- [Commonlib event manager at the investigated revision](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/managers/StorageEventManager.ts)
|
||||
- [Commonlib Offline Scanner at the investigated revision](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/serviceFeatures/offlineScanner.ts)
|
||||
- [Commonlib file handler at the investigated revision](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/serviceModules/ServiceFileHandlerBase.ts)
|
||||
- [Commonlib scanner policy and stored event interpretation](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/docs/offline-scanner.md)
|
||||
- [Commonlib event admission and reflection contract](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/docs/storage-events-and-reflection.md)
|
||||
- [Existing equal-time provenance regression](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/serviceFeatures/offlineScanner.provenance.unit.spec.ts)
|
||||
+14
-3
@@ -669,17 +669,28 @@ Setup URIs preserve both sleep preferences. Older Setup URIs which do not contai
|
||||
#### Batch database update
|
||||
|
||||
Setting key: batchSave
|
||||
Reducing the frequency with which on-disk changes are reflected into the DB
|
||||
|
||||
Combine repeated local file updates before saving them to the local database in modes other than LiveSync mode. This reduces intermediate database revisions and delays their availability to other devices.
|
||||
|
||||
#### Batch database updates in LiveSync mode
|
||||
|
||||
Setting key: batchSaveInLiveSync
|
||||
|
||||
Apply the same bounded delay in LiveSync mode. This separate choice is disabled by default for both existing settings and new Vaults. An older configuration containing `liveSync: true` and `batchSave: true` does not enable it. The settings interface shows the choice for the current mode, and switching modes preserves both choices. Older clients continue to ignore batching in LiveSync mode.
|
||||
|
||||
Both choices share the delays below. Manual synchronisation, deletion, or renaming can end a batch early. Batching reduces intermediate updates; it does not identify when a template has finished or prevent genuine concurrent edits on different devices.
|
||||
|
||||
#### Minimum delay for batch database updating
|
||||
|
||||
Setting key: batchSaveMinimumDelay
|
||||
Seconds. Saving to the local database will be delayed until this value after we stop typing or saving.
|
||||
|
||||
Seconds of quiet after the latest eligible edit before saving to the local database, subject to the maximum delay below.
|
||||
|
||||
#### Maximum delay for batch database updating
|
||||
|
||||
Setting key: batchSaveMaximumDelay
|
||||
Saving will be performed forcefully after this number of seconds.
|
||||
|
||||
Maximum intentional delay in seconds from the first pending update to a file. Later edits do not restart this limit, and waiting for a processing slot does not add another batch delay. This limit makes the work eligible to run; it does not guarantee database completion or receipt by another device within that time.
|
||||
|
||||
### 4. Deletion Propagation (Advanced)
|
||||
|
||||
|
||||
+8
-1
@@ -1,4 +1,5 @@
|
||||
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
||||
import { LOG_LEVEL_URGENT } from "octagonal-wheels/common/logger";
|
||||
import { getLanguage, Notice, Plugin, type App, type PluginManifest } from "./deps";
|
||||
import { setGetLanguage } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
||||
setGetLanguage(getLanguage);
|
||||
@@ -233,6 +234,12 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
|
||||
void this._startUp();
|
||||
}
|
||||
override onunload(): void {
|
||||
return void this.core.services.control.onUnload();
|
||||
void this.core.services.control.onUnload().catch((error) => {
|
||||
this.core.services.API.addLog(
|
||||
"Failed to unload Self-hosted LiveSync: " + String(error),
|
||||
LOG_LEVEL_URGENT,
|
||||
"shutdown"
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -230,7 +230,13 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
|
||||
}
|
||||
this.isLastHidden = isHidden;
|
||||
|
||||
await this.services.fileProcessing.commitPendingFileEvents();
|
||||
const pendingFileEventsCommitted = await this.services.fileProcessing.commitPendingFileEvents();
|
||||
if (!pendingFileEventsCommitted) {
|
||||
this._log(
|
||||
"Pending file events remain because their commit did not complete before the window visibility changed.",
|
||||
LOG_LEVEL_NOTICE
|
||||
);
|
||||
}
|
||||
|
||||
// Desktop opt-in (LiveSync/Periodic only): keep the background channel running while the
|
||||
// window is hidden, instead of suspending on hide. On hide we skip the suspend for both
|
||||
@@ -287,7 +293,10 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
|
||||
if (this.settings.suspendFileWatching) return;
|
||||
if (!this.settings.isConfigured) return;
|
||||
if (!this.services.appLifecycle.isReady()) return;
|
||||
await this.services.fileProcessing.commitPendingFileEvents();
|
||||
const pendingFileEventsCommitted = await this.services.fileProcessing.commitPendingFileEvents();
|
||||
if (!pendingFileEventsCommitted) {
|
||||
return;
|
||||
}
|
||||
if (file == null) {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ import { describe, it, expect, vi, afterEach } from "vitest";
|
||||
import { ModuleObsidianEvents } from "./ModuleObsidianEvents";
|
||||
import { DEFAULT_SETTINGS, REMOTE_COUCHDB } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { reactiveSource } from "octagonal-wheels/dataobject/reactive";
|
||||
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||
|
||||
type SetupOptions = {
|
||||
settings?: Partial<typeof DEFAULT_SETTINGS>;
|
||||
@@ -23,6 +24,8 @@ function setup(opts: SetupOptions) {
|
||||
onResumed: vi.fn(async () => true),
|
||||
};
|
||||
const fileProcessing = { commitPendingFileEvents: vi.fn(async () => true) };
|
||||
const replication = { replicateUnattendedByEvent: vi.fn(async () => undefined) };
|
||||
const conflict = { queueCheckForIfOpen: vi.fn(async () => undefined) };
|
||||
const boundedRemoteActivityCount = reactiveSource(0);
|
||||
const boundedLocalApplicationActivityCount = reactiveSource(0);
|
||||
|
||||
@@ -39,6 +42,8 @@ function setup(opts: SetupOptions) {
|
||||
setting: { saveSettingData: vi.fn(async () => undefined) },
|
||||
appLifecycle,
|
||||
fileProcessing,
|
||||
replication,
|
||||
conflict,
|
||||
replicator: { boundedRemoteActivityCount, boundedLocalApplicationActivityCount },
|
||||
},
|
||||
settings: {
|
||||
@@ -61,6 +66,9 @@ function setup(opts: SetupOptions) {
|
||||
module,
|
||||
appLifecycle,
|
||||
fileProcessing,
|
||||
replication,
|
||||
conflict,
|
||||
api: core._services.API,
|
||||
boundedRemoteActivityCount,
|
||||
boundedLocalApplicationActivityCount,
|
||||
};
|
||||
@@ -303,6 +311,23 @@ describe("watchWindowVisibilityAsync — keepReplicationActiveInBackground", ()
|
||||
expect(appLifecycle.onResumed).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("logs a failed pending-file commit and still suspends on hide", async () => {
|
||||
const { module, appLifecycle, fileProcessing, api } = setup({
|
||||
settings: { keepReplicationActiveInBackground: false, liveSync: true },
|
||||
hidden: true,
|
||||
});
|
||||
fileProcessing.commitPendingFileEvents.mockResolvedValue(false);
|
||||
|
||||
await module.watchWindowVisibilityAsync();
|
||||
|
||||
expect(appLifecycle.onSuspending).toHaveBeenCalledTimes(1);
|
||||
expect(api.addLog).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Pending file events remain"),
|
||||
LOG_LEVEL_NOTICE,
|
||||
expect.any(String)
|
||||
);
|
||||
});
|
||||
|
||||
it("does not apply on mobile even if the flag is set", async () => {
|
||||
const { module, appLifecycle } = setup({
|
||||
settings: { keepReplicationActiveInBackground: true, liveSync: true },
|
||||
@@ -313,3 +338,20 @@ describe("watchWindowVisibilityAsync — keepReplicationActiveInBackground", ()
|
||||
expect(appLifecycle.onSuspending).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("watchWorkspaceOpenAsync — pending file event commit", () => {
|
||||
it("skips replication and conflict checks when pending file events fail to commit", async () => {
|
||||
const { module, fileProcessing, replication, conflict } = setup({
|
||||
settings: { syncOnFileOpen: true },
|
||||
hidden: false,
|
||||
});
|
||||
fileProcessing.commitPendingFileEvents.mockResolvedValue(false);
|
||||
|
||||
await module.watchWorkspaceOpenAsync({ path: "note.md" } as Parameters<
|
||||
typeof module.watchWorkspaceOpenAsync
|
||||
>[0]);
|
||||
|
||||
expect(replication.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(conflict.queueCheckForIfOpen).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -171,20 +171,18 @@ export function paneSyncSettings(
|
||||
}
|
||||
});
|
||||
|
||||
void addPanel(
|
||||
paneEl,
|
||||
$msg("obsidianLiveSyncSettingTab.titleUpdateThinning"),
|
||||
undefined,
|
||||
visibleOnly(() => !this.isConfiguredAs("syncMode", "LIVESYNC"))
|
||||
).then((paneEl) => {
|
||||
new Setting(paneEl).autoWireToggle("batchSave");
|
||||
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleUpdateThinning")).then((paneEl) => {
|
||||
const inLiveSyncMode = () => this.isConfiguredAs("syncMode", "LIVESYNC");
|
||||
const batchEnabled = () => this.isConfiguredAs(inLiveSyncMode() ? "batchSaveInLiveSync" : "batchSave", true);
|
||||
new Setting(paneEl).autoWireToggle("batchSave", { onUpdate: visibleOnly(() => !inLiveSyncMode()) });
|
||||
new Setting(paneEl).autoWireToggle("batchSaveInLiveSync", { onUpdate: visibleOnly(inLiveSyncMode) });
|
||||
new Setting(paneEl).autoWireNumeric("batchSaveMinimumDelay", {
|
||||
acceptZero: true,
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
|
||||
onUpdate: visibleOnly(batchEnabled),
|
||||
});
|
||||
new Setting(paneEl).autoWireNumeric("batchSaveMaximumDelay", {
|
||||
acceptZero: true,
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
|
||||
onUpdate: visibleOnly(batchEnabled),
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab";
|
||||
import type { OnUpdateFunc, PageFunctions } from "./SettingPane";
|
||||
|
||||
const controls = vi.hoisted(() => new Map<string, { onUpdate?: () => { visibility?: boolean } }>());
|
||||
vi.mock("@/common/translation", () => ({ $msg: (key: string) => key }));
|
||||
vi.mock("@vrtmrz/livesync-commonlib/compat/common/logger", () => ({ Logger: vi.fn() }));
|
||||
vi.mock("./LiveSyncSetting.ts", () => ({
|
||||
LiveSyncSetting: class {
|
||||
autoWireDropDown(key: string, options = {}) {
|
||||
controls.set(key, options);
|
||||
return this;
|
||||
}
|
||||
autoWireToggle(key: string, options = {}) {
|
||||
controls.set(key, options);
|
||||
return this;
|
||||
}
|
||||
autoWireNumeric(key: string, options = {}) {
|
||||
controls.set(key, options);
|
||||
return this;
|
||||
}
|
||||
},
|
||||
}));
|
||||
|
||||
import { paneSyncSettings } from "./PaneSyncSettings";
|
||||
|
||||
function setup(liveSync: boolean, batchSave: boolean, batchSaveInLiveSync: boolean) {
|
||||
const callbacks = new Map<string, (value: string) => Promise<void>>();
|
||||
const panels = new Map<string, OnUpdateFunc | undefined>();
|
||||
const tab = {
|
||||
editingSettings: { ...DEFAULT_SETTINGS, liveSync, batchSave, batchSaveInLiveSync },
|
||||
services: { API: { isMobile: () => false }, control: { applySettings: vi.fn(async () => {}) } },
|
||||
addOnSaved: (key: string, callback: (value: string) => Promise<void>) => callbacks.set(key, callback),
|
||||
isConfiguredAs(key: string, value: unknown) {
|
||||
if (key === "syncMode") {
|
||||
return (
|
||||
(this.editingSettings.liveSync
|
||||
? "LIVESYNC"
|
||||
: this.editingSettings.periodicReplication
|
||||
? "PERIODIC"
|
||||
: "ONEVENTS") === value
|
||||
);
|
||||
}
|
||||
return this.editingSettings[key as keyof typeof this.editingSettings] === value;
|
||||
},
|
||||
saveSettings: vi.fn(async () => {}),
|
||||
requestCatalogueRefresh: vi.fn(),
|
||||
};
|
||||
const functions = {
|
||||
addPanel: (_element: HTMLElement, title: string, _callback: unknown, onUpdate?: OnUpdateFunc) => ({
|
||||
then(callback: (element: HTMLElement) => void) {
|
||||
if (!title.endsWith("titleSynchronizationMethod") && !title.endsWith("titleUpdateThinning")) return;
|
||||
panels.set(title, onUpdate);
|
||||
callback({} as HTMLElement);
|
||||
},
|
||||
}),
|
||||
} as unknown as PageFunctions;
|
||||
paneSyncSettings.call(tab as unknown as ObsidianLiveSyncSettingTab, {} as HTMLElement, functions);
|
||||
return { tab, callbacks, panels };
|
||||
}
|
||||
|
||||
function visible(key: string): boolean {
|
||||
const control = controls.get(key);
|
||||
if (!control) throw new Error(`Missing setting control: ${key}`);
|
||||
return control.onUpdate?.().visibility !== false;
|
||||
}
|
||||
|
||||
beforeEach(() => controls.clear());
|
||||
|
||||
describe("mode-specific database batching settings", () => {
|
||||
it.each([
|
||||
{ liveSync: true, batchSave: true, batchSaveInLiveSync: false, delays: false },
|
||||
{ liveSync: true, batchSave: false, batchSaveInLiveSync: true, delays: true },
|
||||
{ liveSync: false, batchSave: true, batchSaveInLiveSync: false, delays: true },
|
||||
{ liveSync: false, batchSave: false, batchSaveInLiveSync: true, delays: false },
|
||||
])("shows the current mode choice and its shared delays: %j", ({ delays, ...settings }) => {
|
||||
const { panels } = setup(settings.liveSync, settings.batchSave, settings.batchSaveInLiveSync);
|
||||
expect(panels.get("obsidianLiveSyncSettingTab.titleUpdateThinning")?.().visibility).not.toBe(false);
|
||||
expect(visible("batchSave")).toBe(!settings.liveSync);
|
||||
expect(visible("batchSaveInLiveSync")).toBe(settings.liveSync);
|
||||
expect(visible("batchSaveMinimumDelay")).toBe(delays);
|
||||
expect(visible("batchSaveMaximumDelay")).toBe(delays);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[true, false],
|
||||
[false, true],
|
||||
])("retains both batching choices across mode changes: %j", async (batchSave, batchSaveInLiveSync) => {
|
||||
const { tab, callbacks } = setup(false, batchSave, batchSaveInLiveSync);
|
||||
const switchMode = callbacks.get("syncMode")!;
|
||||
for (const mode of ["LIVESYNC", "PERIODIC", "ONEVENTS", "LIVESYNC"]) {
|
||||
await switchMode(mode);
|
||||
expect(tab.editingSettings).toMatchObject({ batchSave, batchSaveInLiveSync });
|
||||
}
|
||||
expect(tab.services.control.applySettings).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,961 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { createServer } from "node:net";
|
||||
import { join } from "node:path";
|
||||
import type { Page } from "playwright";
|
||||
import {
|
||||
assertCouchDbReachable,
|
||||
createCouchDbDatabase,
|
||||
deleteCouchDbDatabase,
|
||||
loadCouchDbConfig,
|
||||
makeUniqueDatabaseName,
|
||||
} from "../runner/couchdb.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertE2eCompatibilityUnpaused,
|
||||
configureCouchDb,
|
||||
createE2eCouchDbPluginData,
|
||||
createE2eObsidianDeviceLocalState,
|
||||
prepareRemote,
|
||||
waitForLiveSyncCoreReady,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { withObsidianPage } from "../runner/ui.ts";
|
||||
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
type Mode = "normal" | "ignored-batch-setting" | "enabled-batch";
|
||||
type Slot = { name: string; vault: TemporaryVault; session?: ObsidianLiveSyncSession };
|
||||
type Leaf = { rev: string; ancestry: string[]; logicalDeleted: boolean; pouchDeleted: boolean; body: string | null };
|
||||
type Snapshot = {
|
||||
path: string;
|
||||
rev: string | null;
|
||||
content: string | null;
|
||||
fileMtime: number | null;
|
||||
databaseMtime: number | null;
|
||||
exists: boolean;
|
||||
leaves: Leaf[];
|
||||
waiting: Array<{ path: string; type: string; since: number }>;
|
||||
queued: Array<{ path: string | null; type: string }>;
|
||||
shouldBatchSave: boolean;
|
||||
};
|
||||
type Pair = { a: Page; b: Page };
|
||||
|
||||
const diagnostics = process.env.E2E_OBSIDIAN_DIAGNOSTICS_DIR ?? "/tmp/issue1207-livesync-batch";
|
||||
const artifactRoot = process.env.ISSUE1207_ARTIFACT_ROOT ?? process.cwd();
|
||||
const evidence: Record<string, unknown> = { cases: [], observations: [] };
|
||||
const cases = evidence.cases as Array<Record<string, unknown>>;
|
||||
const observations = evidence.observations as unknown[];
|
||||
const pause = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));
|
||||
const prefix = "issue1207-batch-";
|
||||
|
||||
function check(condition: unknown, message: string): asserts condition {
|
||||
if (!condition) throw new Error(message);
|
||||
}
|
||||
|
||||
function hashes() {
|
||||
return Object.fromEntries(
|
||||
["main.js", "manifest.json", "styles.css"].map((name) => [
|
||||
name,
|
||||
createHash("sha256")
|
||||
.update(new Uint8Array(readFileSync(join(artifactRoot, name))))
|
||||
.digest("hex"),
|
||||
])
|
||||
);
|
||||
}
|
||||
|
||||
async function availablePort(): Promise<number> {
|
||||
const server = createServer();
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(0, "127.0.0.1", resolve);
|
||||
});
|
||||
const address = server.address();
|
||||
check(address && typeof address !== "string", "Missing debug port");
|
||||
await new Promise<void>((resolve, reject) => server.close((error) => (error ? reject(error) : resolve())));
|
||||
return address.port;
|
||||
}
|
||||
|
||||
async function evaluate<T>(page: Page, code: string): Promise<T> {
|
||||
return await page.evaluate(`(async()=>{
|
||||
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||
const manager=core.serviceModules.storageAccess.vaultManager;
|
||||
${code}
|
||||
})()`);
|
||||
}
|
||||
|
||||
async function installProbe(page: Page): Promise<void> {
|
||||
await evaluate(
|
||||
page,
|
||||
`
|
||||
if(globalThis.__issue1207Batch) return;
|
||||
const probe={events:[],changes:[]};
|
||||
globalThis.__issue1207Batch=probe;
|
||||
for(const type of ['create','modify','rename','delete']) {
|
||||
app.vault.on(type,(file,oldPath)=>{
|
||||
const path=file.path;
|
||||
if(!path.startsWith(${JSON.stringify(prefix)})) return;
|
||||
const item={type,path,oldPath:oldPath??null,at:Date.now(),content:null};
|
||||
probe.events.push(item);
|
||||
if(type!=='delete' && file.stat) {
|
||||
void app.vault.read(file).then(body=>item.content=body).catch(error=>item.error=String(error));
|
||||
}
|
||||
});
|
||||
}
|
||||
probe.feed=core.localDatabase.localDatabase.changes({live:true,since:'now',include_docs:true})
|
||||
.on('change',change=>{
|
||||
if(typeof change.doc?.path==='string' && change.doc.path.startsWith(${JSON.stringify(prefix)})) {
|
||||
probe.changes.push({path:change.doc.path,rev:change.doc._rev,
|
||||
logicalDeleted:!!change.doc.deleted,pouchDeleted:!!change.doc._deleted,at:Date.now()});
|
||||
}
|
||||
}).on('error',error=>probe.changes.push({error:String(error),at:Date.now()}));
|
||||
`
|
||||
);
|
||||
}
|
||||
|
||||
async function options(page: Page, mode: Mode, minimum = 2, maximum = 6): Promise<void> {
|
||||
await evaluate(
|
||||
page,
|
||||
`
|
||||
const settings=core.services.setting.currentSettings();
|
||||
Object.assign(settings,{liveSync:true,periodicReplication:false,
|
||||
batchSave:${mode === "ignored-batch-setting"},batchSaveInLiveSync:${mode === "enabled-batch"},
|
||||
batchSaveMinimumDelay:${minimum},batchSaveMaximumDelay:${maximum},deleteMetadataOfDeletedFiles:false});
|
||||
return {liveSync:settings.liveSync,batchSave:settings.batchSave,
|
||||
batchSaveInLiveSync:settings.batchSaveInLiveSync,shouldBatchSave:manager.shouldBatchSave};
|
||||
`
|
||||
);
|
||||
}
|
||||
|
||||
async function snapshot(page: Page, path: string): Promise<Snapshot> {
|
||||
return await evaluate<Snapshot>(
|
||||
page,
|
||||
`
|
||||
const path=${JSON.stringify(path)};
|
||||
const id=await core.services.path.path2id(path);
|
||||
const row=(await core.localDatabase.allDocsRaw({keys:[id],include_docs:true})).rows[0];
|
||||
let leaves=[];
|
||||
if(row?.value?.rev) {
|
||||
const rawLeaves=await core.localDatabase.getRaw(id,{open_revs:'all',revs:true});
|
||||
for(const item of rawLeaves) {
|
||||
if(!item.ok) continue;
|
||||
const raw=item.ok;
|
||||
let body=null;
|
||||
if(!raw._deleted) {
|
||||
const entry=await core.localDatabase.getDBEntryFromMeta(raw,false,false,true);
|
||||
if(entry!==false) body=Array.isArray(entry.data)?entry.data.join(''):entry.data;
|
||||
}
|
||||
leaves.push({rev:raw._rev,logicalDeleted:!!raw.deleted,pouchDeleted:!!raw._deleted,body,
|
||||
ancestry:raw._revisions?.ids.map((hash,i)=>(raw._revisions.start-i)+'-'+hash)??[]});
|
||||
}
|
||||
}
|
||||
const file=app.vault.getAbstractFileByPath(path);
|
||||
return {path,rev:row?.value?.rev??null,exists:!!file?.stat,
|
||||
fileMtime:file?.stat?.mtime??null,databaseMtime:row?.doc?.mtime??null,
|
||||
content:file?.stat?await app.vault.read(file):null,leaves,
|
||||
waiting:[...manager._waitingMap].map(([key,value])=>({path:key,type:value.type,since:value.since})),
|
||||
queued:manager.bufferedQueuedItems.map(item=>({path:item.args?.file?.path??null,type:item.type})),
|
||||
shouldBatchSave:!!manager.shouldBatchSave};
|
||||
`
|
||||
);
|
||||
}
|
||||
|
||||
function liveLeaves(state: Snapshot) {
|
||||
return state.leaves.filter((leaf) => !leaf.logicalDeleted && !leaf.pouchDeleted);
|
||||
}
|
||||
|
||||
async function waitFor(
|
||||
page: Page,
|
||||
path: string,
|
||||
predicate: (state: Snapshot) => boolean,
|
||||
timeout = 20000
|
||||
): Promise<Snapshot> {
|
||||
const deadline = Date.now() + timeout;
|
||||
let state: Snapshot;
|
||||
do {
|
||||
state = await snapshot(page, path);
|
||||
if (predicate(state)) return state;
|
||||
await pause(100);
|
||||
} while (Date.now() < deadline);
|
||||
throw new Error(`State did not settle: ${JSON.stringify(state)}`);
|
||||
}
|
||||
|
||||
async function settled(pair: Pair, path: string, body: string): Promise<[Snapshot, Snapshot]> {
|
||||
const ready = (state: Snapshot) =>
|
||||
state.content === body &&
|
||||
liveLeaves(state).length === 1 &&
|
||||
liveLeaves(state)[0].body === body &&
|
||||
state.leaves.length === 1 &&
|
||||
!state.waiting.some((item) => item.path === path);
|
||||
const deadline = Date.now() + 20000;
|
||||
let states: [Snapshot, Snapshot];
|
||||
do {
|
||||
states = await Promise.all([snapshot(pair.a, path), snapshot(pair.b, path)]);
|
||||
// The row, leaves, and storage are read asynchronously. Require each
|
||||
// sample to agree internally, then compare both current devices.
|
||||
if (
|
||||
states.every((state) => ready(state) && state.rev === liveLeaves(state)[0].rev) &&
|
||||
states[0].rev === states[1].rev
|
||||
)
|
||||
return states;
|
||||
await pause(100);
|
||||
} while (Date.now() < deadline);
|
||||
throw new Error(`Devices did not converge: ${JSON.stringify(states)}`);
|
||||
}
|
||||
|
||||
async function create(page: Page, path: string, body: string): Promise<void> {
|
||||
await evaluate(page, `await app.vault.create(${JSON.stringify(path)},${JSON.stringify(body)});`);
|
||||
}
|
||||
|
||||
async function modify(page: Page, path: string, body: string): Promise<void> {
|
||||
await evaluate(
|
||||
page,
|
||||
`await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),${JSON.stringify(body)});`
|
||||
);
|
||||
}
|
||||
|
||||
async function trace(page: Page, path: string) {
|
||||
return await evaluate(
|
||||
page,
|
||||
`return {
|
||||
events:globalThis.__issue1207Batch.events.filter(item=>item.path===${JSON.stringify(path)}||item.oldPath===${JSON.stringify(path)}),
|
||||
changes:globalThis.__issue1207Batch.changes.filter(item=>item.path===${JSON.stringify(path)}||item.error)};`
|
||||
);
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
await mkdir(diagnostics, { recursive: true });
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
check(cli.binary, "Missing Obsidian CLI");
|
||||
const cliBinary = cli.binary;
|
||||
const config = await loadCouchDbConfig(process.env.ISSUE1207_COUCHDB_ENV);
|
||||
check(["127.0.0.1", "localhost"].includes(new URL(config.uri).hostname), "Use a disposable local CouchDB");
|
||||
await assertCouchDbReachable(config);
|
||||
const dbName = makeUniqueDatabaseName(config.dbPrefix, "live-batch");
|
||||
const couch = { ...config, dbName };
|
||||
const overrides = {
|
||||
doctorProcessedVersion: "1.0.0",
|
||||
notifyThresholdOfRemoteStorageSize: -1,
|
||||
liveSync: false,
|
||||
periodicReplication: false,
|
||||
batchSave: false,
|
||||
batchSaveInLiveSync: false,
|
||||
syncOnStart: false,
|
||||
syncOnSave: false,
|
||||
syncOnFileOpen: false,
|
||||
syncOnEditorSave: false,
|
||||
syncAfterMerge: false,
|
||||
checkConflictOnlyOnOpen: true,
|
||||
showMergeDialogOnlyOnActive: true,
|
||||
disableMarkdownAutoMerge: true,
|
||||
resolveConflictsByNewerFile: false,
|
||||
writeDocumentsIfConflicted: false,
|
||||
writeLogToTheFile: false,
|
||||
};
|
||||
const before = hashes();
|
||||
evidence.artefactsBefore = before;
|
||||
evidence.manifest = JSON.parse(await readFile(join(artifactRoot, "manifest.json"), "utf8"));
|
||||
evidence.commonlibVersion = JSON.parse(
|
||||
await readFile("node_modules/@vrtmrz/livesync-commonlib/package.json", "utf8")
|
||||
).version;
|
||||
evidence.sourceBaseRevision = "73a14d4e4b0b414a1acfe170d01500abb3dca2e9";
|
||||
evidence.unreleasedImplementation = true;
|
||||
if (process.env.ISSUE1207_COMMONLIB_ARTEFACT) {
|
||||
evidence.commonlibArtefactSha256 = createHash("sha256")
|
||||
.update(new Uint8Array(readFileSync(process.env.ISSUE1207_COMMONLIB_ARTEFACT)))
|
||||
.digest("hex");
|
||||
}
|
||||
evidence.timing = { minimumSeconds: 2, maximumSeconds: 6, incomingAndManualSeconds: { minimum: 5, maximum: 60 } };
|
||||
const slots: Slot[] = [];
|
||||
let ownedDatabase = false;
|
||||
|
||||
function redact(value: unknown): string {
|
||||
return JSON.stringify(value, null, 2)
|
||||
.split(config.password)
|
||||
.join("<redacted>")
|
||||
.split(Buffer.from(`${config.username}:${config.password}`).toString("base64"))
|
||||
.join("<redacted>");
|
||||
}
|
||||
|
||||
async function start(slot: Slot, restart = false) {
|
||||
console.log(JSON.stringify({ phase: "starting", device: slot.name, restart }));
|
||||
slot.session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary,
|
||||
artifactRoot,
|
||||
vault: slot.vault,
|
||||
startupGraceMs: 1000,
|
||||
pluginData: restart ? undefined : createE2eCouchDbPluginData(couch, overrides),
|
||||
localStorageEntries: createE2eObsidianDeviceLocalState(slot.vault.name),
|
||||
env: { E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT: String(await availablePort()) },
|
||||
lifecycle: {
|
||||
beforeLaunch: async () => {
|
||||
for (const registry of [
|
||||
join(slot.vault.homePath, ".config", "obsidian", "obsidian.json"),
|
||||
join(slot.vault.homePath, "Library", "Application Support", "obsidian", "obsidian.json"),
|
||||
join(slot.vault.xdgConfigPath, "obsidian", "obsidian.json"),
|
||||
join(slot.vault.userDataPath, "obsidian.json"),
|
||||
]) {
|
||||
const state = JSON.parse(await readFile(registry, "utf8"));
|
||||
state.updateDisabled = true;
|
||||
await writeFile(registry, JSON.stringify(state));
|
||||
}
|
||||
},
|
||||
},
|
||||
});
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
phase: "device-started",
|
||||
device: slot.name,
|
||||
port: slot.session.remoteDebuggingPort,
|
||||
vault: slot.vault.path,
|
||||
})
|
||||
);
|
||||
await waitForLiveSyncCoreReady(cliBinary, slot.session.cliEnv);
|
||||
await assertE2eCompatibilityUnpaused(cliBinary, slot.session.cliEnv, slot.session.remoteDebuggingPort);
|
||||
if (!restart) {
|
||||
await configureCouchDb(cliBinary, slot.session.cliEnv, couch, overrides);
|
||||
await waitForLiveSyncCoreReady(cliBinary, slot.session.cliEnv);
|
||||
await prepareRemote(cliBinary, slot.session.cliEnv);
|
||||
}
|
||||
}
|
||||
|
||||
async function stop(slot: Slot) {
|
||||
if (!slot.session) return;
|
||||
await slot.session.app.stop();
|
||||
await writeFile(
|
||||
join(diagnostics, `process-${slot.name}-${slot.session.remoteDebuggingPort}.json`),
|
||||
redact(slot.session.app.output())
|
||||
);
|
||||
slot.session = undefined;
|
||||
}
|
||||
|
||||
async function withPair<T>(callback: (pair: Pair) => Promise<T>): Promise<T> {
|
||||
check(slots[0].session && slots[1].session, "Both devices must be running");
|
||||
return await withObsidianPage(
|
||||
slots[0].session.remoteDebuggingPort,
|
||||
async (a) =>
|
||||
await withObsidianPage(slots[1].session!.remoteDebuggingPort, async (b) => {
|
||||
await installProbe(a);
|
||||
await installProbe(b);
|
||||
return await callback({ a, b });
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
async function run(name: string, body: () => Promise<unknown>) {
|
||||
if (process.env.ISSUE1207_CASE_FILTER && !new RegExp(process.env.ISSUE1207_CASE_FILTER).test(name)) return;
|
||||
console.log(JSON.stringify({ phase: "case-start", name }));
|
||||
try {
|
||||
const result = await body();
|
||||
cases.push({ name, passed: true, result });
|
||||
console.log(JSON.stringify({ phase: "case-passed", name }));
|
||||
} catch (error) {
|
||||
const failure = error instanceof Error ? error.stack : String(error);
|
||||
cases.push({ name, passed: false, error: failure });
|
||||
console.log(JSON.stringify({ phase: "case-failed", name, error: failure }));
|
||||
await withPair(async (pair) => {
|
||||
let flushFailure: unknown;
|
||||
for (const [device, page] of Object.entries(pair)) {
|
||||
observations.push({
|
||||
name,
|
||||
device,
|
||||
probe: await evaluate(
|
||||
page,
|
||||
`return {
|
||||
events:globalThis.__issue1207Batch.events,changes:globalThis.__issue1207Batch.changes,
|
||||
pending:(await manager.adapter.persistence.loadSnapshot()).map(item=>({
|
||||
type:item.type,path:item.args?.file?.path,oldPath:item.args?.oldPath}))};`
|
||||
),
|
||||
});
|
||||
await page.screenshot({ path: join(diagnostics, `${name}-${device}-failure.png`) });
|
||||
try {
|
||||
await evaluate(page, "await manager.waitForIdle();");
|
||||
} catch (error) {
|
||||
observations.push({ name, device, flushFailure: String(error) });
|
||||
flushFailure ??= error;
|
||||
}
|
||||
}
|
||||
if (flushFailure) throw flushFailure;
|
||||
});
|
||||
await pause(500);
|
||||
}
|
||||
await writeFile(join(diagnostics, "evidence.json"), redact(evidence));
|
||||
}
|
||||
|
||||
try {
|
||||
await createCouchDbDatabase(config, dbName);
|
||||
ownedDatabase = true;
|
||||
for (const name of ["a", "b"]) {
|
||||
const slot = { name, vault: await createTemporaryVault(`issue1207-live-batch-${name}-`) };
|
||||
slots.push(slot);
|
||||
await start(slot);
|
||||
}
|
||||
evidence.runtime = await withPair(async (pair) => {
|
||||
const versions = [];
|
||||
for (const page of [pair.a, pair.b]) {
|
||||
versions.push(
|
||||
await evaluate(
|
||||
page,
|
||||
`let appVersion=null;
|
||||
try {appVersion=JSON.parse(require('fs').readFileSync(require('path').join(process.resourcesPath,'app.asar','package.json'),'utf8')).version;} catch {}
|
||||
return {appVersion,electronVersion:process.versions.electron,
|
||||
pluginVersion:app.plugins.plugins['obsidian-livesync'].manifest.version};`
|
||||
)
|
||||
);
|
||||
await options(page, "normal");
|
||||
const outcome = await evaluate(
|
||||
page,
|
||||
`return await core.services.replication.startContinuous({
|
||||
trigger:'daemon',interaction:{kind:'forbidden'}});`
|
||||
);
|
||||
check(
|
||||
(outcome as { status?: string })?.status === "completed",
|
||||
`Continuous start failed: ${JSON.stringify(outcome)}`
|
||||
);
|
||||
}
|
||||
return versions;
|
||||
});
|
||||
|
||||
evidence.connectivity = await withPair(async (pair) => {
|
||||
const path = `${prefix}connectivity.md`;
|
||||
await create(pair.a, path, "Connectivity probe\n");
|
||||
const first = await settled(pair, path, "Connectivity probe\n");
|
||||
await modify(pair.b, path, "Bidirectional connectivity probe\n");
|
||||
const second = await settled(pair, path, "Bidirectional connectivity probe\n");
|
||||
return { first, second };
|
||||
});
|
||||
|
||||
for (const mode of ["normal", "ignored-batch-setting", "enabled-batch"] as const) {
|
||||
for (let repeat = 0; repeat < 2; repeat++) {
|
||||
await run(`${mode}-coalesce-${repeat}`, () =>
|
||||
withPair(async (pair) => {
|
||||
for (const page of [pair.a, pair.b]) await options(page, mode);
|
||||
const source = repeat === 0 ? pair.a : pair.b;
|
||||
const receiver = repeat === 0 ? pair.b : pair.a;
|
||||
const path = `${prefix}${mode}-edits-${repeat}.md`;
|
||||
await create(source, path, "Initial body\n");
|
||||
const initial = (await settled(pair, path, "Initial body\n"))[0];
|
||||
for (let edit = 0; edit < 6; edit++) {
|
||||
await modify(source, path, `Edit ${edit}\n`);
|
||||
await pause(150);
|
||||
}
|
||||
const during = await snapshot(source, path);
|
||||
const final = await settled(pair, path, "Edit 5\n");
|
||||
const finalSource = repeat === 0 ? final[0] : final[1];
|
||||
const delta = Number(finalSource.rev?.split("-")[0]) - Number(initial.rev?.split("-")[0]);
|
||||
if (mode === "enabled-batch") {
|
||||
check(
|
||||
during.waiting.some((item) => item.path === path),
|
||||
"Batch did not retain the pending edit"
|
||||
);
|
||||
check(delta === 1, `Repeated edits created ${delta} revisions`);
|
||||
} else {
|
||||
check(!during.shouldBatchSave, "LiveSync batching was enabled without an explicit choice");
|
||||
check(delta >= 3, `The control did not expose enough edits: ${delta}`);
|
||||
}
|
||||
return {
|
||||
path,
|
||||
initial,
|
||||
during,
|
||||
final,
|
||||
revisionDelta: delta,
|
||||
source: await trace(source, path),
|
||||
receiver: await trace(receiver, path),
|
||||
};
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
await run(`${mode}-template-lifecycle`, () =>
|
||||
withPair(async (pair) => {
|
||||
for (const page of [pair.a, pair.b]) await options(page, mode);
|
||||
const oldPath = `${prefix}${mode}-untitled.md`;
|
||||
const path = `${prefix}${mode}-completed.md`;
|
||||
await create(pair.a, oldPath, "");
|
||||
await pause(300);
|
||||
await modify(pair.a, oldPath, "Template in progress\n");
|
||||
await pause(300);
|
||||
await evaluate(
|
||||
pair.a,
|
||||
`await app.fileManager.renameFile(app.vault.getFileByPath(${JSON.stringify(oldPath)}),${JSON.stringify(path)});`
|
||||
);
|
||||
await pause(300);
|
||||
await modify(pair.a, path, "Completed template\n");
|
||||
const during = await snapshot(pair.a, path);
|
||||
const final = await settled(pair, path, "Completed template\n");
|
||||
await waitFor(pair.b, oldPath, (state) => !state.exists && liveLeaves(state).length === 0);
|
||||
return {
|
||||
oldPath,
|
||||
path,
|
||||
during,
|
||||
final,
|
||||
oldSource: await snapshot(pair.a, oldPath),
|
||||
oldReceiver: await snapshot(pair.b, oldPath),
|
||||
source: await trace(pair.a, path),
|
||||
receiver: await trace(pair.b, path),
|
||||
oldSourceTrace: await trace(pair.a, oldPath),
|
||||
oldReceiverTrace: await trace(pair.b, oldPath),
|
||||
};
|
||||
})
|
||||
);
|
||||
|
||||
for (const physical of [false, true]) {
|
||||
await run(`${mode}-recreate-${physical}`, () =>
|
||||
withPair(async (pair) => {
|
||||
for (const page of [pair.a, pair.b]) {
|
||||
await options(page, mode);
|
||||
await evaluate(
|
||||
page,
|
||||
`core.services.setting.currentSettings().deleteMetadataOfDeletedFiles=${physical};`
|
||||
);
|
||||
}
|
||||
const path = `${prefix}${mode}-recreate-${physical}.md`;
|
||||
await create(pair.a, path, "Transient body\n");
|
||||
await pause(400);
|
||||
await evaluate(
|
||||
pair.a,
|
||||
`await app.vault.delete(app.vault.getFileByPath(${JSON.stringify(path)}),true);`
|
||||
);
|
||||
await pause(400);
|
||||
await create(pair.a, path, "Recreated body\n");
|
||||
const final = await settled(pair, path, "Recreated body\n");
|
||||
return {
|
||||
physical,
|
||||
final,
|
||||
source: await trace(pair.a, path),
|
||||
receiver: await trace(pair.b, path),
|
||||
};
|
||||
})
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const incoming of ["identical", "different", "delete"] as const) {
|
||||
await run(`enabled-pending-incoming-${incoming}`, () =>
|
||||
withPair(async (pair) => {
|
||||
await options(pair.a, "enabled-batch", 5, 60);
|
||||
await options(pair.b, "normal", 5, 60);
|
||||
const path = `${prefix}pending-${incoming}.md`;
|
||||
await create(pair.b, path, "Shared base\n");
|
||||
const initial = await settled(pair, path, "Shared base\n");
|
||||
const local = "Local edit\n";
|
||||
await modify(pair.a, path, local);
|
||||
const pending = await waitFor(pair.a, path, (state) =>
|
||||
state.waiting.some((item) => item.path === path)
|
||||
);
|
||||
check(pending.rev === initial[0].rev, "Local edit reached the DB before incoming delivery");
|
||||
const sentAt = Date.now();
|
||||
if (incoming === "delete") {
|
||||
await evaluate(
|
||||
pair.b,
|
||||
`await app.vault.delete(app.vault.getFileByPath(${JSON.stringify(path)}),true);`
|
||||
);
|
||||
} else await modify(pair.b, path, incoming === "identical" ? local : "Remote edit\n");
|
||||
const arrived = await waitFor(
|
||||
pair.a,
|
||||
path,
|
||||
(state) =>
|
||||
incoming === "identical"
|
||||
? state.rev !== initial[0].rev && liveLeaves(state).some((leaf) => leaf.body === local)
|
||||
: state.leaves.some((leaf) => leaf.body === local) &&
|
||||
state.leaves.some((leaf) =>
|
||||
incoming === "delete" ? leaf.logicalDeleted : leaf.body === "Remote edit\n"
|
||||
),
|
||||
4000
|
||||
);
|
||||
check(
|
||||
arrived.waiting.some((item) => item.path === path),
|
||||
"The incoming assertion did not occur during Batch waiting"
|
||||
);
|
||||
const leavesBefore = arrived.leaves.map((leaf) => leaf.rev).sort();
|
||||
if (incoming !== "identical") {
|
||||
check(arrived.content === local, "Incoming reflection overwrote the pending local edit");
|
||||
check(arrived.leaves.length === 2, "Expected exactly two genuine concurrent branches");
|
||||
check(
|
||||
arrived.leaves.every((leaf) => leaf.ancestry[1] === initial[0].rev),
|
||||
"Concurrent branches lost their shared base"
|
||||
);
|
||||
}
|
||||
await pause(5600);
|
||||
const after = await snapshot(pair.a, path);
|
||||
const peer = await snapshot(pair.b, path);
|
||||
check(!after.waiting.some((item) => item.path === path), "Batch timer did not complete");
|
||||
check(
|
||||
JSON.stringify(after.leaves.map((leaf) => leaf.rev).sort()) === JSON.stringify(leavesBefore),
|
||||
"The delayed event created another revision"
|
||||
);
|
||||
check(
|
||||
JSON.stringify(peer.leaves.map((leaf) => leaf.rev).sort()) === JSON.stringify(leavesBefore),
|
||||
"Both branches did not converge to the peer"
|
||||
);
|
||||
if (incoming === "identical")
|
||||
check(after.leaves.length === 1, "Identical content created a branch");
|
||||
return { initial, pending, arrived, after, peer, incomingLatencyMs: Date.now() - sentAt - 5600 };
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
await run("enabled-maximum-delay", () =>
|
||||
withPair(async (pair) => {
|
||||
await options(pair.a, "enabled-batch", 2, 6);
|
||||
await options(pair.b, "normal");
|
||||
const path = `${prefix}maximum.md`;
|
||||
await create(pair.b, path, "Initial maximum body\n");
|
||||
const initial = await settled(pair, path, "Initial maximum body\n");
|
||||
const started = Date.now();
|
||||
let finalBody = "";
|
||||
for (let edit = 0; edit < 26; edit++) {
|
||||
finalBody = `Continuous edit ${edit}\n`;
|
||||
await modify(pair.a, path, finalBody);
|
||||
await pause(350);
|
||||
}
|
||||
const stopped = Date.now();
|
||||
const final = await settled(pair, path, finalBody);
|
||||
const changes = await evaluate<Array<{ at: number; rev: string }>>(
|
||||
pair.a,
|
||||
`return globalThis.__issue1207Batch.changes.filter(item=>item.path===${JSON.stringify(path)}&&item.rev!==${JSON.stringify(initial[0].rev)});`
|
||||
);
|
||||
check(changes.length >= 2, "Continuous edits did not produce bounded intermediate and final saves");
|
||||
const first = changes[0].at - started;
|
||||
check(first >= 5000 && first <= 8000, `Unexpected first maximum-delay flush: ${first}ms`);
|
||||
check(changes[0].at < stopped - 1000, "The maximum flush only occurred after typing stopped");
|
||||
return { started, stopped, firstSaveMs: first, changes, final };
|
||||
})
|
||||
);
|
||||
|
||||
await run("enabled-many-paths", () =>
|
||||
withPair(async (pair) => {
|
||||
await options(pair.a, "enabled-batch", 2, 6);
|
||||
await options(pair.b, "normal");
|
||||
const started = Date.now();
|
||||
for (let index = 0; index < 12; index++)
|
||||
await create(pair.a, `${prefix}many-${index}.md`, `File ${index}\n`);
|
||||
const final = [];
|
||||
for (let index = 0; index < 12; index++)
|
||||
final.push(await settled(pair, `${prefix}many-${index}.md`, `File ${index}\n`));
|
||||
return { elapsedMs: Date.now() - started, final };
|
||||
})
|
||||
);
|
||||
|
||||
await run("enabled-manual-sync", () =>
|
||||
withPair(async (pair) => {
|
||||
await options(pair.a, "enabled-batch", 5, 60);
|
||||
await options(pair.b, "normal");
|
||||
const path = `${prefix}manual.md`;
|
||||
await create(pair.b, path, "Before manual sync\n");
|
||||
await settled(pair, path, "Before manual sync\n");
|
||||
await modify(pair.a, path, "After manual sync\n");
|
||||
const pending = await waitFor(pair.a, path, (state) =>
|
||||
state.waiting.some((item) => item.path === path)
|
||||
);
|
||||
const started = Date.now();
|
||||
const outcome = await evaluate(pair.a, "return await core.services.replication.replicate(true);");
|
||||
const atReturn = await snapshot(pair.a, path);
|
||||
const final = await settled(pair, path, "After manual sync\n");
|
||||
const elapsedMs = Date.now() - started;
|
||||
check(elapsedMs < 4000, `Manual sync waited for the Batch timer: ${elapsedMs}ms`);
|
||||
return { pending, outcome, atReturn, final, elapsedMs };
|
||||
})
|
||||
);
|
||||
|
||||
const saveAfterEdit = process.env.ISSUE1207_RESTART_SAVE_AFTER_EDIT === "true";
|
||||
const restartBucket = process.env.ISSUE1207_RESTART_BUCKET;
|
||||
check(
|
||||
!restartBucket || ["same", "exact", "different"].includes(restartBucket),
|
||||
"Unknown restart timestamp condition"
|
||||
);
|
||||
const restartName = restartBucket
|
||||
? `enabled-inflight-restart-${restartBucket}`
|
||||
: saveAfterEdit
|
||||
? "enabled-settings-save-restart"
|
||||
: "enabled-restart";
|
||||
await run(restartName, async () => {
|
||||
const path = `${prefix}restart.md`;
|
||||
const beforeRestart = await withPair(async (pair) => {
|
||||
await options(pair.a, "enabled-batch", 5, 60);
|
||||
await options(pair.b, "normal");
|
||||
if (!saveAfterEdit)
|
||||
await evaluate(
|
||||
pair.a,
|
||||
"await core.services.setting.saveSettingData(); await core.services.control.applySettings();"
|
||||
);
|
||||
await create(pair.b, path, "Before restart\n");
|
||||
const initial = await settled(pair, path, "Before restart\n");
|
||||
if (restartBucket) {
|
||||
check(initial[0].databaseMtime !== null, "Missing initial database mtime");
|
||||
const mtime =
|
||||
restartBucket === "exact"
|
||||
? initial[0].databaseMtime
|
||||
: Math.floor(initial[0].databaseMtime / 2000) * 2000 +
|
||||
(restartBucket === "same" ? 1000 : 3000);
|
||||
await evaluate(
|
||||
pair.a,
|
||||
`const original=manager.requestProcessQueue.bind(manager);
|
||||
manager.requestProcessQueue=async item=>{
|
||||
if(item.args?.file?.path===${JSON.stringify(path)}) {
|
||||
globalThis.__issue1207Batch.inflightPaused=true;
|
||||
await new Promise(resolve=>setTimeout(resolve,30000));
|
||||
}
|
||||
return await original(item);
|
||||
};
|
||||
await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),"Retained after restart\\n",{mtime:${mtime}});`
|
||||
);
|
||||
} else await modify(pair.a, path, "Retained after restart\n");
|
||||
const pending = await waitFor(pair.a, path, (state) =>
|
||||
state.waiting.some((item) => item.path === path)
|
||||
);
|
||||
if (saveAfterEdit) await evaluate(pair.a, "await core.services.setting.saveSettingData();");
|
||||
if (restartBucket)
|
||||
await evaluate(
|
||||
pair.a,
|
||||
`const deadline=Date.now()+3000;
|
||||
while(!globalThis.__issue1207Batch.inflightPaused&&Date.now()<deadline) await new Promise(resolve=>setTimeout(resolve,10));
|
||||
if(!globalThis.__issue1207Batch.inflightPaused) throw new Error('The in-flight test gate did not start');`
|
||||
);
|
||||
await evaluate(pair.a, "await manager._takeSnapshot();");
|
||||
check(pending.rev === initial[0].rev, "Restart test did not retain a pending edit");
|
||||
if (restartBucket) {
|
||||
const inFlight = await snapshot(pair.a, path);
|
||||
const saved = await evaluate<Array<{ path: string }>>(
|
||||
pair.a,
|
||||
`return (await manager.adapter.persistence.loadSnapshot()).map(item=>({path:item.args?.file?.path}));`
|
||||
);
|
||||
check(
|
||||
inFlight.rev === initial[0].rev && !inFlight.waiting.some((item) => item.path === path),
|
||||
"The in-flight fixture did not stop between gate release and database storage"
|
||||
);
|
||||
check(
|
||||
saved.some((item) => item.path === path),
|
||||
"The in-flight event was lost from the recoverable snapshot"
|
||||
);
|
||||
check(inFlight.fileMtime !== null && inFlight.databaseMtime !== null, "Missing in-flight mtimes");
|
||||
const sameBucket =
|
||||
Math.floor(inFlight.fileMtime / 2000) === Math.floor(inFlight.databaseMtime / 2000);
|
||||
check(sameBucket === (restartBucket !== "different"), "The timestamp condition was not applied");
|
||||
if (restartBucket === "exact")
|
||||
check(inFlight.fileMtime === inFlight.databaseMtime, "Raw mtimes differ");
|
||||
observations.push({
|
||||
phase: "inflight-before-stop",
|
||||
restartBucket,
|
||||
pauseBeforeDatabaseMs: 30000,
|
||||
inFlight,
|
||||
saved,
|
||||
sameBucket,
|
||||
});
|
||||
}
|
||||
if (!saveAfterEdit && !restartBucket) {
|
||||
const strictState = await snapshot(pair.a, path);
|
||||
const saved = await evaluate<Array<{ path: string }>>(
|
||||
pair.a,
|
||||
`return (await manager.adapter.persistence.loadSnapshot()).map(item=>({path:item.args?.file?.path}));`
|
||||
);
|
||||
check(
|
||||
strictState.rev === initial[0].rev && strictState.waiting.some((item) => item.path === path),
|
||||
"Restart fixture flushed before shutdown"
|
||||
);
|
||||
check(
|
||||
saved.some((item) => item.path === path),
|
||||
"Restart fixture did not persist the pending event"
|
||||
);
|
||||
observations.push({
|
||||
phase: "restart-verified-pending-snapshot",
|
||||
strictState,
|
||||
saved,
|
||||
initialized: await evaluate(pair.a, "return await core.kvDB.get('initialized');"),
|
||||
});
|
||||
}
|
||||
if (!saveAfterEdit)
|
||||
observations.push({
|
||||
phase: "restart-before-stop",
|
||||
initial,
|
||||
pending,
|
||||
persisted: await evaluate(
|
||||
pair.a,
|
||||
`const events=await manager.adapter.persistence.loadSnapshot();
|
||||
const settings=core.services.setting.currentSettings();
|
||||
return {initialized:await core.kvDB.get('initialized'),restorationRequested:manager.snapShotRestored!==null,
|
||||
settings:{isConfigured:settings.isConfigured,liveSync:settings.liveSync,batchSave:settings.batchSave,batchSaveInLiveSync:settings.batchSaveInLiveSync,
|
||||
suspendFileWatching:settings.suspendFileWatching},
|
||||
events:events?.map(item=>({type:item.type,path:item.args?.file?.path,
|
||||
stat:item.args?.file?.stat,skipBatchWait:item.skipBatchWait}))};`
|
||||
),
|
||||
});
|
||||
return { initial, pending };
|
||||
});
|
||||
await stop(slots[0]);
|
||||
const persistedSettings = JSON.parse(
|
||||
await readFile(join(slots[0].vault.path, ".obsidian/plugins/obsidian-livesync/data.json"), "utf8")
|
||||
);
|
||||
observations.push({
|
||||
phase: "restart-on-disk-settings",
|
||||
settings: Object.fromEntries(
|
||||
[
|
||||
"isConfigured",
|
||||
"liveSync",
|
||||
"batchSave",
|
||||
"batchSaveInLiveSync",
|
||||
"suspendFileWatching",
|
||||
"settingVersion",
|
||||
].map((key) => [key, persistedSettings[key]])
|
||||
),
|
||||
});
|
||||
await start(slots[0], true);
|
||||
const afterRestart = await withPair(async (pair) => {
|
||||
observations.push({
|
||||
phase: "restart-after-boot",
|
||||
state: await snapshot(pair.a, path),
|
||||
persisted: await evaluate(
|
||||
pair.a,
|
||||
`const events=await manager.adapter.persistence.loadSnapshot();
|
||||
const settings=core.services.setting.currentSettings();
|
||||
return {initialized:await core.kvDB.get('initialized'),restorationRequested:manager.snapShotRestored!==null,
|
||||
settings:{isConfigured:settings.isConfigured,liveSync:settings.liveSync,batchSave:settings.batchSave,batchSaveInLiveSync:settings.batchSaveInLiveSync,
|
||||
suspendFileWatching:settings.suspendFileWatching},
|
||||
events:events?.map(item=>({type:item.type,path:item.args?.file?.path,stat:item.args?.file?.stat}))};`
|
||||
),
|
||||
});
|
||||
const result = await settled(pair, path, "Retained after restart\n");
|
||||
check(result[0].shouldBatchSave, "The saved LiveSync batching choice was not restored");
|
||||
return { result, restoredGuard: result[0].shouldBatchSave };
|
||||
});
|
||||
return { beforeRestart, afterRestart };
|
||||
});
|
||||
|
||||
await run("enabled-after-write-restart", async () => {
|
||||
const path = `${prefix}after-write.md`;
|
||||
const saved = await withPair(async (pair) => {
|
||||
await options(pair.a, "enabled-batch", 1, 3);
|
||||
await options(pair.b, "normal");
|
||||
await evaluate(
|
||||
pair.a,
|
||||
"await core.services.setting.saveSettingData(); await core.services.control.applySettings();"
|
||||
);
|
||||
await create(pair.b, path, "Before the durable write\n");
|
||||
const initial = await settled(pair, path, "Before the durable write\n");
|
||||
await evaluate(
|
||||
pair.a,
|
||||
`
|
||||
const original=manager.requestProcessQueue.bind(manager);
|
||||
manager.requestProcessQueue=async item=>{
|
||||
await original(item);
|
||||
if(item.args?.file?.path===${JSON.stringify(path)}) {
|
||||
globalThis.__issue1207Batch.afterWritePaused=true;
|
||||
await new Promise(resolve=>setTimeout(resolve,30000));
|
||||
}
|
||||
};
|
||||
await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),"Durable before restart\\n");
|
||||
const deadline=Date.now()+6000;
|
||||
while(!globalThis.__issue1207Batch.afterWritePaused&&Date.now()<deadline) await new Promise(resolve=>setTimeout(resolve,20));
|
||||
if(!globalThis.__issue1207Batch.afterWritePaused) throw new Error('The after-write gate did not start');
|
||||
await manager._takeSnapshot();
|
||||
`
|
||||
);
|
||||
const durable = await snapshot(pair.a, path);
|
||||
const pending = await evaluate<Array<{ path: string }>>(
|
||||
pair.a,
|
||||
"return (await manager.adapter.persistence.loadSnapshot()).map(item=>({path:item.args?.file?.path}));"
|
||||
);
|
||||
check(durable.rev !== initial[0].rev, "The database write did not complete before interruption");
|
||||
check(
|
||||
pending.some((item) => item.path === path),
|
||||
"The after-write operation was not recoverable"
|
||||
);
|
||||
return { initial, durable, pending };
|
||||
});
|
||||
const restarts = [];
|
||||
for (let attempt = 0; attempt < 2; attempt++) {
|
||||
await stop(slots[0]);
|
||||
await start(slots[0], true);
|
||||
const final = await withPair((pair) => settled(pair, path, "Durable before restart\n"));
|
||||
check(final[0].rev === saved.durable.rev, "Replay created a duplicate database revision");
|
||||
restarts.push(final);
|
||||
}
|
||||
const later = await withPair(async (pair) => {
|
||||
await modify(pair.a, path, "A later edit survives\n");
|
||||
return await settled(pair, path, "A later edit survives\n");
|
||||
});
|
||||
return { saved, restarts, later };
|
||||
});
|
||||
|
||||
await run("enabled-before-snapshot-restart-exact", async () => {
|
||||
const path = `${prefix}before-snapshot.md`;
|
||||
const beforeStop = await withPair(async (pair) => {
|
||||
await options(pair.a, "enabled-batch", 5, 60);
|
||||
await options(pair.b, "normal");
|
||||
await evaluate(
|
||||
pair.a,
|
||||
"await core.services.setting.saveSettingData(); await core.services.control.applySettings();"
|
||||
);
|
||||
await create(pair.b, path, "Before the first snapshot\n");
|
||||
const initial = await settled(pair, path, "Before the first snapshot\n");
|
||||
check(initial[0].databaseMtime !== null, "Missing initial database mtime");
|
||||
await evaluate(
|
||||
pair.a,
|
||||
`
|
||||
await manager.waitForIdle();
|
||||
const persistence=manager.adapter.persistence;
|
||||
const original=persistence.saveSnapshot.bind(persistence);
|
||||
persistence.saveSnapshot=async events=>{
|
||||
if(events.some(item=>item.type==='CHANGED'&&item.args?.file?.path===${JSON.stringify(path)})) {
|
||||
globalThis.__issue1207Batch.beforeSnapshotPaused=true;
|
||||
await new Promise(resolve=>setTimeout(resolve,30000));
|
||||
}
|
||||
return await original(events);
|
||||
};
|
||||
await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),
|
||||
"Recovered without an event snapshot\\n",{mtime:${initial[0].databaseMtime}});
|
||||
const deadline=Date.now()+3000;
|
||||
while(!globalThis.__issue1207Batch.beforeSnapshotPaused&&Date.now()<deadline)
|
||||
await new Promise(resolve=>setTimeout(resolve,10));
|
||||
if(!globalThis.__issue1207Batch.beforeSnapshotPaused)
|
||||
throw new Error('The snapshot persistence gate did not start');
|
||||
`
|
||||
);
|
||||
const state = await snapshot(pair.a, path);
|
||||
const persisted = await evaluate<Array<{ path: string }>>(
|
||||
pair.a,
|
||||
"return (await manager.adapter.persistence.loadSnapshot()??[]).map(item=>({path:item.args?.file?.path}));"
|
||||
);
|
||||
check(state.rev === initial[0].rev, "The unsaved edit reached the database before interruption");
|
||||
check(state.fileMtime === state.databaseMtime, "Raw mtimes differ before interruption");
|
||||
check(!persisted.some((item) => item.path === path), "The interrupted edit already has a saved event");
|
||||
return { initial, state, persisted };
|
||||
});
|
||||
await stop(slots[0]);
|
||||
await start(slots[0], true);
|
||||
const recovered = await withPair((pair) => settled(pair, path, "Recovered without an event snapshot\n"));
|
||||
check(recovered[0].shouldBatchSave, "The saved LiveSync batching choice was not restored");
|
||||
return { beforeStop, recovered };
|
||||
});
|
||||
|
||||
evidence.artefactsAfter = hashes();
|
||||
check(
|
||||
JSON.stringify(before) === JSON.stringify(evidence.artefactsAfter),
|
||||
"Plug-in artefacts changed during testing"
|
||||
);
|
||||
evidence.passed = cases.every((item) => item.passed);
|
||||
await writeFile(join(diagnostics, "evidence.json"), redact(evidence));
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
phase: "completed",
|
||||
passed: evidence.passed,
|
||||
cases: cases.length,
|
||||
failed: cases.filter((item) => !item.passed).map((item) => item.name),
|
||||
})
|
||||
);
|
||||
if (!evidence.passed) process.exitCode = 1;
|
||||
} finally {
|
||||
for (const slot of slots) await stop(slot);
|
||||
for (const slot of slots) await slot.vault.dispose();
|
||||
if (ownedDatabase) await deleteCouchDbDatabase(config, dbName);
|
||||
evidence.cleanup = { devicesStopped: slots.every((slot) => !slot.session), databaseRemoved: ownedDatabase };
|
||||
await writeFile(join(diagnostics, "evidence.json"), redact(evidence));
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
+15
@@ -12,6 +12,21 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Synchronisation
|
||||
|
||||
#### New Feature
|
||||
|
||||
- We can enable **Batch database updates in LiveSync mode** to combine repeated local edits before sending them to other devices.
|
||||
- The choice starts disabled, including when older settings contain the previously ignored batch option. Each mode retains its own choice and uses the same minimum and maximum delays.
|
||||
- Batching delays remote visibility and reduces intermediate updates. It does not establish when a template has finished or prevent genuine concurrent edits.
|
||||
|
||||
#### Fixed
|
||||
|
||||
- Local changes remain available for recovery while their database save is running, including when the application stops after a batch delay ends.
|
||||
- Manual synchronisation and settings application now wait for the accepted local saves to finish and report failures. Pending events retain the existing snapshot format.
|
||||
- Start-up reconciliation checks content when modification times are equal, so an edit within the same timestamp interval is not mistaken for an unchanged file.
|
||||
- Existing revision provenance preserves the correct edit branch. Unreadable or continually changing content remains unresolved rather than being reported as successfully reconciled.
|
||||
|
||||
## 1.0.40
|
||||
|
||||
24th October, 2026
|
||||
|
||||
Reference in new issue
Block a user