mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-10-27 03:23:20 +00:00
Merge main and verify comparison status with published Commonlib
This commit is contained in:
commit
c9381d6a59
38 files changed
+4592
-96
No files matched your search
@@ -0,0 +1,387 @@
|
|||||||
|
---
|
||||||
|
date: 2026-10-24
|
||||||
|
commonlib-version: "0.2.0"
|
||||||
|
self-hosted-livesync-version: "1.0.40"
|
||||||
|
status: accepted-unreleased
|
||||||
|
---
|
||||||
|
|
||||||
|
# Architectural Decision Record: Local File Save Completion and Batching
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Accepted on 2026-10-25. The package versions above identify the investigated
|
||||||
|
baseline, rather than a released implementation of this decision.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
[Issue #1207](https://github.com/vrtmrz/obsidian-livesync/issues/1207) includes
|
||||||
|
conflicts around automated note creation, editing, and renaming. Batching local
|
||||||
|
file updates can reduce the intermediate revisions which another device receives.
|
||||||
|
It also extends the period during which the Vault contains an edit which the
|
||||||
|
local database has not saved. Recovery and incoming-file protection must remain
|
||||||
|
correct throughout that period.
|
||||||
|
|
||||||
|
Two-device Obsidian E2E tests using Self-hosted LiveSync 1.0.40 and Commonlib
|
||||||
|
0.2.0 found that a test-only removal of the LiveSync mode restriction reduced
|
||||||
|
six consecutive edits from six additional revisions to one, in both directions.
|
||||||
|
Incoming identical content produced no extra conflict, while genuinely different
|
||||||
|
edits or a deletion preserved the necessary branches. A synthetic
|
||||||
|
create–edit–rename sequence still delivered intermediate content. These tests
|
||||||
|
used Vault operations, without Templater or Modal Forms themselves.
|
||||||
|
|
||||||
|
Restart tests exposed a separate recovery gap:
|
||||||
|
|
||||||
|
1. The event manager removes an event from its waiting map when its timer is
|
||||||
|
released. It also removes a buffered event before dispatching it. Its saved
|
||||||
|
snapshot covers buffered and waiting events, but omits work in progress.
|
||||||
|
2. `waitForIdle()` waits for the waiting gates to open, rather than for the file
|
||||||
|
handlers to finish. `ServiceFileAccessBase` reports that pending events have
|
||||||
|
been committed after this gate-only wait. Some handler failures also settle
|
||||||
|
without propagating a failure to that caller.
|
||||||
|
3. If the process stops before the database write, the file can retain the new
|
||||||
|
content while the database retains the old content and the saved event
|
||||||
|
snapshot is empty.
|
||||||
|
4. At start-up, the Offline Scanner's `EVEN` branch tries to record missing
|
||||||
|
provenance, then reports completion. It ignores that helper's Boolean result.
|
||||||
|
Different content can consequently remain unsaved when both modification
|
||||||
|
times fall in the same two-second comparison bucket.
|
||||||
|
|
||||||
|
The last condition was reproduced by pausing the target operation immediately
|
||||||
|
before database processing, then restarting the same isolated profile. A
|
||||||
|
different timestamp bucket recovered the edit; the same bucket did not. The
|
||||||
|
changed-condition success demonstrates the boundary, rather than a fix. This
|
||||||
|
does not establish the cause of the reporter's historical conflicts.
|
||||||
|
|
||||||
|
The runtime evidence covers Linux aarch64, Obsidian 1.12.7, and CouchDB 3.5.0.
|
||||||
|
Windows, mobile lifecycles, and the reporter's full automation remain outside
|
||||||
|
that evidence. The inspected Commonlib source is commit
|
||||||
|
`aa54282937c20d3db7f43778d240feeb1586cced`.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Make local file processing recoverable until its result is established, use
|
||||||
|
content and provenance to reconcile ambiguous timestamps, and enable batching
|
||||||
|
in LiveSync mode only through an explicit choice after those contracts pass
|
||||||
|
verification. Apply the recovery contract with batching enabled or disabled.
|
||||||
|
|
||||||
|
The content-reconciliation change targets selected ordinary files in the
|
||||||
|
Offline Scanner's `NEWER_WINS` / `sync-newer` action. Keep the existing action
|
||||||
|
matrix, explicit `DB_APPLY` policy, selection and size limits, conflict guards,
|
||||||
|
and recovery suspensions. Hidden and internal files retain their owning
|
||||||
|
feature's recovery contract; an ordinary file scan cannot recover those paths
|
||||||
|
on its behalf.
|
||||||
|
|
||||||
|
### 1. Distinguish local persistence from replication
|
||||||
|
|
||||||
|
The following boundaries have different meanings:
|
||||||
|
|
||||||
|
| Boundary | Meaning |
|
||||||
|
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| Batch wait released | The file operation may start. Its content can still be absent from the local database. |
|
||||||
|
| Local processing completed | The intended change is represented durably by the local database and its required content, or has been proved already represented or safely superseded. |
|
||||||
|
| Replication completed | A finite replication reports its own outcome. A local save alone does not prove remote receipt or reflection into another Vault. |
|
||||||
|
|
||||||
|
Ordinary file saving by Obsidian remains independent of database saving. Pending
|
||||||
|
and processing indicators must include work in progress and unresolved failures,
|
||||||
|
so an empty timer map cannot imply that local processing is complete.
|
||||||
|
|
||||||
|
### Optional file ownership and completion
|
||||||
|
|
||||||
|
Hidden File Sync and Customisation Sync register the new
|
||||||
|
`processOptionalFileEventWithResult` contract together. `firstHandled` tries
|
||||||
|
providers in order until one returns `completed` or `veto`; only
|
||||||
|
`not-applicable` advances to the next provider. Exceptions propagate unchanged.
|
||||||
|
`HandlerOutcome<TValue, TVetoReason>` gives completion data and refusal reasons
|
||||||
|
independent types. These saves use `void` completion data and string reasons.
|
||||||
|
|
||||||
|
A provider returns `completed` after its database operation succeeds or the
|
||||||
|
current state is already represented. Failed writes and deletions throw, and
|
||||||
|
the manager retains the event. Hidden File Sync acknowledges its processed
|
||||||
|
fingerprint after successful processing. Customisation Sync waits for the real
|
||||||
|
save after its existing 100ms quiet period. All notifications coalesced into a
|
||||||
|
V1 group receive that save's result; V2 files retain separate save groups.
|
||||||
|
|
||||||
|
A finite flush resolves each captured event's `whenFlushRequested` signal. This
|
||||||
|
releases its customisation delay and freezes that group; later groups do not
|
||||||
|
extend its completion. During start-up replay or suspension, providers use
|
||||||
|
physical database readiness rather than waiting for application readiness or
|
||||||
|
resume. Hidden File Sync loads its existing per-database caches before replay
|
||||||
|
and reuses them during the following initialisation hook.
|
||||||
|
|
||||||
|
The existing boolean optional API remains a legacy notification contract for
|
||||||
|
other callers and older hosts. Once a host registers typed providers, the
|
||||||
|
manager dispatches through that chain exclusively. Without a typed provider,
|
||||||
|
a resolved legacy notification completes that dispatch only; its boolean does
|
||||||
|
not establish a database save. Snapshot-write failures remain visible under
|
||||||
|
both contracts. No persistent journal, new snapshot schema, or automatic retry
|
||||||
|
loop is introduced.
|
||||||
|
|
||||||
|
### 2. Keep a recoverable record through processing
|
||||||
|
|
||||||
|
The event manager owns each operation from queue admission through completion.
|
||||||
|
Its existing persistent snapshot must represent buffered, waiting, and
|
||||||
|
in-progress work in an order which preserves deletion and rename dependencies.
|
||||||
|
Moving work between those states must not create a persistent gap.
|
||||||
|
|
||||||
|
An event can leave the recoverable set only after its effect is durable, current
|
||||||
|
state proves that the effect is already represented, or a durably recorded
|
||||||
|
successor safely subsumes it. Finishing an older operation must not remove a
|
||||||
|
newer operation for the same path. Cancellation of a batch timer alone is not
|
||||||
|
evidence that its content is safe to forget.
|
||||||
|
|
||||||
|
Snapshot writes must preserve mutation order. An older asynchronous write must
|
||||||
|
not replace a newer snapshot, and a failed snapshot write must remain visible
|
||||||
|
to the lifecycle caller. Reuse the existing device-local snapshot store; a
|
||||||
|
second journal and a transaction spanning the file system and database are not
|
||||||
|
part of this proposal.
|
||||||
|
|
||||||
|
The first implementation should retain the existing event snapshot format,
|
||||||
|
including event keys and rename intent, and add pending ownership within the
|
||||||
|
event manager. The persistence adapter remains responsible for storing the
|
||||||
|
ordered snapshots. Handler outcomes must distinguish a failed attempt from a
|
||||||
|
policy-approved skip before the manager can acknowledge either safely.
|
||||||
|
|
||||||
|
A crash after the database write but before snapshot removal can replay an
|
||||||
|
operation. Replay must check current storage and revision provenance, retain
|
||||||
|
the existing protection against stale restored events, and avoid duplicate
|
||||||
|
revisions or deletion of a newer same-path file. It must be safe to interrupt
|
||||||
|
replay again. Existing saved snapshots remain readable.
|
||||||
|
|
||||||
|
Boolean failure, an exception, and an unavailable comparison are not successful
|
||||||
|
completion. Keep unresolved work recoverable and report the failure through the
|
||||||
|
existing service and diagnostic boundaries. Explicit exclusions or a verified
|
||||||
|
superseding change may finish work without a new revision; a generic handler
|
||||||
|
failure may not. Recovery may continue with independent paths, while retaining
|
||||||
|
the failed path and its dependencies. This proposal adds no unbounded retry loop.
|
||||||
|
|
||||||
|
The recoverable snapshot is an operation record, not a backup of every transient
|
||||||
|
editor buffer. File content already saved by the host remains available for
|
||||||
|
start-up reconciliation if the process stops before the first snapshot write.
|
||||||
|
Crash guarantees cannot include bytes or events which the host never persisted
|
||||||
|
or delivered, or repair arbitrary storage failure.
|
||||||
|
|
||||||
|
### 3. Give flush a finite, truthful completion boundary
|
||||||
|
|
||||||
|
An explicit commit of pending file events captures the work accepted before
|
||||||
|
that request, releases its batch delays, and waits for the actual processing
|
||||||
|
results, including buffered and already-running work. Later edits cannot extend
|
||||||
|
that boundary indefinitely. A newer coalesced operation may satisfy earlier work
|
||||||
|
only when its successful result covers that work.
|
||||||
|
|
||||||
|
The internal ordering barrier before a delete or rename waits only for its
|
||||||
|
predecessors. It must not wait for itself or later events in the same dispatch
|
||||||
|
loop. Replacing every gate wait with a wait for global idleness would risk a
|
||||||
|
dependency cycle or starvation.
|
||||||
|
|
||||||
|
Manual synchronisation and replication readiness use the completed local
|
||||||
|
boundary. Settings application must observe its result before reporting success
|
||||||
|
or resuming work which depends on it. Persisting a settings file is a separate
|
||||||
|
operation: `saveSettingData()` returning is not a file-processing completion
|
||||||
|
signal, and making every settings-saved notification synchronous is unnecessary.
|
||||||
|
|
||||||
|
When a host permits orderly shutdown, stop admitting watcher work before
|
||||||
|
draining accepted operations or preserving their pending records. Keep their
|
||||||
|
storage available until that work settles. Suspension, forced termination, and
|
||||||
|
Obsidian's synchronous unload callback cannot guarantee that asynchronous work
|
||||||
|
finishes; correctness must also follow from the persistent record and start-up
|
||||||
|
reconciliation. Remote availability must not determine whether local pending
|
||||||
|
work survives shutdown.
|
||||||
|
|
||||||
|
### 4. Reconcile content when timestamps are inconclusive
|
||||||
|
|
||||||
|
Retain the existing timestamp tolerance for platforms with coarse modification
|
||||||
|
times. Timestamp equality, including exactly equal raw timestamps, does not
|
||||||
|
prove content equality. The Offline Scanner must establish one of the following
|
||||||
|
outcomes through the ordinary non-forced file handler and its provenance rules:
|
||||||
|
|
||||||
|
| Evidence | Required outcome |
|
||||||
|
| ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| Vault content already matches an eligible current revision | Record or retain valid provenance as appropriate; create no duplicate revision. |
|
||||||
|
| Vault content matches its recorded revision or saved content evidence, while the current database revision has advanced | Apply the database state through the existing protected reflection path. Do not resurrect the unchanged older content as an edit. |
|
||||||
|
| Vault content differs from its recorded revision | Preserve the edit on that recorded branch. Keep independent concurrent branches available for conflict resolution. |
|
||||||
|
| Vault content has no provable origin and matches no current live leaf | Preserve it according to the existing independent-branch rules. Do not attach it to an arbitrary database winner. |
|
||||||
|
| Required content or provenance cannot be read, or state changes during comparison | Retain unresolved work and report or re-evaluate it. Do not mark it reconciled merely because the timestamp comparison is `EVEN`. |
|
||||||
|
|
||||||
|
Revalidate the relevant file and database state before acting on a comparison.
|
||||||
|
Reuse the existing content and provenance checks; avoid a separate conflict
|
||||||
|
algorithm in the scanner. Metadata continues to reference Chunks containing the
|
||||||
|
file content.
|
||||||
|
|
||||||
|
`tryRecordUntrackedFileRevision()` answers whether it could safely record
|
||||||
|
provenance. Its `false` also covers an already-populated record, conflicting
|
||||||
|
history, unavailable content, and a concurrent change. It is not an 'edited
|
||||||
|
file' predicate. A fallback which treats every `false` as permission for a
|
||||||
|
forced save would violate the required outcomes.
|
||||||
|
|
||||||
|
Propagate the file handler's actual outcome into the pair result, including for
|
||||||
|
storage-to-database writes. Preserve the scanner's `completed`, `skipped`, and
|
||||||
|
`failed` distinctions and its existing `continueOnFileFailure` policy. Ordinary
|
||||||
|
Obsidian start-up can report `completed-with-file-failures` and allow unaffected
|
||||||
|
files to proceed; an unresolved pair must not become `completed` to achieve
|
||||||
|
that result. Keep the stricter callers' existing completion requirements.
|
||||||
|
|
||||||
|
This deliberately extends the current equal-time scanner contract. Existing
|
||||||
|
provenance tests expect no database write during that scan, including for an
|
||||||
|
edited file. Preserve their unchanged-file and later incoming-protection
|
||||||
|
scenarios; change the edited-file expectation to require preservation during
|
||||||
|
the scan itself. Test both identical timestamps and distinct raw timestamps
|
||||||
|
within one comparison bucket.
|
||||||
|
|
||||||
|
### 5. Make batch timing and activation explicit
|
||||||
|
|
||||||
|
Batching is a bounded delay before local database processing. The quiet-period
|
||||||
|
timer follows the latest eligible content update; the maximum intentional delay
|
||||||
|
starts with the first uncommitted update and is not reset by later edits or time
|
||||||
|
spent awaiting a processing slot. Once due, work must become eligible to run.
|
||||||
|
The maximum delay is not a guarantee of database completion, network delivery,
|
||||||
|
or another device's reflection time. Waiting timers should not consume the
|
||||||
|
capacity needed to process unrelated ready files.
|
||||||
|
|
||||||
|
For ordinary local content updates, queue the changed path and operation intent
|
||||||
|
without reading the file body merely to wait. Acquire the current content when
|
||||||
|
the operation becomes due, after coalescing eligible events. Reuse that content
|
||||||
|
snapshot within the operation for comparison, provenance evidence, and Chunk
|
||||||
|
creation. This retains the current ordinary-save boundary; it does not require
|
||||||
|
reading and hashing the file for every keystroke or queued notification.
|
||||||
|
|
||||||
|
Content reuse within one operation is distinct from keeping a body cache across
|
||||||
|
edits. A cached body would need a proven invalidation and freshness contract;
|
||||||
|
mtime and size alone cannot establish that it is current. A saved provenance
|
||||||
|
hash records evidence about earlier content, rather than replacing acquisition
|
||||||
|
of the current bytes. Restart recovery, an incoming update, or a comparison
|
||||||
|
which needs revalidation can still require additional reads. Preserve those
|
||||||
|
checks rather than making a global promise of one read per path.
|
||||||
|
|
||||||
|
Batching therefore aims to reduce repeated content acquisition, comparison,
|
||||||
|
Chunk processing, and local database updates, as well as intermediate revisions
|
||||||
|
and subsequent replication and reflection work. Event filtering, bookkeeping,
|
||||||
|
and pending-operation persistence still occur while waiting. The revision
|
||||||
|
reduction measured by the existing E2E does not quantify file reads, CPU cost,
|
||||||
|
physical disk I/O, or network traffic; measure those costs separately. Host or
|
||||||
|
operating-system caching also means that a read API call is not a physical disk
|
||||||
|
read count.
|
||||||
|
|
||||||
|
Manual flush and required deletion or rename ordering can end a batch early.
|
||||||
|
Ordinary incoming replication continues, using the existing protection for
|
||||||
|
unsaved local content. It must not require all unrelated local batches to flush
|
||||||
|
for every incoming document.
|
||||||
|
|
||||||
|
Add an independent `batchSaveInLiveSync` Boolean, defaulting to `false`
|
||||||
|
for both existing settings and new Vaults. In LiveSync mode, this value selects
|
||||||
|
batching; other modes continue to use `batchSave`. Both choices use the existing
|
||||||
|
minimum and maximum delay settings. The interface presents the choice for the
|
||||||
|
current mode. An ordinary mode switch preserves each mode's choice.
|
||||||
|
|
||||||
|
This adds one persisted setting to distinguish a new explicit choice from old
|
||||||
|
`liveSync: true, batchSave: true` configurations, where batching was ignored.
|
||||||
|
Neither upgrade nor import of that legacy pair may activate batching. An older
|
||||||
|
client continues to apply its existing LiveSync restriction. Include the new
|
||||||
|
property in Commonlib's setting policy and import/export tests; do not infer
|
||||||
|
consent from the presence of the old `batchSave` property.
|
||||||
|
|
||||||
|
The setting and scheduling changes follow the recovery work. Recovery in the
|
||||||
|
currently supported modes must pass verification before the new mode is enabled.
|
||||||
|
|
||||||
|
### 6. Keep batching separate from automation completion
|
||||||
|
|
||||||
|
The intended benefit is fewer short-lived intermediate revisions. Batching
|
||||||
|
cannot discover when a third-party template has finished, particularly while a
|
||||||
|
dialogue is open or another device also runs automation. A maximum delay or a
|
||||||
|
rename boundary can legitimately publish intermediate content.
|
||||||
|
|
||||||
|
Coalescing `CREATE` followed by `CHANGED` for the same still-existing file is a
|
||||||
|
useful follow-up, but it needs its own ordering and interruption tests. Preserve
|
||||||
|
the existing cross-path rename rule: store the destination before deleting the
|
||||||
|
proven source branch. Do not retarget or discard pending source operations
|
||||||
|
without proving the effect is already covered.
|
||||||
|
|
||||||
|
A guarantee that a multi-step template appears remotely only after completion
|
||||||
|
would need an explicit producer completion contract and its failure behaviour.
|
||||||
|
That is a separate design. Enabling batching alone must not be described as
|
||||||
|
eliminating genuine concurrent-edit conflicts or as a confirmed fix for #1207.
|
||||||
|
|
||||||
|
## Ownership and implementation order
|
||||||
|
|
||||||
|
1. **Commonlib recovery:** event ownership, ordered snapshot persistence,
|
||||||
|
successful completion and failure propagation, finite flush boundaries, and
|
||||||
|
same-time content reconciliation. Add focused failing regressions before
|
||||||
|
changing production code. Reuse the existing event manager, file handler,
|
||||||
|
scanner, and provenance store.
|
||||||
|
2. **Host integration:** keep the Obsidian persistence adapter and lifecycle
|
||||||
|
wiring aligned with those contracts. Verify callers of
|
||||||
|
`commitPendingFileEvents`, including settings application and database
|
||||||
|
preparation, and preserve their failure outcomes. Validate the packed
|
||||||
|
Commonlib artefact and update the consumer's exact dependency.
|
||||||
|
3. **LiveSync mode batching:** add the explicit setting and its primary-language
|
||||||
|
presentation only after the recovery regressions pass. Establish the stated
|
||||||
|
timing semantics and repeat the two-device comparisons with the actual
|
||||||
|
package, including start-up with batching already enabled.
|
||||||
|
4. **Further coalescing:** assess same-path creation/editing and any proposed
|
||||||
|
rename optimisation separately, against the original automation problem.
|
||||||
|
|
||||||
|
Keep these changes independently reviewable. Shared behaviour belongs in
|
||||||
|
Commonlib; the plug-in must not acquire a source mirror or patch installed
|
||||||
|
package files. Snapshot format changes, a second persistent store, or a new
|
||||||
|
automation transaction API require a further design decision.
|
||||||
|
|
||||||
|
## Verification required
|
||||||
|
|
||||||
|
Use controlled promises at the manager and persistence boundaries for focused
|
||||||
|
regressions, and real PouchDB tests for content and ancestry. The acceptance
|
||||||
|
matrix is:
|
||||||
|
|
||||||
|
| Scenario | Required observation |
|
||||||
|
| -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| Stop after dequeue, timer release, or during a database write | Pending work remains recoverable; restart eventually represents the saved Vault edit. |
|
||||||
|
| Database write succeeds before snapshot cleanup; restart twice | Replay creates no duplicate revision and preserves later edits. |
|
||||||
|
| Old and new events share a path; snapshot writes overlap | Completing old work cannot remove new work or regress the durable snapshot. |
|
||||||
|
| Handler returns false, throws, or snapshot persistence fails | Flush reports failure; pending work is retained; unrelated recoverable work can continue. |
|
||||||
|
| Buffered or in-progress work exists with no waiting timer | Flush still waits for actual completion. |
|
||||||
|
| A flush overlaps new typing or a rename/delete barrier | The captured work completes without waiting for later work or deadlocking. |
|
||||||
|
| Equal timestamps: unchanged, edited, untracked, pruned provenance, unavailable Chunks, or concurrent recheck changes | Preserve correct ancestry and bytes, avoid duplicate revisions, and distinguish unresolved comparisons from successful reconciliation. |
|
||||||
|
| Settings save followed by stop; ordinary unload; interruption during replay | No operation is forgotten because a settings callback or shutdown released its timer. |
|
||||||
|
| Sustained edits and more paths than available processing slots | Observe the intentional-delay deadline and progress of unrelated ready files. |
|
||||||
|
| Old settings, mode changes, explicit opt-in, and settings import | Existing LiveSync mode users retain immediate saving until they choose batching. |
|
||||||
|
| Incoming identical content, an edit, a deletion, or delete/recreate while waiting | Preserve genuine branches without extra conflicts or loss of local work. |
|
||||||
|
|
||||||
|
Keep the existing E2E comparison between batching disabled, an ignored legacy
|
||||||
|
setting, and explicitly enabled batching. Repeat both directions, manual
|
||||||
|
synchronisation, saved-snapshot restart, and controlled in-progress restart with
|
||||||
|
the fix. Record the original failure separately from repaired results; a pass
|
||||||
|
with a different timestamp bucket or a test-only bundle is not verification of
|
||||||
|
the production fix. Run platform-specific lifecycle checks before extending the
|
||||||
|
evidence to Windows or mobile.
|
||||||
|
|
||||||
|
## Alternatives and consequences
|
||||||
|
|
||||||
|
- **Remove only the LiveSync guard:** leaves recovery gaps and can activate a
|
||||||
|
previously ignored setting. The explicit choice follows correctness work.
|
||||||
|
- **Wait only at unload:** cannot cover abrupt termination and hosts which do
|
||||||
|
not await asynchronous unload. Continuous recoverability is still required.
|
||||||
|
- **Repair only the scanner:** improves surviving-file recovery, but does not
|
||||||
|
establish event ordering, deletion intent, or truthful flush completion.
|
||||||
|
- **Retain only in-progress events:** closes a persistence gap, but still needs
|
||||||
|
success/failure handling, replay checks, and recovery before the first snapshot.
|
||||||
|
- **Remove timestamp rounding or force the Vault copy into the current winner:**
|
||||||
|
fails for exactly equal timestamps and can destroy branch intent. Content and
|
||||||
|
provenance determine the safe action.
|
||||||
|
|
||||||
|
Keeping work recoverable adds snapshot bookkeeping and can make local flush
|
||||||
|
wait longer because it now waits for real processing. Same-time reconciliation
|
||||||
|
can require additional content reads. Batching trades prompt remote visibility
|
||||||
|
for fewer database updates and may increase the window for genuine concurrent
|
||||||
|
edits. Measure these costs without weakening the content-preservation contract.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [Conflict resolution and revision provenance](../specs_conflict_resolution.md)
|
||||||
|
- [Batch database update settings](../settings.md#batch-database-update)
|
||||||
|
- [Common library package boundary](2026_07_common_library_package_boundary.md)
|
||||||
|
- [Fast Fetch persistence and completion](2026_08_fast_fetch_persistence_and_completion.md)
|
||||||
|
- [Two-device Batch save reproduction](../../test/e2e-obsidian/scripts/issue1207-livesync-batch.ts)
|
||||||
|
- [Commonlib event manager at the investigated revision](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/managers/StorageEventManager.ts)
|
||||||
|
- [Commonlib Offline Scanner at the investigated revision](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/serviceFeatures/offlineScanner.ts)
|
||||||
|
- [Commonlib file handler at the investigated revision](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/serviceModules/ServiceFileHandlerBase.ts)
|
||||||
|
- [Commonlib scanner policy and stored event interpretation](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/docs/offline-scanner.md)
|
||||||
|
- [Commonlib event admission and reflection contract](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/docs/storage-events-and-reflection.md)
|
||||||
|
- [Existing equal-time provenance regression](https://github.com/vrtmrz/livesync-commonlib/blob/aa54282937c20d3db7f43778d240feeb1586cced/src/serviceFeatures/offlineScanner.provenance.unit.spec.ts)
|
||||||
+16
-3
@@ -669,17 +669,28 @@ Setup URIs preserve both sleep preferences. Older Setup URIs which do not contai
|
|||||||
#### Batch database update
|
#### Batch database update
|
||||||
|
|
||||||
Setting key: batchSave
|
Setting key: batchSave
|
||||||
Reducing the frequency with which on-disk changes are reflected into the DB
|
|
||||||
|
Combine repeated local file updates before saving them to the local database in modes other than LiveSync mode. This reduces intermediate database revisions and delays their availability to other devices.
|
||||||
|
|
||||||
|
#### Batch database updates in LiveSync mode
|
||||||
|
|
||||||
|
Setting key: batchSaveInLiveSync
|
||||||
|
|
||||||
|
Apply the same bounded delay in LiveSync mode. This separate choice is disabled by default for both existing settings and new Vaults. An older configuration containing `liveSync: true` and `batchSave: true` does not enable it. The settings interface shows the choice for the current mode, and switching modes preserves both choices. Older clients continue to ignore batching in LiveSync mode.
|
||||||
|
|
||||||
|
Both choices share the delays below. Manual synchronisation, deletion, or renaming can end a batch early. Batching reduces intermediate updates; it does not identify when a template has finished or prevent genuine concurrent edits on different devices.
|
||||||
|
|
||||||
#### Minimum delay for batch database updating
|
#### Minimum delay for batch database updating
|
||||||
|
|
||||||
Setting key: batchSaveMinimumDelay
|
Setting key: batchSaveMinimumDelay
|
||||||
Seconds. Saving to the local database will be delayed until this value after we stop typing or saving.
|
|
||||||
|
Seconds of quiet after the latest eligible edit before saving to the local database, subject to the maximum delay below.
|
||||||
|
|
||||||
#### Maximum delay for batch database updating
|
#### Maximum delay for batch database updating
|
||||||
|
|
||||||
Setting key: batchSaveMaximumDelay
|
Setting key: batchSaveMaximumDelay
|
||||||
Saving will be performed forcefully after this number of seconds.
|
|
||||||
|
Maximum intentional delay in seconds from the first pending update to a file. Later edits do not restart this limit, and waiting for a processing slot does not add another batch delay. This limit makes the work eligible to run; it does not guarantee database completion or receipt by another device within that time.
|
||||||
|
|
||||||
### 4. Deletion Propagation (Advanced)
|
### 4. Deletion Propagation (Advanced)
|
||||||
|
|
||||||
@@ -795,6 +806,8 @@ When a saved setting changes whether a normal file can be reflected, LiveSync re
|
|||||||
|
|
||||||
Customisation Sync is a supported, advanced opt-in feature. Its current per-file implementation is covered by a two-Vault real-Obsidian workflow for snippets, configuration files, and plug-in files. Hidden File Sync is a separate feature with different setup, selection, and conflict behaviour; do not use both features to manage the same files.
|
Customisation Sync is a supported, advanced opt-in feature. Its current per-file implementation is covered by a two-Vault real-Obsidian workflow for snippets, configuration files, and plug-in files. Hidden File Sync is a separate feature with different setup, selection, and conflict behaviour; do not use both features to manage the same files.
|
||||||
|
|
||||||
|
**Automatic** delegates an item's files to Hidden File Sync. Enable and initialise Hidden File Sync in **Sync Settings** on this device first, after reviewing its [target and ignore patterns](./tips/hidden-file-sync.md#review-the-file-selection). Choosing Automatic while Hidden File Sync is disabled displays a Notice and leaves the current mode unchanged. Existing Automatic assignments, including those imported from another device, remain saved while Hidden File Sync is disabled.
|
||||||
|
|
||||||
### 1. Customisation Sync
|
### 1. Customisation Sync
|
||||||
|
|
||||||
#### Device name
|
#### Device name
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ export secret_key=<SECRET KEY>
|
|||||||
export bucket=vault-data
|
export bucket=vault-data
|
||||||
export region=auto
|
export region=auto
|
||||||
export bucket_prefix=my-vault
|
export bucket_prefix=my-vault
|
||||||
export passphrase=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||||
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
|
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
|
||||||
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -128,6 +128,9 @@ export database=obsidiannotes
|
|||||||
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
|
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
|
||||||
```
|
```
|
||||||
|
|
||||||
|
> [!TIP]
|
||||||
|
> In fish, `hostname` is read-only and cannot be exported. Pass the values through `env` instead: `curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | env hostname=http://localhost:5984 username=<...> password=<...> database=obsidiannotes bash`.
|
||||||
|
|
||||||
If it results like the following:
|
If it results like the following:
|
||||||
```
|
```
|
||||||
CouchDB provisioning completed.
|
CouchDB provisioning completed.
|
||||||
@@ -173,20 +176,20 @@ Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our serv
|
|||||||
|
|
||||||
### 1. Generate the setup URI on a desktop device or server
|
### 1. Generate the setup URI on a desktop device or server
|
||||||
|
|
||||||
The `username` and `password` here are the credentials which Self-hosted LiveSync will store for routine access to this database. They may be the administrator credentials from step 2. If you have separately configured a CouchDB account with the required access to this database, use that account instead. Neither the provisioning wrapper nor the Setup URI generator creates that separate account.
|
The `LIVESYNC_USER` and `LIVESYNC_PASS` here are the credentials which Self-hosted LiveSync will store for routine access to this database. They may be the administrator credentials from step 2. If you have separately configured a CouchDB account with the required access to this database, use that account instead. Neither the provisioning wrapper nor the Setup URI generator creates that separate account.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
export hostname=https://tiles-photograph-routine-groundwater.trycloudflare.com
|
export LIVESYNC_HOST=https://tiles-photograph-routine-groundwater.trycloudflare.com
|
||||||
export database=obsidiannotes
|
export LIVESYNC_DB=obsidiannotes
|
||||||
export username=<INSERT COUCHDB USERNAME FOR LIVESYNC>
|
export LIVESYNC_USER=<INSERT COUCHDB USERNAME FOR LIVESYNC>
|
||||||
export password=<INSERT THE COUCHDB PASSWORD>
|
export LIVESYNC_PASS=<INSERT THE COUCHDB PASSWORD>
|
||||||
export passphrase=<INSERT A STRONG VAULT ENCRYPTION PASSPHRASE>
|
export LIVESYNC_PASSPHRASE=<INSERT A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||||
export uri_passphrase=<INSERT A SEPARATE SETUP URI PASSPHRASE> # Optional
|
export uri_passphrase=<INSERT A SEPARATE SETUP URI PASSPHRASE> # Optional
|
||||||
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
||||||
```
|
```
|
||||||
|
|
||||||
> [!TIP]
|
> [!TIP]
|
||||||
> `passphrase` protects the synchronised Vault data with end-to-end encryption. `uri_passphrase` protects only the Setup URI. Use different values, store both securely, and do not send the Setup URI and its passphrase through the same channel.
|
> `LIVESYNC_PASSPHRASE` protects the synchronised Vault data with end-to-end encryption. `uri_passphrase` protects only the Setup URI. Use different values, store both securely, and do not send the Setup URI and its passphrase through the same channel.
|
||||||
>
|
>
|
||||||
> If `uri_passphrase` is omitted, the generator creates a cryptographically random value and prints it once.
|
> If `uri_passphrase` is omitted, the generator creates a cryptographically random value and prints it once.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -130,7 +130,7 @@ export remote_type=p2p
|
|||||||
export p2p_relays=wss://relay.example.com
|
export p2p_relays=wss://relay.example.com
|
||||||
export p2p_room_id=<A PRIVATE ROOM ID> # Optional; generated when omitted
|
export p2p_room_id=<A PRIVATE ROOM ID> # Optional; generated when omitted
|
||||||
export p2p_passphrase=<A PRIVATE P2P PASSPHRASE> # Optional; generated when omitted
|
export p2p_passphrase=<A PRIVATE P2P PASSPHRASE> # Optional; generated when omitted
|
||||||
export passphrase=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||||
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
|
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
|
||||||
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -72,6 +72,12 @@ Hidden File Sync must be enabled independently after ordinary setup on each devi
|
|||||||
|
|
||||||
If you later run Rebuild or Fetch as a recovery operation, expect optional features to be disabled again. Complete ordinary recovery first, then repeat this guide deliberately.
|
If you later run Rebuild or Fetch as a recovery operation, expect optional features to be disabled again. Complete ordinary recovery first, then repeat this guide deliberately.
|
||||||
|
|
||||||
|
## Automatic in Customisation Sync
|
||||||
|
|
||||||
|
**Automatic** delegates an item's files to Hidden File Sync. Enable Hidden File Sync on this device through the steps above before choosing Automatic. If Hidden File Sync is disabled, the choice displays a Notice without changing the item's mode or starting initialisation. Previously saved Automatic assignments remain saved, so this prerequisite also applies after importing settings from another device.
|
||||||
|
|
||||||
|
Once Hidden File Sync is enabled, choose the item's initial action in Customisation Sync. Selecting a whole plug-in applies that action to both its **MAIN** and **DATA** selections.
|
||||||
|
|
||||||
## Related settings
|
## Related settings
|
||||||
|
|
||||||
The [settings reference](../settings.md#7-hidden-files-advanced) describes periodic scanning, scanning before replication, notification suppression, selectors, and overwrite patterns. Change those controls only after the basic Hidden File Sync path is working.
|
The [settings reference](../settings.md#7-hidden-files-advanced) describes periodic scanning, scanning before replication, notification suppression, selectors, and overwrite patterns. Change those controls only after the basic Hidden File Sync path is working.
|
||||||
Generated
+9
-9
@@ -23,13 +23,13 @@
|
|||||||
"@smithy/types": "^4.14.3",
|
"@smithy/types": "^4.14.3",
|
||||||
"@smithy/util-retry": "^4.4.5",
|
"@smithy/util-retry": "^4.4.5",
|
||||||
"@vrtmrz/browser-ui-kit": "0.1.0",
|
"@vrtmrz/browser-ui-kit": "0.1.0",
|
||||||
"@vrtmrz/livesync-commonlib": "0.2.0",
|
"@vrtmrz/livesync-commonlib": "0.2.1",
|
||||||
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
|
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
|
||||||
"@vrtmrz/ui-interactions": "0.1.2",
|
"@vrtmrz/ui-interactions": "0.1.2",
|
||||||
"@vrtmrz/value-store": "0.1.0",
|
"@vrtmrz/value-store": "0.1.0",
|
||||||
"diff-match-patch": "^1.0.5",
|
"diff-match-patch": "^1.0.5",
|
||||||
"idb": "^8.0.3",
|
"idb": "^8.0.3",
|
||||||
"markdown-it": "^14.3.2",
|
"markdown-it": "^14.3.3",
|
||||||
"minimatch": "^10.2.5",
|
"minimatch": "^10.2.5",
|
||||||
"obsidian": "^1.13.1",
|
"obsidian": "^1.13.1",
|
||||||
"octagonal-wheels": "^0.1.55",
|
"octagonal-wheels": "^0.1.55",
|
||||||
@@ -4116,9 +4116,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/@vrtmrz/livesync-commonlib": {
|
"node_modules/@vrtmrz/livesync-commonlib": {
|
||||||
"version": "0.2.0",
|
"version": "0.2.1",
|
||||||
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.2.0.tgz",
|
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.2.1.tgz",
|
||||||
"integrity": "sha512-K24Pkke+FTjZI7DjIV7qkZk+BmC9VebN3da+2cVD9SWOgoGl7OvDyXTHDafbrmmM4vi4ydaydAEh9glO1xHF6A==",
|
"integrity": "sha512-fnQb2FcEG+NqIQX9KhTstea4B7C4VAL0g9ry5rp41qnJDuLaR5zfCz995L+l3Khyrp/9kcPw4VX5RUmV+8GVog==",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@aws-crypto/sha256-js": "5.2.0",
|
"@aws-crypto/sha256-js": "5.2.0",
|
||||||
@@ -4134,7 +4134,7 @@
|
|||||||
"events": "^3.3.0",
|
"events": "^3.3.0",
|
||||||
"fflate": "^0.8.2",
|
"fflate": "^0.8.2",
|
||||||
"idb": "^8.0.3",
|
"idb": "^8.0.3",
|
||||||
"markdown-it": "^14.3.2",
|
"markdown-it": "^14.3.3",
|
||||||
"minimatch": "^10.2.6",
|
"minimatch": "^10.2.6",
|
||||||
"octagonal-wheels": "^0.1.55",
|
"octagonal-wheels": "^0.1.55",
|
||||||
"pouchdb-adapter-http": "^9.0.0",
|
"pouchdb-adapter-http": "^9.0.0",
|
||||||
@@ -8263,9 +8263,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/markdown-it": {
|
"node_modules/markdown-it": {
|
||||||
"version": "14.3.2",
|
"version": "14.3.3",
|
||||||
"resolved": "https://registry.npmjs.org/markdown-it/-/markdown-it-14.3.2.tgz",
|
"resolved": "https://registry.npmjs.org/markdown-it/-/markdown-it-14.3.3.tgz",
|
||||||
"integrity": "sha512-sHHjZ5fJKlgrG4qns2YwVcdNep35h5fERrfkD2YNsb9UFk0UIHarbiTaHKVMlPuWAoiilyK8Fv/jAm11slsY7Q==",
|
"integrity": "sha512-L//tDAkjTjJCU6pBUSBTgs3Kg4InVaZuUZog/b6V12UKhcDQ7S78877ibbCanRExwPuYP1CZD7Gd2dxSy1GXRg==",
|
||||||
"funding": [
|
"funding": [
|
||||||
{
|
{
|
||||||
"type": "github",
|
"type": "github",
|
||||||
|
|||||||
+5
-2
@@ -97,6 +97,9 @@
|
|||||||
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
|
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
|
||||||
"test:e2e:obsidian:hidden-file-selectors": "tsx test/e2e-obsidian/scripts/hidden-file-selectors.ts",
|
"test:e2e:obsidian:hidden-file-selectors": "tsx test/e2e-obsidian/scripts/hidden-file-selectors.ts",
|
||||||
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
|
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
|
||||||
|
"test:e2e:obsidian:customisation-catalogue-reload": "tsx test/e2e-obsidian/scripts/customisation-catalogue-reload.ts",
|
||||||
|
"test:e2e:obsidian:optional-file-save-completion": "tsx test/e2e-obsidian/scripts/optional-file-save-completion.ts",
|
||||||
|
"test:e2e:obsidian:livesync-batch": "tsx test/e2e-obsidian/scripts/issue1207-livesync-batch.ts",
|
||||||
"test:e2e:obsidian:received-change-readiness": "tsx test/e2e-obsidian/scripts/received-change-readiness.ts",
|
"test:e2e:obsidian:received-change-readiness": "tsx test/e2e-obsidian/scripts/received-change-readiness.ts",
|
||||||
"test:e2e:obsidian:remote-feature-change": "tsx test/e2e-obsidian/scripts/remote-feature-change.ts",
|
"test:e2e:obsidian:remote-feature-change": "tsx test/e2e-obsidian/scripts/remote-feature-change.ts",
|
||||||
"test:e2e:obsidian:internal-metadata-migration": "tsx test/e2e-obsidian/scripts/internal-metadata-migration.ts",
|
"test:e2e:obsidian:internal-metadata-migration": "tsx test/e2e-obsidian/scripts/internal-metadata-migration.ts",
|
||||||
@@ -197,13 +200,13 @@
|
|||||||
"@smithy/types": "^4.14.3",
|
"@smithy/types": "^4.14.3",
|
||||||
"@smithy/util-retry": "^4.4.5",
|
"@smithy/util-retry": "^4.4.5",
|
||||||
"@vrtmrz/browser-ui-kit": "0.1.0",
|
"@vrtmrz/browser-ui-kit": "0.1.0",
|
||||||
"@vrtmrz/livesync-commonlib": "0.2.0",
|
"@vrtmrz/livesync-commonlib": "0.2.1",
|
||||||
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
|
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
|
||||||
"@vrtmrz/ui-interactions": "0.1.2",
|
"@vrtmrz/ui-interactions": "0.1.2",
|
||||||
"@vrtmrz/value-store": "0.1.0",
|
"@vrtmrz/value-store": "0.1.0",
|
||||||
"diff-match-patch": "^1.0.5",
|
"diff-match-patch": "^1.0.5",
|
||||||
"idb": "^8.0.3",
|
"idb": "^8.0.3",
|
||||||
"markdown-it": "^14.3.2",
|
"markdown-it": "^14.3.3",
|
||||||
"minimatch": "^10.2.5",
|
"minimatch": "^10.2.5",
|
||||||
"obsidian": "^1.13.1",
|
"obsidian": "^1.13.1",
|
||||||
"octagonal-wheels": "^0.1.55",
|
"octagonal-wheels": "^0.1.55",
|
||||||
|
|||||||
@@ -7,6 +7,9 @@
|
|||||||
* remove it from this map in the same change.
|
* remove it from this map in the same change.
|
||||||
*/
|
*/
|
||||||
export const liveSyncProvisionalEnglishMessages = {
|
export const liveSyncProvisionalEnglishMessages = {
|
||||||
|
"Hidden File Sync is disabled on this device. Enable it in Sync Settings before using Automatic or initialising hidden files.":
|
||||||
|
"Hidden File Sync is disabled on this device. Enable it in Sync Settings before using Automatic or initialising hidden files.",
|
||||||
|
"Hidden File Sync initialisation was not started.": "Hidden File Sync initialisation was not started.",
|
||||||
"(RegExp) Limit normal files to matching paths. Leave empty to allow all normal file paths. Hidden File Sync uses its own target patterns.":
|
"(RegExp) Limit normal files to matching paths. Leave empty to allow all normal file paths. Hidden File Sync uses its own target patterns.":
|
||||||
"(RegExp) Limit normal files to matching paths. Leave empty to allow all normal file paths. Hidden File Sync uses its own target patterns.",
|
"(RegExp) Limit normal files to matching paths. Leave empty to allow all normal file paths. Hidden File Sync uses its own target patterns.",
|
||||||
"(RegExp) Skip changes to matching normal and hidden files, locally and remotely.":
|
"(RegExp) Skip changes to matching normal and hidden files, locally and remotely.":
|
||||||
|
|||||||
@@ -0,0 +1,298 @@
|
|||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
|
||||||
|
vi.mock("@/deps.ts", () => ({
|
||||||
|
addIcon: vi.fn(),
|
||||||
|
diff_match_patch: class DiffMatchPatch {},
|
||||||
|
normalizePath: (path: string) => path,
|
||||||
|
parseYaml: vi.fn(),
|
||||||
|
Platform: {},
|
||||||
|
}));
|
||||||
|
vi.mock("./PluginDialogModal.ts", () => ({ PluginDialogModal: class PluginDialogModal {} }));
|
||||||
|
vi.mock("@/features/HiddenFileCommon/JsonResolveModal.ts", () => ({ JsonResolveModal: class JsonResolveModal {} }));
|
||||||
|
vi.mock("@/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts", () => ({
|
||||||
|
ConflictResolveModal: class ConflictResolveModal {},
|
||||||
|
}));
|
||||||
|
vi.mock("@/modules/features/ModuleLog.ts", () => ({ MARK_DONE: "" }));
|
||||||
|
vi.mock("@/common/utils.ts", () => ({
|
||||||
|
cancelTask: vi.fn(),
|
||||||
|
EVEN: Symbol("even"),
|
||||||
|
isCustomisationSyncMetadata: vi.fn(),
|
||||||
|
isPluginMetadata: vi.fn(),
|
||||||
|
scheduleTask: vi.fn(),
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/PeriodicProcessor.ts", () => ({ PeriodicProcessor: class PeriodicProcessor {} }));
|
||||||
|
vi.mock("@/common/events.ts", () => ({
|
||||||
|
EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG: "open-plugin-sync",
|
||||||
|
eventHub: { onEvent: vi.fn() },
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/translation", () => ({ $msg: (message: string) => message }));
|
||||||
|
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({ getObsidianCommunityPluginManager: vi.fn() }));
|
||||||
|
|
||||||
|
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
|
import { createOptionalFileSyncDatabase, optionalFileEntry } from "../../../test/fixtures/optionalFileSyncDatabase";
|
||||||
|
import { ConfigSync, PluginDataExDisplayV2, pluginManifests } from "./CmdConfigSync";
|
||||||
|
|
||||||
|
const filenames = ["main.js", "manifest.json", "styles.css"];
|
||||||
|
const content: Record<string, string> = {
|
||||||
|
"main.js": "module.exports = class ExamplePlugin {};",
|
||||||
|
"manifest.json": JSON.stringify({ id: "example", name: "Example", version: "1.0.0" }),
|
||||||
|
"styles.css": ".example { color: red; }",
|
||||||
|
};
|
||||||
|
const fixtureOwners: Awaited<ReturnType<typeof createOptionalFileSyncDatabase>>[] = [];
|
||||||
|
|
||||||
|
function deferred() {
|
||||||
|
let resolve!: () => void;
|
||||||
|
const promise = new Promise<void>((done) => {
|
||||||
|
resolve = done;
|
||||||
|
});
|
||||||
|
return { promise, resolve };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function catalogue() {
|
||||||
|
const fixture = await createOptionalFileSyncDatabase();
|
||||||
|
fixtureOwners.push(fixture);
|
||||||
|
fixture.settings.syncOnlyRegEx = "\\.(?:md|png|pdf)$" as typeof fixture.settings.syncOnlyRegEx;
|
||||||
|
const paths = filenames.map((name) => `ix:device/PLUGIN_MAIN/example%${name}` as FilePathWithPrefix);
|
||||||
|
for (const [index, path] of paths.entries()) {
|
||||||
|
const wireContent = `fixture-header\u200b\n\u200c${btoa(content[filenames[index]])}`;
|
||||||
|
await expect(fixture.localDatabase.putDBEntry(optionalFileEntry(path, wireContent))).resolves.toMatchObject({
|
||||||
|
ok: true,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
const gates = new Map(paths.map((path) => [path, deferred()]));
|
||||||
|
const readEntry = fixture.localDatabase.getDBEntry.bind(fixture.localDatabase);
|
||||||
|
vi.spyOn(fixture.localDatabase, "getDBEntry").mockImplementation(async (path, ...args) => {
|
||||||
|
await gates.get(path)?.promise;
|
||||||
|
return readEntry(path, ...args);
|
||||||
|
});
|
||||||
|
const written = new Map<string, string>();
|
||||||
|
const configSync = Object.create(ConfigSync.prototype) as ConfigSync;
|
||||||
|
Object.assign(configSync, {
|
||||||
|
core: {
|
||||||
|
localDatabase: fixture.localDatabase,
|
||||||
|
settings: fixture.settings,
|
||||||
|
services: fixture.services,
|
||||||
|
storageAccess: {
|
||||||
|
ensureDir: vi.fn(async () => true),
|
||||||
|
readHiddenFileText: vi.fn(async (path: string) => written.get(path) ?? ""),
|
||||||
|
writeHiddenFileAuto: vi.fn(async (path: string, data: string) => {
|
||||||
|
written.set(path, data);
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
pluginList: [],
|
||||||
|
pluginListGeneration: 0,
|
||||||
|
loadedManifest_mTime: new Map(),
|
||||||
|
updatingV2Count: 0,
|
||||||
|
pluginScanProcessor: { clearQueue: vi.fn(), enqueue: vi.fn() },
|
||||||
|
_log: vi.fn(),
|
||||||
|
storeCustomisationFileV2: vi.fn(async () => true),
|
||||||
|
});
|
||||||
|
const updates: Promise<void>[] = [];
|
||||||
|
const update = configSync.updatePluginListV2.bind(configSync);
|
||||||
|
vi.spyOn(configSync, "updatePluginListV2").mockImplementation((...args) => {
|
||||||
|
const promise = update(...args);
|
||||||
|
updates.push(promise);
|
||||||
|
return promise;
|
||||||
|
});
|
||||||
|
const release = (order = filenames) => {
|
||||||
|
for (const name of order) gates.get(`ix:device/PLUGIN_MAIN/example%${name}` as FilePathWithPrefix)!.resolve();
|
||||||
|
};
|
||||||
|
return { ...fixture, configSync, updates, paths, release, written };
|
||||||
|
}
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
vi.restoreAllMocks();
|
||||||
|
pluginManifests.clear();
|
||||||
|
for (const fixture of fixtureOwners.splice(0)) await fixture.dispose();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("issue #1271: rebuilding a multi-file Customisation Sync item", () => {
|
||||||
|
it.each([
|
||||||
|
{ order: filenames },
|
||||||
|
{ order: [...filenames].reverse() },
|
||||||
|
{ order: ["manifest.json", "main.js", "styles.css"] },
|
||||||
|
])("retains all per-file documents when reads are released as $order", async ({ order }) => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
await fixture.configSync.reloadPluginList(false);
|
||||||
|
fixture.release(order);
|
||||||
|
await Promise.all(fixture.updates);
|
||||||
|
|
||||||
|
const documents = await fixture.database.allDocs({ keys: fixture.paths });
|
||||||
|
expect(documents.rows).toHaveLength(3);
|
||||||
|
expect(fixture.configSync.pluginList).toHaveLength(1);
|
||||||
|
expect(fixture.configSync.pluginList[0].files.map((file) => file.filename).sort()).toEqual(
|
||||||
|
filenames.map((name) => `plugins/example/${name}`).sort()
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("applies all three files after the first reload", async () => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
await fixture.configSync.reloadPluginList(false);
|
||||||
|
fixture.release();
|
||||||
|
await Promise.all(fixture.updates);
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
fixture.configSync.applyDataV2(fixture.configSync.pluginList[0] as PluginDataExDisplayV2)
|
||||||
|
).resolves.toBe(true);
|
||||||
|
expect([...fixture.written.keys()].sort()).toEqual(
|
||||||
|
filenames.map((name) => `.obsidian/plugins/example/${name}`).sort()
|
||||||
|
);
|
||||||
|
for (const name of filenames)
|
||||||
|
expect(fixture.written.get(`.obsidian/plugins/example/${name}`)).toBe(content[name]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fills an existing row during a second enumeration", async () => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
await fixture.configSync.reloadPluginList(false);
|
||||||
|
fixture.release();
|
||||||
|
await Promise.all(fixture.updates);
|
||||||
|
await fixture.configSync.updatePluginList(false);
|
||||||
|
await Promise.all(fixture.updates);
|
||||||
|
|
||||||
|
expect(fixture.configSync.pluginList[0].files.map((file) => file.filename).sort()).toEqual(
|
||||||
|
filenames.map((name) => `plugins/example/${name}`).sort()
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("retains all files when item updates run sequentially", async () => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
fixture.release();
|
||||||
|
for (const path of fixture.paths) await fixture.configSync.updatePluginListV2(false, path);
|
||||||
|
|
||||||
|
expect(fixture.configSync.pluginList[0].files.map((file) => file.filename).sort()).toEqual(
|
||||||
|
filenames.map((name) => `plugins/example/${name}`).sort()
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("updates a different item while another item's reads are pending", async () => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
const otherPath = "ix:device/PLUGIN_MAIN/other%main.js" as FilePathWithPrefix;
|
||||||
|
await fixture.localDatabase.putDBEntry(
|
||||||
|
optionalFileEntry(otherPath, `fixture-header\u200b\n\u200c${btoa(content["main.js"])}`)
|
||||||
|
);
|
||||||
|
const pending = fixture.paths.map((path) => fixture.configSync.updatePluginListV2(false, path));
|
||||||
|
try {
|
||||||
|
await fixture.configSync.updatePluginListV2(false, otherPath);
|
||||||
|
expect(fixture.configSync.pluginList.map((entry) => entry.name)).toEqual(["other"]);
|
||||||
|
expect(fixture.configSync.updatingV2Count).toBe(3);
|
||||||
|
} finally {
|
||||||
|
fixture.release();
|
||||||
|
await Promise.all(pending);
|
||||||
|
}
|
||||||
|
expect(fixture.configSync.updatingV2Count).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("continues updating an item after an earlier read fails", async () => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
fixture.release();
|
||||||
|
vi.spyOn(fixture.configSync, "createPluginDataExFileV2").mockRejectedValueOnce(new Error("Read failed"));
|
||||||
|
const failed = fixture.configSync.updatePluginListV2(false, fixture.paths[0]);
|
||||||
|
const next = fixture.configSync.updatePluginListV2(false, fixture.paths[1]);
|
||||||
|
|
||||||
|
await expect(failed).rejects.toThrow("Read failed");
|
||||||
|
await next;
|
||||||
|
expect(fixture.configSync.pluginList[0].files.map((file) => file.filename)).toEqual([
|
||||||
|
"plugins/example/manifest.json",
|
||||||
|
]);
|
||||||
|
expect(fixture.configSync.updatingV2Count).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["read", "file update"])("reload discards cached files while an earlier %s is pending", async (phase) => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
fixture.release();
|
||||||
|
for (const path of fixture.paths) await fixture.configSync.updatePluginListV2(false, path);
|
||||||
|
const earlierRead = deferred();
|
||||||
|
const freshRead = deferred();
|
||||||
|
const entered = deferred();
|
||||||
|
const createFile = fixture.configSync.createPluginDataExFileV2.bind(fixture.configSync);
|
||||||
|
let reads = 0;
|
||||||
|
vi.spyOn(fixture.configSync, "createPluginDataExFileV2").mockImplementation(async (path) => {
|
||||||
|
if (reads++ === 0) {
|
||||||
|
if (phase === "read") {
|
||||||
|
entered.resolve();
|
||||||
|
await earlierRead.promise;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
await freshRead.promise;
|
||||||
|
}
|
||||||
|
return createFile(path);
|
||||||
|
});
|
||||||
|
if (phase === "file update") {
|
||||||
|
const entry = fixture.configSync.pluginList[0] as PluginDataExDisplayV2;
|
||||||
|
const setFile = entry.setFile.bind(entry);
|
||||||
|
vi.spyOn(entry, "setFile").mockImplementationOnce(async (file) => {
|
||||||
|
entered.resolve();
|
||||||
|
await earlierRead.promise;
|
||||||
|
await setFile(file);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
const pending = fixture.configSync.updatePluginListV2(false, fixture.paths[0]);
|
||||||
|
try {
|
||||||
|
await entered.promise;
|
||||||
|
for (const path of fixture.paths.slice(1)) await fixture.database.remove(await fixture.database.get(path));
|
||||||
|
await fixture.configSync.reloadPluginList(false);
|
||||||
|
earlierRead.resolve();
|
||||||
|
await pending;
|
||||||
|
} finally {
|
||||||
|
earlierRead.resolve();
|
||||||
|
freshRead.resolve();
|
||||||
|
await Promise.all(fixture.updates);
|
||||||
|
}
|
||||||
|
expect(fixture.configSync.pluginList[0].files.map((file) => file.filename)).toEqual([
|
||||||
|
"plugins/example/main.js",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("clearing the disabled feature prevents pending updates from repopulating the list", async () => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
const pending = fixture.paths.map((path) => fixture.configSync.updatePluginListV2(false, path));
|
||||||
|
fixture.settings.usePluginSync = false;
|
||||||
|
await fixture.configSync.updatePluginList(false);
|
||||||
|
fixture.release();
|
||||||
|
await Promise.all(pending);
|
||||||
|
|
||||||
|
expect(fixture.configSync.pluginList).toEqual([]);
|
||||||
|
expect(fixture.configSync.updatingV2Count).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("a superseded enumeration cannot reintroduce a deleted item", async () => {
|
||||||
|
const fixture = await catalogue();
|
||||||
|
const obsoletePath = "ix:device/PLUGIN_MAIN/obsolete%main.js" as FilePathWithPrefix;
|
||||||
|
await fixture.localDatabase.putDBEntry(
|
||||||
|
optionalFileEntry(obsoletePath, `fixture-header\u200b\n\u200c${btoa(content["main.js"])}`)
|
||||||
|
);
|
||||||
|
const enumerate = fixture.localDatabase.findEntries.bind(fixture.localDatabase);
|
||||||
|
type EnumeratedEntry = ReturnType<typeof enumerate> extends AsyncGenerator<infer Entry> ? Entry : never;
|
||||||
|
const obsoleteDocs: EnumeratedEntry[] = [];
|
||||||
|
for await (const doc of enumerate(obsoletePath, `${obsoletePath}\u{10ffff}`, { include_docs: true })) {
|
||||||
|
obsoleteDocs.push(doc);
|
||||||
|
}
|
||||||
|
const entered = deferred();
|
||||||
|
const release = deferred();
|
||||||
|
let scans = 0;
|
||||||
|
vi.spyOn(fixture.localDatabase, "findEntries").mockImplementation((...args) => {
|
||||||
|
if (scans++ !== 0) return enumerate(...args);
|
||||||
|
return (async function* () {
|
||||||
|
entered.resolve();
|
||||||
|
await release.promise;
|
||||||
|
yield* obsoleteDocs;
|
||||||
|
})();
|
||||||
|
});
|
||||||
|
const pending = fixture.configSync.updatePluginList(false);
|
||||||
|
try {
|
||||||
|
await entered.promise;
|
||||||
|
await fixture.database.remove(await fixture.database.get(obsoletePath));
|
||||||
|
await fixture.configSync.reloadPluginList(false);
|
||||||
|
} finally {
|
||||||
|
release.resolve();
|
||||||
|
fixture.release();
|
||||||
|
await pending;
|
||||||
|
await Promise.all(fixture.updates);
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(fixture.configSync.pluginList.map((entry) => entry.name)).toEqual(["example"]);
|
||||||
|
expect(fixture.configSync.pluginList[0].files).toHaveLength(3);
|
||||||
|
expect(fixture.configSync.updatingV2Count).toBe(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,220 @@
|
|||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
|
||||||
|
vi.mock("@/deps.ts", () => ({
|
||||||
|
addIcon: vi.fn(),
|
||||||
|
diff_match_patch: class DiffMatchPatch {},
|
||||||
|
normalizePath: vi.fn((path: string) => path),
|
||||||
|
parseYaml: vi.fn(),
|
||||||
|
Platform: {},
|
||||||
|
}));
|
||||||
|
vi.mock("./PluginDialogModal.ts", () => ({
|
||||||
|
PluginDialogModal: class PluginDialogModal {},
|
||||||
|
}));
|
||||||
|
vi.mock("@/features/HiddenFileCommon/JsonResolveModal.ts", () => ({
|
||||||
|
JsonResolveModal: class JsonResolveModal {},
|
||||||
|
}));
|
||||||
|
vi.mock("@/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts", () => ({
|
||||||
|
ConflictResolveModal: class ConflictResolveModal {},
|
||||||
|
}));
|
||||||
|
// The readiness helpers come from the real LiveSyncCommands, so only its log view dependency is replaced.
|
||||||
|
vi.mock("@/modules/features/ModuleLog.ts", () => ({
|
||||||
|
MARK_DONE: "",
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/types.ts", () => ({
|
||||||
|
ICXHeader: "ix:",
|
||||||
|
PERIODIC_PLUGIN_SWEEP: 60,
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/utils.ts", () => ({
|
||||||
|
cancelTask: vi.fn(),
|
||||||
|
EVEN: Symbol("even"),
|
||||||
|
isCustomisationSyncMetadata: vi.fn(),
|
||||||
|
isPluginMetadata: vi.fn(),
|
||||||
|
scheduleTask: vi.fn(),
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/PeriodicProcessor.ts", () => ({
|
||||||
|
PeriodicProcessor: class PeriodicProcessor {},
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/events.ts", () => ({
|
||||||
|
EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG: "open-plugin-sync",
|
||||||
|
eventHub: {
|
||||||
|
onEvent: vi.fn(),
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/translation", () => ({
|
||||||
|
$msg: vi.fn((message: string) => message),
|
||||||
|
}));
|
||||||
|
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
|
||||||
|
getObsidianCommunityPluginManager: vi.fn(),
|
||||||
|
}));
|
||||||
|
|
||||||
|
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
|
import { getDocData } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||||
|
import { ConfigSync } from "./CmdConfigSync";
|
||||||
|
import { createOptionalFileSyncDatabase } from "../../../test/fixtures/optionalFileSyncDatabase";
|
||||||
|
|
||||||
|
const path = ".obsidian/snippets/test.css" as FilePath;
|
||||||
|
const fixtures: Awaited<ReturnType<typeof createOptionalFileSyncDatabase>>[] = [];
|
||||||
|
|
||||||
|
async function createConfigSync(useV2 = false) {
|
||||||
|
const fixture = await createOptionalFileSyncDatabase();
|
||||||
|
fixtures.push(fixture);
|
||||||
|
fixture.settings.usePluginSyncV2 = useV2;
|
||||||
|
const files = new Map<string, { content: string; mtime: number }>([
|
||||||
|
[path, { content: "body { color: red; }", mtime: 2000 }],
|
||||||
|
]);
|
||||||
|
const storageAccess = {
|
||||||
|
statHidden: vi.fn(async (file: string) => {
|
||||||
|
const entry = files.get(file);
|
||||||
|
return entry
|
||||||
|
? { type: "file" as const, ctime: 1000, mtime: entry.mtime, size: entry.content.length }
|
||||||
|
: null;
|
||||||
|
}),
|
||||||
|
readHiddenFileBinary: vi.fn(async (file: string) => {
|
||||||
|
const entry = files.get(file);
|
||||||
|
if (!entry) throw new Error(`Missing fixture: ${file}`);
|
||||||
|
return new TextEncoder().encode(entry.content).buffer;
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
const services = {
|
||||||
|
...fixture.services,
|
||||||
|
appLifecycle: { isReady: () => false, isSuspended: () => true },
|
||||||
|
database: { ...fixture.services.database, isDatabaseReady: vi.fn(() => true) },
|
||||||
|
path: {
|
||||||
|
...fixture.services.path,
|
||||||
|
isMarkedAsSameChanges: vi.fn(),
|
||||||
|
markChangesAreSame: vi.fn(),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const config = Object.create(ConfigSync.prototype) as ConfigSync;
|
||||||
|
Object.assign(config, {
|
||||||
|
core: { settings: fixture.settings, localDatabase: fixture.localDatabase, storageAccess, services },
|
||||||
|
_log: vi.fn(),
|
||||||
|
updatePluginList: vi.fn(async () => undefined),
|
||||||
|
updatePluginListV2: vi.fn(async () => undefined),
|
||||||
|
recentProcessedInternalFiles: [],
|
||||||
|
});
|
||||||
|
const databasePath = useV2 ? config.filenameWithUnifiedKey(path) : config.filenameToUnifiedKey(path);
|
||||||
|
return { fixture, config, files, services, storageAccess, databasePath };
|
||||||
|
}
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
vi.restoreAllMocks();
|
||||||
|
for (const fixture of fixtures.splice(0)) await fixture.dispose();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Customisation Sync optional save completion", () => {
|
||||||
|
const flush = { whenFlushRequested: Promise.resolve() };
|
||||||
|
|
||||||
|
it("waits for the database write while the application is suspended or still starting", async () => {
|
||||||
|
const { fixture, config } = await createConfigSync();
|
||||||
|
let release!: () => void;
|
||||||
|
const gate = new Promise<void>((resolve) => {
|
||||||
|
release = resolve;
|
||||||
|
});
|
||||||
|
const put = fixture.localDatabase.putDBEntry.bind(fixture.localDatabase);
|
||||||
|
const spy = vi.spyOn(fixture.localDatabase, "putDBEntry").mockImplementation(async (...args) => {
|
||||||
|
await gate;
|
||||||
|
return await put(...args);
|
||||||
|
});
|
||||||
|
let completed = false;
|
||||||
|
const result = config._processOptionalFileEventWithResult(path, flush).then((outcome) => {
|
||||||
|
completed = true;
|
||||||
|
return outcome;
|
||||||
|
});
|
||||||
|
await vi.waitFor(() => expect(spy).toHaveBeenCalledOnce());
|
||||||
|
expect(completed).toBe(false);
|
||||||
|
release();
|
||||||
|
await expect(result).resolves.toEqual({ kind: "completed" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([false, true])("confirms identical contents and stores a same-time edit (V2: %s)", async (useV2) => {
|
||||||
|
const { fixture, config, databasePath, files } = await createConfigSync(useV2);
|
||||||
|
await expect(config._processOptionalFileEventWithResult(path, flush)).resolves.toEqual({ kind: "completed" });
|
||||||
|
const first = await fixture.database.get(databasePath);
|
||||||
|
await config._processOptionalFileEventWithResult(path, flush);
|
||||||
|
expect((await fixture.database.get(databasePath))._rev).toBe(first._rev);
|
||||||
|
files.get(path)!.content = "body { color: tan; }";
|
||||||
|
await config._processOptionalFileEventWithResult(path, flush);
|
||||||
|
expect((await fixture.database.get(databasePath))._rev).not.toBe(first._rev);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
{ useV2: false, throws: false },
|
||||||
|
{ useV2: true, throws: false },
|
||||||
|
{ useV2: false, throws: true },
|
||||||
|
{ useV2: true, throws: true },
|
||||||
|
])("rejects a failed write and permits the unchanged notification to retry: %j", async ({ useV2, throws }) => {
|
||||||
|
const { fixture, config, databasePath } = await createConfigSync(useV2);
|
||||||
|
const failure = new Error("temporary write failure");
|
||||||
|
const put = vi.spyOn(fixture.localDatabase, "putDBEntry");
|
||||||
|
if (throws) put.mockRejectedValueOnce(failure);
|
||||||
|
else put.mockResolvedValueOnce(false);
|
||||||
|
const failed = config._processOptionalFileEventWithResult(path, flush);
|
||||||
|
if (throws) await expect(failed).rejects.toBe(failure);
|
||||||
|
else await expect(failed).rejects.toThrow("Could not save customisation");
|
||||||
|
await expect(fixture.localDatabase.getDBEntry(databasePath)).resolves.toBe(false);
|
||||||
|
await expect(config._processOptionalFileEventWithResult(path, flush)).resolves.toEqual({ kind: "completed" });
|
||||||
|
expect(put).toHaveBeenCalledTimes(2);
|
||||||
|
expect(await fixture.localDatabase.getDBEntry(databasePath)).not.toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([false, true])("confirms deletion and treats a repeated deletion as complete (V2: %s)", async (useV2) => {
|
||||||
|
const { fixture, config, databasePath, files } = await createConfigSync(useV2);
|
||||||
|
await config._processOptionalFileEventWithResult(path, flush);
|
||||||
|
files.delete(path);
|
||||||
|
const put = vi.spyOn(fixture.localDatabase, "putRaw");
|
||||||
|
put.mockResolvedValueOnce({ ok: false, id: "failed", rev: "" });
|
||||||
|
await expect(config._processOptionalFileEventWithResult(path, flush)).rejects.toThrow(
|
||||||
|
"Could not delete customisation"
|
||||||
|
);
|
||||||
|
await expect(config._processOptionalFileEventWithResult(path, flush)).resolves.toEqual({ kind: "completed" });
|
||||||
|
const deleted = await fixture.database.get(databasePath);
|
||||||
|
expect("deleted" in deleted && deleted.deleted).toBe(true);
|
||||||
|
await expect(config._processOptionalFileEventWithResult(path, flush)).resolves.toEqual({ kind: "completed" });
|
||||||
|
expect((await fixture.database.get(databasePath))._rev).toBe(deleted._rev);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("distinguishes a different owner and unmet prerequisites", async () => {
|
||||||
|
const { config, services, storageAccess } = await createConfigSync();
|
||||||
|
await expect(config._processOptionalFileEventWithResult("notes.md" as FilePath, flush)).resolves.toEqual({
|
||||||
|
kind: "not-applicable",
|
||||||
|
});
|
||||||
|
services.database.isDatabaseReady.mockReturnValue(false);
|
||||||
|
await expect(config._processOptionalFileEventWithResult(path, flush)).resolves.toEqual({
|
||||||
|
kind: "veto",
|
||||||
|
value: "database-not-ready",
|
||||||
|
});
|
||||||
|
services.database.isDatabaseReady.mockReturnValue(true);
|
||||||
|
vi.spyOn(services.setting, "getDeviceAndVaultName").mockReturnValue("");
|
||||||
|
await expect(config._processOptionalFileEventWithResult(path, flush)).resolves.toEqual({
|
||||||
|
kind: "veto",
|
||||||
|
value: "device-name-not-configured",
|
||||||
|
});
|
||||||
|
expect(storageAccess.readHiddenFileBinary).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves every V2 file when neighbouring plug-in files arrive together", async () => {
|
||||||
|
const { fixture, config, files } = await createConfigSync(true);
|
||||||
|
const main = ".obsidian/plugins/example/main.js" as FilePath;
|
||||||
|
const style = ".obsidian/plugins/example/styles.css" as FilePath;
|
||||||
|
files.set(main, { content: "module.exports = {};", mtime: 3000 });
|
||||||
|
files.set(style, { content: "body {}", mtime: 3000 });
|
||||||
|
await Promise.all([
|
||||||
|
config._processOptionalFileEventWithResult(main, flush),
|
||||||
|
config._processOptionalFileEventWithResult(style, flush),
|
||||||
|
]);
|
||||||
|
expect(await fixture.localDatabase.getDBEntry(config.filenameWithUnifiedKey(main))).not.toBe(false);
|
||||||
|
expect(await fixture.localDatabase.getDBEntry(config.filenameWithUnifiedKey(style))).not.toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not acknowledge a database deletion which was not accepted", async () => {
|
||||||
|
const { fixture, config, databasePath } = await createConfigSync();
|
||||||
|
await config.storeCustomizationFiles(path);
|
||||||
|
const original = await fixture.localDatabase.getDBEntry(databasePath);
|
||||||
|
expect(original).not.toBe(false);
|
||||||
|
vi.spyOn(fixture.localDatabase, "putRaw").mockResolvedValueOnce({ ok: false, id: "failed", rev: "" });
|
||||||
|
await expect(config.deleteConfigOnDatabase(databasePath)).resolves.toBe(false);
|
||||||
|
const unchanged = await fixture.localDatabase.getDBEntry(databasePath);
|
||||||
|
expect(unchanged && getDocData(unchanged.data)).toBe(original && getDocData(original.data));
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -69,6 +69,10 @@ import { LiveSyncError } from "@vrtmrz/livesync-commonlib/compat/common/LSError"
|
|||||||
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
|
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
|
||||||
import { getObsidianCommunityPluginManager } from "@/common/obsidianCommunityPlugins.ts";
|
import { getObsidianCommunityPluginManager } from "@/common/obsidianCommunityPlugins.ts";
|
||||||
|
|
||||||
|
import type { HandlerOutcome } from "@vrtmrz/livesync-commonlib/compat/services/lib/HandlerUtils";
|
||||||
|
import type { OptionalFileEventOptions } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
||||||
|
import { PendingConfigSaves } from "./PendingConfigSaves";
|
||||||
|
|
||||||
const d = "\u200b";
|
const d = "\u200b";
|
||||||
const d2 = "\n";
|
const d2 = "\n";
|
||||||
const UPDATED_CONFIGURATION_NOTICE_KEY = "config-sync:updated-configuration";
|
const UPDATED_CONFIGURATION_NOTICE_KEY = "config-sync:updated-configuration";
|
||||||
@@ -412,6 +416,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
periodicPluginSweepProcessor = new PeriodicProcessor(this.core, async () => await this.scanAllConfigFiles(false));
|
periodicPluginSweepProcessor = new PeriodicProcessor(this.core, async () => await this.scanAllConfigFiles(false));
|
||||||
|
|
||||||
pluginList: IPluginDataExDisplay[] = [];
|
pluginList: IPluginDataExDisplay[] = [];
|
||||||
|
private pluginListGeneration = 0;
|
||||||
showPluginSyncModal() {
|
showPluginSyncModal() {
|
||||||
if (!this.isThisModuleEnabled()) {
|
if (!this.isThisModuleEnabled()) {
|
||||||
return;
|
return;
|
||||||
@@ -532,6 +537,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
return Promise.resolve(true);
|
return Promise.resolve(true);
|
||||||
}
|
}
|
||||||
async reloadPluginList(showMessage: boolean) {
|
async reloadPluginList(showMessage: boolean) {
|
||||||
|
this.pluginListGeneration++;
|
||||||
this.pluginList = [];
|
this.pluginList = [];
|
||||||
this.loadedManifest_mTime.clear();
|
this.loadedManifest_mTime.clear();
|
||||||
pluginList.set(this.pluginList);
|
pluginList.set(this.pluginList);
|
||||||
@@ -761,12 +767,16 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
updatingV2Count = 0;
|
updatingV2Count = 0;
|
||||||
|
|
||||||
async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
|
async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
|
||||||
|
const generation = this.pluginListGeneration;
|
||||||
|
const isCurrent = () => generation === this.pluginListGeneration && this.isThisModuleEnabled();
|
||||||
try {
|
try {
|
||||||
this.updatingV2Count++;
|
this.updatingV2Count++;
|
||||||
pluginV2Progress.set(this.updatingV2Count);
|
pluginV2Progress.set(this.updatingV2Count);
|
||||||
// const unifiedFilenameWithKey = this.filenameWithUnifiedKey(updatedDocumentPath);
|
// const unifiedFilenameWithKey = this.filenameWithUnifiedKey(updatedDocumentPath);
|
||||||
const { pathV1 } = this.parseUnifiedPath(unifiedFilenameWithKey);
|
const { pathV1 } = this.parseUnifiedPath(unifiedFilenameWithKey);
|
||||||
|
|
||||||
|
await serialized(`config:catalogue:${pathV1}`, async () => {
|
||||||
|
if (!isCurrent()) return;
|
||||||
const oldEntry = this.pluginList.find((e) => e.documentPath == pathV1);
|
const oldEntry = this.pluginList.find((e) => e.documentPath == pathV1);
|
||||||
let entry: PluginDataExDisplayV2 | undefined = undefined;
|
let entry: PluginDataExDisplayV2 | undefined = undefined;
|
||||||
|
|
||||||
@@ -780,6 +790,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
}
|
}
|
||||||
if (!entry) return;
|
if (!entry) return;
|
||||||
const file = await this.createPluginDataExFileV2(unifiedFilenameWithKey);
|
const file = await this.createPluginDataExFileV2(unifiedFilenameWithKey);
|
||||||
|
if (!isCurrent()) return;
|
||||||
if (file) {
|
if (file) {
|
||||||
await entry.setFile(file);
|
await entry.setFile(file);
|
||||||
} else {
|
} else {
|
||||||
@@ -788,6 +799,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
this.pluginList = this.pluginList.filter((e) => e.documentPath != pathV1);
|
this.pluginList = this.pluginList.filter((e) => e.documentPath != pathV1);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if (!isCurrent()) return;
|
||||||
const newList = this.pluginList.filter((e) => e.documentPath != entry.documentPath);
|
const newList = this.pluginList.filter((e) => e.documentPath != entry.documentPath);
|
||||||
newList.push(entry);
|
newList.push(entry);
|
||||||
this.pluginList = newList;
|
this.pluginList = newList;
|
||||||
@@ -795,6 +807,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
scheduleTask("updatePluginListV2", 100, () => {
|
scheduleTask("updatePluginListV2", 100, () => {
|
||||||
pluginList.set(this.pluginList);
|
pluginList.set(this.pluginList);
|
||||||
});
|
});
|
||||||
|
});
|
||||||
} finally {
|
} finally {
|
||||||
this.updatingV2Count--;
|
this.updatingV2Count--;
|
||||||
pluginV2Progress.set(this.updatingV2Count);
|
pluginV2Progress.set(this.updatingV2Count);
|
||||||
@@ -871,11 +884,13 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
|
|
||||||
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
|
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
|
||||||
if (!this.isThisModuleEnabled()) {
|
if (!this.isThisModuleEnabled()) {
|
||||||
|
this.pluginListGeneration++;
|
||||||
this.pluginScanProcessor.clearQueue();
|
this.pluginScanProcessor.clearQueue();
|
||||||
this.pluginList = [];
|
this.pluginList = [];
|
||||||
pluginList.set(this.pluginList);
|
pluginList.set(this.pluginList);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
const generation = this.pluginListGeneration;
|
||||||
try {
|
try {
|
||||||
this.updatingV2Count++;
|
this.updatingV2Count++;
|
||||||
pluginV2Progress.set(this.updatingV2Count);
|
pluginV2Progress.set(this.updatingV2Count);
|
||||||
@@ -888,6 +903,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
})
|
})
|
||||||
: this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
|
: this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
|
||||||
for await (const v of plugins) {
|
for await (const v of plugins) {
|
||||||
|
if (generation !== this.pluginListGeneration || !this.isThisModuleEnabled()) return;
|
||||||
if (v.deleted || v._deleted) continue;
|
if (v.deleted || v._deleted) continue;
|
||||||
if (v.path.indexOf("%") !== -1) {
|
if (v.path.indexOf("%") !== -1) {
|
||||||
fireAndForget(() => this.updatePluginListV2(showMessage, v.path));
|
fireAndForget(() => this.updatePluginListV2(showMessage, v.path));
|
||||||
@@ -1206,7 +1222,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
}
|
}
|
||||||
|
|
||||||
recentProcessedInternalFiles = [] as string[];
|
recentProcessedInternalFiles = [] as string[];
|
||||||
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
|
async makeEntryFromFile(path: FilePath, requireCompletion = false): Promise<false | PluginDataExFile> {
|
||||||
const stat = await this.core.storageAccess.statHidden(path);
|
const stat = await this.core.storageAccess.statHidden(path);
|
||||||
let version: string | undefined;
|
let version: string | undefined;
|
||||||
let displayName: string | undefined;
|
let displayName: string | undefined;
|
||||||
@@ -1240,6 +1256,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log(`The file ${path} could not be encoded`);
|
this._log(`The file ${path} could not be encoded`);
|
||||||
this._log(ex, LOG_LEVEL_VERBOSE);
|
this._log(ex, LOG_LEVEL_VERBOSE);
|
||||||
|
if (requireCompletion) throw ex;
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
const mtime = stat.mtime;
|
const mtime = stat.mtime;
|
||||||
@@ -1254,6 +1271,15 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
|
async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
|
||||||
|
return await this.storeCustomisationFileV2Core(path, term, force, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async storeCustomisationFileV2Core(
|
||||||
|
path: FilePath,
|
||||||
|
term: string,
|
||||||
|
force: boolean,
|
||||||
|
requireCompletion: boolean
|
||||||
|
) {
|
||||||
const vf = this.filenameWithUnifiedKey(path, term);
|
const vf = this.filenameWithUnifiedKey(path, term);
|
||||||
return await serialized(`plugin-${vf}`, async () => {
|
return await serialized(`plugin-${vf}`, async () => {
|
||||||
const prefixedFileName = vf;
|
const prefixedFileName = vf;
|
||||||
@@ -1261,7 +1287,8 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
const id = await this.path2id(prefixedFileName);
|
const id = await this.path2id(prefixedFileName);
|
||||||
const stat = await this.core.storageAccess.statHidden(path);
|
const stat = await this.core.storageAccess.statHidden(path);
|
||||||
if (!stat) {
|
if (!stat) {
|
||||||
return false;
|
if (!requireCompletion) return false;
|
||||||
|
return await this.deleteConfigOnDatabaseCore(vf, force, true);
|
||||||
}
|
}
|
||||||
const mtime = stat.mtime;
|
const mtime = stat.mtime;
|
||||||
const content = await this.core.storageAccess.readHiddenFileBinary(path);
|
const content = await this.core.storageAccess.readHiddenFileBinary(path);
|
||||||
@@ -1285,7 +1312,10 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
eden: {},
|
eden: {},
|
||||||
};
|
};
|
||||||
} else {
|
} else {
|
||||||
if (this.services.path.isMarkedAsSameChanges(prefixedFileName, [old.mtime, mtime + 1]) == EVEN) {
|
if (
|
||||||
|
!requireCompletion &&
|
||||||
|
this.services.path.isMarkedAsSameChanges(prefixedFileName, [old.mtime, mtime + 1]) == EVEN
|
||||||
|
) {
|
||||||
this._log(
|
this._log(
|
||||||
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`,
|
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`,
|
||||||
LOG_LEVEL_DEBUG
|
LOG_LEVEL_DEBUG
|
||||||
@@ -1300,7 +1330,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
const dataStart = dataSrc.indexOf(DUMMY_END);
|
const dataStart = dataSrc.indexOf(DUMMY_END);
|
||||||
const oldContent = dataSrc.substring(dataStart + DUMMY_END.length);
|
const oldContent = dataSrc.substring(dataStart + DUMMY_END.length);
|
||||||
const oldContentArray = base64ToArrayBuffer(oldContent);
|
const oldContentArray = base64ToArrayBuffer(oldContent);
|
||||||
if (await isDocContentSame(oldContentArray, content)) {
|
if (!old.deleted && (await isDocContentSame(oldContentArray, content))) {
|
||||||
this._log(
|
this._log(
|
||||||
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`,
|
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`,
|
||||||
LOG_LEVEL_VERBOSE
|
LOG_LEVEL_VERBOSE
|
||||||
@@ -1320,24 +1350,36 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
const ret = await this.localDatabase.putDBEntry(saveData);
|
const ret = await this.localDatabase.putDBEntry(saveData);
|
||||||
|
if (requireCompletion && (!ret || !ret.ok))
|
||||||
|
throw new LiveSyncError(`Could not save customisation: ${path}`);
|
||||||
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
|
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
|
||||||
fireAndForget(() => this.updatePluginListV2(false, this.filenameWithUnifiedKey(path)));
|
fireAndForget(() => this.updatePluginListV2(false, this.filenameWithUnifiedKey(path)));
|
||||||
return ret;
|
return ret;
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
|
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
|
||||||
this._log(ex, LOG_LEVEL_VERBOSE);
|
this._log(ex, LOG_LEVEL_VERBOSE);
|
||||||
|
if (requireCompletion) throw ex;
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
|
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
|
||||||
|
return await this.storeCustomizationFilesCore(path, termOverRide, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async storeCustomizationFilesCore(
|
||||||
|
path: FilePath,
|
||||||
|
termOverRide: string | undefined,
|
||||||
|
requireCompletion: boolean
|
||||||
|
) {
|
||||||
const term = termOverRide || this.services.setting.getDeviceAndVaultName();
|
const term = termOverRide || this.services.setting.getDeviceAndVaultName();
|
||||||
if (term == "") {
|
if (term == "") {
|
||||||
this._log($msg("We have to configure the device name"), LOG_LEVEL_NOTICE);
|
this._log($msg("We have to configure the device name"), LOG_LEVEL_NOTICE);
|
||||||
|
if (requireCompletion) throw new LiveSyncError("The device name is not configured");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
if (this.useV2) {
|
if (this.useV2) {
|
||||||
return await this.storeCustomisationFileV2(path, term);
|
return await this.storeCustomisationFileV2Core(path, term, false, requireCompletion);
|
||||||
}
|
}
|
||||||
const vf = this.filenameToUnifiedKey(path, term);
|
const vf = this.filenameToUnifiedKey(path, term);
|
||||||
// console.warn(`Storing ${path} to ${bareVF} :--> ${keyedVF}`);
|
// console.warn(`Storing ${path} to ${bareVF} :--> ${keyedVF}`);
|
||||||
@@ -1378,7 +1420,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
fileTargets = ["manifest.json", "theme.css"].map((e) => `${parentPath}/${e}` as FilePath);
|
fileTargets = ["manifest.json", "theme.css"].map((e) => `${parentPath}/${e}` as FilePath);
|
||||||
}
|
}
|
||||||
for (const target of fileTargets) {
|
for (const target of fileTargets) {
|
||||||
const data = await this.makeEntryFromFile(target);
|
const data = await this.makeEntryFromFile(target, requireCompletion);
|
||||||
if (data == false) {
|
if (data == false) {
|
||||||
this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
|
this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
|
||||||
continue;
|
continue;
|
||||||
@@ -1398,9 +1440,9 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
// this._log(`Configuration saving: ${prefixedFileName}`);
|
// this._log(`Configuration saving: ${prefixedFileName}`);
|
||||||
if (dt.files.length == 0) {
|
if (dt.files.length == 0) {
|
||||||
this._log(`Nothing left: deleting.. ${path}`);
|
this._log(`Nothing left: deleting.. ${path}`);
|
||||||
await this.deleteConfigOnDatabase(prefixedFileName);
|
const deleted = await this.deleteConfigOnDatabaseCore(prefixedFileName, false, requireCompletion);
|
||||||
await this.updatePluginList(false, prefixedFileName);
|
await this.updatePluginList(false, prefixedFileName);
|
||||||
return;
|
return requireCompletion ? deleted : undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
const content = createTextBlob(serialize(dt));
|
const content = createTextBlob(serialize(dt));
|
||||||
@@ -1422,7 +1464,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
eden: {},
|
eden: {},
|
||||||
};
|
};
|
||||||
} else {
|
} else {
|
||||||
if (old.mtime == mtime) {
|
if (!requireCompletion && old.mtime == mtime) {
|
||||||
// this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
|
// this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
@@ -1443,7 +1485,7 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
const isSame = (await Promise.all(diffs)).every((e) => e == true);
|
const isSame = (await Promise.all(diffs)).every((e) => e == true);
|
||||||
if (isSame) {
|
if (isSame && !old.deleted) {
|
||||||
this._log(
|
this._log(
|
||||||
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`,
|
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`,
|
||||||
LOG_LEVEL_VERBOSE
|
LOG_LEVEL_VERBOSE
|
||||||
@@ -1464,16 +1506,51 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
const ret = await this.localDatabase.putDBEntry(saveData);
|
const ret = await this.localDatabase.putDBEntry(saveData);
|
||||||
|
if (requireCompletion && (!ret || !ret.ok))
|
||||||
|
throw new LiveSyncError(`Could not save customisation: ${path}`);
|
||||||
await this.updatePluginList(false, saveData.path);
|
await this.updatePluginList(false, saveData.path);
|
||||||
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
|
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
|
||||||
return ret;
|
return ret;
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
|
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
|
||||||
this._log(ex, LOG_LEVEL_VERBOSE);
|
this._log(ex, LOG_LEVEL_VERBOSE);
|
||||||
|
if (requireCompletion) throw ex;
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
private pendingOptionalSaves?: PendingConfigSaves;
|
||||||
|
|
||||||
|
async _processOptionalFileEventWithResult(
|
||||||
|
path: FilePath,
|
||||||
|
options?: OptionalFileEventOptions
|
||||||
|
): Promise<HandlerOutcome<void, string>> {
|
||||||
|
if (!this.isThisModuleEnabled() || !this.isTargetPath(path)) return { kind: "not-applicable" };
|
||||||
|
const configDir = normalizePath(this.configDir);
|
||||||
|
const handledElsewhere = Object.values(this.settings.pluginSyncExtendedSetting)
|
||||||
|
.filter((entry) => entry.mode !== MODE_SELECTIVE && entry.mode !== MODE_SHINY)
|
||||||
|
.flatMap((entry) => entry.files)
|
||||||
|
.some((file) => `${configDir}/${file}`.toLowerCase().startsWith(path.toLowerCase()));
|
||||||
|
if (handledElsewhere) return { kind: "not-applicable" };
|
||||||
|
if (!this._isDatabaseReady()) return { kind: "veto", value: "database-not-ready" };
|
||||||
|
const term = this.services.setting.getDeviceAndVaultName();
|
||||||
|
if (!term) return { kind: "veto", value: "device-name-not-configured" };
|
||||||
|
const stat = await this.core.storageAccess.statHidden(path);
|
||||||
|
if (stat && stat.type !== "file") return { kind: "not-applicable" };
|
||||||
|
|
||||||
|
// V1 saves a complete file group; V2 persists each file separately.
|
||||||
|
const key = this.useV2 ? this.filenameWithUnifiedKey(path, term) : this.filenameToUnifiedKey(path, term);
|
||||||
|
this.pendingOptionalSaves ??= new PendingConfigSaves();
|
||||||
|
await this.pendingOptionalSaves.enqueue(
|
||||||
|
key,
|
||||||
|
async () => {
|
||||||
|
await this.storeCustomizationFilesCore(path, term, true);
|
||||||
|
},
|
||||||
|
options?.whenFlushRequested
|
||||||
|
);
|
||||||
|
return { kind: "completed" };
|
||||||
|
}
|
||||||
|
|
||||||
async _anyProcessOptionalFileEvent(path: FilePath): Promise<boolean> {
|
async _anyProcessOptionalFileEvent(path: FilePath): Promise<boolean> {
|
||||||
return await this.watchVaultRawEventsAsync(path);
|
return await this.watchVaultRawEventsAsync(path);
|
||||||
}
|
}
|
||||||
@@ -1615,6 +1692,14 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) {
|
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) {
|
||||||
|
return await this.deleteConfigOnDatabaseCore(prefixedFileName, forceWrite, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async deleteConfigOnDatabaseCore(
|
||||||
|
prefixedFileName: FilePathWithPrefix,
|
||||||
|
forceWrite: boolean,
|
||||||
|
requireCompletion: boolean
|
||||||
|
) {
|
||||||
// const id = await this.path2id(prefixedFileName);
|
// const id = await this.path2id(prefixedFileName);
|
||||||
const mtime = new Date().getTime();
|
const mtime = new Date().getTime();
|
||||||
return await serialized("file-x-" + prefixedFileName, async () => {
|
return await serialized("file-x-" + prefixedFileName, async () => {
|
||||||
@@ -1640,13 +1725,15 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
type: "newnote",
|
type: "newnote",
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
await this.localDatabase.putRaw(saveData);
|
const result = await this.localDatabase.putRaw(saveData);
|
||||||
|
if (!result.ok) throw new LiveSyncError(`Could not delete customisation: ${prefixedFileName}`);
|
||||||
await this.updatePluginList(false, prefixedFileName);
|
await this.updatePluginList(false, prefixedFileName);
|
||||||
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
|
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
|
||||||
return true;
|
return true;
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
|
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
|
||||||
this._log(ex, LOG_LEVEL_VERBOSE);
|
this._log(ex, LOG_LEVEL_VERBOSE);
|
||||||
|
if (requireCompletion) throw ex;
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -1756,6 +1843,9 @@ export class ConfigSync extends LiveSyncCommands {
|
|||||||
}
|
}
|
||||||
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
|
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
|
||||||
services.fileProcessing.processOptionalFileEvent.addHandler(this._anyProcessOptionalFileEvent.bind(this));
|
services.fileProcessing.processOptionalFileEvent.addHandler(this._anyProcessOptionalFileEvent.bind(this));
|
||||||
|
services.fileProcessing.processOptionalFileEventWithResult?.addHandler(
|
||||||
|
this._processOptionalFileEventWithResult.bind(this)
|
||||||
|
);
|
||||||
services.conflict.getOptionalConflictCheckMethod.addHandler(this._anyGetOptionalConflictCheckMethod.bind(this));
|
services.conflict.getOptionalConflictCheckMethod.addHandler(this._anyGetOptionalConflictCheckMethod.bind(this));
|
||||||
services.replication.processVirtualDocument.addHandler(this._anyModuleParsedReplicationResultItem.bind(this));
|
services.replication.processVirtualDocument.addHandler(this._anyModuleParsedReplicationResultItem.bind(this));
|
||||||
services.setting.onRealiseSetting.addHandler(this._everyRealizeSettingSyncMode.bind(this));
|
services.setting.onRealiseSetting.addHandler(this._everyRealizeSettingSyncMode.bind(this));
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
import { compatGlobal, type CompatTimeoutHandle } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
||||||
|
|
||||||
|
type PendingSave = {
|
||||||
|
save: () => Promise<void>;
|
||||||
|
completion: Promise<void>;
|
||||||
|
resolve: () => void;
|
||||||
|
reject: (error: unknown) => void;
|
||||||
|
timer?: CompatTimeoutHandle;
|
||||||
|
started: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Owns the quiet-period delay and completion of coalesced customisation saves. */
|
||||||
|
export class PendingConfigSaves {
|
||||||
|
private readonly pending = new Map<string, PendingSave>();
|
||||||
|
|
||||||
|
enqueue(key: string, save: () => Promise<void>, whenFlushRequested?: Promise<void>): Promise<void> {
|
||||||
|
let group = this.pending.get(key);
|
||||||
|
if (!group) {
|
||||||
|
let resolve!: () => void;
|
||||||
|
let reject!: (error: unknown) => void;
|
||||||
|
const completion = new Promise<void>((onResolve, onReject) => {
|
||||||
|
resolve = onResolve;
|
||||||
|
reject = onReject;
|
||||||
|
});
|
||||||
|
group = { save, completion, resolve, reject, started: false };
|
||||||
|
this.pending.set(key, group);
|
||||||
|
} else {
|
||||||
|
group.save = save;
|
||||||
|
if (group.timer !== undefined) compatGlobal.clearTimeout(group.timer);
|
||||||
|
}
|
||||||
|
const captured = group;
|
||||||
|
group.timer = compatGlobal.setTimeout(() => this.run(key, captured), 100);
|
||||||
|
// Capture this group: a late flush of an earlier event must not release
|
||||||
|
// a newer group which happens to use the same customisation key.
|
||||||
|
void whenFlushRequested?.then(
|
||||||
|
() => this.run(key, captured),
|
||||||
|
() => {}
|
||||||
|
);
|
||||||
|
return group.completion;
|
||||||
|
}
|
||||||
|
|
||||||
|
private run(key: string, group: PendingSave): void {
|
||||||
|
if (group.started) return;
|
||||||
|
group.started = true;
|
||||||
|
if (group.timer !== undefined) compatGlobal.clearTimeout(group.timer);
|
||||||
|
this.pending.delete(key);
|
||||||
|
void Promise.resolve().then(group.save).then(group.resolve, group.reject);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { PendingConfigSaves } from "./PendingConfigSaves";
|
||||||
|
|
||||||
|
function deferred() {
|
||||||
|
let resolve!: () => void;
|
||||||
|
let reject!: (error: unknown) => void;
|
||||||
|
const promise = new Promise<void>((onResolve, onReject) => {
|
||||||
|
resolve = onResolve;
|
||||||
|
reject = onReject;
|
||||||
|
});
|
||||||
|
return { promise, resolve, reject };
|
||||||
|
}
|
||||||
|
|
||||||
|
afterEach(() => vi.useRealTimers());
|
||||||
|
|
||||||
|
describe("pending customisation saves", () => {
|
||||||
|
it("retains the quiet period and completes every coalesced notification after the actual save", async () => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
const pending = new PendingConfigSaves();
|
||||||
|
const save = deferred();
|
||||||
|
const firstSave = vi.fn(async () => {});
|
||||||
|
const lastSave = vi.fn(() => save.promise);
|
||||||
|
const first = pending.enqueue("plugin", firstSave);
|
||||||
|
await vi.advanceTimersByTimeAsync(80);
|
||||||
|
const second = pending.enqueue("plugin", lastSave);
|
||||||
|
let completed = false;
|
||||||
|
const all = Promise.all([first, second]).then(() => {
|
||||||
|
completed = true;
|
||||||
|
});
|
||||||
|
await vi.advanceTimersByTimeAsync(99);
|
||||||
|
expect(lastSave).not.toHaveBeenCalled();
|
||||||
|
await vi.advanceTimersByTimeAsync(1);
|
||||||
|
expect(firstSave).not.toHaveBeenCalled();
|
||||||
|
expect(lastSave).toHaveBeenCalledOnce();
|
||||||
|
expect(completed).toBe(false);
|
||||||
|
save.resolve();
|
||||||
|
await all;
|
||||||
|
expect(completed).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("flushes the captured group without waiting for or releasing later notifications", async () => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
const pending = new PendingConfigSaves();
|
||||||
|
const flush = deferred();
|
||||||
|
const lateFlush = deferred();
|
||||||
|
const firstSave = deferred();
|
||||||
|
const work = vi.fn(() => firstSave.promise);
|
||||||
|
const first = pending.enqueue("plugin", work, flush.promise);
|
||||||
|
const coalesced = pending.enqueue("plugin", work, lateFlush.promise);
|
||||||
|
flush.resolve();
|
||||||
|
await vi.advanceTimersByTimeAsync(0);
|
||||||
|
expect(work).toHaveBeenCalledOnce();
|
||||||
|
const nextWork = vi.fn(async () => {});
|
||||||
|
const later = pending.enqueue("plugin", nextWork);
|
||||||
|
lateFlush.resolve();
|
||||||
|
await vi.advanceTimersByTimeAsync(0);
|
||||||
|
expect(nextWork).not.toHaveBeenCalled();
|
||||||
|
firstSave.resolve();
|
||||||
|
await Promise.all([first, coalesced]);
|
||||||
|
expect(nextWork).not.toHaveBeenCalled();
|
||||||
|
await vi.advanceTimersByTimeAsync(100);
|
||||||
|
await later;
|
||||||
|
expect(nextWork).toHaveBeenCalledOnce();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects every coalesced notification with the original failure and permits a later retry", async () => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
const pending = new PendingConfigSaves();
|
||||||
|
const failure = new Error("database write failed");
|
||||||
|
const save = async () => {
|
||||||
|
throw failure;
|
||||||
|
};
|
||||||
|
const first = pending.enqueue("plugin", save);
|
||||||
|
const second = pending.enqueue("plugin", save);
|
||||||
|
const outcomes = Promise.allSettled([first, second]);
|
||||||
|
await vi.advanceTimersByTimeAsync(100);
|
||||||
|
expect(await outcomes).toEqual([
|
||||||
|
{ status: "rejected", reason: failure },
|
||||||
|
{ status: "rejected", reason: failure },
|
||||||
|
]);
|
||||||
|
const retry = pending.enqueue("plugin", async () => {}, Promise.resolve());
|
||||||
|
await retry;
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -20,10 +20,14 @@
|
|||||||
MODE_SHINY,
|
MODE_SHINY,
|
||||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
import { normalizePath } from "@/deps";
|
import { normalizePath } from "@/deps";
|
||||||
import { HiddenFileSync } from "@/features/HiddenFileSync/CmdHiddenFileSync.ts";
|
import {
|
||||||
|
HiddenFileSync,
|
||||||
|
HIDDEN_FILE_SYNC_DISABLED_NOTICE,
|
||||||
|
} from "@/features/HiddenFileSync/CmdHiddenFileSync.ts";
|
||||||
import { LOG_LEVEL_NOTICE, Logger } from "octagonal-wheels/common/logger";
|
import { LOG_LEVEL_NOTICE, Logger } from "octagonal-wheels/common/logger";
|
||||||
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore.ts";
|
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore.ts";
|
||||||
import { $msg as translateMessage } from "@/common/translation";
|
import { $msg as translateMessage } from "@/common/translation";
|
||||||
|
import { getAutomaticSyncTargetFiles } from "./getAutomaticSyncTargetFiles.ts";
|
||||||
import {
|
import {
|
||||||
REPLICATION_PROGRESS_PRESENTATIONS,
|
REPLICATION_PROGRESS_PRESENTATIONS,
|
||||||
USER_INITIATED_REPLICATION_AUTHORITY,
|
USER_INITIATED_REPLICATION_AUTHORITY,
|
||||||
@@ -139,6 +143,11 @@
|
|||||||
async function deleteData(data: IPluginDataExDisplay): Promise<boolean> {
|
async function deleteData(data: IPluginDataExDisplay): Promise<boolean> {
|
||||||
return await addOn.deleteData(data);
|
return await addOn.deleteData(data);
|
||||||
}
|
}
|
||||||
|
function checkAutomaticSyncAvailable() {
|
||||||
|
if (addOnHiddenFileSync.isThisModuleEnabled()) return true;
|
||||||
|
Logger(translateMessage(HIDDEN_FILE_SYNC_DISABLED_NOTICE), LOG_LEVEL_NOTICE);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
function askMode(evt: MouseEvent, title: string, key: string) {
|
function askMode(evt: MouseEvent, title: string, key: string) {
|
||||||
const menu = new Menu();
|
const menu = new Menu();
|
||||||
menu.addItem((item) => item.setTitle(title).setIsLabel(true));
|
menu.addItem((item) => item.setTitle(title).setIsLabel(true));
|
||||||
@@ -149,6 +158,7 @@
|
|||||||
item.setTitle(`${getIcon(mode as SYNC_MODE)}:${TITLES[mode]}`)
|
item.setTitle(`${getIcon(mode as SYNC_MODE)}:${TITLES[mode]}`)
|
||||||
.onClick((e) => {
|
.onClick((e) => {
|
||||||
if (mode === MODE_AUTOMATIC) {
|
if (mode === MODE_AUTOMATIC) {
|
||||||
|
if (!checkAutomaticSyncAvailable()) return;
|
||||||
askOverwriteModeForAutomatic(evt, key);
|
askOverwriteModeForAutomatic(evt, key);
|
||||||
} else {
|
} else {
|
||||||
setMode(key, mode as SYNC_MODE);
|
setMode(key, mode as SYNC_MODE);
|
||||||
@@ -161,10 +171,11 @@
|
|||||||
menu.showAtMouseEvent(evt);
|
menu.showAtMouseEvent(evt);
|
||||||
}
|
}
|
||||||
function applyAutomaticSync(key: string, direction: "pushForce" | "pullForce" | "safe") {
|
function applyAutomaticSync(key: string, direction: "pushForce" | "pullForce" | "safe") {
|
||||||
|
if (!checkAutomaticSyncAvailable()) return;
|
||||||
setMode(key, MODE_AUTOMATIC);
|
setMode(key, MODE_AUTOMATIC);
|
||||||
const configDir = normalizePath(plugin.core.services.API.getSystemConfigDir());
|
const configDir = normalizePath(plugin.core.services.API.getSystemConfigDir());
|
||||||
const files = (plugin.core.settings.pluginSyncExtendedSetting[key]?.files ?? []).map((e) => `${configDir}/${e}`);
|
const files = getAutomaticSyncTargetFiles(key, configDir, plugin.core.settings.pluginSyncExtendedSetting);
|
||||||
addOnHiddenFileSync.initialiseInternalFileSync(direction, true, files);
|
void addOnHiddenFileSync.initialiseInternalFileSync(direction, true, files);
|
||||||
}
|
}
|
||||||
function askOverwriteModeForAutomatic(evt: MouseEvent, key: string) {
|
function askOverwriteModeForAutomatic(evt: MouseEvent, key: string) {
|
||||||
const menu = new Menu();
|
const menu = new Menu();
|
||||||
|
|||||||
@@ -0,0 +1,13 @@
|
|||||||
|
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
|
|
||||||
|
export function getAutomaticSyncTargetFiles(
|
||||||
|
key: string,
|
||||||
|
configDir: string,
|
||||||
|
entries: ObsidianLiveSyncSettings["pluginSyncExtendedSetting"]
|
||||||
|
): string[] {
|
||||||
|
const keys = key.startsWith("PLUGIN_ALL/")
|
||||||
|
? [`PLUGIN_DATA/${key.substring("PLUGIN_ALL/".length)}`, `PLUGIN_MAIN/${key.substring("PLUGIN_ALL/".length)}`]
|
||||||
|
: [key];
|
||||||
|
const files = new Set(keys.flatMap((entryKey) => entries[entryKey]?.files ?? []));
|
||||||
|
return [...files].map((file) => `${configDir}/${file}`);
|
||||||
|
}
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { MODE_AUTOMATIC, type ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
|
import { getAutomaticSyncTargetFiles } from "./getAutomaticSyncTargetFiles.ts";
|
||||||
|
|
||||||
|
function createEntries(): ObsidianLiveSyncSettings["pluginSyncExtendedSetting"] {
|
||||||
|
return {
|
||||||
|
"CONFIG/app.json": { key: "CONFIG/app.json", mode: MODE_AUTOMATIC, files: ["app.json"] },
|
||||||
|
"PLUGIN_MAIN/example": {
|
||||||
|
key: "PLUGIN_MAIN/example",
|
||||||
|
mode: MODE_AUTOMATIC,
|
||||||
|
files: ["plugins/example/main.js", "plugins/example/manifest.json"],
|
||||||
|
},
|
||||||
|
"PLUGIN_DATA/example": {
|
||||||
|
key: "PLUGIN_DATA/example",
|
||||||
|
mode: MODE_AUTOMATIC,
|
||||||
|
files: ["plugins/example/data.json"],
|
||||||
|
},
|
||||||
|
"PLUGIN_ETC/example": {
|
||||||
|
key: "PLUGIN_ETC/example",
|
||||||
|
mode: MODE_AUTOMATIC,
|
||||||
|
files: ["plugins/example/extra.json"],
|
||||||
|
},
|
||||||
|
"PLUGIN_MAIN/other": {
|
||||||
|
key: "PLUGIN_MAIN/other",
|
||||||
|
mode: MODE_AUTOMATIC,
|
||||||
|
files: ["plugins/other/main.js"],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("getAutomaticSyncTargetFiles", () => {
|
||||||
|
it("initialises a single selected item in the current configuration directory", () => {
|
||||||
|
expect(getAutomaticSyncTargetFiles("CONFIG/app.json", ".custom-config", createEntries())).toEqual([
|
||||||
|
".custom-config/app.json",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("initialises both saved child selections when the whole plug-in is selected", () => {
|
||||||
|
expect(getAutomaticSyncTargetFiles("PLUGIN_ALL/example", ".obsidian", createEntries())).toEqual([
|
||||||
|
".obsidian/plugins/example/data.json",
|
||||||
|
".obsidian/plugins/example/main.js",
|
||||||
|
".obsidian/plugins/example/manifest.json",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("retains the available child files when the other child has no saved entry", () => {
|
||||||
|
const entries = createEntries();
|
||||||
|
delete entries["PLUGIN_DATA/example"];
|
||||||
|
expect(getAutomaticSyncTargetFiles("PLUGIN_ALL/example", ".obsidian", entries)).toEqual([
|
||||||
|
".obsidian/plugins/example/main.js",
|
||||||
|
".obsidian/plugins/example/manifest.json",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps an empty selection empty rather than including unrelated plug-ins", () => {
|
||||||
|
expect(getAutomaticSyncTargetFiles("PLUGIN_ALL/absent", ".obsidian", createEntries())).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,260 @@
|
|||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
|
||||||
|
|
||||||
|
vi.mock("@/deps.ts", () => ({}));
|
||||||
|
vi.mock("@/features/HiddenFileCommon/JsonResolveModal.ts", () => ({ JsonResolveModal: class JsonResolveModal {} }));
|
||||||
|
vi.mock("@/modules/features/ModuleLog.ts", () => ({ MARK_DONE: "" }));
|
||||||
|
vi.mock("./configureHiddenFileSyncMode.ts", () => ({ configureHiddenFileSyncMode: vi.fn() }));
|
||||||
|
|
||||||
|
import type { FilePath, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
|
import { readContent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||||
|
import { HiddenFileSync } from "./CmdHiddenFileSync";
|
||||||
|
import { createOptionalFileSyncDatabase, optionalFileEntry } from "../../../test/fixtures/optionalFileSyncDatabase";
|
||||||
|
|
||||||
|
const path = ".obsidian/plugins/example/main.js" as FilePath;
|
||||||
|
const content = "module.exports = class ExamplePlugin {};";
|
||||||
|
const fixtureOwners: Awaited<ReturnType<typeof createOptionalFileSyncDatabase>>[] = [];
|
||||||
|
|
||||||
|
async function getText(entry: Parameters<typeof readContent>[0]) {
|
||||||
|
const value = await readContent(entry);
|
||||||
|
return typeof value === "string" ? value : new TextDecoder().decode(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function createHiddenFileSync() {
|
||||||
|
const fixture = await createOptionalFileSyncDatabase();
|
||||||
|
fixtureOwners.push(fixture);
|
||||||
|
fixture.settings.usePluginSync = false;
|
||||||
|
fixture.settings.syncInternalFilesTargetPatterns = "" as typeof fixture.settings.syncInternalFilesTargetPatterns;
|
||||||
|
fixture.settings.syncInternalFilesIgnorePatterns = "" as typeof fixture.settings.syncInternalFilesIgnorePatterns;
|
||||||
|
let fileContent = content;
|
||||||
|
let stat: UXStat | null = {
|
||||||
|
ctime: 1_000,
|
||||||
|
mtime: 2_000,
|
||||||
|
size: fileContent.length,
|
||||||
|
type: "file",
|
||||||
|
};
|
||||||
|
const values = new Map<string, unknown>();
|
||||||
|
const kvDB = {
|
||||||
|
get: vi.fn(async <T>(key: string) => values.get(key) as T | undefined),
|
||||||
|
set: vi.fn(async <T>(key: string, value: T) => {
|
||||||
|
values.set(key, value);
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
const storageAccess = {
|
||||||
|
statHidden: vi.fn(async () => stat),
|
||||||
|
readHiddenFileAuto: vi.fn(async () => fileContent),
|
||||||
|
writeHiddenFileAuto: vi.fn(async () => undefined),
|
||||||
|
removeHidden: vi.fn(async () => true),
|
||||||
|
isExistsIncludeHidden: vi.fn(async () => true),
|
||||||
|
ensureDir: vi.fn(async () => undefined),
|
||||||
|
triggerHiddenFile: vi.fn(async () => undefined),
|
||||||
|
setFile(nextContent: string | null, mtime = 2_000) {
|
||||||
|
fileContent = nextContent ?? "";
|
||||||
|
stat = nextContent === null ? null : { ctime: 1_000, mtime, size: fileContent.length, type: "file" };
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const databaseService = { isDatabaseReady: vi.fn(() => true) };
|
||||||
|
const appLifecycle = { isReady: vi.fn(() => false), isSuspended: vi.fn(() => true) };
|
||||||
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
|
Object.assign(hiddenFileSync, {
|
||||||
|
core: {
|
||||||
|
settings: fixture.settings,
|
||||||
|
localDatabase: fixture.localDatabase,
|
||||||
|
kvDB,
|
||||||
|
storageAccess,
|
||||||
|
services: {
|
||||||
|
...fixture.services,
|
||||||
|
database: databaseService,
|
||||||
|
appLifecycle,
|
||||||
|
path: {
|
||||||
|
...fixture.services.path,
|
||||||
|
markChangesAreSame: vi.fn(),
|
||||||
|
unmarkChanges: vi.fn(),
|
||||||
|
},
|
||||||
|
vault: { isIgnoredByIgnoreFile: vi.fn(async () => false) },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
cacheFileRegExps: new Map(),
|
||||||
|
cacheCustomisationSyncIgnoredFiles: new Map(),
|
||||||
|
_fileInfoLastProcessed: new Map(),
|
||||||
|
_databaseInfoLastProcessed: new Map(),
|
||||||
|
_fileInfoLastKnown: new Map(),
|
||||||
|
semaphore: Semaphore(10),
|
||||||
|
_log: vi.fn(),
|
||||||
|
});
|
||||||
|
return { fixture, hiddenFileSync, storageAccess, kvDB, databaseService, appLifecycle };
|
||||||
|
}
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
vi.restoreAllMocks();
|
||||||
|
for (const fixture of fixtureOwners.splice(0)) await fixture.dispose();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Hidden File Sync optional save outcomes", () => {
|
||||||
|
it("retries a failed database write when the storage fingerprint is unchanged", async () => {
|
||||||
|
const { fixture, hiddenFileSync } = await createHiddenFileSync();
|
||||||
|
await expect(hiddenFileSync.isTargetFile(path)).resolves.toBe(true);
|
||||||
|
vi.spyOn(fixture.localDatabase, "putDBEntry").mockRejectedValueOnce(new Error("temporary database failure"));
|
||||||
|
|
||||||
|
await expect(hiddenFileSync.trackStorageFileModification(path)).resolves.toBe(false);
|
||||||
|
await expect(hiddenFileSync.trackStorageFileModification(path)).resolves.toBe(true);
|
||||||
|
|
||||||
|
expect(fixture.localDatabase.putDBEntry).toHaveBeenCalledTimes(2);
|
||||||
|
const saved = await fixture.localDatabase.getDBEntry(`i:${path}` as FilePath);
|
||||||
|
expect(saved).not.toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects a failed strict write and retries the same fingerprint without delaying readiness", async () => {
|
||||||
|
const { fixture, hiddenFileSync, appLifecycle } = await createHiddenFileSync();
|
||||||
|
const putDBEntry = vi
|
||||||
|
.spyOn(fixture.localDatabase, "putDBEntry")
|
||||||
|
.mockRejectedValueOnce(new Error("temporary database failure"));
|
||||||
|
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).rejects.toThrow(
|
||||||
|
"temporary database failure"
|
||||||
|
);
|
||||||
|
expect(hiddenFileSync.getLastProcessedFileKey(path)).toBeUndefined();
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
hiddenFileSync._processOptionalFileEventWithResult(path, {
|
||||||
|
whenFlushRequested: new Promise<void>(() => undefined),
|
||||||
|
})
|
||||||
|
).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
expect(putDBEntry).toHaveBeenCalledTimes(2);
|
||||||
|
const saved = await fixture.localDatabase.getDBEntry(`i:${path}` as FilePath);
|
||||||
|
expect(saved).not.toBe(false);
|
||||||
|
if (saved) expect(await getText(saved)).toBe(content);
|
||||||
|
expect(appLifecycle.isReady()).toBe(false);
|
||||||
|
expect(appLifecycle.isSuspended()).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects a false database-write result and can store a later edit after a same-content event", async () => {
|
||||||
|
const { fixture, hiddenFileSync, storageAccess } = await createHiddenFileSync();
|
||||||
|
const putDBEntry = vi
|
||||||
|
.spyOn(fixture.localDatabase, "putDBEntry")
|
||||||
|
.mockImplementationOnce(async () => false as never);
|
||||||
|
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).rejects.toThrow(
|
||||||
|
`Failed to store hidden file in the database: ${path}`
|
||||||
|
);
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
const first = await fixture.localDatabase.getDBEntry(`i:${path}` as FilePath);
|
||||||
|
expect(first).not.toBe(false);
|
||||||
|
if (first) {
|
||||||
|
const firstRevision = first._rev;
|
||||||
|
hiddenFileSync._fileInfoLastProcessed.delete(path);
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
const unchanged = await fixture.localDatabase.getDBEntry(`i:${path}` as FilePath);
|
||||||
|
expect(unchanged).not.toBe(false);
|
||||||
|
if (unchanged) expect(unchanged._rev).toBe(firstRevision);
|
||||||
|
}
|
||||||
|
|
||||||
|
storageAccess.setFile("module.exports = class UpdatedExamplePlugin {};", 3_000);
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
const updated = await fixture.localDatabase.getDBEntry(`i:${path}` as FilePath);
|
||||||
|
expect(updated).not.toBe(false);
|
||||||
|
if (updated) expect(await getText(updated)).toBe("module.exports = class UpdatedExamplePlugin {};");
|
||||||
|
expect(putDBEntry).toHaveBeenCalledTimes(3);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("vetoes when the physical database is unavailable and ignores disabled or filtered events", async () => {
|
||||||
|
const { fixture, hiddenFileSync, databaseService } = await createHiddenFileSync();
|
||||||
|
|
||||||
|
databaseService.isDatabaseReady.mockReturnValue(false);
|
||||||
|
await expect(
|
||||||
|
hiddenFileSync._processOptionalFileEventWithResult(path, {
|
||||||
|
whenFlushRequested: new Promise<void>(() => undefined),
|
||||||
|
})
|
||||||
|
).resolves.toMatchObject({ kind: "veto", value: expect.any(String) });
|
||||||
|
databaseService.isDatabaseReady.mockReturnValue(true);
|
||||||
|
|
||||||
|
fixture.settings.syncInternalFiles = false;
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "not-applicable",
|
||||||
|
});
|
||||||
|
fixture.settings.syncInternalFiles = true;
|
||||||
|
fixture.settings.syncInternalFilesTargetPatterns =
|
||||||
|
"^\\.obsidian/snippets/" as typeof fixture.settings.syncInternalFilesTargetPatterns;
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "not-applicable",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("treats an absent replayed deletion as complete and leaves cache initialisation intact", async () => {
|
||||||
|
const { hiddenFileSync, storageAccess, kvDB } = await createHiddenFileSync();
|
||||||
|
storageAccess.setFile(null);
|
||||||
|
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
expect(hiddenFileSync.getLastProcessedFileKey(path)).toBe("0-0");
|
||||||
|
const lifecycle = hiddenFileSync as unknown as {
|
||||||
|
_everyOnDatabaseInitialized(showNotice: boolean): Promise<boolean>;
|
||||||
|
};
|
||||||
|
hiddenFileSync.core.settings.syncInternalFiles = false;
|
||||||
|
await lifecycle._everyOnDatabaseInitialized(false);
|
||||||
|
expect(kvDB.get).toHaveBeenCalledTimes(3);
|
||||||
|
expect(hiddenFileSync.getLastProcessedFileKey(path)).toBe("0-0");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects actual deletion failures and completes duplicate deletions without another write", async () => {
|
||||||
|
const { fixture, hiddenFileSync, storageAccess } = await createHiddenFileSync();
|
||||||
|
await fixture.localDatabase.putDBEntry(optionalFileEntry(`i:${path}`, content));
|
||||||
|
storageAccess.setFile(null);
|
||||||
|
const putRaw = vi
|
||||||
|
.spyOn(fixture.localDatabase, "putRaw")
|
||||||
|
.mockRejectedValueOnce(new Error("temporary deletion failure"));
|
||||||
|
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).rejects.toThrow(
|
||||||
|
"temporary deletion failure"
|
||||||
|
);
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
hiddenFileSync.resetLastProcessedFile([path]);
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
expect(putRaw).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("recreates an empty hidden file when its database entry is logically deleted", async () => {
|
||||||
|
const { fixture, hiddenFileSync, storageAccess } = await createHiddenFileSync();
|
||||||
|
await fixture.localDatabase.putDBEntry(optionalFileEntry(`i:${path}`, ""));
|
||||||
|
await expect(hiddenFileSync.deleteInternalFileOnDatabase(path)).resolves.toBe(true);
|
||||||
|
const deletedBase = await fixture.localDatabase.getDBEntry(
|
||||||
|
`i:${path}` as FilePath,
|
||||||
|
undefined,
|
||||||
|
false,
|
||||||
|
true,
|
||||||
|
true
|
||||||
|
);
|
||||||
|
expect(deletedBase).not.toBe(false);
|
||||||
|
if (deletedBase) expect(deletedBase.deleted).toBe(true);
|
||||||
|
if (!deletedBase) throw new Error("The logically deleted test entry could not be loaded");
|
||||||
|
vi.spyOn(fixture.localDatabase, "getDBEntry").mockResolvedValueOnce(deletedBase);
|
||||||
|
|
||||||
|
storageAccess.setFile("", 4_000);
|
||||||
|
const putDBEntry = vi.spyOn(fixture.localDatabase, "putDBEntry");
|
||||||
|
|
||||||
|
await expect(hiddenFileSync._processOptionalFileEventWithResult(path)).resolves.toMatchObject({
|
||||||
|
kind: "completed",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(putDBEntry).toHaveBeenCalledTimes(1);
|
||||||
|
const recreated = await fixture.localDatabase.getDBEntry(`i:${path}` as FilePath);
|
||||||
|
expect(recreated).not.toBe(false);
|
||||||
|
if (recreated) {
|
||||||
|
expect(recreated.deleted).not.toBe(true);
|
||||||
|
expect(await getText(recreated)).toBe("");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -54,6 +54,8 @@ import { EVENT_SETTING_SAVED, eventHub } from "@/common/events.ts";
|
|||||||
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
|
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
|
||||||
import type { LiveSyncCore } from "@/main.ts";
|
import type { LiveSyncCore } from "@/main.ts";
|
||||||
import { tryGetFilePath } from "@vrtmrz/livesync-commonlib/compat/common/utils.doc";
|
import { tryGetFilePath } from "@vrtmrz/livesync-commonlib/compat/common/utils.doc";
|
||||||
|
import type { HandlerOutcome } from "@vrtmrz/livesync-commonlib/compat/services/lib/HandlerUtils";
|
||||||
|
import type { OptionalFileEventOptions } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
||||||
import { configureHiddenFileSyncMode, type ConfigureHiddenFileSyncResult } from "./configureHiddenFileSyncMode.ts";
|
import { configureHiddenFileSyncMode, type ConfigureHiddenFileSyncResult } from "./configureHiddenFileSyncMode.ts";
|
||||||
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
|
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
|
||||||
import { getObsidianCommunityPluginManager } from "@/common/obsidianCommunityPlugins.ts";
|
import { getObsidianCommunityPluginManager } from "@/common/obsidianCommunityPlugins.ts";
|
||||||
@@ -69,6 +71,9 @@ type HiddenFileInitialisationProgress = {
|
|||||||
const HIDDEN_FILE_NOTICE_GROUP = "hidden-file-changes";
|
const HIDDEN_FILE_NOTICE_GROUP = "hidden-file-changes";
|
||||||
const HIDDEN_FILE_NOTICE_DURATION_MS = 20_000;
|
const HIDDEN_FILE_NOTICE_DURATION_MS = 20_000;
|
||||||
|
|
||||||
|
export const HIDDEN_FILE_SYNC_DISABLED_NOTICE =
|
||||||
|
"Hidden File Sync is disabled on this device. Enable it in Sync Settings before using Automatic or initialising hidden files.";
|
||||||
|
|
||||||
function getComparingMTime(
|
function getComparingMTime(
|
||||||
doc: (MetaEntry | LoadedEntry | false) | UXFileInfo | UXStat | null | undefined,
|
doc: (MetaEntry | LoadedEntry | false) | UXFileInfo | UXStat | null | undefined,
|
||||||
includeDeleted = false
|
includeDeleted = false
|
||||||
@@ -161,9 +166,7 @@ export class HiddenFileSync extends LiveSyncCommands {
|
|||||||
// return true;
|
// return true;
|
||||||
// }
|
// }
|
||||||
private async _everyOnDatabaseInitialized(showNotice: boolean) {
|
private async _everyOnDatabaseInitialized(showNotice: boolean) {
|
||||||
this._fileInfoLastProcessed = await autosaveCache(this.kvDB, "hidden-file-lastProcessed");
|
await this.ensureProcessedCaches();
|
||||||
this._databaseInfoLastProcessed = await autosaveCache(this.kvDB, "hidden-file-lastProcessed-database");
|
|
||||||
this._fileInfoLastKnown = await autosaveCache(this.kvDB, "hidden-file-lastKnown");
|
|
||||||
if (this.isThisModuleEnabled()) {
|
if (this.isThisModuleEnabled()) {
|
||||||
if (this._fileInfoLastProcessed.size == 0 && this._fileInfoLastProcessed.size == 0) {
|
if (this._fileInfoLastProcessed.size == 0 && this._fileInfoLastProcessed.size == 0) {
|
||||||
this._log(`No cache found. Performing startup scan.`, LOG_LEVEL_VERBOSE);
|
this._log(`No cache found. Performing startup scan.`, LOG_LEVEL_VERBOSE);
|
||||||
@@ -249,6 +252,26 @@ export class HiddenFileSync extends LiveSyncCommands {
|
|||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async _processOptionalFileEventWithResult(
|
||||||
|
path: FilePath,
|
||||||
|
_options?: OptionalFileEventOptions
|
||||||
|
): Promise<HandlerOutcome<void, string>> {
|
||||||
|
if (!this.isThisModuleEnabled()) return { kind: "not-applicable" };
|
||||||
|
if (!(await this.isTargetFile(path))) return { kind: "not-applicable" };
|
||||||
|
if (!this._isDatabaseReady()) {
|
||||||
|
return { kind: "veto", value: "The local database is not ready" };
|
||||||
|
}
|
||||||
|
|
||||||
|
await this.ensureProcessedCaches();
|
||||||
|
if (!this.isThisModuleEnabled() || !(await this.isTargetFile(path))) {
|
||||||
|
return { kind: "not-applicable" };
|
||||||
|
}
|
||||||
|
if (!(await this.trackStorageFileModificationStrict(path))) {
|
||||||
|
return { kind: "not-applicable" };
|
||||||
|
}
|
||||||
|
return { kind: "completed" };
|
||||||
|
}
|
||||||
|
|
||||||
_anyGetOptionalConflictCheckMethod(path: FilePathWithPrefix): Promise<boolean | "newer"> {
|
_anyGetOptionalConflictCheckMethod(path: FilePathWithPrefix): Promise<boolean | "newer"> {
|
||||||
if (isInternalMetadata(path)) {
|
if (isInternalMetadata(path)) {
|
||||||
this.queueConflictCheck(path);
|
this.queueConflictCheck(path);
|
||||||
@@ -317,6 +340,51 @@ export class HiddenFileSync extends LiveSyncCommands {
|
|||||||
_fileInfoLastKnown!: MapLike<string, number>;
|
_fileInfoLastKnown!: MapLike<string, number>;
|
||||||
_databaseInfoLastProcessed!: MapLike<string, string>;
|
_databaseInfoLastProcessed!: MapLike<string, string>;
|
||||||
|
|
||||||
|
private processedCacheStore: unknown;
|
||||||
|
private processedCacheStoreReady = false;
|
||||||
|
private processedCacheLoadStore: unknown;
|
||||||
|
private processedCacheLoad?: Promise<void>;
|
||||||
|
|
||||||
|
private async ensureProcessedCaches() {
|
||||||
|
const store = this.kvDB;
|
||||||
|
if (!store) throw new Error("The Hidden File Sync cache store is unavailable");
|
||||||
|
if (this.processedCacheStoreReady && this.processedCacheStore === store) return;
|
||||||
|
if (this.processedCacheLoad && this.processedCacheLoadStore === store) {
|
||||||
|
await this.processedCacheLoad;
|
||||||
|
if (!this.processedCacheStoreReady || this.processedCacheStore !== this.kvDB) {
|
||||||
|
throw new Error("The Hidden File Sync cache store changed while loading its processing state");
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
let load!: Promise<void>;
|
||||||
|
load = Promise.all([
|
||||||
|
autosaveCache<string, string>(store, "hidden-file-lastProcessed"),
|
||||||
|
autosaveCache<string, string>(store, "hidden-file-lastProcessed-database"),
|
||||||
|
autosaveCache<string, number>(store, "hidden-file-lastKnown"),
|
||||||
|
])
|
||||||
|
.then(([fileInfoLastProcessed, databaseInfoLastProcessed, fileInfoLastKnown]) => {
|
||||||
|
if (this.processedCacheLoad !== load || this.kvDB !== store) return;
|
||||||
|
this._fileInfoLastProcessed = fileInfoLastProcessed;
|
||||||
|
this._databaseInfoLastProcessed = databaseInfoLastProcessed;
|
||||||
|
this._fileInfoLastKnown = fileInfoLastKnown;
|
||||||
|
this.processedCacheStore = store;
|
||||||
|
this.processedCacheStoreReady = true;
|
||||||
|
})
|
||||||
|
.finally(() => {
|
||||||
|
if (this.processedCacheLoad === load) {
|
||||||
|
this.processedCacheLoad = undefined;
|
||||||
|
this.processedCacheLoadStore = undefined;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
this.processedCacheLoadStore = store;
|
||||||
|
this.processedCacheLoad = load;
|
||||||
|
await load;
|
||||||
|
if (!this.processedCacheStoreReady || this.processedCacheStore !== this.kvDB) {
|
||||||
|
throw new Error("The Hidden File Sync cache store changed while loading its processing state");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
statToKey(stat: UXStat | null) {
|
statToKey(stat: UXStat | null) {
|
||||||
return `${stat?.mtime ?? 0}-${stat?.size ?? 0}`;
|
return `${stat?.mtime ?? 0}-${stat?.size ?? 0}`;
|
||||||
}
|
}
|
||||||
@@ -600,7 +668,6 @@ Offline Changed files: ${processFiles.length}`;
|
|||||||
this._log(`Hidden file:${path} is changed.`, LOG_LEVEL_VERBOSE);
|
this._log(`Hidden file:${path} is changed.`, LOG_LEVEL_VERBOSE);
|
||||||
stat = cache.stat;
|
stat = cache.stat;
|
||||||
}
|
}
|
||||||
this.updateLastProcessedFile(path, stat!);
|
|
||||||
const lastIsNotFound = !lastKey || lastKey.endsWith("-0-0");
|
const lastIsNotFound = !lastKey || lastKey.endsWith("-0-0");
|
||||||
const nowIsNotFound = cache.deleted;
|
const nowIsNotFound = cache.deleted;
|
||||||
const type = lastIsNotFound && nowIsNotFound ? "invalid" : nowIsNotFound ? "delete" : "modified";
|
const type = lastIsNotFound && nowIsNotFound ? "invalid" : nowIsNotFound ? "delete" : "modified";
|
||||||
@@ -651,6 +718,27 @@ Offline Changed files: ${processFiles.length}`;
|
|||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private async trackStorageFileModificationStrict(path: FilePath): Promise<boolean> {
|
||||||
|
return await this.serializedForEvent(path, async () => {
|
||||||
|
if (!this.isThisModuleEnabled() || !(await this.isTargetFile(path))) return false;
|
||||||
|
const stat = await this.core.storageAccess.statHidden(path);
|
||||||
|
if (stat != null && stat.type != "file") return true;
|
||||||
|
const key = await this.fileToStatKey(path, stat);
|
||||||
|
if (this.getLastProcessedFileKey(path) === key) {
|
||||||
|
this._log(`${path} Already processed.`, LOG_LEVEL_DEBUG);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
const fileInfo = await this.loadFileWithInfo(path);
|
||||||
|
if (fileInfo.deleted) {
|
||||||
|
await this.deleteInternalFileOnDatabaseStrict(path);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
await this.storeInternalFileToDatabaseStrict(fileInfo);
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
// --> Conflict processing
|
// --> Conflict processing
|
||||||
|
|
||||||
// Keep one in-flight conflict check per path so repeated sync events do not close the active merge dialogue.
|
// Keep one in-flight conflict check per path so repeated sync events do not close the active merge dialogue.
|
||||||
@@ -1444,6 +1532,11 @@ Offline Changed files: ${files.length}`;
|
|||||||
initialisationProgress?: HiddenFileInitialisationProgress
|
initialisationProgress?: HiddenFileInitialisationProgress
|
||||||
) {
|
) {
|
||||||
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
|
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
|
||||||
|
if (!this.isThisModuleEnabled()) {
|
||||||
|
this._log($msg(HIDDEN_FILE_SYNC_DISABLED_NOTICE), logLevel);
|
||||||
|
initialisationProgress?.done($msg("Hidden File Sync initialisation was not started."));
|
||||||
|
return false;
|
||||||
|
}
|
||||||
const p = initialisationProgress ?? this._progress("[⚙ Initialise]\n", logLevel);
|
const p = initialisationProgress ?? this._progress("[⚙ Initialise]\n", logLevel);
|
||||||
// p.log("Resolving conflicts before starting...");
|
// p.log("Resolving conflicts before starting...");
|
||||||
// await this.resolveConflictOnInternalFiles();
|
// await this.resolveConflictOnInternalFiles();
|
||||||
@@ -1494,12 +1587,25 @@ Offline Changed files: ${files.length}`;
|
|||||||
await this.scanAllDatabaseChanges(showChildNotices, true, false);
|
await this.scanAllDatabaseChanges(showChildNotices, true, false);
|
||||||
}
|
}
|
||||||
p.done();
|
p.done();
|
||||||
|
return true;
|
||||||
}
|
}
|
||||||
// <-- Initialization functions
|
// <-- Initialization functions
|
||||||
|
|
||||||
// --> Storage To Database Functions
|
// --> Storage To Database Functions
|
||||||
|
|
||||||
async __loadBaseSaveData(file: FilePath, includeContent = true): Promise<LoadedEntry | false> {
|
async __loadBaseSaveData(file: FilePath, includeContent = true): Promise<LoadedEntry | false> {
|
||||||
|
return await this.loadBaseSaveData(file, includeContent, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadBaseSaveDataStrict(file: FilePath, includeContent = true): Promise<LoadedEntry | false> {
|
||||||
|
return await this.loadBaseSaveData(file, includeContent, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadBaseSaveData(
|
||||||
|
file: FilePath,
|
||||||
|
includeContent: boolean,
|
||||||
|
strict: boolean
|
||||||
|
): Promise<LoadedEntry | false> {
|
||||||
const prefixedFileName = addPrefix(file, ICHeader);
|
const prefixedFileName = addPrefix(file, ICHeader);
|
||||||
const id = await this.path2id(prefixedFileName, ICHeader);
|
const id = await this.path2id(prefixedFileName, ICHeader);
|
||||||
try {
|
try {
|
||||||
@@ -1525,6 +1631,7 @@ Offline Changed files: ${files.length}`;
|
|||||||
return old;
|
return old;
|
||||||
}
|
}
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
|
if (strict) throw ex;
|
||||||
this._log(`Getting base save data failed`);
|
this._log(`Getting base save data failed`);
|
||||||
this._log(ex, LOG_LEVEL_VERBOSE);
|
this._log(ex, LOG_LEVEL_VERBOSE);
|
||||||
return false;
|
return false;
|
||||||
@@ -1552,6 +1659,18 @@ Offline Changed files: ${files.length}`;
|
|||||||
}
|
}
|
||||||
|
|
||||||
async storeInternalFileToDatabase(file: InternalFileInfo | UXFileInfo, forceWrite = false) {
|
async storeInternalFileToDatabase(file: InternalFileInfo | UXFileInfo, forceWrite = false) {
|
||||||
|
return await this.storeInternalFileToDatabaseCore(file, forceWrite, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async storeInternalFileToDatabaseStrict(file: InternalFileInfo | UXFileInfo, forceWrite = false) {
|
||||||
|
return await this.storeInternalFileToDatabaseCore(file, forceWrite, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async storeInternalFileToDatabaseCore(
|
||||||
|
file: InternalFileInfo | UXFileInfo,
|
||||||
|
forceWrite: boolean,
|
||||||
|
strict: boolean
|
||||||
|
) {
|
||||||
const storeFilePath = stripAllPrefixes(file.path);
|
const storeFilePath = stripAllPrefixes(file.path);
|
||||||
const storageFilePath = file.path;
|
const storageFilePath = file.path;
|
||||||
if (await this.services.vault.isIgnoredByIgnoreFile(storageFilePath)) {
|
if (await this.services.vault.isIgnoredByIgnoreFile(storageFilePath)) {
|
||||||
@@ -1565,9 +1684,11 @@ Offline Changed files: ${files.length}`;
|
|||||||
if (fileInfo.deleted) {
|
if (fileInfo.deleted) {
|
||||||
throw new Error(`Hidden file:${storeFilePath} is deleted. This should not be occurred.`);
|
throw new Error(`Hidden file:${storeFilePath} is deleted. This should not be occurred.`);
|
||||||
}
|
}
|
||||||
const baseData = await this.__loadBaseSaveData(storeFilePath, true);
|
const baseData = strict
|
||||||
|
? await this.loadBaseSaveDataStrict(storeFilePath, true)
|
||||||
|
: await this.__loadBaseSaveData(storeFilePath, true);
|
||||||
if (baseData === false) throw new Error("Failed to load base data");
|
if (baseData === false) throw new Error("Failed to load base data");
|
||||||
if (baseData._rev && !forceWrite) {
|
if (baseData._rev && !baseData.deleted && !baseData._deleted && !forceWrite) {
|
||||||
// Not newly created, we should check the content is actually modified.
|
// Not newly created, we should check the content is actually modified.
|
||||||
const isSame = await isDocContentSame(readAsBlob(baseData), fileInfo.body);
|
const isSame = await isDocContentSame(readAsBlob(baseData), fileInfo.body);
|
||||||
if (isSame) {
|
if (isSame) {
|
||||||
@@ -1592,11 +1713,15 @@ Offline Changed files: ${files.length}`;
|
|||||||
this.updateLastProcessed(storeFilePath, saveData, fileInfo.stat);
|
this.updateLastProcessed(storeFilePath, saveData, fileInfo.stat);
|
||||||
}
|
}
|
||||||
const success = ret && ret.ok;
|
const success = ret && ret.ok;
|
||||||
|
if (!success && strict) {
|
||||||
|
throw new Error(`Failed to store hidden file in the database: ${storeFilePath}`);
|
||||||
|
}
|
||||||
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) ${success ? "Done" : "Failed"}`);
|
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) ${success ? "Done" : "Failed"}`);
|
||||||
return success;
|
return success;
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Failed`);
|
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Failed`);
|
||||||
this._log(ex, LOG_LEVEL_VERBOSE);
|
this._log(ex, LOG_LEVEL_VERBOSE);
|
||||||
|
if (strict) throw ex;
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -1676,6 +1801,14 @@ Offline Changed files: ${files.length}`;
|
|||||||
}
|
}
|
||||||
|
|
||||||
async deleteInternalFileOnDatabase(filenameSrc: FilePath, forceWrite = false) {
|
async deleteInternalFileOnDatabase(filenameSrc: FilePath, forceWrite = false) {
|
||||||
|
return await this.deleteInternalFileOnDatabaseCore(filenameSrc, forceWrite, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async deleteInternalFileOnDatabaseStrict(filenameSrc: FilePath, forceWrite = false) {
|
||||||
|
return await this.deleteInternalFileOnDatabaseCore(filenameSrc, forceWrite, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async deleteInternalFileOnDatabaseCore(filenameSrc: FilePath, _forceWrite: boolean, strict: boolean) {
|
||||||
const storeFilePath = filenameSrc;
|
const storeFilePath = filenameSrc;
|
||||||
const storageFilePath = filenameSrc;
|
const storageFilePath = filenameSrc;
|
||||||
const displayFileName = filenameSrc;
|
const displayFileName = filenameSrc;
|
||||||
@@ -1686,18 +1819,26 @@ Offline Changed files: ${files.length}`;
|
|||||||
}
|
}
|
||||||
return await serialized("file-" + prefixedFileName, async () => {
|
return await serialized("file-" + prefixedFileName, async () => {
|
||||||
try {
|
try {
|
||||||
const baseData = await this.__loadBaseSaveData(storeFilePath, false);
|
const baseData = strict
|
||||||
|
? await this.loadBaseSaveDataStrict(storeFilePath, false)
|
||||||
|
: await this.__loadBaseSaveData(storeFilePath, false);
|
||||||
if (baseData === false) throw new Error("Failed to load base data during deleting");
|
if (baseData === false) throw new Error("Failed to load base data during deleting");
|
||||||
if (baseData._conflicts !== undefined) {
|
if (baseData._conflicts !== undefined) {
|
||||||
for (const conflictRev of baseData._conflicts) {
|
for (const conflictRev of baseData._conflicts) {
|
||||||
await this.localDatabase.removeRevision(baseData._id, conflictRev);
|
const removed = await this.localDatabase.removeRevision(baseData._id, conflictRev);
|
||||||
|
if (strict && removed === false) {
|
||||||
|
throw new Error(`Failed to remove conflicted hidden-file revision ${conflictRev}`);
|
||||||
|
}
|
||||||
this._log(
|
this._log(
|
||||||
`STORAGE -x> DB: ${displayFileName}: (hidden) conflict removed ${baseData._rev} => ${conflictRev}`,
|
`STORAGE -x> DB: ${displayFileName}: (hidden) conflict removed ${baseData._rev} => ${conflictRev}`,
|
||||||
LOG_LEVEL_VERBOSE
|
LOG_LEVEL_VERBOSE
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (baseData.deleted) {
|
const alreadyDeleted = strict
|
||||||
|
? !baseData._rev || baseData.deleted || baseData._deleted
|
||||||
|
: baseData.deleted;
|
||||||
|
if (alreadyDeleted) {
|
||||||
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) already deleted`, LOG_LEVEL_VERBOSE);
|
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) already deleted`, LOG_LEVEL_VERBOSE);
|
||||||
this.updateLastProcessedDeletion(storeFilePath, baseData);
|
this.updateLastProcessedDeletion(storeFilePath, baseData);
|
||||||
return true;
|
return true;
|
||||||
@@ -1718,11 +1859,13 @@ Offline Changed files: ${files.length}`;
|
|||||||
return true;
|
return true;
|
||||||
} else {
|
} else {
|
||||||
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
|
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
|
||||||
|
if (strict) throw new Error(`Failed to delete hidden file from the database: ${storeFilePath}`);
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
} catch (ex) {
|
} catch (ex) {
|
||||||
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
|
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
|
||||||
this._log(ex, LOG_LEVEL_VERBOSE);
|
this._log(ex, LOG_LEVEL_VERBOSE);
|
||||||
|
if (strict) throw ex;
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -1925,6 +2068,7 @@ Offline Changed files: ${files.length}`;
|
|||||||
|
|
||||||
async configureHiddenFileSync(mode: OptionalSyncFeatureMode) {
|
async configureHiddenFileSync(mode: OptionalSyncFeatureMode) {
|
||||||
let initialisationProgress: HiddenFileInitialisationProgress | undefined;
|
let initialisationProgress: HiddenFileInitialisationProgress | undefined;
|
||||||
|
let initialised = false;
|
||||||
let result: ConfigureHiddenFileSyncResult;
|
let result: ConfigureHiddenFileSyncResult;
|
||||||
try {
|
try {
|
||||||
result = await configureHiddenFileSyncMode(mode, {
|
result = await configureHiddenFileSyncMode(mode, {
|
||||||
@@ -1954,7 +2098,7 @@ Offline Changed files: ${files.length}`;
|
|||||||
);
|
);
|
||||||
},
|
},
|
||||||
initialise: async (direction) => {
|
initialise: async (direction) => {
|
||||||
await this.initialiseInternalFileSync(direction, true, false, initialisationProgress);
|
initialised = await this.initialiseInternalFileSync(direction, true, false, initialisationProgress);
|
||||||
initialisationProgress = undefined;
|
initialisationProgress = undefined;
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1962,7 +2106,7 @@ Offline Changed files: ${files.length}`;
|
|||||||
initialisationProgress?.done("Failed");
|
initialisationProgress?.done("Failed");
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
if (result == "ignored" || result == "disabled") {
|
if (result == "ignored" || result == "disabled" || !initialised) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
// this.plugin.settings.useAdvancedMode = true;
|
// this.plugin.settings.useAdvancedMode = true;
|
||||||
@@ -2084,6 +2228,9 @@ Offline Changed files: ${files.length}`;
|
|||||||
// services.databaseEvents.handleOnDatabaseInitialisation(this._everyOnInitializeDatabase.bind(this));
|
// services.databaseEvents.handleOnDatabaseInitialisation(this._everyOnInitializeDatabase.bind(this));
|
||||||
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));
|
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));
|
||||||
services.fileProcessing.processOptionalFileEvent.addHandler(this._anyProcessOptionalFileEvent.bind(this));
|
services.fileProcessing.processOptionalFileEvent.addHandler(this._anyProcessOptionalFileEvent.bind(this));
|
||||||
|
services.fileProcessing.processOptionalFileEventWithResult?.addHandler(
|
||||||
|
this._processOptionalFileEventWithResult.bind(this)
|
||||||
|
);
|
||||||
services.conflict.getOptionalConflictCheckMethod.addHandler(this._anyGetOptionalConflictCheckMethod.bind(this));
|
services.conflict.getOptionalConflictCheckMethod.addHandler(this._anyGetOptionalConflictCheckMethod.bind(this));
|
||||||
services.replication.processOptionalSynchroniseResult.addHandler(this._anyProcessOptionalSyncFiles.bind(this));
|
services.replication.processOptionalSynchroniseResult.addHandler(this._anyProcessOptionalSyncFiles.bind(this));
|
||||||
services.setting.onRealiseSetting.addHandler(this._everyRealizeSettingSyncMode.bind(this));
|
services.setting.onRealiseSetting.addHandler(this._everyRealizeSettingSyncMode.bind(this));
|
||||||
|
|||||||
@@ -99,6 +99,115 @@ function createHiddenRevisionOperation() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
describe("HiddenFileSync configuration-change notices", () => {
|
describe("HiddenFileSync configuration-change notices", () => {
|
||||||
|
it.each(["pushForce", "pullForce", "safe"] as const)(
|
||||||
|
"refuses %s initialisation before processing files while Hidden File Sync is disabled",
|
||||||
|
async (direction) => {
|
||||||
|
const progress = { log: vi.fn(), once: vi.fn(), done: vi.fn() };
|
||||||
|
const path = ".obsidian/example.json" as FilePath;
|
||||||
|
const operations = {
|
||||||
|
rebuildFromStorage: vi.fn(async () => [path]),
|
||||||
|
rebuildFromDatabase: vi.fn(async () => []),
|
||||||
|
rebuildMerging: vi.fn(async () => [path]),
|
||||||
|
adoptCurrentStorageFilesAsProcessed: vi.fn(async () => undefined),
|
||||||
|
adoptCurrentDatabaseFilesAsProcessed: vi.fn(async () => undefined),
|
||||||
|
scanAllStorageChanges: vi.fn(async () => undefined),
|
||||||
|
scanAllDatabaseChanges: vi.fn(async () => undefined),
|
||||||
|
};
|
||||||
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
|
const settings = { syncInternalFiles: false };
|
||||||
|
const createProgress = vi.fn(() => progress);
|
||||||
|
const log = vi.fn();
|
||||||
|
Object.assign(hiddenFileSync, {
|
||||||
|
core: { settings },
|
||||||
|
_progress: createProgress,
|
||||||
|
_log: log,
|
||||||
|
...operations,
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = await hiddenFileSync.initialiseInternalFileSync(direction, true, [path]);
|
||||||
|
|
||||||
|
for (const operation of Object.values(operations)) {
|
||||||
|
expect(operation).not.toHaveBeenCalled();
|
||||||
|
}
|
||||||
|
expect(result).toBe(false);
|
||||||
|
expect(settings.syncInternalFiles).toBe(false);
|
||||||
|
expect(createProgress).not.toHaveBeenCalled();
|
||||||
|
expect(log).toHaveBeenCalledWith(expect.stringContaining("Hidden File Sync is disabled"), LOG_LEVEL_NOTICE);
|
||||||
|
expect(progress.done).not.toHaveBeenCalled();
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it("finishes an existing preparation Notice as not started when initialisation is disabled", async () => {
|
||||||
|
const progress = { log: vi.fn(), once: vi.fn(), done: vi.fn() };
|
||||||
|
const createProgress = vi.fn();
|
||||||
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
|
Object.assign(hiddenFileSync, {
|
||||||
|
core: { settings: { syncInternalFiles: false } },
|
||||||
|
_progress: createProgress,
|
||||||
|
_log: vi.fn(),
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = await hiddenFileSync.initialiseInternalFileSync("safe", true, false, progress);
|
||||||
|
|
||||||
|
expect(result).toBe(false);
|
||||||
|
expect(createProgress).not.toHaveBeenCalled();
|
||||||
|
expect(progress.done).toHaveBeenCalledExactlyOnceWith("Hidden File Sync initialisation was not started.");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not apply a stored deletion when initialisation is requested while disabled", async () => {
|
||||||
|
const path = ".obsidian/example.json" as FilePath;
|
||||||
|
const deleted: MetaEntry = {
|
||||||
|
_id: "i:example" as DocumentID,
|
||||||
|
_rev: "2-deleted",
|
||||||
|
path: `i:${path}` as FilePathWithPrefix,
|
||||||
|
type: "plain",
|
||||||
|
ctime: 1,
|
||||||
|
mtime: 10,
|
||||||
|
size: 0,
|
||||||
|
deleted: true,
|
||||||
|
children: [],
|
||||||
|
eden: {},
|
||||||
|
};
|
||||||
|
const progress = { log: vi.fn(), once: vi.fn(), done: vi.fn() };
|
||||||
|
const deleteFile = vi.fn(async () => "OK" as const);
|
||||||
|
const updateDeletion = vi.fn();
|
||||||
|
const getFiles = vi.fn(async () => [deleted]);
|
||||||
|
const processedStorage = new Map([[path, "storage-processed"]]);
|
||||||
|
const processedDatabase = new Map([[path, "database-processed"]]);
|
||||||
|
const resetProcessed = vi.fn(() => processedDatabase.clear());
|
||||||
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
|
Object.assign(hiddenFileSync, {
|
||||||
|
core: {
|
||||||
|
settings: { syncInternalFiles: false },
|
||||||
|
services: { vault: { isIgnoredByIgnoreFile: vi.fn(async () => false) } },
|
||||||
|
},
|
||||||
|
_progress: vi.fn(() => progress),
|
||||||
|
_log: vi.fn(),
|
||||||
|
_verbose: vi.fn(),
|
||||||
|
_fileInfoLastProcessed: processedStorage,
|
||||||
|
_databaseInfoLastProcessed: processedDatabase,
|
||||||
|
getAllDatabaseFiles: getFiles,
|
||||||
|
getPath: (doc: MetaEntry) => doc.path,
|
||||||
|
resetLastProcessedDatabase: resetProcessed,
|
||||||
|
serializedForEvent: async (_path: unknown, operation: () => Promise<unknown>) => await operation(),
|
||||||
|
__deleteFile: deleteFile,
|
||||||
|
updateLastProcessedDeletion: updateDeletion,
|
||||||
|
adoptCurrentStorageFilesAsProcessed: vi.fn(),
|
||||||
|
adoptCurrentDatabaseFilesAsProcessed: vi.fn(),
|
||||||
|
scanAllStorageChanges: vi.fn(),
|
||||||
|
scanAllDatabaseChanges: vi.fn(),
|
||||||
|
});
|
||||||
|
|
||||||
|
await hiddenFileSync.initialiseInternalFileSync("pullForce", true);
|
||||||
|
|
||||||
|
expect(deleteFile).not.toHaveBeenCalled();
|
||||||
|
expect(updateDeletion).not.toHaveBeenCalled();
|
||||||
|
expect(getFiles).not.toHaveBeenCalled();
|
||||||
|
expect(resetProcessed).not.toHaveBeenCalled();
|
||||||
|
expect(processedStorage).toEqual(new Map([[path, "storage-processed"]]));
|
||||||
|
expect(processedDatabase).toEqual(new Map([[path, "database-processed"]]));
|
||||||
|
});
|
||||||
|
|
||||||
it("shows manual Hidden File Sync commands only when the feature, Advanced mode, and runtime are ready", () => {
|
it("shows manual Hidden File Sync commands only when the feature, Advanced mode, and runtime are ready", () => {
|
||||||
const commands: Array<{
|
const commands: Array<{
|
||||||
id: string;
|
id: string;
|
||||||
@@ -136,6 +245,13 @@ describe("HiddenFileSync configuration-change notices", () => {
|
|||||||
expect(command?.checkCallback?.(true)).toBe(false);
|
expect(command?.checkCallback?.(true)).toBe(false);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
settings.useAdvancedMode = true;
|
||||||
|
for (const commandId of commandIds) {
|
||||||
|
const command = commands.find(({ id }) => id === commandId);
|
||||||
|
expect(command?.checkCallback?.(true)).toBe(false);
|
||||||
|
expect(command?.checkCallback?.(false)).toBe(false);
|
||||||
|
}
|
||||||
|
|
||||||
settings.syncInternalFiles = true;
|
settings.syncInternalFiles = true;
|
||||||
settings.useAdvancedMode = true;
|
settings.useAdvancedMode = true;
|
||||||
for (const commandId of commandIds) {
|
for (const commandId of commandIds) {
|
||||||
@@ -252,6 +368,7 @@ describe("HiddenFileSync configuration-change notices", () => {
|
|||||||
const scanAllDatabaseChanges = vi.fn(async () => undefined);
|
const scanAllDatabaseChanges = vi.fn(async () => undefined);
|
||||||
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
Object.assign(hiddenFileSync, {
|
Object.assign(hiddenFileSync, {
|
||||||
|
core: { settings: { syncInternalFiles: true } },
|
||||||
_progress: vi.fn(() => progress),
|
_progress: vi.fn(() => progress),
|
||||||
rebuildMerging,
|
rebuildMerging,
|
||||||
adoptCurrentStorageFilesAsProcessed,
|
adoptCurrentStorageFilesAsProcessed,
|
||||||
@@ -260,14 +377,83 @@ describe("HiddenFileSync configuration-change notices", () => {
|
|||||||
scanAllDatabaseChanges,
|
scanAllDatabaseChanges,
|
||||||
});
|
});
|
||||||
|
|
||||||
await hiddenFileSync.initialiseInternalFileSync("safe", true);
|
const result = await hiddenFileSync.initialiseInternalFileSync("safe", true);
|
||||||
|
|
||||||
|
expect(result).toBe(true);
|
||||||
expect(rebuildMerging).toHaveBeenCalledWith(false, false);
|
expect(rebuildMerging).toHaveBeenCalledWith(false, false);
|
||||||
expect(scanAllStorageChanges).toHaveBeenCalledWith(false, true, false);
|
expect(scanAllStorageChanges).toHaveBeenCalledWith(false, true, false);
|
||||||
expect(scanAllDatabaseChanges).toHaveBeenCalledWith(false, true, false);
|
expect(scanAllDatabaseChanges).toHaveBeenCalledWith(false, true, false);
|
||||||
expect(progress.done).toHaveBeenCalledOnce();
|
expect(progress.done).toHaveBeenCalledOnce();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it.each(["pushForce", "pullForce", "safe"] as const)(
|
||||||
|
"retains the selected files and completes %s initialisation when enabled",
|
||||||
|
async (direction) => {
|
||||||
|
const files = [".obsidian/plugins/example/main.js", ".obsidian/plugins/example/data.json"] as FilePath[];
|
||||||
|
const entries = files.map((path) => ({ path: `i:${path}` as FilePathWithPrefix }));
|
||||||
|
const progress = { log: vi.fn(), once: vi.fn(), done: vi.fn() };
|
||||||
|
const rebuildFromStorage = vi.fn(async () => files);
|
||||||
|
const rebuildFromDatabase = vi.fn(async () => entries);
|
||||||
|
const rebuildMerging = vi.fn(async () => files);
|
||||||
|
const adoptStorage = vi.fn();
|
||||||
|
const adoptDatabase = vi.fn();
|
||||||
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
|
Object.assign(hiddenFileSync, {
|
||||||
|
core: { settings: { syncInternalFiles: true } },
|
||||||
|
_progress: vi.fn(() => progress),
|
||||||
|
getPath: (entry: { path: FilePathWithPrefix }) => entry.path,
|
||||||
|
rebuildFromStorage,
|
||||||
|
rebuildFromDatabase,
|
||||||
|
rebuildMerging,
|
||||||
|
adoptCurrentStorageFilesAsProcessed: adoptStorage,
|
||||||
|
adoptCurrentDatabaseFilesAsProcessed: adoptDatabase,
|
||||||
|
scanAllStorageChanges: vi.fn(),
|
||||||
|
scanAllDatabaseChanges: vi.fn(),
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = await hiddenFileSync.initialiseInternalFileSync(direction, true, files);
|
||||||
|
|
||||||
|
expect(result).toBe(true);
|
||||||
|
if (direction === "pushForce") {
|
||||||
|
expect(rebuildFromStorage).toHaveBeenCalledExactlyOnceWith(false, files, false);
|
||||||
|
expect(rebuildFromDatabase).not.toHaveBeenCalled();
|
||||||
|
expect(rebuildMerging).not.toHaveBeenCalled();
|
||||||
|
} else if (direction === "pullForce") {
|
||||||
|
expect(rebuildFromDatabase).toHaveBeenCalledExactlyOnceWith(false, files, false);
|
||||||
|
expect(rebuildFromStorage).not.toHaveBeenCalled();
|
||||||
|
expect(rebuildMerging).not.toHaveBeenCalled();
|
||||||
|
} else {
|
||||||
|
expect(rebuildMerging).toHaveBeenCalledExactlyOnceWith(false, files);
|
||||||
|
expect(rebuildFromStorage).not.toHaveBeenCalled();
|
||||||
|
expect(rebuildFromDatabase).not.toHaveBeenCalled();
|
||||||
|
}
|
||||||
|
expect(adoptStorage).toHaveBeenCalledExactlyOnceWith(files);
|
||||||
|
expect(adoptDatabase).toHaveBeenCalledExactlyOnceWith(files);
|
||||||
|
expect(progress.done).toHaveBeenCalledOnce();
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it("does not report completion when the initialisation entry point refuses to start", async () => {
|
||||||
|
vi.mocked(configureHiddenFileSyncMode).mockImplementation(async (_mode, handlers) => {
|
||||||
|
await handlers.enable();
|
||||||
|
await handlers.initialise("safe");
|
||||||
|
return "enabled";
|
||||||
|
});
|
||||||
|
const progress = { log: vi.fn(), once: vi.fn(), done: vi.fn() };
|
||||||
|
const log = vi.fn();
|
||||||
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
|
Object.assign(hiddenFileSync, {
|
||||||
|
core: { services: { setting: { applyPartial: vi.fn() } } },
|
||||||
|
_progress: vi.fn(() => progress),
|
||||||
|
_log: log,
|
||||||
|
initialiseInternalFileSync: vi.fn(async () => false),
|
||||||
|
});
|
||||||
|
|
||||||
|
await hiddenFileSync.configureHiddenFileSync("MERGE");
|
||||||
|
|
||||||
|
expect(log).not.toHaveBeenCalledWith("Hidden File Sync initialisation completed.", expect.any(Number));
|
||||||
|
});
|
||||||
|
|
||||||
it("retirement guard: does not restore separate gathering and restart Notices", async () => {
|
it("retirement guard: does not restore separate gathering and restart Notices", async () => {
|
||||||
vi.mocked(configureHiddenFileSyncMode).mockImplementation(async (_mode, handlers) => {
|
vi.mocked(configureHiddenFileSyncMode).mockImplementation(async (_mode, handlers) => {
|
||||||
await handlers.enable();
|
await handlers.enable();
|
||||||
@@ -286,7 +472,7 @@ describe("HiddenFileSync configuration-change notices", () => {
|
|||||||
const applyPartial = vi.fn(async () => {
|
const applyPartial = vi.fn(async () => {
|
||||||
events.push("apply-settings");
|
events.push("apply-settings");
|
||||||
});
|
});
|
||||||
const initialiseInternalFileSync = vi.fn(async () => undefined);
|
const initialiseInternalFileSync = vi.fn(async () => true);
|
||||||
const log = vi.fn();
|
const log = vi.fn();
|
||||||
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
|
||||||
Object.assign(hiddenFileSync, {
|
Object.assign(hiddenFileSync, {
|
||||||
|
|||||||
+8
-1
@@ -1,4 +1,5 @@
|
|||||||
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
||||||
|
import { LOG_LEVEL_URGENT } from "octagonal-wheels/common/logger";
|
||||||
import { getLanguage, Notice, Plugin, type App, type PluginManifest } from "./deps";
|
import { getLanguage, Notice, Plugin, type App, type PluginManifest } from "./deps";
|
||||||
import { setGetLanguage } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
import { setGetLanguage } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
||||||
setGetLanguage(getLanguage);
|
setGetLanguage(getLanguage);
|
||||||
@@ -233,6 +234,12 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
|
|||||||
void this._startUp();
|
void this._startUp();
|
||||||
}
|
}
|
||||||
override onunload(): void {
|
override onunload(): void {
|
||||||
return void this.core.services.control.onUnload();
|
void this.core.services.control.onUnload().catch((error) => {
|
||||||
|
this.core.services.API.addLog(
|
||||||
|
"Failed to unload Self-hosted LiveSync: " + String(error),
|
||||||
|
LOG_LEVEL_URGENT,
|
||||||
|
"shutdown"
|
||||||
|
);
|
||||||
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -230,7 +230,13 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
|
|||||||
}
|
}
|
||||||
this.isLastHidden = isHidden;
|
this.isLastHidden = isHidden;
|
||||||
|
|
||||||
await this.services.fileProcessing.commitPendingFileEvents();
|
const pendingFileEventsCommitted = await this.services.fileProcessing.commitPendingFileEvents();
|
||||||
|
if (!pendingFileEventsCommitted) {
|
||||||
|
this._log(
|
||||||
|
"Pending file events remain because their commit did not complete before the window visibility changed.",
|
||||||
|
LOG_LEVEL_NOTICE
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
// Desktop opt-in (LiveSync/Periodic only): keep the background channel running while the
|
// Desktop opt-in (LiveSync/Periodic only): keep the background channel running while the
|
||||||
// window is hidden, instead of suspending on hide. On hide we skip the suspend for both
|
// window is hidden, instead of suspending on hide. On hide we skip the suspend for both
|
||||||
@@ -287,7 +293,10 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
|
|||||||
if (this.settings.suspendFileWatching) return;
|
if (this.settings.suspendFileWatching) return;
|
||||||
if (!this.settings.isConfigured) return;
|
if (!this.settings.isConfigured) return;
|
||||||
if (!this.services.appLifecycle.isReady()) return;
|
if (!this.services.appLifecycle.isReady()) return;
|
||||||
await this.services.fileProcessing.commitPendingFileEvents();
|
const pendingFileEventsCommitted = await this.services.fileProcessing.commitPendingFileEvents();
|
||||||
|
if (!pendingFileEventsCommitted) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (file == null) {
|
if (file == null) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import { describe, it, expect, vi, afterEach } from "vitest";
|
|||||||
import { ModuleObsidianEvents } from "./ModuleObsidianEvents";
|
import { ModuleObsidianEvents } from "./ModuleObsidianEvents";
|
||||||
import { DEFAULT_SETTINGS, REMOTE_COUCHDB } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
import { DEFAULT_SETTINGS, REMOTE_COUCHDB } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
import { reactiveSource } from "octagonal-wheels/dataobject/reactive";
|
import { reactiveSource } from "octagonal-wheels/dataobject/reactive";
|
||||||
|
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||||
|
|
||||||
type SetupOptions = {
|
type SetupOptions = {
|
||||||
settings?: Partial<typeof DEFAULT_SETTINGS>;
|
settings?: Partial<typeof DEFAULT_SETTINGS>;
|
||||||
@@ -23,6 +24,8 @@ function setup(opts: SetupOptions) {
|
|||||||
onResumed: vi.fn(async () => true),
|
onResumed: vi.fn(async () => true),
|
||||||
};
|
};
|
||||||
const fileProcessing = { commitPendingFileEvents: vi.fn(async () => true) };
|
const fileProcessing = { commitPendingFileEvents: vi.fn(async () => true) };
|
||||||
|
const replication = { replicateUnattendedByEvent: vi.fn(async () => undefined) };
|
||||||
|
const conflict = { queueCheckForIfOpen: vi.fn(async () => undefined) };
|
||||||
const boundedRemoteActivityCount = reactiveSource(0);
|
const boundedRemoteActivityCount = reactiveSource(0);
|
||||||
const boundedLocalApplicationActivityCount = reactiveSource(0);
|
const boundedLocalApplicationActivityCount = reactiveSource(0);
|
||||||
|
|
||||||
@@ -39,6 +42,8 @@ function setup(opts: SetupOptions) {
|
|||||||
setting: { saveSettingData: vi.fn(async () => undefined) },
|
setting: { saveSettingData: vi.fn(async () => undefined) },
|
||||||
appLifecycle,
|
appLifecycle,
|
||||||
fileProcessing,
|
fileProcessing,
|
||||||
|
replication,
|
||||||
|
conflict,
|
||||||
replicator: { boundedRemoteActivityCount, boundedLocalApplicationActivityCount },
|
replicator: { boundedRemoteActivityCount, boundedLocalApplicationActivityCount },
|
||||||
},
|
},
|
||||||
settings: {
|
settings: {
|
||||||
@@ -61,6 +66,9 @@ function setup(opts: SetupOptions) {
|
|||||||
module,
|
module,
|
||||||
appLifecycle,
|
appLifecycle,
|
||||||
fileProcessing,
|
fileProcessing,
|
||||||
|
replication,
|
||||||
|
conflict,
|
||||||
|
api: core._services.API,
|
||||||
boundedRemoteActivityCount,
|
boundedRemoteActivityCount,
|
||||||
boundedLocalApplicationActivityCount,
|
boundedLocalApplicationActivityCount,
|
||||||
};
|
};
|
||||||
@@ -303,6 +311,23 @@ describe("watchWindowVisibilityAsync — keepReplicationActiveInBackground", ()
|
|||||||
expect(appLifecycle.onResumed).toHaveBeenCalledTimes(1);
|
expect(appLifecycle.onResumed).toHaveBeenCalledTimes(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("logs a failed pending-file commit and still suspends on hide", async () => {
|
||||||
|
const { module, appLifecycle, fileProcessing, api } = setup({
|
||||||
|
settings: { keepReplicationActiveInBackground: false, liveSync: true },
|
||||||
|
hidden: true,
|
||||||
|
});
|
||||||
|
fileProcessing.commitPendingFileEvents.mockResolvedValue(false);
|
||||||
|
|
||||||
|
await module.watchWindowVisibilityAsync();
|
||||||
|
|
||||||
|
expect(appLifecycle.onSuspending).toHaveBeenCalledTimes(1);
|
||||||
|
expect(api.addLog).toHaveBeenCalledWith(
|
||||||
|
expect.stringContaining("Pending file events remain"),
|
||||||
|
LOG_LEVEL_NOTICE,
|
||||||
|
expect.any(String)
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
it("does not apply on mobile even if the flag is set", async () => {
|
it("does not apply on mobile even if the flag is set", async () => {
|
||||||
const { module, appLifecycle } = setup({
|
const { module, appLifecycle } = setup({
|
||||||
settings: { keepReplicationActiveInBackground: true, liveSync: true },
|
settings: { keepReplicationActiveInBackground: true, liveSync: true },
|
||||||
@@ -313,3 +338,20 @@ describe("watchWindowVisibilityAsync — keepReplicationActiveInBackground", ()
|
|||||||
expect(appLifecycle.onSuspending).toHaveBeenCalledTimes(1);
|
expect(appLifecycle.onSuspending).toHaveBeenCalledTimes(1);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("watchWorkspaceOpenAsync — pending file event commit", () => {
|
||||||
|
it("skips replication and conflict checks when pending file events fail to commit", async () => {
|
||||||
|
const { module, fileProcessing, replication, conflict } = setup({
|
||||||
|
settings: { syncOnFileOpen: true },
|
||||||
|
hidden: false,
|
||||||
|
});
|
||||||
|
fileProcessing.commitPendingFileEvents.mockResolvedValue(false);
|
||||||
|
|
||||||
|
await module.watchWorkspaceOpenAsync({ path: "note.md" } as Parameters<
|
||||||
|
typeof module.watchWorkspaceOpenAsync
|
||||||
|
>[0]);
|
||||||
|
|
||||||
|
expect(replication.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||||
|
expect(conflict.queueCheckForIfOpen).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -171,20 +171,18 @@ export function paneSyncSettings(
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
void addPanel(
|
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleUpdateThinning")).then((paneEl) => {
|
||||||
paneEl,
|
const inLiveSyncMode = () => this.isConfiguredAs("syncMode", "LIVESYNC");
|
||||||
$msg("obsidianLiveSyncSettingTab.titleUpdateThinning"),
|
const batchEnabled = () => this.isConfiguredAs(inLiveSyncMode() ? "batchSaveInLiveSync" : "batchSave", true);
|
||||||
undefined,
|
new Setting(paneEl).autoWireToggle("batchSave", { onUpdate: visibleOnly(() => !inLiveSyncMode()) });
|
||||||
visibleOnly(() => !this.isConfiguredAs("syncMode", "LIVESYNC"))
|
new Setting(paneEl).autoWireToggle("batchSaveInLiveSync", { onUpdate: visibleOnly(inLiveSyncMode) });
|
||||||
).then((paneEl) => {
|
|
||||||
new Setting(paneEl).autoWireToggle("batchSave");
|
|
||||||
new Setting(paneEl).autoWireNumeric("batchSaveMinimumDelay", {
|
new Setting(paneEl).autoWireNumeric("batchSaveMinimumDelay", {
|
||||||
acceptZero: true,
|
acceptZero: true,
|
||||||
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
|
onUpdate: visibleOnly(batchEnabled),
|
||||||
});
|
});
|
||||||
new Setting(paneEl).autoWireNumeric("batchSaveMaximumDelay", {
|
new Setting(paneEl).autoWireNumeric("batchSaveMaximumDelay", {
|
||||||
acceptZero: true,
|
acceptZero: true,
|
||||||
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
|
onUpdate: visibleOnly(batchEnabled),
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,98 @@
|
|||||||
|
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
|
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab";
|
||||||
|
import type { OnUpdateFunc, PageFunctions } from "./SettingPane";
|
||||||
|
|
||||||
|
const controls = vi.hoisted(() => new Map<string, { onUpdate?: () => { visibility?: boolean } }>());
|
||||||
|
vi.mock("@/common/translation", () => ({ $msg: (key: string) => key }));
|
||||||
|
vi.mock("@vrtmrz/livesync-commonlib/compat/common/logger", () => ({ Logger: vi.fn() }));
|
||||||
|
vi.mock("./LiveSyncSetting.ts", () => ({
|
||||||
|
LiveSyncSetting: class {
|
||||||
|
autoWireDropDown(key: string, options = {}) {
|
||||||
|
controls.set(key, options);
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
autoWireToggle(key: string, options = {}) {
|
||||||
|
controls.set(key, options);
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
autoWireNumeric(key: string, options = {}) {
|
||||||
|
controls.set(key, options);
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
|
||||||
|
import { paneSyncSettings } from "./PaneSyncSettings";
|
||||||
|
|
||||||
|
function setup(liveSync: boolean, batchSave: boolean, batchSaveInLiveSync: boolean) {
|
||||||
|
const callbacks = new Map<string, (value: string) => Promise<void>>();
|
||||||
|
const panels = new Map<string, OnUpdateFunc | undefined>();
|
||||||
|
const tab = {
|
||||||
|
editingSettings: { ...DEFAULT_SETTINGS, liveSync, batchSave, batchSaveInLiveSync },
|
||||||
|
services: { API: { isMobile: () => false }, control: { applySettings: vi.fn(async () => {}) } },
|
||||||
|
addOnSaved: (key: string, callback: (value: string) => Promise<void>) => callbacks.set(key, callback),
|
||||||
|
isConfiguredAs(key: string, value: unknown) {
|
||||||
|
if (key === "syncMode") {
|
||||||
|
return (
|
||||||
|
(this.editingSettings.liveSync
|
||||||
|
? "LIVESYNC"
|
||||||
|
: this.editingSettings.periodicReplication
|
||||||
|
? "PERIODIC"
|
||||||
|
: "ONEVENTS") === value
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return this.editingSettings[key as keyof typeof this.editingSettings] === value;
|
||||||
|
},
|
||||||
|
saveSettings: vi.fn(async () => {}),
|
||||||
|
requestCatalogueRefresh: vi.fn(),
|
||||||
|
};
|
||||||
|
const functions = {
|
||||||
|
addPanel: (_element: HTMLElement, title: string, _callback: unknown, onUpdate?: OnUpdateFunc) => ({
|
||||||
|
then(callback: (element: HTMLElement) => void) {
|
||||||
|
if (!title.endsWith("titleSynchronizationMethod") && !title.endsWith("titleUpdateThinning")) return;
|
||||||
|
panels.set(title, onUpdate);
|
||||||
|
callback({} as HTMLElement);
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
} as unknown as PageFunctions;
|
||||||
|
paneSyncSettings.call(tab as unknown as ObsidianLiveSyncSettingTab, {} as HTMLElement, functions);
|
||||||
|
return { tab, callbacks, panels };
|
||||||
|
}
|
||||||
|
|
||||||
|
function visible(key: string): boolean {
|
||||||
|
const control = controls.get(key);
|
||||||
|
if (!control) throw new Error(`Missing setting control: ${key}`);
|
||||||
|
return control.onUpdate?.().visibility !== false;
|
||||||
|
}
|
||||||
|
|
||||||
|
beforeEach(() => controls.clear());
|
||||||
|
|
||||||
|
describe("mode-specific database batching settings", () => {
|
||||||
|
it.each([
|
||||||
|
{ liveSync: true, batchSave: true, batchSaveInLiveSync: false, delays: false },
|
||||||
|
{ liveSync: true, batchSave: false, batchSaveInLiveSync: true, delays: true },
|
||||||
|
{ liveSync: false, batchSave: true, batchSaveInLiveSync: false, delays: true },
|
||||||
|
{ liveSync: false, batchSave: false, batchSaveInLiveSync: true, delays: false },
|
||||||
|
])("shows the current mode choice and its shared delays: %j", ({ delays, ...settings }) => {
|
||||||
|
const { panels } = setup(settings.liveSync, settings.batchSave, settings.batchSaveInLiveSync);
|
||||||
|
expect(panels.get("obsidianLiveSyncSettingTab.titleUpdateThinning")?.().visibility).not.toBe(false);
|
||||||
|
expect(visible("batchSave")).toBe(!settings.liveSync);
|
||||||
|
expect(visible("batchSaveInLiveSync")).toBe(settings.liveSync);
|
||||||
|
expect(visible("batchSaveMinimumDelay")).toBe(delays);
|
||||||
|
expect(visible("batchSaveMaximumDelay")).toBe(delays);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
[true, false],
|
||||||
|
[false, true],
|
||||||
|
])("retains both batching choices across mode changes: %j", async (batchSave, batchSaveInLiveSync) => {
|
||||||
|
const { tab, callbacks } = setup(false, batchSave, batchSaveInLiveSync);
|
||||||
|
const switchMode = callbacks.get("syncMode")!;
|
||||||
|
for (const mode of ["LIVESYNC", "PERIODIC", "ONEVENTS", "LIVESYNC"]) {
|
||||||
|
await switchMode(mode);
|
||||||
|
expect(tab.editingSettings).toMatchObject({ batchSave, batchSaveInLiveSync });
|
||||||
|
}
|
||||||
|
expect(tab.services.control.applySettings).toHaveBeenCalledTimes(4);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -88,7 +88,7 @@ npm run test:e2e:obsidian:focused -- replication-status
|
|||||||
|
|
||||||
The wrapper accepts only maintained real-Obsidian scenario names; run it with `--help` for the current list. It deliberately does not manage CouchDB, Object Storage, or the P2P signalling relay. Start the required fixture first, or use the complete service-managed suite.
|
The wrapper accepts only maintained real-Obsidian scenario names; run it with `--help` for the current list. It deliberately does not manage CouchDB, Object Storage, or the P2P signalling relay. Start the required fixture first, or use the complete service-managed suite.
|
||||||
|
|
||||||
`replication-status` uses an isolated Vault without a remote service. It injects synthetic replication statistics into the real plug-in to check the ⏩ icon, the comparison count and its three-second expiry, independently styled numeric items, stable rendered digit widths, and unchanged DOM nodes for repeated values. It verifies the inactive class, retained inactive content and width, and the ability to override the default hiding behaviour through CSS. A separate session enters mobile emulation before loading the plug-in and measures the editor item gaps at a 390px viewport width. The desktop checks also ensure that the status bar keeps its compact spacing. The mobile spacing fixture uses the existing overlay placement method before measurement; it does not verify Android window events. It switches between split panes, opens a popout editor, checks log updates there, and returns to the main window. Where Electron exposes native window control, it also hides the main window and verifies that popout logs still update before restoring it. This checks display behaviour; Commonlib owns the tests which establish that PouchDB produces the underlying statistics.
|
`replication-status` uses an isolated Vault without a remote service. It injects synthetic replication statistics into the real plug-in to check the ⏩ icon, the comparison count and its three-second expiry, independently styled numeric items, stable rendered digit widths, computed right alignment, and unchanged DOM nodes for repeated values. It also feeds the existing queue sources and checks their values in both displays, the separate P2P line, file warnings and their compact icon, and the corresponding plain-text status. It verifies the inactive class, retained inactive content and width, and the ability to override the default hiding behaviour through CSS. A separate session enters mobile emulation before loading the plug-in and measures the editor item gaps at a 390px viewport width. The desktop checks also ensure that the status bar keeps its compact spacing. The mobile spacing fixture uses the existing overlay placement method before measurement; it does not verify Android window events. It switches between split panes, opens a popout editor, checks log updates there, and returns to the main window. Where Electron exposes native window control, it also hides the main window and verifies that popout logs still update before restoring it. This checks display behaviour; Commonlib owns the tests which establish that PouchDB produces the underlying statistics.
|
||||||
|
|
||||||
`database-diagnostics` needs no remote service. It checks the Usage command and Hatch integrity action with an isolated Vault, including visible radio choices for original names and pseudonyms, their default and keyboard navigation, the default-on integrity-only remote option, retained local findings when the remote is unavailable, cancellation during inspection with earlier rows retained, and unchanged settings, local revisions, and Vault files. It captures desktop results and the choice and result dialogues in mobile emulation, checking layout at 800×600 and 375×667, radio choices for available formats, their touch targets, and export controls. Copy and download payloads are captured inside the isolated renderer, so the scenario leaves the host clipboard and download directory unchanged.
|
`database-diagnostics` needs no remote service. It checks the Usage command and Hatch integrity action with an isolated Vault, including visible radio choices for original names and pseudonyms, their default and keyboard navigation, the default-on integrity-only remote option, retained local findings when the remote is unavailable, cancellation during inspection with earlier rows retained, and unchanged settings, local revisions, and Vault files. It captures desktop results and the choice and result dialogues in mobile emulation, checking layout at 800×600 and 375×667, radio choices for available formats, their touch targets, and export controls. Copy and download payloads are captured inside the isolated renderer, so the scenario leaves the host clipboard and download directory unchanged.
|
||||||
|
|
||||||
@@ -130,7 +130,7 @@ The underlying `test:e2e:obsidian:<scenario>` scripts remain available for an im
|
|||||||
|
|
||||||
`test:e2e:obsidian:joined-devices` uses a temporary Vault and isolated CouchDB database to open Joined devices from Remote Configuration, a version Notice, and its registered command. `test:e2e:obsidian:joined-devices-object-storage` runs the same assertions against a unique Object Storage prefix, including the conditional removal supported by the test S3 service. It checks that newer and older Self-hosted LiveSync versions share one Notice with one recommendation and review action, that the current device appears first with current metadata, and that locked-out and other-device records are shown. It confirms that a manual refresh reads an updated remote record and that cancelling the inline confirmation leaves the remote unchanged before forgetting one device. It verifies the raw milestone retains the other device, remote state, and preferred-ID tweaks, then checks mobile scrolling, safe-area bounds, and touch targets. It captures the version Notice, desktop list, forget confirmation, completed removal, and mobile dialogue; unrelated Notices are dismissed before the list captures.
|
`test:e2e:obsidian:joined-devices` uses a temporary Vault and isolated CouchDB database to open Joined devices from Remote Configuration, a version Notice, and its registered command. `test:e2e:obsidian:joined-devices-object-storage` runs the same assertions against a unique Object Storage prefix, including the conditional removal supported by the test S3 service. It checks that newer and older Self-hosted LiveSync versions share one Notice with one recommendation and review action, that the current device appears first with current metadata, and that locked-out and other-device records are shown. It confirms that a manual refresh reads an updated remote record and that cancelling the inline confirmation leaves the remote unchanged before forgetting one device. It verifies the raw milestone retains the other device, remote state, and preferred-ID tweaks, then checks mobile scrolling, safe-area bounds, and touch targets. It captures the version Notice, desktop list, forget confirmation, completed removal, and mobile dialogue; unrelated Notices are dismissed before the list captures.
|
||||||
|
|
||||||
`test:e2e:obsidian:settings-ui` starts with a pending compatibility review and verifies the dedicated pause summary, its detailed explanation, and the explicit resume action in a temporary real Obsidian session. It captures the desktop summary and the iPhone-sized summary and detail dialogues; the mobile checks cover viewport containment, horizontal overflow, safe-area containment, and the close control's touch target. It confirms that the acknowledged internal version advances only after the review is accepted, and checks that the Change Log contains no acknowledgement control. It then enables Advanced mode and persists one numeric Advanced setting. On Obsidian before 1.13, this covers the imperative settings fallback used by the `SettingSpec` proof. On Obsidian 1.13 or later, it verifies the task-oriented landing order, the separate Synchronisation group containing Remote Configuration and Sync Settings, and the General Settings group containing Appearance, Logging, and Extra menus. It reopens and cancels onboarding from Quick Setup, opens all 14 nested settings pages, searches globally for the Advanced control, captures the landing page, search result, Advanced page, and mobile landing page, and verifies that the value remains after the settings dialogue is closed and reopened. The mobile landing check confirms that Remote Configuration is reachable without initial scrolling. Finally, it selects the Synchronisation Settings page and verifies that the deletion panel still exposes the effective 'Keep empty folder' setting without presenting the legacy `trashInsteadDelete` control, whose value no longer changes Obsidian deletion behaviour.
|
`test:e2e:obsidian:settings-ui` starts with a pending compatibility review and verifies the dedicated pause summary, its detailed explanation, and the explicit resume action in a temporary real Obsidian session. It captures the desktop summary and the iPhone-sized summary and detail dialogues; the mobile checks cover viewport containment, horizontal overflow, safe-area containment, and the close control's touch target. It confirms that the acknowledged internal version advances only after the review is accepted, and checks that the Change Log contains no acknowledgement control. It then enables Advanced mode and persists one numeric Advanced setting. It also checks that batching starts disabled, that each Sync Mode shows its own batching choice and the appropriate shared delay controls, and that switching modes and reopening settings preserves both choices. It reads the saved plug-in data to confirm that the choices are persisted independently. On Obsidian before 1.13, this covers the imperative settings fallback used by the `SettingSpec` proof. On Obsidian 1.13 or later, it verifies the task-oriented landing order, the separate Synchronisation group containing Remote Configuration and Sync Settings, and the General Settings group containing Appearance, Logging, and Extra menus. It reopens and cancels onboarding from Quick Setup, opens all 14 nested settings pages, searches globally for the Advanced control, captures the landing page, search result, Advanced page, and mobile landing page, and verifies that the value remains after the settings dialogue is closed and reopened. The mobile landing check confirms that Remote Configuration is reachable without initial scrolling. Finally, it selects the Synchronisation Settings page and verifies that the deletion panel still exposes the effective 'Keep empty folder' setting without presenting the legacy `trashInsteadDelete` control, whose value no longer changes Obsidian deletion behaviour.
|
||||||
|
|
||||||
The default runner uses the maintained pre-1.13 Obsidian fixture and therefore exercises the complete compatibility-review, mobile-layout, and imperative-settings path. To exercise only the native settings contract against an additional Obsidian 1.13-or-later installation, supply its executable and companion CLI explicitly:
|
The default runner uses the maintained pre-1.13 Obsidian fixture and therefore exercises the complete compatibility-review, mobile-layout, and imperative-settings path. To exercise only the native settings contract against an additional Obsidian 1.13-or-later installation, supply its executable and companion CLI explicitly:
|
||||||
|
|
||||||
@@ -255,10 +255,20 @@ This proves in real Obsidian the plug-in behaviour shared by supported platforms
|
|||||||
|
|
||||||
`test:e2e:obsidian:hidden-file-selectors` stores and restores selected JS, CSS, and JSON files with a normal-file allow-list set, and compares both directions with an empty allow-list. It uses one temporary Vault and a local database, and also runs in the complete local suite. Run it through the focused wrapper to build the current bundle first; no remote service is required. Set `E2E_HIDDEN_FILE_SELECTORS_OUTPUT` to choose the JSON report path.
|
`test:e2e:obsidian:hidden-file-selectors` stores and restores selected JS, CSS, and JSON files with a normal-file allow-list set, and compares both directions with an empty allow-list. It uses one temporary Vault and a local database, and also runs in the complete local suite. Run it through the focused wrapper to build the current bundle first; no remote service is required. Set `E2E_HIDDEN_FILE_SELECTORS_OUTPUT` to choose the JSON report path.
|
||||||
|
|
||||||
|
Before enablement, the scenario directly requests initialisation while Hidden File Sync is disabled. It requires a refused result, unchanged local files, Metadata revisions, and processed markers, and no initialisation progress Notice.
|
||||||
|
|
||||||
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
|
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
|
||||||
|
|
||||||
|
`test:e2e:obsidian:customisation-catalogue-reload` opens the Customisation Sync dialogue, enables Maintenance mode, and clicks Reload. It verifies that this first Reload lists all three files of a sample plug-in, then selects the source copy and clicks Apply to restore their contents. It also checks a second enumeration and Apply through the existing methods in one temporary Vault with a local database. Run it through the focused wrapper to build the current bundle first; no remote service is required. Set `E2E_CUSTOMISATION_CATALOGUE_OUTPUT` to choose the JSON report path.
|
||||||
|
|
||||||
The workflow also opens the Customisation Sync dialogue. `--case=visibility` checks that **Hide not applicable items** hides an identical snippet, preserves a different snippet and its source selector, and restores the identical snippet when cleared. `--case=mtime` gives the multi-file plug-in fixture modern millisecond timestamps and requires an older remote copy to remain labelled **Older** and unselected by **Select All Shiny**. With no case argument, both checks run before the existing apply, update, and deletion workflow.
|
The workflow also opens the Customisation Sync dialogue. `--case=visibility` checks that **Hide not applicable items** hides an identical snippet, preserves a different snippet and its source selector, and restores the identical snippet when cleared. `--case=mtime` gives the multi-file plug-in fixture modern millisecond timestamps and requires an older remote copy to remain labelled **Older** and unselected by **Select All Shiny**. With no case argument, both checks run before the existing apply, update, and deletion workflow.
|
||||||
|
|
||||||
|
`--case=automatic` separately checks that choosing **Automatic** while Hidden File Sync is disabled displays a Notice without changing modes, files, Metadata revisions, or processed markers. The fixture starts with a saved Automatic selection and requires it to survive start-up and both rejected actions. It also checks disabling the feature after opening **Initial Action**. After enablement, selecting a whole plug-in must initialise both **MAIN** and **DATA**, and transfer their files to the other Vault. Separate temporary Vaults exercise **Overwrite Remote**, **Overwrite Local**, and **Use newer**. The latter two start with different local and database contents; deterministic per-file modification times require **Use newer** to keep the newer database MAIN content and the newer local DATA content.
|
||||||
|
|
||||||
|
`test:e2e:obsidian:livesync-batch` uses two real Obsidian Vaults and a disposable local CouchDB to compare batching disabled, the older `batchSave` choice alone, and the explicit LiveSync batching choice. It covers repeated edits in both directions, template-like writes, deletion and recreation, incoming changes, maximum delay, concurrent paths, manual flush, and restart recovery before and after a database write. Restarts use the saved profile with natural plug-in start-up. `ISSUE1207_CASE_FILTER` selects case names; `ISSUE1207_RESTART_BUCKET=same|exact|different` exercises interrupted processing at the selected timestamp condition. `ISSUE1207_RESTART_SAVE_AFTER_EDIT=true` separately checks a settings save after the pending edit. Use `npm run test:e2e:obsidian:focused -- livesync-batch` to build and run the default matrix. This scenario requires a localhost CouchDB and is outside the complete local suite.
|
||||||
|
|
||||||
|
`test:e2e:obsidian:optional-file-save-completion` exercises Hidden File Sync and both Customisation Sync formats in one isolated real Obsidian Vault. It holds a local database write to verify that its event stays in the snapshot and that flush waits for the write, then checks unchanged notifications, subsequent edits, and repeated deletions. Injected save failures remain pending and must replay successfully before the next start reaches readiness. This local scenario disables replication and requires no remote service. It is outside `test:e2e:obsidian:local-suite`.
|
||||||
|
|
||||||
`test:e2e:obsidian:received-change-readiness` uses two sequential real Obsidian sessions and isolated CouchDB databases. The source creates ordinary notes and their Chunks; the target starts continuous replication, resets readiness through the public lifecycle service, and receives those documents while its Vault files remain absent. Marking the target ready twice must emit one readiness event and reflect the queued content. A second note remains absent after readiness while database reflecting is explicitly suspended, then appears when that setting is resumed. This focused scenario is outside `test:e2e:obsidian:local-suite`.
|
`test:e2e:obsidian:received-change-readiness` uses two sequential real Obsidian sessions and isolated CouchDB databases. The source creates ordinary notes and their Chunks; the target starts continuous replication, resets readiness through the public lifecycle service, and receives those documents while its Vault files remain absent. Marking the target ready twice must emit one readiness event and reflect the queued content. A second note remains absent after readiness while database reflecting is explicitly suspended, then appears when that setting is resumed. This focused scenario is outside `test:e2e:obsidian:local-suite`.
|
||||||
|
|
||||||
`test:e2e:obsidian:remote-feature-change` starts real Obsidian with continuous CouchDB replication, then changes the remote version document from generation 12 to generation 13 with an unknown feature. It waits for the control document to reach the local database and the active Replicator to retire, checks that another replication is refused, and verifies that an already accepted Vault note remains intact. After restarting the same Vault, the current remote declaration still blocks finite replication and the actual continuous connection attempt. No KV feature history is involved. It is a focused test outside `test:e2e:obsidian:local-suite`; recovery with a future compatible client remains a separate validation boundary.
|
`test:e2e:obsidian:remote-feature-change` starts real Obsidian with continuous CouchDB replication, then changes the remote version document from generation 12 to generation 13 with an unknown feature. It waits for the control document to reach the local database and the active Replicator to retire, checks that another replication is refused, and verifies that an already accepted Vault note remains intact. After restarting the same Vault, the current remote declaration still blocks finite replication and the actual continuous connection attempt. No KV feature history is involved. It is a focused test outside `test:e2e:obsidian:local-suite`; recovery with a future compatible client remains a separate validation boundary.
|
||||||
|
|||||||
@@ -0,0 +1,246 @@
|
|||||||
|
import { execFileSync } from "node:child_process";
|
||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { dirname, join } from "node:path";
|
||||||
|
import { isDeepStrictEqual } from "node:util";
|
||||||
|
import { fileURLToPath } from "node:url";
|
||||||
|
import { evalObsidianJson } from "../runner/cli.ts";
|
||||||
|
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||||
|
import {
|
||||||
|
createE2eCouchDbPluginData,
|
||||||
|
createE2eObsidianDeviceLocalState,
|
||||||
|
waitForLiveSyncCoreReady,
|
||||||
|
} from "../runner/liveSyncWorkflow.ts";
|
||||||
|
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||||
|
import { waitForVisibleObsidianDialogue, withObsidianPage } from "../runner/ui.ts";
|
||||||
|
import { createTemporaryVault } from "../runner/vault.ts";
|
||||||
|
|
||||||
|
const pluginName = "livesync-catalogue-example";
|
||||||
|
const sourceTerm = "catalogue-source";
|
||||||
|
const pluginRoot = `.obsidian/plugins/${pluginName}`;
|
||||||
|
const pluginFiles = {
|
||||||
|
[`${pluginRoot}/main.js`]: "module.exports = class ExamplePlugin {};\n",
|
||||||
|
[`${pluginRoot}/manifest.json`]: JSON.stringify({
|
||||||
|
id: pluginName,
|
||||||
|
name: "Catalogue Example",
|
||||||
|
version: "1.0.0",
|
||||||
|
minAppVersion: "1.0.0",
|
||||||
|
description: "An isolated test fixture",
|
||||||
|
author: "Self-hosted LiveSync",
|
||||||
|
}),
|
||||||
|
[`${pluginRoot}/styles.css`]: ".livesync-catalogue-example { color: red; }\n",
|
||||||
|
};
|
||||||
|
const allowList = "\\.(?:md|png|pdf)$";
|
||||||
|
const coreCode = "const core=app.plugins.plugins['obsidian-livesync'].core;";
|
||||||
|
const configCode = `${coreCode}const addOn=core.getAddOn('ConfigSync');`;
|
||||||
|
const settleCode = `
|
||||||
|
const settle=async()=>{
|
||||||
|
await new Promise(resolve=>setTimeout(resolve,100));
|
||||||
|
for(let attempt=0;attempt<200;attempt++){
|
||||||
|
if(addOn.updatingV2Count===0)return;
|
||||||
|
await new Promise(resolve=>setTimeout(resolve,25));
|
||||||
|
}
|
||||||
|
throw new Error('Customisation Sync did not settle');
|
||||||
|
};
|
||||||
|
`;
|
||||||
|
const sourceRowCode = `
|
||||||
|
const row=addOn.pluginList.find(entry=>entry.category==='PLUGIN_MAIN'&&entry.name===${JSON.stringify(pluginName)}&&entry.term===${JSON.stringify(sourceTerm)});
|
||||||
|
if(!row)throw new Error('The source catalogue row is missing');
|
||||||
|
`;
|
||||||
|
|
||||||
|
type Observation = { name: string; expected: unknown; actual: unknown; passed: boolean };
|
||||||
|
|
||||||
|
async function writeFixtures(vaultPath: string, files: Record<string, string>) {
|
||||||
|
for (const [path, content] of Object.entries(files)) {
|
||||||
|
await mkdir(dirname(join(vaultPath, path)), { recursive: true });
|
||||||
|
await writeFile(join(vaultPath, path), content);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function removeFixtures(vaultPath: string, files: Record<string, string>) {
|
||||||
|
for (const path of Object.keys(files)) await rm(join(vaultPath, path), { force: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
async function matchingFiles(vaultPath: string, files: Record<string, string>) {
|
||||||
|
const matches: string[] = [];
|
||||||
|
for (const [path, content] of Object.entries(files)) {
|
||||||
|
try {
|
||||||
|
if ((await readFile(join(vaultPath, path), "utf8")) === content) matches.push(path);
|
||||||
|
} catch (error) {
|
||||||
|
if (!(error instanceof Error) || !("code" in error) || error.code !== "ENOENT") throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return matches.sort();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitForMatchingFiles(vaultPath: string, files: Record<string, string>) {
|
||||||
|
const deadline = Date.now() + 10_000;
|
||||||
|
let matches = await matchingFiles(vaultPath, files);
|
||||||
|
while (matches.length !== Object.keys(files).length && Date.now() < deadline) {
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||||
|
matches = await matchingFiles(vaultPath, files);
|
||||||
|
}
|
||||||
|
return matches;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const binary = requireObsidianBinary();
|
||||||
|
const cli = discoverObsidianCli();
|
||||||
|
if (!cli.binary) throw new Error("The Obsidian CLI is unavailable");
|
||||||
|
const cliBinary = cli.binary;
|
||||||
|
const output =
|
||||||
|
process.env.E2E_CUSTOMISATION_CATALOGUE_OUTPUT ?? join(tmpdir(), "livesync-customisation-catalogue.json");
|
||||||
|
const observations: Observation[] = [];
|
||||||
|
const report: Record<string, unknown> = {
|
||||||
|
revision: execFileSync("git", ["rev-parse", "HEAD"], { encoding: "utf8" }).trim(),
|
||||||
|
sourceDiffSha256: createHash("sha256")
|
||||||
|
.update(new Uint8Array(execFileSync("git", ["diff", "--binary", "HEAD"])))
|
||||||
|
.digest("hex"),
|
||||||
|
scenarioSha256: createHash("sha256")
|
||||||
|
.update(new Uint8Array(await readFile(fileURLToPath(import.meta.url))))
|
||||||
|
.digest("hex"),
|
||||||
|
commonlibEntrySha256: createHash("sha256")
|
||||||
|
.update(
|
||||||
|
new Uint8Array(
|
||||||
|
await readFile(
|
||||||
|
"node_modules/@vrtmrz/livesync-commonlib/dist/managers/EntryManager/EntryManagerImpls.js"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.digest("hex"),
|
||||||
|
bundleSha256: createHash("sha256")
|
||||||
|
.update(new Uint8Array(await readFile("main.js")))
|
||||||
|
.digest("hex"),
|
||||||
|
commonlibVersion: JSON.parse(await readFile("node_modules/@vrtmrz/livesync-commonlib/package.json", "utf8"))
|
||||||
|
.version,
|
||||||
|
observations,
|
||||||
|
};
|
||||||
|
const vault = await createTemporaryVault("livesync-customisation-catalogue-");
|
||||||
|
let session: ObsidianLiveSyncSession | undefined;
|
||||||
|
const record = (name: string, actual: unknown, expected: unknown) => {
|
||||||
|
const observation = { name, expected, actual, passed: isDeepStrictEqual(actual, expected) };
|
||||||
|
observations.push(observation);
|
||||||
|
console.log(JSON.stringify(observation));
|
||||||
|
};
|
||||||
|
const evaluate = async <T>(code: string): Promise<T> => {
|
||||||
|
if (!session) throw new Error("The isolated Obsidian session has not started");
|
||||||
|
return await evalObsidianJson<T>(cliBinary, `(async()=>{${code}})()`, session.cliEnv, 60_000);
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
session = await startObsidianLiveSyncSession({
|
||||||
|
binary,
|
||||||
|
cliBinary,
|
||||||
|
vault,
|
||||||
|
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
|
||||||
|
pluginData: createE2eCouchDbPluginData(
|
||||||
|
{ uri: "http://127.0.0.1:9", username: "", password: "", dbName: "catalogue-local-only" },
|
||||||
|
{ periodicReplication: false, syncOnFileOpen: false, useOnlyLocalChunk: true }
|
||||||
|
),
|
||||||
|
});
|
||||||
|
report.pluginVersion = session.readiness.pluginVersion;
|
||||||
|
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||||
|
report.runtime = await evaluate(`${coreCode}
|
||||||
|
core.services.setting.setDeviceAndVaultName('catalogue-local');
|
||||||
|
Object.assign(core.settings,{
|
||||||
|
isConfigured:true,usePluginSync:true,usePluginSyncV2:true,syncInternalFiles:false,
|
||||||
|
autoSweepPlugins:false,autoSweepPluginsPeriodic:false,watchInternalFileChanges:false,
|
||||||
|
syncOnlyRegEx:${JSON.stringify(allowList)},syncIgnoreRegEx:''
|
||||||
|
});
|
||||||
|
return JSON.stringify({userAgent:navigator.userAgent,databaseReady:core.localDatabase.isReady});
|
||||||
|
`);
|
||||||
|
|
||||||
|
await writeFixtures(vault.path, pluginFiles);
|
||||||
|
await evaluate(`${configCode}${settleCode}
|
||||||
|
for(const path of ${JSON.stringify(Object.keys(pluginFiles))}){
|
||||||
|
const stored=await addOn.storeCustomisationFileV2(path,${JSON.stringify(sourceTerm)});
|
||||||
|
if(stored===false)throw new Error('Could not seed a Customisation Sync document');
|
||||||
|
}
|
||||||
|
await settle();
|
||||||
|
return JSON.stringify({seeded:true});
|
||||||
|
`);
|
||||||
|
const opened = await evaluate<{ opened: boolean }>(
|
||||||
|
`return JSON.stringify({opened:app.commands.executeCommandById('obsidian-livesync:livesync-plugin-dialog-ex')});`
|
||||||
|
);
|
||||||
|
if (!opened.opened) throw new Error("Could not open the Customisation Sync dialogue command");
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
const dialogue = await waitForVisibleObsidianDialogue(page, "Customization Sync (Beta3)");
|
||||||
|
await dialogue
|
||||||
|
.locator("label")
|
||||||
|
.filter({ hasText: "Maintenance mode" })
|
||||||
|
.locator('input[type="checkbox"]')
|
||||||
|
.check();
|
||||||
|
await dialogue.getByRole("button", { name: "Reload", exact: true }).click();
|
||||||
|
});
|
||||||
|
const catalogue = await evaluate<{ files: string[]; documents: number }>(`${configCode}${settleCode}
|
||||||
|
await settle();
|
||||||
|
${sourceRowCode}
|
||||||
|
let documents=0;
|
||||||
|
for await(const doc of core.localDatabase.findEntries('ix:','ix;')){
|
||||||
|
if(doc.path.startsWith(${JSON.stringify(`ix:${sourceTerm}/PLUGIN_MAIN/${pluginName}%`)})&&!doc.deleted&&!doc._deleted)documents++;
|
||||||
|
}
|
||||||
|
return JSON.stringify({files:row.files.map(file=>file.filename).sort(),documents});
|
||||||
|
`);
|
||||||
|
record("#1271: all source documents remain in the database", catalogue.documents, 3);
|
||||||
|
record(
|
||||||
|
"#1271: one reload lists all files",
|
||||||
|
catalogue.files,
|
||||||
|
Object.keys(pluginFiles)
|
||||||
|
.map((path) => path.substring(".obsidian/".length))
|
||||||
|
.sort()
|
||||||
|
);
|
||||||
|
await removeFixtures(vault.path, pluginFiles);
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
const dialogue = await waitForVisibleObsidianDialogue(page, "Customization Sync (Beta3)");
|
||||||
|
const row = dialogue.locator(`.filerow:has(select option[value="${sourceTerm}"])`);
|
||||||
|
await row.locator("select").selectOption(sourceTerm);
|
||||||
|
await row.locator(".body").getByRole("button", { name: "✓", exact: true }).click();
|
||||||
|
});
|
||||||
|
report.firstApply = { action: "UI Apply", sourceTerm };
|
||||||
|
record(
|
||||||
|
"#1271: first Apply restores all file contents",
|
||||||
|
await waitForMatchingFiles(vault.path, pluginFiles),
|
||||||
|
Object.keys(pluginFiles).sort()
|
||||||
|
);
|
||||||
|
|
||||||
|
const secondPass = await evaluate<string[]>(`${configCode}${settleCode}
|
||||||
|
await addOn.updatePluginList(false);await settle();${sourceRowCode}
|
||||||
|
return JSON.stringify(row.files.map(file=>file.filename).sort());
|
||||||
|
`);
|
||||||
|
record(
|
||||||
|
"#1271 control: a second pass lists all files",
|
||||||
|
secondPass,
|
||||||
|
Object.keys(pluginFiles)
|
||||||
|
.map((path) => path.substring(".obsidian/".length))
|
||||||
|
.sort()
|
||||||
|
);
|
||||||
|
await removeFixtures(vault.path, pluginFiles);
|
||||||
|
await evaluate(`${configCode}${sourceRowCode}return JSON.stringify({applied:await addOn.applyDataV2(row)});`);
|
||||||
|
record(
|
||||||
|
"#1271 control: Apply after a second pass restores all contents",
|
||||||
|
await matchingFiles(vault.path, pluginFiles),
|
||||||
|
Object.keys(pluginFiles).sort()
|
||||||
|
);
|
||||||
|
} catch (error) {
|
||||||
|
report.runtimeError = error instanceof Error ? error.stack : String(error);
|
||||||
|
if (session) report.processOutput = JSON.stringify(session.app.output()).slice(-4_000);
|
||||||
|
throw error;
|
||||||
|
} finally {
|
||||||
|
try {
|
||||||
|
await writeFile(output, JSON.stringify(report, null, 2) + "\n");
|
||||||
|
console.log(`Verification report: ${output}`);
|
||||||
|
} finally {
|
||||||
|
try {
|
||||||
|
if (session) await session.app.stop();
|
||||||
|
} finally {
|
||||||
|
await vault.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (observations.some((observation) => !observation.passed)) process.exitCode = 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((error: unknown) => {
|
||||||
|
console.error(error instanceof Error ? error.stack : error);
|
||||||
|
process.exitCode = 1;
|
||||||
|
});
|
||||||
@@ -1,6 +1,12 @@
|
|||||||
import { mkdir, readFile, rm, utimes, writeFile } from "node:fs/promises";
|
import { mkdir, readFile, rm, utimes, writeFile } from "node:fs/promises";
|
||||||
import { dirname, join } from "node:path";
|
import { dirname, join } from "node:path";
|
||||||
import { VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
import {
|
||||||
|
MODE_AUTOMATIC,
|
||||||
|
VERSIONING_DOCID,
|
||||||
|
type ObsidianLiveSyncSettings,
|
||||||
|
type LoadedEntry,
|
||||||
|
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
|
import { readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||||
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
|
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
|
||||||
import { evalObsidianJson } from "../runner/cli.ts";
|
import { evalObsidianJson } from "../runner/cli.ts";
|
||||||
import {
|
import {
|
||||||
@@ -23,11 +29,12 @@ import {
|
|||||||
prepareRemote,
|
prepareRemote,
|
||||||
pushLocalChanges,
|
pushLocalChanges,
|
||||||
waitForLiveSyncCoreReady,
|
waitForLiveSyncCoreReady,
|
||||||
|
waitForLocalDatabaseEntry,
|
||||||
} from "../runner/liveSyncWorkflow.ts";
|
} from "../runner/liveSyncWorkflow.ts";
|
||||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||||
import { waitForVisibleObsidianDialogue, withObsidianPage } from "../runner/ui.ts";
|
import { waitForVisibleObsidianDialogue, withObsidianPage } from "../runner/ui.ts";
|
||||||
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
||||||
import type { Locator } from "playwright";
|
import type { Locator, Page } from "playwright";
|
||||||
|
|
||||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||||
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
|
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
|
||||||
@@ -59,6 +66,18 @@ const pluginDir = ".obsidian/plugins/livesync-e2e-sample";
|
|||||||
const pluginManifestPath = `${pluginDir}/manifest.json`;
|
const pluginManifestPath = `${pluginDir}/manifest.json`;
|
||||||
const pluginMainPath = `${pluginDir}/main.js`;
|
const pluginMainPath = `${pluginDir}/main.js`;
|
||||||
const pluginStylesPath = `${pluginDir}/styles.css`;
|
const pluginStylesPath = `${pluginDir}/styles.css`;
|
||||||
|
const pluginDisplayName = "LiveSync E2E Sample";
|
||||||
|
const pluginDataPath = `${pluginDir}/data.json`;
|
||||||
|
const pluginDataContent = JSON.stringify({ source: "customisation-sync-a", enabled: true }, null, 4) + "\n";
|
||||||
|
const targetPluginDataContent = JSON.stringify({ source: "customisation-sync-b", enabled: true }, null, 4) + "\n";
|
||||||
|
const existingAutomaticKey = "SNIPPET/livesync-e2e-existing-automatic.css";
|
||||||
|
const existingAutomaticSelections: ObsidianLiveSyncSettings["pluginSyncExtendedSetting"] = {
|
||||||
|
[existingAutomaticKey]: {
|
||||||
|
key: existingAutomaticKey,
|
||||||
|
mode: MODE_AUTOMATIC,
|
||||||
|
files: ["snippets/livesync-e2e-existing-automatic.css"],
|
||||||
|
},
|
||||||
|
};
|
||||||
const identicalSnippetPath = ".obsidian/snippets/livesync-customisation-e2e-identical.css";
|
const identicalSnippetPath = ".obsidian/snippets/livesync-customisation-e2e-identical.css";
|
||||||
const identicalSnippetContent = ".livesync-customisation-identical { color: #3d6f54; }\n";
|
const identicalSnippetContent = ".livesync-customisation-identical { color: #3d6f54; }\n";
|
||||||
const differentSnippetPath = ".obsidian/snippets/livesync-customisation-e2e-different.css";
|
const differentSnippetPath = ".obsidian/snippets/livesync-customisation-e2e-different.css";
|
||||||
@@ -104,7 +123,8 @@ const pluginFixtures = [
|
|||||||
{ path: pluginStylesPath, source: pluginStylesContent, target: targetPluginStylesContent },
|
{ path: pluginStylesPath, source: pluginStylesContent, target: targetPluginStylesContent },
|
||||||
] as const;
|
] as const;
|
||||||
|
|
||||||
type CustomisationSyncCase = "all" | "visibility" | "mtime";
|
type CustomisationSyncCase = "all" | "visibility" | "mtime" | "automatic";
|
||||||
|
type AutomaticInitialAction = "Overwrite Remote" | "Overwrite Local" | "Use newer";
|
||||||
|
|
||||||
type CustomisationSyncTestGlobal = typeof globalThis & {
|
type CustomisationSyncTestGlobal = typeof globalThis & {
|
||||||
app?: { commands?: { executeCommandById(commandId: string): boolean } };
|
app?: { commands?: { executeCommandById(commandId: string): boolean } };
|
||||||
@@ -131,6 +151,14 @@ type CustomisationScanResult = {
|
|||||||
files: string[];
|
files: string[];
|
||||||
};
|
};
|
||||||
|
|
||||||
|
type AutomaticSyncSnapshot = {
|
||||||
|
syncInternalFiles: boolean;
|
||||||
|
pluginSyncExtendedSetting: Record<string, { key: string; mode: number; files: string[] }>;
|
||||||
|
hiddenFiles: { path: string; rev: string; deleted: boolean }[];
|
||||||
|
processedFiles: [string, string | undefined][];
|
||||||
|
processedDatabaseFiles: [string, string | undefined][];
|
||||||
|
};
|
||||||
|
|
||||||
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
|
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
|
||||||
const fullPath = join(vaultPath, path);
|
const fullPath = join(vaultPath, path);
|
||||||
await mkdir(dirname(fullPath), { recursive: true });
|
await mkdir(dirname(fullPath), { recursive: true });
|
||||||
@@ -147,10 +175,11 @@ function selectedCase(): CustomisationSyncCase {
|
|||||||
if (args.length === 0) return "all";
|
if (args.length === 0) return "all";
|
||||||
const arg = args[0];
|
const arg = args[0];
|
||||||
if (args.length !== 1 || !arg?.startsWith("--case=")) {
|
if (args.length !== 1 || !arg?.startsWith("--case=")) {
|
||||||
throw new Error("Usage: test:e2e:obsidian:customisation-sync [--case=all|visibility|mtime]");
|
throw new Error("Usage: test:e2e:obsidian:customisation-sync [--case=all|visibility|mtime|automatic]");
|
||||||
}
|
}
|
||||||
const selected = arg.slice("--case=".length);
|
const selected = arg.slice("--case=".length);
|
||||||
if (selected === "all" || selected === "visibility" || selected === "mtime") return selected;
|
if (selected === "all" || selected === "visibility" || selected === "mtime" || selected === "automatic")
|
||||||
|
return selected;
|
||||||
throw new Error(`Unknown Customisation Sync E2E case: ${selected}`);
|
throw new Error(`Unknown Customisation Sync E2E case: ${selected}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -244,6 +273,282 @@ async function inspectPluginFreshnessAndNewestSelection(session: ObsidianLiveSyn
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function openAutomaticModeForPlugin(page: Page, pluginName: string): Promise<Locator> {
|
||||||
|
const opened = await page.evaluate(() =>
|
||||||
|
(globalThis as CustomisationSyncTestGlobal).app?.commands?.executeCommandById(
|
||||||
|
"obsidian-livesync:livesync-plugin-dialog-ex"
|
||||||
|
)
|
||||||
|
);
|
||||||
|
if (opened !== true) throw new Error("Could not open the Customisation Sync dialogue command.");
|
||||||
|
const dialogue = await waitForVisibleObsidianDialogue(page, "Customization Sync (Beta3)");
|
||||||
|
const pluginRow = dialogue.locator(".labelrow").filter({ hasText: pluginName }).first();
|
||||||
|
await pluginRow.waitFor({ state: "visible", timeout: 10000 });
|
||||||
|
await pluginRow.locator("button.status").click();
|
||||||
|
const automaticOption = page.locator(".menu-item").filter({ hasText: "Automatic" }).last();
|
||||||
|
await automaticOption.waitFor({ state: "visible", timeout: 10000 });
|
||||||
|
await automaticOption.click();
|
||||||
|
return dialogue;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function selectAutomaticInitialAction(page: Page, label: AutomaticInitialAction): Promise<void> {
|
||||||
|
const action = page.locator(".menu-item").filter({ hasText: label }).last();
|
||||||
|
await action.waitFor({ state: "visible", timeout: 10000 });
|
||||||
|
await action.click();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readAutomaticSyncSnapshot(cliBinary: string, env: NodeJS.ProcessEnv): Promise<AutomaticSyncSnapshot> {
|
||||||
|
return await evalObsidianJson<AutomaticSyncSnapshot>(
|
||||||
|
cliBinary,
|
||||||
|
[
|
||||||
|
"(async()=>{",
|
||||||
|
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||||
|
"const hidden=core.getAddOn('HiddenFileSync');",
|
||||||
|
"const files=await hidden.getAllDatabaseFiles();",
|
||||||
|
"const processed=(cache)=>Array.from(cache.keys()).sort().map((path)=>[path,cache.get(path)]);",
|
||||||
|
"return JSON.stringify({",
|
||||||
|
"syncInternalFiles:core.settings.syncInternalFiles,",
|
||||||
|
"pluginSyncExtendedSetting:core.settings.pluginSyncExtendedSetting,",
|
||||||
|
"hiddenFiles:files.map((entry)=>{const path=hidden.getPath(entry);return {path:path.startsWith('i:')?path.slice(2):path,rev:entry._rev??'',deleted:!!(entry.deleted||entry._deleted)};}).sort((a,b)=>a.path.localeCompare(b.path)),",
|
||||||
|
"processedFiles:processed(hidden._fileInfoLastProcessed),",
|
||||||
|
"processedDatabaseFiles:processed(hidden._databaseInfoLastProcessed),",
|
||||||
|
"});",
|
||||||
|
"})()",
|
||||||
|
].join(""),
|
||||||
|
env
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readAutomaticFileContents(context: RunnerContext, session: ObsidianLiveSyncSession) {
|
||||||
|
const files = await evalObsidianJson<
|
||||||
|
{ path: string; entry: LoadedEntry | false; local: string; localMtime: number }[]
|
||||||
|
>(
|
||||||
|
context.cliBinary,
|
||||||
|
[
|
||||||
|
"(async()=>{",
|
||||||
|
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||||
|
`const paths=${JSON.stringify([pluginStylesPath, pluginDataPath])};`,
|
||||||
|
"const files=await Promise.all(paths.map(async(path)=>{",
|
||||||
|
"const entry=await core.localDatabase.getDBEntry('i:'+path,undefined,false,true);",
|
||||||
|
"const stat=await app.vault.adapter.stat(path);",
|
||||||
|
"if(!stat) throw new Error('Missing Automatic fixture file: '+path);",
|
||||||
|
"return {path,entry,local:await app.vault.adapter.read(path),localMtime:stat.mtime};",
|
||||||
|
"}));",
|
||||||
|
"return JSON.stringify(files);",
|
||||||
|
"})()",
|
||||||
|
].join(""),
|
||||||
|
session.cliEnv
|
||||||
|
);
|
||||||
|
return await Promise.all(
|
||||||
|
files.map(async (file) => ({
|
||||||
|
path: file.path,
|
||||||
|
stored: file.entry ? await readAsBlob(file.entry).text() : null,
|
||||||
|
storedMtime: file.entry ? file.entry.mtime : 0,
|
||||||
|
local: file.local,
|
||||||
|
localMtime: file.localMtime,
|
||||||
|
}))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function setHiddenSyncEnabledForFixture(
|
||||||
|
cliBinary: string,
|
||||||
|
env: NodeJS.ProcessEnv,
|
||||||
|
enabled: boolean
|
||||||
|
): Promise<void> {
|
||||||
|
await evalObsidianJson<unknown>(
|
||||||
|
cliBinary,
|
||||||
|
[
|
||||||
|
"(async()=>{",
|
||||||
|
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||||
|
`core.settings.syncInternalFiles=${JSON.stringify(enabled)};`,
|
||||||
|
"return JSON.stringify({syncInternalFiles:core.settings.syncInternalFiles});",
|
||||||
|
"})()",
|
||||||
|
].join(""),
|
||||||
|
env
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function inspectAutomaticSyncSelection(
|
||||||
|
context: RunnerContext,
|
||||||
|
session: ObsidianLiveSyncSession,
|
||||||
|
vault: TemporaryVault,
|
||||||
|
pluginName: string,
|
||||||
|
initialAction: AutomaticInitialAction
|
||||||
|
): Promise<void> {
|
||||||
|
const noticeText =
|
||||||
|
"Hidden File Sync is disabled on this device. Enable it in Sync Settings before using Automatic or initialising hidden files.";
|
||||||
|
const trackedFiles = [pluginStylesPath, pluginDataPath];
|
||||||
|
const contentsBefore = await Promise.all(trackedFiles.map((path) => readVaultFile(vault.path, path)));
|
||||||
|
const disabledBefore = await readAutomaticSyncSnapshot(context.cliBinary, session.cliEnv);
|
||||||
|
assertEqual(
|
||||||
|
JSON.stringify(disabledBefore.pluginSyncExtendedSetting[existingAutomaticKey]),
|
||||||
|
JSON.stringify(existingAutomaticSelections[existingAutomaticKey]),
|
||||||
|
"The fixture did not retain the saved Automatic selection while Hidden File Sync was disabled."
|
||||||
|
);
|
||||||
|
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
const dialogue = await openAutomaticModeForPlugin(page, pluginDisplayName);
|
||||||
|
const notice = page.locator(".notice:visible").filter({ hasText: noticeText }).last();
|
||||||
|
await notice.waitFor({ state: "visible", timeout: 10000 });
|
||||||
|
const initialActionMenu = page.locator(".menu:visible").filter({ hasText: "Initial Action" });
|
||||||
|
if ((await initialActionMenu.count()) !== 0) {
|
||||||
|
throw new Error("Automatic displayed an Initial Action menu while Hidden File Sync was disabled.");
|
||||||
|
}
|
||||||
|
await page.keyboard.press("Escape");
|
||||||
|
await dialogue.waitFor({ state: "hidden", timeout: 10000 });
|
||||||
|
await notice.click({ force: true });
|
||||||
|
await notice.waitFor({ state: "hidden", timeout: 10000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
const disabledAfter = await readAutomaticSyncSnapshot(context.cliBinary, session.cliEnv);
|
||||||
|
const contentsAfter = await Promise.all(trackedFiles.map((path) => readVaultFile(vault.path, path)));
|
||||||
|
if (JSON.stringify(disabledAfter) !== JSON.stringify(disabledBefore)) {
|
||||||
|
throw new Error(
|
||||||
|
`Automatic changed Hidden File Sync state while it was disabled: ${JSON.stringify({
|
||||||
|
before: disabledBefore,
|
||||||
|
after: disabledAfter,
|
||||||
|
})}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assertEqual(
|
||||||
|
JSON.stringify(contentsAfter),
|
||||||
|
JSON.stringify(contentsBefore),
|
||||||
|
"Automatic changed plug-in files while Hidden File Sync was disabled."
|
||||||
|
);
|
||||||
|
|
||||||
|
// Exercise the second guard: the user may open Initial Action while enabled,
|
||||||
|
// then disable Hidden File Sync before choosing a direction.
|
||||||
|
await setHiddenSyncEnabledForFixture(context.cliBinary, session.cliEnv, true);
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
await openAutomaticModeForPlugin(page, pluginDisplayName);
|
||||||
|
const initialActionMenu = page.locator(".menu:visible").filter({ hasText: "Initial Action" });
|
||||||
|
await initialActionMenu.waitFor({ state: "visible", timeout: 10000 });
|
||||||
|
});
|
||||||
|
await setHiddenSyncEnabledForFixture(context.cliBinary, session.cliEnv, false);
|
||||||
|
const beforeDisabledAction = await readAutomaticSyncSnapshot(context.cliBinary, session.cliEnv);
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
await selectAutomaticInitialAction(page, "Overwrite Remote");
|
||||||
|
const notice = page.locator(".notice:visible").filter({ hasText: noticeText }).last();
|
||||||
|
await notice.waitFor({ state: "visible", timeout: 10000 });
|
||||||
|
await notice.click({ force: true });
|
||||||
|
await notice.waitFor({ state: "hidden", timeout: 10000 });
|
||||||
|
});
|
||||||
|
const afterDisabledAction = await readAutomaticSyncSnapshot(context.cliBinary, session.cliEnv);
|
||||||
|
if (JSON.stringify(afterDisabledAction) !== JSON.stringify(beforeDisabledAction)) {
|
||||||
|
throw new Error(
|
||||||
|
`Automatic changed Hidden File Sync state after it was disabled from Initial Action: ${JSON.stringify({
|
||||||
|
before: beforeDisabledAction,
|
||||||
|
after: afterDisabledAction,
|
||||||
|
})}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assertEqual(
|
||||||
|
JSON.stringify(await Promise.all(trackedFiles.map((path) => readVaultFile(vault.path, path)))),
|
||||||
|
JSON.stringify(contentsBefore),
|
||||||
|
"Automatic changed plug-in files after Hidden File Sync was disabled from Initial Action."
|
||||||
|
);
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
await page.keyboard.press("Escape");
|
||||||
|
});
|
||||||
|
|
||||||
|
await setHiddenSyncEnabledForFixture(context.cliBinary, session.cliEnv, true);
|
||||||
|
if (initialAction !== "Overwrite Remote") {
|
||||||
|
const fixtures = [
|
||||||
|
{
|
||||||
|
path: pluginStylesPath,
|
||||||
|
stored: pluginStylesContent,
|
||||||
|
local: targetPluginStylesContent,
|
||||||
|
storedIsNewer: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
path: pluginDataPath,
|
||||||
|
stored: pluginDataContent,
|
||||||
|
local: targetPluginDataContent,
|
||||||
|
storedIsNewer: false,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
const actualFiles = await readAutomaticFileContents(context, session);
|
||||||
|
for (const fixture of fixtures) {
|
||||||
|
const actual = actualFiles.find((file) => file.path === fixture.path);
|
||||||
|
if (
|
||||||
|
!actual ||
|
||||||
|
actual.stored !== fixture.stored ||
|
||||||
|
actual.local !== fixture.local ||
|
||||||
|
actual.storedMtime > actual.localMtime !== fixture.storedIsNewer ||
|
||||||
|
actual.storedMtime === actual.localMtime
|
||||||
|
) {
|
||||||
|
throw new Error(
|
||||||
|
`Automatic direction fixture did not retain distinct contents and modification times: ${JSON.stringify(actual)}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
const dialogue = await openAutomaticModeForPlugin(page, pluginDisplayName);
|
||||||
|
await selectAutomaticInitialAction(page, initialAction);
|
||||||
|
for (const label of ["MAIN", "DATA"]) {
|
||||||
|
const row = dialogue.locator(".filerow").filter({ hasText: label }).first();
|
||||||
|
await row.locator(".statusnote").filter({ hasText: "Automatic" }).waitFor({ state: "visible" });
|
||||||
|
}
|
||||||
|
await page.keyboard.press("Escape");
|
||||||
|
await dialogue.waitFor({ state: "hidden", timeout: 10000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
for (const path of [pluginManifestPath, pluginMainPath, pluginStylesPath, pluginDataPath]) {
|
||||||
|
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
|
||||||
|
}
|
||||||
|
const enabled = await readAutomaticSyncSnapshot(context.cliBinary, session.cliEnv);
|
||||||
|
const mainMode = enabled.pluginSyncExtendedSetting[`PLUGIN_MAIN/${pluginName}`];
|
||||||
|
const dataMode = enabled.pluginSyncExtendedSetting[`PLUGIN_DATA/${pluginName}`];
|
||||||
|
if (mainMode?.mode !== MODE_AUTOMATIC || dataMode?.mode !== MODE_AUTOMATIC) {
|
||||||
|
throw new Error(
|
||||||
|
`Automatic did not save both plug-in group child settings: ${JSON.stringify({ mainMode, dataMode })}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assertEqual(
|
||||||
|
JSON.stringify([...mainMode.files].sort()),
|
||||||
|
JSON.stringify([
|
||||||
|
`plugins/${pluginName}/main.js`,
|
||||||
|
`plugins/${pluginName}/manifest.json`,
|
||||||
|
`plugins/${pluginName}/styles.css`,
|
||||||
|
]),
|
||||||
|
"Automatic did not select the PLUGIN_MAIN files."
|
||||||
|
);
|
||||||
|
assertEqual(
|
||||||
|
JSON.stringify([...dataMode.files].sort()),
|
||||||
|
JSON.stringify([`plugins/${pluginName}/data.json`]),
|
||||||
|
"Automatic did not select the PLUGIN_DATA file."
|
||||||
|
);
|
||||||
|
assertEqual(
|
||||||
|
enabled.hiddenFiles.filter((entry) => entry.path.startsWith(pluginDir + "/")).length,
|
||||||
|
4,
|
||||||
|
"Automatic did not initialise all files in the PLUGIN_ALL group."
|
||||||
|
);
|
||||||
|
const expectedFiles = [
|
||||||
|
{
|
||||||
|
path: pluginStylesPath,
|
||||||
|
content: initialAction === "Overwrite Remote" ? targetPluginStylesContent : pluginStylesContent,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
path: pluginDataPath,
|
||||||
|
content: initialAction === "Overwrite Local" ? pluginDataContent : targetPluginDataContent,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
const deadline = Date.now() + 15000;
|
||||||
|
let actualFiles = await readAutomaticFileContents(context, session);
|
||||||
|
const contentsMatch = () =>
|
||||||
|
actualFiles.every(
|
||||||
|
(file, index) => file.stored === expectedFiles[index].content && file.local === expectedFiles[index].content
|
||||||
|
);
|
||||||
|
while (!contentsMatch() && Date.now() < deadline) {
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||||
|
actualFiles = await readAutomaticFileContents(context, session);
|
||||||
|
}
|
||||||
|
if (!contentsMatch()) {
|
||||||
|
throw new Error(`${initialAction} did not apply both MAIN and DATA: ${JSON.stringify(actualFiles)}`);
|
||||||
|
}
|
||||||
|
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||||
|
}
|
||||||
|
|
||||||
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
|
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
|
||||||
await rm(join(vaultPath, path), { force: true });
|
await rm(join(vaultPath, path), { force: true });
|
||||||
}
|
}
|
||||||
@@ -287,7 +592,8 @@ async function waitForPathContent(
|
|||||||
async function startConfiguredSession(
|
async function startConfiguredSession(
|
||||||
context: RunnerContext,
|
context: RunnerContext,
|
||||||
vault: TemporaryVault,
|
vault: TemporaryVault,
|
||||||
deviceName: string
|
deviceName: string,
|
||||||
|
settings: Partial<Pick<ObsidianLiveSyncSettings, "syncInternalFiles" | "pluginSyncExtendedSetting">> = {}
|
||||||
): Promise<ObsidianLiveSyncSession> {
|
): Promise<ObsidianLiveSyncSession> {
|
||||||
const couchDbSettings = {
|
const couchDbSettings = {
|
||||||
uri: context.couchDb.uri,
|
uri: context.couchDb.uri,
|
||||||
@@ -306,6 +612,7 @@ async function startConfiguredSession(
|
|||||||
autoSweepPlugins: false,
|
autoSweepPlugins: false,
|
||||||
autoSweepPluginsPeriodic: false,
|
autoSweepPluginsPeriodic: false,
|
||||||
syncInternalFiles: false,
|
syncInternalFiles: false,
|
||||||
|
...settings,
|
||||||
};
|
};
|
||||||
const session = await startObsidianLiveSyncSession({
|
const session = await startObsidianLiveSyncSession({
|
||||||
binary: context.binary,
|
binary: context.binary,
|
||||||
@@ -319,6 +626,18 @@ async function startConfiguredSession(
|
|||||||
});
|
});
|
||||||
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
||||||
await assertE2eCompatibilityMarker(context.cliBinary, session.cliEnv);
|
await assertE2eCompatibilityMarker(context.cliBinary, session.cliEnv);
|
||||||
|
if (Object.keys(settings.pluginSyncExtendedSetting ?? {}).length > 0) {
|
||||||
|
const restoredSelections = await evalObsidianJson<ObsidianLiveSyncSettings["pluginSyncExtendedSetting"]>(
|
||||||
|
context.cliBinary,
|
||||||
|
"JSON.stringify(app.plugins.plugins['obsidian-livesync'].core.settings.pluginSyncExtendedSetting)",
|
||||||
|
session.cliEnv
|
||||||
|
);
|
||||||
|
assertEqual(
|
||||||
|
JSON.stringify(restoredSelections),
|
||||||
|
JSON.stringify(settings.pluginSyncExtendedSetting),
|
||||||
|
"The saved Automatic selection changed during start-up."
|
||||||
|
);
|
||||||
|
}
|
||||||
await configureCouchDb(context.cliBinary, session.cliEnv, couchDbSettings, customisationSettings);
|
await configureCouchDb(context.cliBinary, session.cliEnv, couchDbSettings, customisationSettings);
|
||||||
await evalObsidianJson<unknown>(
|
await evalObsidianJson<unknown>(
|
||||||
context.cliBinary,
|
context.cliBinary,
|
||||||
@@ -389,7 +708,7 @@ async function storeCustomisationFile(cliBinary: string, env: NodeJS.ProcessEnv,
|
|||||||
async function deleteCustomisationSyncEntry(
|
async function deleteCustomisationSyncEntry(
|
||||||
cliBinary: string,
|
cliBinary: string,
|
||||||
env: NodeJS.ProcessEnv,
|
env: NodeJS.ProcessEnv,
|
||||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_DATA",
|
||||||
name: string,
|
name: string,
|
||||||
term?: string
|
term?: string
|
||||||
): Promise<void> {
|
): Promise<void> {
|
||||||
@@ -418,7 +737,7 @@ async function deleteCustomisationSyncEntry(
|
|||||||
async function waitForCustomisationEntry(
|
async function waitForCustomisationEntry(
|
||||||
cliBinary: string,
|
cliBinary: string,
|
||||||
env: NodeJS.ProcessEnv,
|
env: NodeJS.ProcessEnv,
|
||||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_DATA",
|
||||||
name: string,
|
name: string,
|
||||||
term?: string,
|
term?: string,
|
||||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
|
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
|
||||||
@@ -430,7 +749,7 @@ async function waitForCustomisationEntry(
|
|||||||
async function waitForCustomisationEntries(
|
async function waitForCustomisationEntries(
|
||||||
cliBinary: string,
|
cliBinary: string,
|
||||||
env: NodeJS.ProcessEnv,
|
env: NodeJS.ProcessEnv,
|
||||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_DATA",
|
||||||
name: string,
|
name: string,
|
||||||
count: number,
|
count: number,
|
||||||
term?: string,
|
term?: string,
|
||||||
@@ -570,10 +889,13 @@ async function applyRemoteCustomisationGroup(
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
async function main(): Promise<void> {
|
async function runWorkflow(
|
||||||
const testCase = selectedCase();
|
testCase: CustomisationSyncCase,
|
||||||
|
initialAction: AutomaticInitialAction = "Overwrite Remote"
|
||||||
|
): Promise<void> {
|
||||||
const testVisibility = testCase === "all" || testCase === "visibility";
|
const testVisibility = testCase === "all" || testCase === "visibility";
|
||||||
const testMtime = testCase === "all" || testCase === "mtime";
|
const testMtime = testCase === "all" || testCase === "mtime";
|
||||||
|
const testAutomatic = testCase === "automatic";
|
||||||
const binary = requireObsidianBinary();
|
const binary = requireObsidianBinary();
|
||||||
const cli = discoverObsidianCli();
|
const cli = discoverObsidianCli();
|
||||||
if (!cli.binary) {
|
if (!cli.binary) {
|
||||||
@@ -604,6 +926,11 @@ async function main(): Promise<void> {
|
|||||||
await writeVaultFile(vaultA.path, pluginManifestPath, pluginManifestContent);
|
await writeVaultFile(vaultA.path, pluginManifestPath, pluginManifestContent);
|
||||||
await writeVaultFile(vaultA.path, pluginMainPath, pluginMainContent);
|
await writeVaultFile(vaultA.path, pluginMainPath, pluginMainContent);
|
||||||
await writeVaultFile(vaultA.path, pluginStylesPath, pluginStylesContent);
|
await writeVaultFile(vaultA.path, pluginStylesPath, pluginStylesContent);
|
||||||
|
if (testAutomatic) {
|
||||||
|
await writeVaultFile(vaultA.path, pluginDataPath, pluginDataContent);
|
||||||
|
await setVaultFileMtime(vaultA.path, pluginStylesPath, targetPluginMtime);
|
||||||
|
await setVaultFileMtime(vaultA.path, pluginDataPath, sourcePluginMtime);
|
||||||
|
}
|
||||||
if (testVisibility) {
|
if (testVisibility) {
|
||||||
for (const fixture of visibilityFixtures) {
|
for (const fixture of visibilityFixtures) {
|
||||||
await writeVaultFile(vaultA.path, fixture.path, fixture.source);
|
await writeVaultFile(vaultA.path, fixture.path, fixture.source);
|
||||||
@@ -627,6 +954,14 @@ async function main(): Promise<void> {
|
|||||||
await setVaultFileMtime(vaultB.path, fixture.path, targetPluginMtime);
|
await setVaultFileMtime(vaultB.path, fixture.path, targetPluginMtime);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if (testAutomatic) {
|
||||||
|
for (const fixture of pluginFixtures) {
|
||||||
|
await writeVaultFile(vaultB.path, fixture.path, fixture.target);
|
||||||
|
}
|
||||||
|
await writeVaultFile(vaultB.path, pluginDataPath, targetPluginDataContent);
|
||||||
|
await setVaultFileMtime(vaultB.path, pluginStylesPath, sourcePluginMtime);
|
||||||
|
await setVaultFileMtime(vaultB.path, pluginDataPath, targetPluginMtime);
|
||||||
|
}
|
||||||
|
|
||||||
let session = await startConfiguredSession(context, vaultA, sourceDeviceName);
|
let session = await startConfiguredSession(context, vaultA, sourceDeviceName);
|
||||||
const scanResult = await scanCustomisations(context.cliBinary, session.cliEnv);
|
const scanResult = await scanCustomisations(context.cliBinary, session.cliEnv);
|
||||||
@@ -636,6 +971,9 @@ async function main(): Promise<void> {
|
|||||||
for (const fixture of pluginFixtures) {
|
for (const fixture of pluginFixtures) {
|
||||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, fixture.path);
|
await storeCustomisationFile(context.cliBinary, session.cliEnv, fixture.path);
|
||||||
}
|
}
|
||||||
|
if (testAutomatic) {
|
||||||
|
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginDataPath);
|
||||||
|
}
|
||||||
if (testVisibility) {
|
if (testVisibility) {
|
||||||
for (const fixture of visibilityFixtures) {
|
for (const fixture of visibilityFixtures) {
|
||||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, fixture.path);
|
await storeCustomisationFile(context.cliBinary, session.cliEnv, fixture.path);
|
||||||
@@ -650,15 +988,35 @@ async function main(): Promise<void> {
|
|||||||
pluginName,
|
pluginName,
|
||||||
3
|
3
|
||||||
);
|
);
|
||||||
|
const pluginDataEntry = testAutomatic
|
||||||
|
? await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "PLUGIN_DATA", pluginName)
|
||||||
|
: undefined;
|
||||||
|
if (testAutomatic && initialAction !== "Overwrite Remote") {
|
||||||
|
await setHiddenSyncEnabledForFixture(context.cliBinary, session.cliEnv, true);
|
||||||
|
await evalObsidianJson<unknown>(
|
||||||
|
context.cliBinary,
|
||||||
|
[
|
||||||
|
"(async()=>{",
|
||||||
|
"const hidden=app.plugins.plugins['obsidian-livesync'].core.getAddOn('HiddenFileSync');",
|
||||||
|
`const paths=${JSON.stringify([pluginManifestPath, pluginMainPath, pluginStylesPath, pluginDataPath])};`,
|
||||||
|
"const result=await hidden.initialiseInternalFileSync('pushForce',false,paths);",
|
||||||
|
"if(result!==true) throw new Error('Could not prepare source hidden-file documents.');",
|
||||||
|
"return JSON.stringify({initialised:result});",
|
||||||
|
"})()",
|
||||||
|
].join(""),
|
||||||
|
session.cliEnv
|
||||||
|
);
|
||||||
|
await setHiddenSyncEnabledForFixture(context.cliBinary, session.cliEnv, false);
|
||||||
|
}
|
||||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||||
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
|
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
|
||||||
const ids = new Set(docs.map((doc) => doc._id));
|
const ids = new Set(docs.map((doc) => doc._id));
|
||||||
const entries = [entry, configEntry, ...pluginEntries];
|
const entries = [entry, configEntry, ...pluginEntries, ...(pluginDataEntry ? [pluginDataEntry] : [])];
|
||||||
return entries.every(
|
return entries.every(
|
||||||
(target) => ids.has(target.id) && target.children.every((childId) => ids.has(childId))
|
(target) => ids.has(target.id) && target.children.every((childId) => ids.has(childId))
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
for (const target of [entry, configEntry, ...pluginEntries]) {
|
for (const target of [entry, configEntry, ...pluginEntries, ...(pluginDataEntry ? [pluginDataEntry] : [])]) {
|
||||||
const remoteEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, target.id);
|
const remoteEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, target.id);
|
||||||
if (
|
if (
|
||||||
!remoteEntry.path?.startsWith("/\\:") ||
|
!remoteEntry.path?.startsWith("/\\:") ||
|
||||||
@@ -678,8 +1036,15 @@ async function main(): Promise<void> {
|
|||||||
throw new Error("The remote feature list does not declare encrypted internal Metadata.");
|
throw new Error("The remote feature list does not declare encrypted internal Metadata.");
|
||||||
}
|
}
|
||||||
await session.app.stop();
|
await session.app.stop();
|
||||||
|
if (testAutomatic && initialAction === "Overwrite Remote") {
|
||||||
|
for (const path of [pluginManifestPath, pluginMainPath, pluginStylesPath, pluginDataPath]) {
|
||||||
|
await removeVaultFile(vaultA.path, path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
session = await startConfiguredSession(context, vaultB, targetDeviceName);
|
session = await startConfiguredSession(context, vaultB, targetDeviceName, {
|
||||||
|
pluginSyncExtendedSetting: testAutomatic ? existingAutomaticSelections : {},
|
||||||
|
});
|
||||||
if (testVisibility) {
|
if (testVisibility) {
|
||||||
for (const fixture of visibilityFixtures) {
|
for (const fixture of visibilityFixtures) {
|
||||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, fixture.path);
|
await storeCustomisationFile(context.cliBinary, session.cliEnv, fixture.path);
|
||||||
@@ -692,6 +1057,15 @@ async function main(): Promise<void> {
|
|||||||
}
|
}
|
||||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||||
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
|
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
|
||||||
|
if (testAutomatic) {
|
||||||
|
await waitForCustomisationEntry(
|
||||||
|
context.cliBinary,
|
||||||
|
session.cliEnv,
|
||||||
|
"PLUGIN_DATA",
|
||||||
|
pluginName,
|
||||||
|
sourceDeviceName
|
||||||
|
);
|
||||||
|
}
|
||||||
if (testVisibility) {
|
if (testVisibility) {
|
||||||
for (const fixture of visibilityFixtures) {
|
for (const fixture of visibilityFixtures) {
|
||||||
const name = fixture.path.split("/").pop() ?? fixture.path;
|
const name = fixture.path.split("/").pop() ?? fixture.path;
|
||||||
@@ -721,10 +1095,43 @@ async function main(): Promise<void> {
|
|||||||
try {
|
try {
|
||||||
if (testVisibility) await inspectCustomisationVisibility(session);
|
if (testVisibility) await inspectCustomisationVisibility(session);
|
||||||
if (testMtime) await inspectPluginFreshnessAndNewestSelection(session);
|
if (testMtime) await inspectPluginFreshnessAndNewestSelection(session);
|
||||||
|
if (testAutomatic) {
|
||||||
|
await inspectAutomaticSyncSelection(context, session, vaultB, pluginName, initialAction);
|
||||||
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
await session.app.stop().catch(() => undefined);
|
await session.app.stop().catch(() => undefined);
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
|
if (testAutomatic) {
|
||||||
|
await session.app.stop();
|
||||||
|
const receivingVault = initialAction === "Overwrite Remote" ? vaultA : await createTemporaryVault();
|
||||||
|
session = await startConfiguredSession(context, receivingVault, sourceDeviceName, {
|
||||||
|
syncInternalFiles: true,
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||||
|
const expectedStyles =
|
||||||
|
initialAction === "Overwrite Remote" ? targetPluginStylesContent : pluginStylesContent;
|
||||||
|
const expectedData = initialAction === "Overwrite Local" ? pluginDataContent : targetPluginDataContent;
|
||||||
|
const receivedStyles = await waitForPathContent(
|
||||||
|
receivingVault.path,
|
||||||
|
pluginStylesPath,
|
||||||
|
(content) => content === expectedStyles
|
||||||
|
);
|
||||||
|
const receivedData = await waitForPathContent(
|
||||||
|
receivingVault.path,
|
||||||
|
pluginDataPath,
|
||||||
|
(content) => content === expectedData
|
||||||
|
);
|
||||||
|
assertEqual(receivedStyles, expectedStyles, "Automatic plug-in group lost PLUGIN_MAIN files.");
|
||||||
|
assertEqual(receivedData, expectedData, "Automatic plug-in group lost the PLUGIN_DATA file.");
|
||||||
|
} finally {
|
||||||
|
await session.app.stop();
|
||||||
|
if (receivingVault !== vaultA) await receivingVault.dispose();
|
||||||
|
}
|
||||||
|
console.log(`Automatic ${initialAction} initialised and transferred both plug-in group children.`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (testCase !== "all") {
|
if (testCase !== "all") {
|
||||||
await session.app.stop();
|
await session.app.stop();
|
||||||
console.log(`Customisation Sync ${testCase} regression case passed.`);
|
console.log(`Customisation Sync ${testCase} regression case passed.`);
|
||||||
@@ -841,6 +1248,18 @@ async function main(): Promise<void> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function main(): Promise<void> {
|
||||||
|
const testCase = selectedCase();
|
||||||
|
if (testCase === "automatic") {
|
||||||
|
for (const action of ["Overwrite Remote", "Overwrite Local", "Use newer"] as const) {
|
||||||
|
console.log(`Checking Automatic initial action: ${action}`);
|
||||||
|
await runWorkflow(testCase, action);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
await runWorkflow(testCase);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
main().catch((error: unknown) => {
|
main().catch((error: unknown) => {
|
||||||
console.error(error instanceof Error ? error.stack : error);
|
console.error(error instanceof Error ? error.stack : error);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
|
|||||||
@@ -63,6 +63,21 @@ const manualMergeJsonPath = ".obsidian/livesync-e2e-manual-merge.json";
|
|||||||
const targetPath = ".obsidian/livesync-targeted/only-a.json";
|
const targetPath = ".obsidian/livesync-targeted/only-a.json";
|
||||||
const hiddenFileCliTimeoutMs = Number(process.env.E2E_OBSIDIAN_HIDDEN_FILE_CLI_TIMEOUT_MS ?? 90000);
|
const hiddenFileCliTimeoutMs = Number(process.env.E2E_OBSIDIAN_HIDDEN_FILE_CLI_TIMEOUT_MS ?? 90000);
|
||||||
const hiddenFileInitialisationStateKey = "__livesyncE2EHiddenFileInitialisation";
|
const hiddenFileInitialisationStateKey = "__livesyncE2EHiddenFileInitialisation";
|
||||||
|
const disabledInitialisationProbeKey = "__livesyncE2EDisabledHiddenInitialisation";
|
||||||
|
|
||||||
|
type DisabledInitialisationSnapshot = {
|
||||||
|
syncInternalFiles: boolean;
|
||||||
|
hiddenFiles: { id: string; path: string; rev: string; deleted: boolean }[];
|
||||||
|
processedFiles: [string, string | undefined][];
|
||||||
|
processedDatabaseFiles: [string, string | undefined][];
|
||||||
|
};
|
||||||
|
|
||||||
|
type DisabledInitialisationObservation = {
|
||||||
|
result: boolean | null;
|
||||||
|
before: DisabledInitialisationSnapshot;
|
||||||
|
after: DisabledInitialisationSnapshot;
|
||||||
|
progressNotices: string[];
|
||||||
|
};
|
||||||
|
|
||||||
type RunnerContext = {
|
type RunnerContext = {
|
||||||
binary: string;
|
binary: string;
|
||||||
@@ -133,6 +148,114 @@ async function waitForPathDeleted(
|
|||||||
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
|
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function assertDisabledInitialisationIsNoOp(
|
||||||
|
context: RunnerContext,
|
||||||
|
session: ObsidianLiveSyncSession,
|
||||||
|
vault: TemporaryVault
|
||||||
|
): Promise<void> {
|
||||||
|
const snippetExisted = await pathExists(vault.path, snippetPath);
|
||||||
|
const originalSnippet = snippetExisted ? await readVaultFile(vault.path, snippetPath) : undefined;
|
||||||
|
const originalJson = await readVaultFile(vault.path, mergeJsonPath);
|
||||||
|
const preservedSnippet = "body { --disabled-hidden-initialisation: preserved; }\n";
|
||||||
|
const preservedJson = JSON.stringify({ disabledInitialisation: "preserved" }, null, 4) + "\n";
|
||||||
|
await writeVaultFile(vault.path, snippetPath, preservedSnippet);
|
||||||
|
await writeVaultFile(vault.path, mergeJsonPath, preservedJson);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
await page.evaluate((stateKey) => {
|
||||||
|
const state = { progressNotices: [] as string[] };
|
||||||
|
const observer = new MutationObserver(() => {
|
||||||
|
for (const notice of document.querySelectorAll<HTMLElement>(".notice")) {
|
||||||
|
const text = notice.textContent?.trim() ?? "";
|
||||||
|
if (text.includes("[⚙") && !state.progressNotices.includes(text)) {
|
||||||
|
state.progressNotices.push(text);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
observer.observe(document.body, { childList: true, subtree: true, characterData: true });
|
||||||
|
(globalThis as unknown as Record<string, { progressNotices: string[]; observer: MutationObserver }>)[
|
||||||
|
stateKey
|
||||||
|
] = { ...state, observer };
|
||||||
|
}, disabledInitialisationProbeKey);
|
||||||
|
});
|
||||||
|
|
||||||
|
const observation = await evalObsidianJson<DisabledInitialisationObservation>(
|
||||||
|
context.cliBinary,
|
||||||
|
[
|
||||||
|
"(async()=>{",
|
||||||
|
`const targetFiles=${JSON.stringify([snippetPath, mergeJsonPath])};`,
|
||||||
|
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||||
|
"const addOn=core.getAddOn('HiddenFileSync');",
|
||||||
|
"const snapshot=async()=>{",
|
||||||
|
"const files=await addOn.getAllDatabaseFiles();",
|
||||||
|
"const markers=(cache)=>Array.from(cache.keys()).sort().map((path)=>[path,cache.get(path)]);",
|
||||||
|
"return {",
|
||||||
|
"syncInternalFiles:core.settings.syncInternalFiles,",
|
||||||
|
"hiddenFiles:files.map((entry)=>({id:entry._id,path:addOn.getPath(entry),rev:entry._rev??'',deleted:!!(entry.deleted||entry._deleted)})).sort((a,b)=>a.path.localeCompare(b.path)),",
|
||||||
|
"processedFiles:markers(addOn._fileInfoLastProcessed),",
|
||||||
|
"processedDatabaseFiles:markers(addOn._databaseInfoLastProcessed),",
|
||||||
|
"};};",
|
||||||
|
"const before=await snapshot();",
|
||||||
|
"const result=await addOn.initialiseInternalFileSync('pullForce',true,targetFiles);",
|
||||||
|
"const after=await snapshot();",
|
||||||
|
"return JSON.stringify({result:result===undefined?null:result,before,after});",
|
||||||
|
"})()",
|
||||||
|
].join(""),
|
||||||
|
session.cliEnv,
|
||||||
|
hiddenFileCliTimeoutMs
|
||||||
|
);
|
||||||
|
const progressNotices = await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
|
return await page.evaluate((stateKey) => {
|
||||||
|
const state = (
|
||||||
|
globalThis as unknown as Record<
|
||||||
|
string,
|
||||||
|
{ progressNotices: string[]; observer: MutationObserver } | undefined
|
||||||
|
>
|
||||||
|
)[stateKey];
|
||||||
|
state?.observer.disconnect();
|
||||||
|
return state?.progressNotices ?? [];
|
||||||
|
}, disabledInitialisationProbeKey);
|
||||||
|
});
|
||||||
|
|
||||||
|
const failures: string[] = [];
|
||||||
|
if (observation.result !== false)
|
||||||
|
failures.push(`expected false result, received ${String(observation.result)}`);
|
||||||
|
if (observation.before.syncInternalFiles !== false || observation.after.syncInternalFiles !== false) {
|
||||||
|
failures.push("Hidden File Sync setting changed while initialisation was disabled");
|
||||||
|
}
|
||||||
|
if (JSON.stringify(observation.before) !== JSON.stringify(observation.after)) {
|
||||||
|
failures.push("local hidden database state or processed markers changed");
|
||||||
|
}
|
||||||
|
if ((await readVaultFile(vault.path, snippetPath)) !== preservedSnippet) {
|
||||||
|
failures.push("the existing local snippet was deleted or replaced");
|
||||||
|
}
|
||||||
|
if ((await readVaultFile(vault.path, mergeJsonPath)) !== preservedJson) {
|
||||||
|
failures.push("the existing local JSON file was replaced");
|
||||||
|
}
|
||||||
|
if (progressNotices.length > 0)
|
||||||
|
failures.push(`initialisation created progress Notices: ${JSON.stringify(progressNotices)}`);
|
||||||
|
if (failures.length > 0) {
|
||||||
|
throw new Error(
|
||||||
|
`Disabled Hidden File Sync initialisation was not a no-op: ${failures.join("; ")}\n${JSON.stringify({
|
||||||
|
observation,
|
||||||
|
progressNotices,
|
||||||
|
})}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
console.log(
|
||||||
|
"Disabled Hidden File Sync initialisation returned false without changing hidden files or cached state."
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
await writeVaultFile(vault.path, mergeJsonPath, originalJson);
|
||||||
|
if (snippetExisted && originalSnippet !== undefined) {
|
||||||
|
await writeVaultFile(vault.path, snippetPath, originalSnippet);
|
||||||
|
} else {
|
||||||
|
await removeVaultFile(vault.path, snippetPath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
function hasJsonValues(content: string, values: Record<string, unknown>): boolean {
|
function hasJsonValues(content: string, values: Record<string, unknown>): boolean {
|
||||||
try {
|
try {
|
||||||
const parsed = JSON.parse(content) as Record<string, unknown>;
|
const parsed = JSON.parse(content) as Record<string, unknown>;
|
||||||
@@ -649,6 +772,7 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
|
|||||||
"Transient start-up Notices remained before the Hidden File Sync initialisation check."
|
"Transient start-up Notices remained before the Hidden File Sync initialisation check."
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
await assertDisabledInitialisationIsNoOp(context, session, vault);
|
||||||
await withObsidianPage(port, async (page) => {
|
await withObsidianPage(port, async (page) => {
|
||||||
await page.evaluate((stateKey) => {
|
await page.evaluate((stateKey) => {
|
||||||
const obsidianApp = (globalThis as typeof globalThis & { app: any }).app;
|
const obsidianApp = (globalThis as typeof globalThis & { app: any }).app;
|
||||||
|
|||||||
@@ -0,0 +1,962 @@
|
|||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||||
|
import { createServer } from "node:net";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import type { Page } from "playwright";
|
||||||
|
import {
|
||||||
|
assertCouchDbReachable,
|
||||||
|
createCouchDbDatabase,
|
||||||
|
deleteCouchDbDatabase,
|
||||||
|
loadCouchDbConfig,
|
||||||
|
makeUniqueDatabaseName,
|
||||||
|
} from "../runner/couchdb.ts";
|
||||||
|
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||||
|
import {
|
||||||
|
assertE2eCompatibilityUnpaused,
|
||||||
|
configureCouchDb,
|
||||||
|
createE2eCouchDbPluginData,
|
||||||
|
createE2eObsidianDeviceLocalState,
|
||||||
|
prepareRemote,
|
||||||
|
waitForLiveSyncCoreReady,
|
||||||
|
} from "../runner/liveSyncWorkflow.ts";
|
||||||
|
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||||
|
import { withObsidianPage } from "../runner/ui.ts";
|
||||||
|
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
||||||
|
|
||||||
|
type Mode = "normal" | "ignored-batch-setting" | "enabled-batch";
|
||||||
|
type Slot = { name: string; vault: TemporaryVault; session?: ObsidianLiveSyncSession };
|
||||||
|
type Leaf = { rev: string; ancestry: string[]; logicalDeleted: boolean; pouchDeleted: boolean; body: string | null };
|
||||||
|
type Snapshot = {
|
||||||
|
path: string;
|
||||||
|
rev: string | null;
|
||||||
|
content: string | null;
|
||||||
|
fileMtime: number | null;
|
||||||
|
databaseMtime: number | null;
|
||||||
|
exists: boolean;
|
||||||
|
leaves: Leaf[];
|
||||||
|
waiting: Array<{ path: string; type: string; since: number }>;
|
||||||
|
queued: Array<{ path: string | null; type: string }>;
|
||||||
|
shouldBatchSave: boolean;
|
||||||
|
};
|
||||||
|
type Pair = { a: Page; b: Page };
|
||||||
|
|
||||||
|
const diagnostics = process.env.E2E_OBSIDIAN_DIAGNOSTICS_DIR ?? "/tmp/issue1207-livesync-batch";
|
||||||
|
const artifactRoot = process.env.ISSUE1207_ARTIFACT_ROOT ?? process.cwd();
|
||||||
|
const evidence: Record<string, unknown> = { cases: [], observations: [] };
|
||||||
|
const cases = evidence.cases as Array<Record<string, unknown>>;
|
||||||
|
const observations = evidence.observations as unknown[];
|
||||||
|
const pause = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));
|
||||||
|
const prefix = "issue1207-batch-";
|
||||||
|
|
||||||
|
function check(condition: unknown, message: string): asserts condition {
|
||||||
|
if (!condition) throw new Error(message);
|
||||||
|
}
|
||||||
|
|
||||||
|
function hashes() {
|
||||||
|
return Object.fromEntries(
|
||||||
|
["main.js", "manifest.json", "styles.css"].map((name) => [
|
||||||
|
name,
|
||||||
|
createHash("sha256")
|
||||||
|
.update(new Uint8Array(readFileSync(join(artifactRoot, name))))
|
||||||
|
.digest("hex"),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function availablePort(): Promise<number> {
|
||||||
|
const server = createServer();
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
server.once("error", reject);
|
||||||
|
server.listen(0, "127.0.0.1", resolve);
|
||||||
|
});
|
||||||
|
const address = server.address();
|
||||||
|
check(address && typeof address !== "string", "Missing debug port");
|
||||||
|
await new Promise<void>((resolve, reject) => server.close((error) => (error ? reject(error) : resolve())));
|
||||||
|
return address.port;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function evaluate<T>(page: Page, code: string): Promise<T> {
|
||||||
|
return await page.evaluate(`(async()=>{
|
||||||
|
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||||
|
const manager=core.serviceModules.storageAccess.vaultManager;
|
||||||
|
${code}
|
||||||
|
})()`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function installProbe(page: Page): Promise<void> {
|
||||||
|
await evaluate(
|
||||||
|
page,
|
||||||
|
`
|
||||||
|
if(globalThis.__issue1207Batch) return;
|
||||||
|
const probe={events:[],changes:[]};
|
||||||
|
globalThis.__issue1207Batch=probe;
|
||||||
|
for(const type of ['create','modify','rename','delete']) {
|
||||||
|
app.vault.on(type,(file,oldPath)=>{
|
||||||
|
const path=file.path;
|
||||||
|
if(!path.startsWith(${JSON.stringify(prefix)})) return;
|
||||||
|
const item={type,path,oldPath:oldPath??null,at:Date.now(),content:null};
|
||||||
|
probe.events.push(item);
|
||||||
|
if(type!=='delete' && file.stat) {
|
||||||
|
void app.vault.read(file).then(body=>item.content=body).catch(error=>item.error=String(error));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
probe.feed=core.localDatabase.localDatabase.changes({live:true,since:'now',include_docs:true})
|
||||||
|
.on('change',change=>{
|
||||||
|
if(typeof change.doc?.path==='string' && change.doc.path.startsWith(${JSON.stringify(prefix)})) {
|
||||||
|
probe.changes.push({path:change.doc.path,rev:change.doc._rev,
|
||||||
|
logicalDeleted:!!change.doc.deleted,pouchDeleted:!!change.doc._deleted,at:Date.now()});
|
||||||
|
}
|
||||||
|
}).on('error',error=>probe.changes.push({error:String(error),at:Date.now()}));
|
||||||
|
`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function options(page: Page, mode: Mode, minimum = 2, maximum = 6): Promise<void> {
|
||||||
|
await evaluate(
|
||||||
|
page,
|
||||||
|
`
|
||||||
|
const settings=core.services.setting.currentSettings();
|
||||||
|
Object.assign(settings,{liveSync:true,periodicReplication:false,
|
||||||
|
batchSave:${mode === "ignored-batch-setting"},batchSaveInLiveSync:${mode === "enabled-batch"},
|
||||||
|
batchSaveMinimumDelay:${minimum},batchSaveMaximumDelay:${maximum},deleteMetadataOfDeletedFiles:false});
|
||||||
|
return {liveSync:settings.liveSync,batchSave:settings.batchSave,
|
||||||
|
batchSaveInLiveSync:settings.batchSaveInLiveSync,shouldBatchSave:manager.shouldBatchSave};
|
||||||
|
`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function snapshot(page: Page, path: string): Promise<Snapshot> {
|
||||||
|
return await evaluate<Snapshot>(
|
||||||
|
page,
|
||||||
|
`
|
||||||
|
const path=${JSON.stringify(path)};
|
||||||
|
const id=await core.services.path.path2id(path);
|
||||||
|
const row=(await core.localDatabase.allDocsRaw({keys:[id],include_docs:true})).rows[0];
|
||||||
|
let leaves=[];
|
||||||
|
if(row?.value?.rev) {
|
||||||
|
const rawLeaves=await core.localDatabase.getRaw(id,{open_revs:'all',revs:true});
|
||||||
|
for(const item of rawLeaves) {
|
||||||
|
if(!item.ok) continue;
|
||||||
|
const raw=item.ok;
|
||||||
|
let body=null;
|
||||||
|
if(!raw._deleted) {
|
||||||
|
const entry=await core.localDatabase.getDBEntryFromMeta(raw,false,false,true);
|
||||||
|
if(entry!==false) body=Array.isArray(entry.data)?entry.data.join(''):entry.data;
|
||||||
|
}
|
||||||
|
leaves.push({rev:raw._rev,logicalDeleted:!!raw.deleted,pouchDeleted:!!raw._deleted,body,
|
||||||
|
ancestry:raw._revisions?.ids.map((hash,i)=>(raw._revisions.start-i)+'-'+hash)??[]});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const file=app.vault.getAbstractFileByPath(path);
|
||||||
|
return {path,rev:row?.value?.rev??null,exists:!!file?.stat,
|
||||||
|
fileMtime:file?.stat?.mtime??null,databaseMtime:row?.doc?.mtime??null,
|
||||||
|
content:file?.stat?await app.vault.read(file):null,leaves,
|
||||||
|
waiting:[...manager._waitingMap].map(([key,value])=>({path:key,type:value.type,since:value.since})),
|
||||||
|
queued:manager.bufferedQueuedItems.map(item=>({path:item.args?.file?.path??null,type:item.type})),
|
||||||
|
shouldBatchSave:!!manager.shouldBatchSave};
|
||||||
|
`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function liveLeaves(state: Snapshot) {
|
||||||
|
return state.leaves.filter((leaf) => !leaf.logicalDeleted && !leaf.pouchDeleted);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitFor(
|
||||||
|
page: Page,
|
||||||
|
path: string,
|
||||||
|
predicate: (state: Snapshot) => boolean,
|
||||||
|
timeout = 20000
|
||||||
|
): Promise<Snapshot> {
|
||||||
|
const deadline = Date.now() + timeout;
|
||||||
|
let state: Snapshot;
|
||||||
|
do {
|
||||||
|
state = await snapshot(page, path);
|
||||||
|
if (predicate(state)) return state;
|
||||||
|
await pause(100);
|
||||||
|
} while (Date.now() < deadline);
|
||||||
|
throw new Error(`State did not settle: ${JSON.stringify(state)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function settled(pair: Pair, path: string, body: string): Promise<[Snapshot, Snapshot]> {
|
||||||
|
const ready = (state: Snapshot) =>
|
||||||
|
state.content === body &&
|
||||||
|
liveLeaves(state).length === 1 &&
|
||||||
|
liveLeaves(state)[0].body === body &&
|
||||||
|
state.leaves.length === 1 &&
|
||||||
|
!state.waiting.some((item) => item.path === path);
|
||||||
|
const deadline = Date.now() + 20000;
|
||||||
|
let states: [Snapshot, Snapshot];
|
||||||
|
do {
|
||||||
|
states = await Promise.all([snapshot(pair.a, path), snapshot(pair.b, path)]);
|
||||||
|
// The row, leaves, and storage are read asynchronously. Require each
|
||||||
|
// sample to agree internally, then compare both current devices.
|
||||||
|
if (
|
||||||
|
states.every((state) => ready(state) && state.rev === liveLeaves(state)[0].rev) &&
|
||||||
|
states[0].rev === states[1].rev
|
||||||
|
)
|
||||||
|
return states;
|
||||||
|
await pause(100);
|
||||||
|
} while (Date.now() < deadline);
|
||||||
|
throw new Error(`Devices did not converge: ${JSON.stringify(states)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function create(page: Page, path: string, body: string): Promise<void> {
|
||||||
|
await evaluate(page, `await app.vault.create(${JSON.stringify(path)},${JSON.stringify(body)});`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function modify(page: Page, path: string, body: string): Promise<void> {
|
||||||
|
await evaluate(
|
||||||
|
page,
|
||||||
|
`await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),${JSON.stringify(body)});`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function trace(page: Page, path: string) {
|
||||||
|
return await evaluate(
|
||||||
|
page,
|
||||||
|
`return {
|
||||||
|
events:globalThis.__issue1207Batch.events.filter(item=>item.path===${JSON.stringify(path)}||item.oldPath===${JSON.stringify(path)}),
|
||||||
|
changes:globalThis.__issue1207Batch.changes.filter(item=>item.path===${JSON.stringify(path)}||item.error)};`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main(): Promise<void> {
|
||||||
|
await mkdir(diagnostics, { recursive: true });
|
||||||
|
const binary = requireObsidianBinary();
|
||||||
|
const cli = discoverObsidianCli();
|
||||||
|
check(cli.binary, "Missing Obsidian CLI");
|
||||||
|
const cliBinary = cli.binary;
|
||||||
|
const config = await loadCouchDbConfig(process.env.ISSUE1207_COUCHDB_ENV);
|
||||||
|
check(["127.0.0.1", "localhost"].includes(new URL(config.uri).hostname), "Use a disposable local CouchDB");
|
||||||
|
await assertCouchDbReachable(config);
|
||||||
|
const dbName = makeUniqueDatabaseName(config.dbPrefix, "live-batch");
|
||||||
|
const couch = { ...config, dbName };
|
||||||
|
const overrides = {
|
||||||
|
doctorProcessedVersion: "1.0.0",
|
||||||
|
notifyThresholdOfRemoteStorageSize: -1,
|
||||||
|
liveSync: false,
|
||||||
|
periodicReplication: false,
|
||||||
|
batchSave: false,
|
||||||
|
batchSaveInLiveSync: false,
|
||||||
|
syncOnStart: false,
|
||||||
|
syncOnSave: false,
|
||||||
|
syncOnFileOpen: false,
|
||||||
|
syncOnEditorSave: false,
|
||||||
|
syncAfterMerge: false,
|
||||||
|
checkConflictOnlyOnOpen: true,
|
||||||
|
showMergeDialogOnlyOnActive: true,
|
||||||
|
disableMarkdownAutoMerge: true,
|
||||||
|
resolveConflictsByNewerFile: false,
|
||||||
|
writeDocumentsIfConflicted: false,
|
||||||
|
writeLogToTheFile: false,
|
||||||
|
};
|
||||||
|
const before = hashes();
|
||||||
|
evidence.artefactsBefore = before;
|
||||||
|
evidence.manifest = JSON.parse(await readFile(join(artifactRoot, "manifest.json"), "utf8"));
|
||||||
|
evidence.commonlibVersion = JSON.parse(
|
||||||
|
await readFile("node_modules/@vrtmrz/livesync-commonlib/package.json", "utf8")
|
||||||
|
).version;
|
||||||
|
evidence.sourceBaseRevision = "73a14d4e4b0b414a1acfe170d01500abb3dca2e9";
|
||||||
|
evidence.unreleasedImplementation = true;
|
||||||
|
if (process.env.ISSUE1207_COMMONLIB_ARTEFACT) {
|
||||||
|
evidence.commonlibArtefactSha256 = createHash("sha256")
|
||||||
|
.update(new Uint8Array(readFileSync(process.env.ISSUE1207_COMMONLIB_ARTEFACT)))
|
||||||
|
.digest("hex");
|
||||||
|
}
|
||||||
|
evidence.timing = { minimumSeconds: 2, maximumSeconds: 6, incomingAndManualSeconds: { minimum: 5, maximum: 60 } };
|
||||||
|
const slots: Slot[] = [];
|
||||||
|
let ownedDatabase = false;
|
||||||
|
|
||||||
|
function redact(value: unknown): string {
|
||||||
|
return JSON.stringify(value, null, 2)
|
||||||
|
.split(config.password)
|
||||||
|
.join("<redacted>")
|
||||||
|
.split(Buffer.from(`${config.username}:${config.password}`).toString("base64"))
|
||||||
|
.join("<redacted>");
|
||||||
|
}
|
||||||
|
|
||||||
|
async function start(slot: Slot, restart = false) {
|
||||||
|
console.log(JSON.stringify({ phase: "starting", device: slot.name, restart }));
|
||||||
|
slot.session = await startObsidianLiveSyncSession({
|
||||||
|
binary,
|
||||||
|
cliBinary,
|
||||||
|
artifactRoot,
|
||||||
|
vault: slot.vault,
|
||||||
|
startupGraceMs: 1000,
|
||||||
|
pluginStartup: restart ? "natural" : undefined,
|
||||||
|
pluginData: restart ? undefined : createE2eCouchDbPluginData(couch, overrides),
|
||||||
|
localStorageEntries: restart ? undefined : createE2eObsidianDeviceLocalState(slot.vault.name),
|
||||||
|
env: { E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT: String(await availablePort()) },
|
||||||
|
lifecycle: {
|
||||||
|
beforeLaunch: async () => {
|
||||||
|
for (const registry of [
|
||||||
|
join(slot.vault.homePath, ".config", "obsidian", "obsidian.json"),
|
||||||
|
join(slot.vault.homePath, "Library", "Application Support", "obsidian", "obsidian.json"),
|
||||||
|
join(slot.vault.xdgConfigPath, "obsidian", "obsidian.json"),
|
||||||
|
join(slot.vault.userDataPath, "obsidian.json"),
|
||||||
|
]) {
|
||||||
|
const state = JSON.parse(await readFile(registry, "utf8"));
|
||||||
|
state.updateDisabled = true;
|
||||||
|
await writeFile(registry, JSON.stringify(state));
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
console.log(
|
||||||
|
JSON.stringify({
|
||||||
|
phase: "device-started",
|
||||||
|
device: slot.name,
|
||||||
|
port: slot.session.remoteDebuggingPort,
|
||||||
|
vault: slot.vault.path,
|
||||||
|
})
|
||||||
|
);
|
||||||
|
await waitForLiveSyncCoreReady(cliBinary, slot.session.cliEnv);
|
||||||
|
await assertE2eCompatibilityUnpaused(cliBinary, slot.session.cliEnv, slot.session.remoteDebuggingPort);
|
||||||
|
if (!restart) {
|
||||||
|
await configureCouchDb(cliBinary, slot.session.cliEnv, couch, overrides);
|
||||||
|
await waitForLiveSyncCoreReady(cliBinary, slot.session.cliEnv);
|
||||||
|
await prepareRemote(cliBinary, slot.session.cliEnv);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function stop(slot: Slot) {
|
||||||
|
if (!slot.session) return;
|
||||||
|
await slot.session.app.stop();
|
||||||
|
await writeFile(
|
||||||
|
join(diagnostics, `process-${slot.name}-${slot.session.remoteDebuggingPort}.json`),
|
||||||
|
redact(slot.session.app.output())
|
||||||
|
);
|
||||||
|
slot.session = undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function withPair<T>(callback: (pair: Pair) => Promise<T>): Promise<T> {
|
||||||
|
check(slots[0].session && slots[1].session, "Both devices must be running");
|
||||||
|
return await withObsidianPage(
|
||||||
|
slots[0].session.remoteDebuggingPort,
|
||||||
|
async (a) =>
|
||||||
|
await withObsidianPage(slots[1].session!.remoteDebuggingPort, async (b) => {
|
||||||
|
await installProbe(a);
|
||||||
|
await installProbe(b);
|
||||||
|
return await callback({ a, b });
|
||||||
|
})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function run(name: string, body: () => Promise<unknown>) {
|
||||||
|
if (process.env.ISSUE1207_CASE_FILTER && !new RegExp(process.env.ISSUE1207_CASE_FILTER).test(name)) return;
|
||||||
|
console.log(JSON.stringify({ phase: "case-start", name }));
|
||||||
|
try {
|
||||||
|
const result = await body();
|
||||||
|
cases.push({ name, passed: true, result });
|
||||||
|
console.log(JSON.stringify({ phase: "case-passed", name }));
|
||||||
|
} catch (error) {
|
||||||
|
const failure = error instanceof Error ? error.stack : String(error);
|
||||||
|
cases.push({ name, passed: false, error: failure });
|
||||||
|
console.log(JSON.stringify({ phase: "case-failed", name, error: failure }));
|
||||||
|
await withPair(async (pair) => {
|
||||||
|
let flushFailure: unknown;
|
||||||
|
for (const [device, page] of Object.entries(pair)) {
|
||||||
|
observations.push({
|
||||||
|
name,
|
||||||
|
device,
|
||||||
|
probe: await evaluate(
|
||||||
|
page,
|
||||||
|
`return {
|
||||||
|
events:globalThis.__issue1207Batch.events,changes:globalThis.__issue1207Batch.changes,
|
||||||
|
pending:(await manager.adapter.persistence.loadSnapshot()).map(item=>({
|
||||||
|
type:item.type,path:item.args?.file?.path,oldPath:item.args?.oldPath}))};`
|
||||||
|
),
|
||||||
|
});
|
||||||
|
await page.screenshot({ path: join(diagnostics, `${name}-${device}-failure.png`) });
|
||||||
|
try {
|
||||||
|
await evaluate(page, "await manager.waitForIdle();");
|
||||||
|
} catch (error) {
|
||||||
|
observations.push({ name, device, flushFailure: String(error) });
|
||||||
|
flushFailure ??= error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (flushFailure) throw flushFailure;
|
||||||
|
});
|
||||||
|
await pause(500);
|
||||||
|
}
|
||||||
|
await writeFile(join(diagnostics, "evidence.json"), redact(evidence));
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await createCouchDbDatabase(config, dbName);
|
||||||
|
ownedDatabase = true;
|
||||||
|
for (const name of ["a", "b"]) {
|
||||||
|
const slot = { name, vault: await createTemporaryVault(`issue1207-live-batch-${name}-`) };
|
||||||
|
slots.push(slot);
|
||||||
|
await start(slot);
|
||||||
|
}
|
||||||
|
evidence.runtime = await withPair(async (pair) => {
|
||||||
|
const versions = [];
|
||||||
|
for (const page of [pair.a, pair.b]) {
|
||||||
|
versions.push(
|
||||||
|
await evaluate(
|
||||||
|
page,
|
||||||
|
`let appVersion=null;
|
||||||
|
try {appVersion=JSON.parse(require('fs').readFileSync(require('path').join(process.resourcesPath,'app.asar','package.json'),'utf8')).version;} catch {}
|
||||||
|
return {appVersion,electronVersion:process.versions.electron,
|
||||||
|
pluginVersion:app.plugins.plugins['obsidian-livesync'].manifest.version};`
|
||||||
|
)
|
||||||
|
);
|
||||||
|
await options(page, "normal");
|
||||||
|
const outcome = await evaluate(
|
||||||
|
page,
|
||||||
|
`return await core.services.replication.startContinuous({
|
||||||
|
trigger:'daemon',interaction:{kind:'forbidden'}});`
|
||||||
|
);
|
||||||
|
check(
|
||||||
|
(outcome as { status?: string })?.status === "completed",
|
||||||
|
`Continuous start failed: ${JSON.stringify(outcome)}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return versions;
|
||||||
|
});
|
||||||
|
|
||||||
|
evidence.connectivity = await withPair(async (pair) => {
|
||||||
|
const path = `${prefix}connectivity.md`;
|
||||||
|
await create(pair.a, path, "Connectivity probe\n");
|
||||||
|
const first = await settled(pair, path, "Connectivity probe\n");
|
||||||
|
await modify(pair.b, path, "Bidirectional connectivity probe\n");
|
||||||
|
const second = await settled(pair, path, "Bidirectional connectivity probe\n");
|
||||||
|
return { first, second };
|
||||||
|
});
|
||||||
|
|
||||||
|
for (const mode of ["normal", "ignored-batch-setting", "enabled-batch"] as const) {
|
||||||
|
for (let repeat = 0; repeat < 2; repeat++) {
|
||||||
|
await run(`${mode}-coalesce-${repeat}`, () =>
|
||||||
|
withPair(async (pair) => {
|
||||||
|
for (const page of [pair.a, pair.b]) await options(page, mode);
|
||||||
|
const source = repeat === 0 ? pair.a : pair.b;
|
||||||
|
const receiver = repeat === 0 ? pair.b : pair.a;
|
||||||
|
const path = `${prefix}${mode}-edits-${repeat}.md`;
|
||||||
|
await create(source, path, "Initial body\n");
|
||||||
|
const initial = (await settled(pair, path, "Initial body\n"))[0];
|
||||||
|
for (let edit = 0; edit < 6; edit++) {
|
||||||
|
await modify(source, path, `Edit ${edit}\n`);
|
||||||
|
await pause(150);
|
||||||
|
}
|
||||||
|
const during = await snapshot(source, path);
|
||||||
|
const final = await settled(pair, path, "Edit 5\n");
|
||||||
|
const finalSource = repeat === 0 ? final[0] : final[1];
|
||||||
|
const delta = Number(finalSource.rev?.split("-")[0]) - Number(initial.rev?.split("-")[0]);
|
||||||
|
if (mode === "enabled-batch") {
|
||||||
|
check(
|
||||||
|
during.waiting.some((item) => item.path === path),
|
||||||
|
"Batch did not retain the pending edit"
|
||||||
|
);
|
||||||
|
check(delta === 1, `Repeated edits created ${delta} revisions`);
|
||||||
|
} else {
|
||||||
|
check(!during.shouldBatchSave, "LiveSync batching was enabled without an explicit choice");
|
||||||
|
check(delta >= 3, `The control did not expose enough edits: ${delta}`);
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
path,
|
||||||
|
initial,
|
||||||
|
during,
|
||||||
|
final,
|
||||||
|
revisionDelta: delta,
|
||||||
|
source: await trace(source, path),
|
||||||
|
receiver: await trace(receiver, path),
|
||||||
|
};
|
||||||
|
})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
await run(`${mode}-template-lifecycle`, () =>
|
||||||
|
withPair(async (pair) => {
|
||||||
|
for (const page of [pair.a, pair.b]) await options(page, mode);
|
||||||
|
const oldPath = `${prefix}${mode}-untitled.md`;
|
||||||
|
const path = `${prefix}${mode}-completed.md`;
|
||||||
|
await create(pair.a, oldPath, "");
|
||||||
|
await pause(300);
|
||||||
|
await modify(pair.a, oldPath, "Template in progress\n");
|
||||||
|
await pause(300);
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`await app.fileManager.renameFile(app.vault.getFileByPath(${JSON.stringify(oldPath)}),${JSON.stringify(path)});`
|
||||||
|
);
|
||||||
|
await pause(300);
|
||||||
|
await modify(pair.a, path, "Completed template\n");
|
||||||
|
const during = await snapshot(pair.a, path);
|
||||||
|
const final = await settled(pair, path, "Completed template\n");
|
||||||
|
await waitFor(pair.b, oldPath, (state) => !state.exists && liveLeaves(state).length === 0);
|
||||||
|
return {
|
||||||
|
oldPath,
|
||||||
|
path,
|
||||||
|
during,
|
||||||
|
final,
|
||||||
|
oldSource: await snapshot(pair.a, oldPath),
|
||||||
|
oldReceiver: await snapshot(pair.b, oldPath),
|
||||||
|
source: await trace(pair.a, path),
|
||||||
|
receiver: await trace(pair.b, path),
|
||||||
|
oldSourceTrace: await trace(pair.a, oldPath),
|
||||||
|
oldReceiverTrace: await trace(pair.b, oldPath),
|
||||||
|
};
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
for (const physical of [false, true]) {
|
||||||
|
await run(`${mode}-recreate-${physical}`, () =>
|
||||||
|
withPair(async (pair) => {
|
||||||
|
for (const page of [pair.a, pair.b]) {
|
||||||
|
await options(page, mode);
|
||||||
|
await evaluate(
|
||||||
|
page,
|
||||||
|
`core.services.setting.currentSettings().deleteMetadataOfDeletedFiles=${physical};`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const path = `${prefix}${mode}-recreate-${physical}.md`;
|
||||||
|
await create(pair.a, path, "Transient body\n");
|
||||||
|
await pause(400);
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`await app.vault.delete(app.vault.getFileByPath(${JSON.stringify(path)}),true);`
|
||||||
|
);
|
||||||
|
await pause(400);
|
||||||
|
await create(pair.a, path, "Recreated body\n");
|
||||||
|
const final = await settled(pair, path, "Recreated body\n");
|
||||||
|
return {
|
||||||
|
physical,
|
||||||
|
final,
|
||||||
|
source: await trace(pair.a, path),
|
||||||
|
receiver: await trace(pair.b, path),
|
||||||
|
};
|
||||||
|
})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const incoming of ["identical", "different", "delete"] as const) {
|
||||||
|
await run(`enabled-pending-incoming-${incoming}`, () =>
|
||||||
|
withPair(async (pair) => {
|
||||||
|
await options(pair.a, "enabled-batch", 5, 60);
|
||||||
|
await options(pair.b, "normal", 5, 60);
|
||||||
|
const path = `${prefix}pending-${incoming}.md`;
|
||||||
|
await create(pair.b, path, "Shared base\n");
|
||||||
|
const initial = await settled(pair, path, "Shared base\n");
|
||||||
|
const local = "Local edit\n";
|
||||||
|
await modify(pair.a, path, local);
|
||||||
|
const pending = await waitFor(pair.a, path, (state) =>
|
||||||
|
state.waiting.some((item) => item.path === path)
|
||||||
|
);
|
||||||
|
check(pending.rev === initial[0].rev, "Local edit reached the DB before incoming delivery");
|
||||||
|
const sentAt = Date.now();
|
||||||
|
if (incoming === "delete") {
|
||||||
|
await evaluate(
|
||||||
|
pair.b,
|
||||||
|
`await app.vault.delete(app.vault.getFileByPath(${JSON.stringify(path)}),true);`
|
||||||
|
);
|
||||||
|
} else await modify(pair.b, path, incoming === "identical" ? local : "Remote edit\n");
|
||||||
|
const arrived = await waitFor(
|
||||||
|
pair.a,
|
||||||
|
path,
|
||||||
|
(state) =>
|
||||||
|
incoming === "identical"
|
||||||
|
? state.rev !== initial[0].rev && liveLeaves(state).some((leaf) => leaf.body === local)
|
||||||
|
: state.leaves.some((leaf) => leaf.body === local) &&
|
||||||
|
state.leaves.some((leaf) =>
|
||||||
|
incoming === "delete" ? leaf.logicalDeleted : leaf.body === "Remote edit\n"
|
||||||
|
),
|
||||||
|
4000
|
||||||
|
);
|
||||||
|
check(
|
||||||
|
arrived.waiting.some((item) => item.path === path),
|
||||||
|
"The incoming assertion did not occur during Batch waiting"
|
||||||
|
);
|
||||||
|
const leavesBefore = arrived.leaves.map((leaf) => leaf.rev).sort();
|
||||||
|
if (incoming !== "identical") {
|
||||||
|
check(arrived.content === local, "Incoming reflection overwrote the pending local edit");
|
||||||
|
check(arrived.leaves.length === 2, "Expected exactly two genuine concurrent branches");
|
||||||
|
check(
|
||||||
|
arrived.leaves.every((leaf) => leaf.ancestry[1] === initial[0].rev),
|
||||||
|
"Concurrent branches lost their shared base"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
await pause(5600);
|
||||||
|
const after = await snapshot(pair.a, path);
|
||||||
|
const peer = await snapshot(pair.b, path);
|
||||||
|
check(!after.waiting.some((item) => item.path === path), "Batch timer did not complete");
|
||||||
|
check(
|
||||||
|
JSON.stringify(after.leaves.map((leaf) => leaf.rev).sort()) === JSON.stringify(leavesBefore),
|
||||||
|
"The delayed event created another revision"
|
||||||
|
);
|
||||||
|
check(
|
||||||
|
JSON.stringify(peer.leaves.map((leaf) => leaf.rev).sort()) === JSON.stringify(leavesBefore),
|
||||||
|
"Both branches did not converge to the peer"
|
||||||
|
);
|
||||||
|
if (incoming === "identical")
|
||||||
|
check(after.leaves.length === 1, "Identical content created a branch");
|
||||||
|
return { initial, pending, arrived, after, peer, incomingLatencyMs: Date.now() - sentAt - 5600 };
|
||||||
|
})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
await run("enabled-maximum-delay", () =>
|
||||||
|
withPair(async (pair) => {
|
||||||
|
await options(pair.a, "enabled-batch", 2, 6);
|
||||||
|
await options(pair.b, "normal");
|
||||||
|
const path = `${prefix}maximum.md`;
|
||||||
|
await create(pair.b, path, "Initial maximum body\n");
|
||||||
|
const initial = await settled(pair, path, "Initial maximum body\n");
|
||||||
|
const started = Date.now();
|
||||||
|
let finalBody = "";
|
||||||
|
for (let edit = 0; edit < 26; edit++) {
|
||||||
|
finalBody = `Continuous edit ${edit}\n`;
|
||||||
|
await modify(pair.a, path, finalBody);
|
||||||
|
await pause(350);
|
||||||
|
}
|
||||||
|
const stopped = Date.now();
|
||||||
|
const final = await settled(pair, path, finalBody);
|
||||||
|
const changes = await evaluate<Array<{ at: number; rev: string }>>(
|
||||||
|
pair.a,
|
||||||
|
`return globalThis.__issue1207Batch.changes.filter(item=>item.path===${JSON.stringify(path)}&&item.rev!==${JSON.stringify(initial[0].rev)});`
|
||||||
|
);
|
||||||
|
check(changes.length >= 2, "Continuous edits did not produce bounded intermediate and final saves");
|
||||||
|
const first = changes[0].at - started;
|
||||||
|
check(first >= 5000 && first <= 8000, `Unexpected first maximum-delay flush: ${first}ms`);
|
||||||
|
check(changes[0].at < stopped - 1000, "The maximum flush only occurred after typing stopped");
|
||||||
|
return { started, stopped, firstSaveMs: first, changes, final };
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
await run("enabled-many-paths", () =>
|
||||||
|
withPair(async (pair) => {
|
||||||
|
await options(pair.a, "enabled-batch", 2, 6);
|
||||||
|
await options(pair.b, "normal");
|
||||||
|
const started = Date.now();
|
||||||
|
for (let index = 0; index < 12; index++)
|
||||||
|
await create(pair.a, `${prefix}many-${index}.md`, `File ${index}\n`);
|
||||||
|
const final = [];
|
||||||
|
for (let index = 0; index < 12; index++)
|
||||||
|
final.push(await settled(pair, `${prefix}many-${index}.md`, `File ${index}\n`));
|
||||||
|
return { elapsedMs: Date.now() - started, final };
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
await run("enabled-manual-sync", () =>
|
||||||
|
withPair(async (pair) => {
|
||||||
|
await options(pair.a, "enabled-batch", 5, 60);
|
||||||
|
await options(pair.b, "normal");
|
||||||
|
const path = `${prefix}manual.md`;
|
||||||
|
await create(pair.b, path, "Before manual sync\n");
|
||||||
|
await settled(pair, path, "Before manual sync\n");
|
||||||
|
await modify(pair.a, path, "After manual sync\n");
|
||||||
|
const pending = await waitFor(pair.a, path, (state) =>
|
||||||
|
state.waiting.some((item) => item.path === path)
|
||||||
|
);
|
||||||
|
const started = Date.now();
|
||||||
|
const outcome = await evaluate(pair.a, "return await core.services.replication.replicate(true);");
|
||||||
|
const atReturn = await snapshot(pair.a, path);
|
||||||
|
const final = await settled(pair, path, "After manual sync\n");
|
||||||
|
const elapsedMs = Date.now() - started;
|
||||||
|
check(elapsedMs < 4000, `Manual sync waited for the Batch timer: ${elapsedMs}ms`);
|
||||||
|
return { pending, outcome, atReturn, final, elapsedMs };
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
const saveAfterEdit = process.env.ISSUE1207_RESTART_SAVE_AFTER_EDIT === "true";
|
||||||
|
const restartBucket = process.env.ISSUE1207_RESTART_BUCKET;
|
||||||
|
check(
|
||||||
|
!restartBucket || ["same", "exact", "different"].includes(restartBucket),
|
||||||
|
"Unknown restart timestamp condition"
|
||||||
|
);
|
||||||
|
const restartName = restartBucket
|
||||||
|
? `enabled-inflight-restart-${restartBucket}`
|
||||||
|
: saveAfterEdit
|
||||||
|
? "enabled-settings-save-restart"
|
||||||
|
: "enabled-restart";
|
||||||
|
await run(restartName, async () => {
|
||||||
|
const path = `${prefix}restart.md`;
|
||||||
|
const beforeRestart = await withPair(async (pair) => {
|
||||||
|
await options(pair.a, "enabled-batch", 5, 60);
|
||||||
|
await options(pair.b, "normal");
|
||||||
|
if (!saveAfterEdit)
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
"await core.services.setting.saveSettingData(); await core.services.control.applySettings();"
|
||||||
|
);
|
||||||
|
await create(pair.b, path, "Before restart\n");
|
||||||
|
const initial = await settled(pair, path, "Before restart\n");
|
||||||
|
if (restartBucket) {
|
||||||
|
check(initial[0].databaseMtime !== null, "Missing initial database mtime");
|
||||||
|
const mtime =
|
||||||
|
restartBucket === "exact"
|
||||||
|
? initial[0].databaseMtime
|
||||||
|
: Math.floor(initial[0].databaseMtime / 2000) * 2000 +
|
||||||
|
(restartBucket === "same" ? 1000 : 3000);
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`const original=manager.requestProcessQueue.bind(manager);
|
||||||
|
manager.requestProcessQueue=async item=>{
|
||||||
|
if(item.args?.file?.path===${JSON.stringify(path)}) {
|
||||||
|
globalThis.__issue1207Batch.inflightPaused=true;
|
||||||
|
await new Promise(resolve=>setTimeout(resolve,30000));
|
||||||
|
}
|
||||||
|
return await original(item);
|
||||||
|
};
|
||||||
|
await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),"Retained after restart\\n",{mtime:${mtime}});`
|
||||||
|
);
|
||||||
|
} else await modify(pair.a, path, "Retained after restart\n");
|
||||||
|
const pending = await waitFor(pair.a, path, (state) =>
|
||||||
|
state.waiting.some((item) => item.path === path)
|
||||||
|
);
|
||||||
|
if (saveAfterEdit) await evaluate(pair.a, "await core.services.setting.saveSettingData();");
|
||||||
|
if (restartBucket)
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`const deadline=Date.now()+core.services.setting.currentSettings().batchSaveMinimumDelay*1000+3000;
|
||||||
|
while(!globalThis.__issue1207Batch.inflightPaused&&Date.now()<deadline) await new Promise(resolve=>setTimeout(resolve,10));
|
||||||
|
if(!globalThis.__issue1207Batch.inflightPaused) throw new Error('The in-flight test gate did not start');`
|
||||||
|
);
|
||||||
|
await evaluate(pair.a, "await manager._takeSnapshot();");
|
||||||
|
check(pending.rev === initial[0].rev, "Restart test did not retain a pending edit");
|
||||||
|
if (restartBucket) {
|
||||||
|
const inFlight = await snapshot(pair.a, path);
|
||||||
|
const saved = await evaluate<Array<{ path: string }>>(
|
||||||
|
pair.a,
|
||||||
|
`return (await manager.adapter.persistence.loadSnapshot()).map(item=>({path:item.args?.file?.path}));`
|
||||||
|
);
|
||||||
|
check(
|
||||||
|
inFlight.rev === initial[0].rev && !inFlight.waiting.some((item) => item.path === path),
|
||||||
|
"The in-flight fixture did not stop between gate release and database storage"
|
||||||
|
);
|
||||||
|
check(
|
||||||
|
saved.some((item) => item.path === path),
|
||||||
|
"The in-flight event was lost from the recoverable snapshot"
|
||||||
|
);
|
||||||
|
check(inFlight.fileMtime !== null && inFlight.databaseMtime !== null, "Missing in-flight mtimes");
|
||||||
|
const sameBucket =
|
||||||
|
Math.floor(inFlight.fileMtime / 2000) === Math.floor(inFlight.databaseMtime / 2000);
|
||||||
|
check(sameBucket === (restartBucket !== "different"), "The timestamp condition was not applied");
|
||||||
|
if (restartBucket === "exact")
|
||||||
|
check(inFlight.fileMtime === inFlight.databaseMtime, "Raw mtimes differ");
|
||||||
|
observations.push({
|
||||||
|
phase: "inflight-before-stop",
|
||||||
|
restartBucket,
|
||||||
|
pauseBeforeDatabaseMs: 30000,
|
||||||
|
inFlight,
|
||||||
|
saved,
|
||||||
|
sameBucket,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (!saveAfterEdit && !restartBucket) {
|
||||||
|
const strictState = await snapshot(pair.a, path);
|
||||||
|
const saved = await evaluate<Array<{ path: string }>>(
|
||||||
|
pair.a,
|
||||||
|
`return (await manager.adapter.persistence.loadSnapshot()).map(item=>({path:item.args?.file?.path}));`
|
||||||
|
);
|
||||||
|
check(
|
||||||
|
strictState.rev === initial[0].rev && strictState.waiting.some((item) => item.path === path),
|
||||||
|
"Restart fixture flushed before shutdown"
|
||||||
|
);
|
||||||
|
check(
|
||||||
|
saved.some((item) => item.path === path),
|
||||||
|
"Restart fixture did not persist the pending event"
|
||||||
|
);
|
||||||
|
observations.push({
|
||||||
|
phase: "restart-verified-pending-snapshot",
|
||||||
|
strictState,
|
||||||
|
saved,
|
||||||
|
initialized: await evaluate(pair.a, "return await core.kvDB.get('initialized');"),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (!saveAfterEdit)
|
||||||
|
observations.push({
|
||||||
|
phase: "restart-before-stop",
|
||||||
|
initial,
|
||||||
|
pending,
|
||||||
|
persisted: await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`const events=await manager.adapter.persistence.loadSnapshot();
|
||||||
|
const settings=core.services.setting.currentSettings();
|
||||||
|
return {initialized:await core.kvDB.get('initialized'),restorationRequested:manager.snapShotRestored!==null,
|
||||||
|
settings:{isConfigured:settings.isConfigured,liveSync:settings.liveSync,batchSave:settings.batchSave,batchSaveInLiveSync:settings.batchSaveInLiveSync,
|
||||||
|
suspendFileWatching:settings.suspendFileWatching},
|
||||||
|
events:events?.map(item=>({type:item.type,path:item.args?.file?.path,
|
||||||
|
stat:item.args?.file?.stat,skipBatchWait:item.skipBatchWait}))};`
|
||||||
|
),
|
||||||
|
});
|
||||||
|
return { initial, pending };
|
||||||
|
});
|
||||||
|
await stop(slots[0]);
|
||||||
|
const persistedSettings = JSON.parse(
|
||||||
|
await readFile(join(slots[0].vault.path, ".obsidian/plugins/obsidian-livesync/data.json"), "utf8")
|
||||||
|
);
|
||||||
|
observations.push({
|
||||||
|
phase: "restart-on-disk-settings",
|
||||||
|
settings: Object.fromEntries(
|
||||||
|
[
|
||||||
|
"isConfigured",
|
||||||
|
"liveSync",
|
||||||
|
"batchSave",
|
||||||
|
"batchSaveInLiveSync",
|
||||||
|
"suspendFileWatching",
|
||||||
|
"settingVersion",
|
||||||
|
].map((key) => [key, persistedSettings[key]])
|
||||||
|
),
|
||||||
|
});
|
||||||
|
await start(slots[0], true);
|
||||||
|
const afterRestart = await withPair(async (pair) => {
|
||||||
|
observations.push({
|
||||||
|
phase: "restart-after-boot",
|
||||||
|
state: await snapshot(pair.a, path),
|
||||||
|
persisted: await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`const events=await manager.adapter.persistence.loadSnapshot();
|
||||||
|
const settings=core.services.setting.currentSettings();
|
||||||
|
return {initialized:await core.kvDB.get('initialized'),restorationRequested:manager.snapShotRestored!==null,
|
||||||
|
settings:{isConfigured:settings.isConfigured,liveSync:settings.liveSync,batchSave:settings.batchSave,batchSaveInLiveSync:settings.batchSaveInLiveSync,
|
||||||
|
suspendFileWatching:settings.suspendFileWatching},
|
||||||
|
events:events?.map(item=>({type:item.type,path:item.args?.file?.path,stat:item.args?.file?.stat}))};`
|
||||||
|
),
|
||||||
|
});
|
||||||
|
const result = await settled(pair, path, "Retained after restart\n");
|
||||||
|
check(result[0].shouldBatchSave, "The saved LiveSync batching choice was not restored");
|
||||||
|
return { result, restoredGuard: result[0].shouldBatchSave };
|
||||||
|
});
|
||||||
|
return { beforeRestart, afterRestart };
|
||||||
|
});
|
||||||
|
|
||||||
|
await run("enabled-after-write-restart", async () => {
|
||||||
|
const path = `${prefix}after-write.md`;
|
||||||
|
const saved = await withPair(async (pair) => {
|
||||||
|
await options(pair.a, "enabled-batch", 1, 3);
|
||||||
|
await options(pair.b, "normal");
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
"await core.services.setting.saveSettingData(); await core.services.control.applySettings();"
|
||||||
|
);
|
||||||
|
await create(pair.b, path, "Before the durable write\n");
|
||||||
|
const initial = await settled(pair, path, "Before the durable write\n");
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`
|
||||||
|
const original=manager.requestProcessQueue.bind(manager);
|
||||||
|
manager.requestProcessQueue=async item=>{
|
||||||
|
await original(item);
|
||||||
|
if(item.args?.file?.path===${JSON.stringify(path)}) {
|
||||||
|
globalThis.__issue1207Batch.afterWritePaused=true;
|
||||||
|
await new Promise(resolve=>setTimeout(resolve,30000));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),"Durable before restart\\n");
|
||||||
|
const deadline=Date.now()+6000;
|
||||||
|
while(!globalThis.__issue1207Batch.afterWritePaused&&Date.now()<deadline) await new Promise(resolve=>setTimeout(resolve,20));
|
||||||
|
if(!globalThis.__issue1207Batch.afterWritePaused) throw new Error('The after-write gate did not start');
|
||||||
|
await manager._takeSnapshot();
|
||||||
|
`
|
||||||
|
);
|
||||||
|
const durable = await snapshot(pair.a, path);
|
||||||
|
const pending = await evaluate<Array<{ path: string }>>(
|
||||||
|
pair.a,
|
||||||
|
"return (await manager.adapter.persistence.loadSnapshot()).map(item=>({path:item.args?.file?.path}));"
|
||||||
|
);
|
||||||
|
check(durable.rev !== initial[0].rev, "The database write did not complete before interruption");
|
||||||
|
check(
|
||||||
|
pending.some((item) => item.path === path),
|
||||||
|
"The after-write operation was not recoverable"
|
||||||
|
);
|
||||||
|
return { initial, durable, pending };
|
||||||
|
});
|
||||||
|
const restarts = [];
|
||||||
|
for (let attempt = 0; attempt < 2; attempt++) {
|
||||||
|
await stop(slots[0]);
|
||||||
|
await start(slots[0], true);
|
||||||
|
const final = await withPair((pair) => settled(pair, path, "Durable before restart\n"));
|
||||||
|
check(final[0].rev === saved.durable.rev, "Replay created a duplicate database revision");
|
||||||
|
restarts.push(final);
|
||||||
|
}
|
||||||
|
const later = await withPair(async (pair) => {
|
||||||
|
await modify(pair.a, path, "A later edit survives\n");
|
||||||
|
return await settled(pair, path, "A later edit survives\n");
|
||||||
|
});
|
||||||
|
return { saved, restarts, later };
|
||||||
|
});
|
||||||
|
|
||||||
|
await run("enabled-before-snapshot-restart-exact", async () => {
|
||||||
|
const path = `${prefix}before-snapshot.md`;
|
||||||
|
const beforeStop = await withPair(async (pair) => {
|
||||||
|
await options(pair.a, "enabled-batch", 5, 60);
|
||||||
|
await options(pair.b, "normal");
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
"await core.services.setting.saveSettingData(); await core.services.control.applySettings();"
|
||||||
|
);
|
||||||
|
await create(pair.b, path, "Before the first snapshot\n");
|
||||||
|
const initial = await settled(pair, path, "Before the first snapshot\n");
|
||||||
|
check(initial[0].databaseMtime !== null, "Missing initial database mtime");
|
||||||
|
await evaluate(
|
||||||
|
pair.a,
|
||||||
|
`
|
||||||
|
await manager.waitForIdle();
|
||||||
|
const persistence=manager.adapter.persistence;
|
||||||
|
const original=persistence.saveSnapshot.bind(persistence);
|
||||||
|
persistence.saveSnapshot=async events=>{
|
||||||
|
if(events.some(item=>item.type==='CHANGED'&&item.args?.file?.path===${JSON.stringify(path)})) {
|
||||||
|
globalThis.__issue1207Batch.beforeSnapshotPaused=true;
|
||||||
|
await new Promise(resolve=>setTimeout(resolve,30000));
|
||||||
|
}
|
||||||
|
return await original(events);
|
||||||
|
};
|
||||||
|
await app.vault.modify(app.vault.getFileByPath(${JSON.stringify(path)}),
|
||||||
|
"Recovered without an event snapshot\\n",{mtime:${initial[0].databaseMtime}});
|
||||||
|
const deadline=Date.now()+3000;
|
||||||
|
while(!globalThis.__issue1207Batch.beforeSnapshotPaused&&Date.now()<deadline)
|
||||||
|
await new Promise(resolve=>setTimeout(resolve,10));
|
||||||
|
if(!globalThis.__issue1207Batch.beforeSnapshotPaused)
|
||||||
|
throw new Error('The snapshot persistence gate did not start');
|
||||||
|
`
|
||||||
|
);
|
||||||
|
const state = await snapshot(pair.a, path);
|
||||||
|
const persisted = await evaluate<Array<{ path: string }>>(
|
||||||
|
pair.a,
|
||||||
|
"return (await manager.adapter.persistence.loadSnapshot()??[]).map(item=>({path:item.args?.file?.path}));"
|
||||||
|
);
|
||||||
|
check(state.rev === initial[0].rev, "The unsaved edit reached the database before interruption");
|
||||||
|
check(state.fileMtime === state.databaseMtime, "Raw mtimes differ before interruption");
|
||||||
|
check(!persisted.some((item) => item.path === path), "The interrupted edit already has a saved event");
|
||||||
|
return { initial, state, persisted };
|
||||||
|
});
|
||||||
|
await stop(slots[0]);
|
||||||
|
await start(slots[0], true);
|
||||||
|
const recovered = await withPair((pair) => settled(pair, path, "Recovered without an event snapshot\n"));
|
||||||
|
check(recovered[0].shouldBatchSave, "The saved LiveSync batching choice was not restored");
|
||||||
|
return { beforeStop, recovered };
|
||||||
|
});
|
||||||
|
|
||||||
|
evidence.artefactsAfter = hashes();
|
||||||
|
check(
|
||||||
|
JSON.stringify(before) === JSON.stringify(evidence.artefactsAfter),
|
||||||
|
"Plug-in artefacts changed during testing"
|
||||||
|
);
|
||||||
|
evidence.passed = cases.every((item) => item.passed);
|
||||||
|
await writeFile(join(diagnostics, "evidence.json"), redact(evidence));
|
||||||
|
console.log(
|
||||||
|
JSON.stringify({
|
||||||
|
phase: "completed",
|
||||||
|
passed: evidence.passed,
|
||||||
|
cases: cases.length,
|
||||||
|
failed: cases.filter((item) => !item.passed).map((item) => item.name),
|
||||||
|
})
|
||||||
|
);
|
||||||
|
if (!evidence.passed) process.exitCode = 1;
|
||||||
|
} finally {
|
||||||
|
for (const slot of slots) await stop(slot);
|
||||||
|
for (const slot of slots) await slot.vault.dispose();
|
||||||
|
if (ownedDatabase) await deleteCouchDbDatabase(config, dbName);
|
||||||
|
evidence.cleanup = { devicesStopped: slots.every((slot) => !slot.session), databaseRemoved: ownedDatabase };
|
||||||
|
await writeFile(join(diagnostics, "evidence.json"), redact(evidence));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((error: unknown) => {
|
||||||
|
console.error(error instanceof Error ? error.stack : error);
|
||||||
|
process.exitCode = 1;
|
||||||
|
});
|
||||||
@@ -0,0 +1,230 @@
|
|||||||
|
/** Exercises optional file admission, actual save completion, and replay in a real Obsidian Vault. */
|
||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { readFile, writeFile } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { evalObsidianJson } from "../runner/cli.ts";
|
||||||
|
import { requireObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||||
|
import {
|
||||||
|
createE2eCouchDbPluginData,
|
||||||
|
createE2eObsidianDeviceLocalState,
|
||||||
|
waitForLiveSyncCoreReady,
|
||||||
|
} from "../runner/liveSyncWorkflow.ts";
|
||||||
|
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||||
|
import { createTemporaryVault } from "../runner/vault.ts";
|
||||||
|
|
||||||
|
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
|
||||||
|
|
||||||
|
async function main(): Promise<void> {
|
||||||
|
const binary = requireObsidianBinary();
|
||||||
|
const cliBinary = requireObsidianCli();
|
||||||
|
const vault = await createTemporaryVault("obsidian-optional-save-");
|
||||||
|
let session: ObsidianLiveSyncSession | undefined;
|
||||||
|
const observations: unknown[] = [];
|
||||||
|
const bundleHash = createHash("sha256")
|
||||||
|
.update(new Uint8Array(await readFile("main.js")))
|
||||||
|
.digest("hex");
|
||||||
|
const settings = createE2eCouchDbPluginData(
|
||||||
|
// These scenarios use only the local database; every replication trigger is disabled.
|
||||||
|
{ uri: "http://127.0.0.1:1", username: "unused", password: "unused", dbName: "optional-save-local" },
|
||||||
|
{
|
||||||
|
syncInternalFiles: true,
|
||||||
|
usePluginSync: true,
|
||||||
|
usePluginSyncV2: true,
|
||||||
|
deviceAndVaultName: "optional-save-device",
|
||||||
|
watchInternalFileChanges: true,
|
||||||
|
syncInternalFilesTargetPatterns: "^\\.obsidian(?:$|/(?:optional-save|snippets)(?:/|$))",
|
||||||
|
syncInternalFilesIgnorePatterns: "",
|
||||||
|
autoSweepPlugins: false,
|
||||||
|
autoSweepPluginsPeriodic: false,
|
||||||
|
syncInternalFilesInterval: 0,
|
||||||
|
liveSync: false,
|
||||||
|
periodicReplication: false,
|
||||||
|
syncOnStart: false,
|
||||||
|
syncOnSave: false,
|
||||||
|
syncOnFileOpen: false,
|
||||||
|
syncOnEditorSave: false,
|
||||||
|
syncAfterMerge: false,
|
||||||
|
showVerboseLog: true,
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
async function start(restart = false): Promise<void> {
|
||||||
|
session = await startObsidianLiveSyncSession({
|
||||||
|
binary,
|
||||||
|
cliBinary: cliBinary,
|
||||||
|
vault,
|
||||||
|
pluginStartup: restart ? "natural" : undefined,
|
||||||
|
pluginData: restart ? undefined : settings,
|
||||||
|
localStorageEntries: restart ? undefined : createE2eObsidianDeviceLocalState(vault.name),
|
||||||
|
lifecycle: {
|
||||||
|
beforeLaunch: async () => {
|
||||||
|
const registryPath = join(vault.userDataPath, "obsidian.json");
|
||||||
|
const registry = JSON.parse(await readFile(registryPath, "utf8"));
|
||||||
|
registry.updateDisabled = true;
|
||||||
|
await writeFile(registryPath, JSON.stringify(registry));
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function evaluate<T>(body: string): Promise<T> {
|
||||||
|
if (!session) throw new Error("The Obsidian session is not running");
|
||||||
|
return await evalObsidianJson<T>(
|
||||||
|
cliBinary,
|
||||||
|
`(async()=>{
|
||||||
|
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||||
|
const manager=core.serviceModules.storageAccess.vaultManager;
|
||||||
|
const config=core.getAddOn('ConfigSync');
|
||||||
|
const sleep=ms=>new Promise(resolve=>setTimeout(resolve,ms));
|
||||||
|
const require=(condition,message)=>{if(!condition)throw new Error(message);};
|
||||||
|
const waitFor=async (predicate,message)=>{
|
||||||
|
const until=Date.now()+10000;
|
||||||
|
while(!(await predicate())) {
|
||||||
|
if(Date.now()>until)throw new Error(message);
|
||||||
|
await sleep(20);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const pending=async path=>(await manager.adapter.persistence.loadSnapshot()??[])
|
||||||
|
.some(event=>event.args?.file?.path===path);
|
||||||
|
const entry=async path=>await core.localDatabase.getDBEntry(path,undefined,false,true);
|
||||||
|
const contents=doc=>Array.isArray(doc.data)?doc.data.join(''):doc.data;
|
||||||
|
const represents=(doc,content)=>doc&& !doc.deleted &&
|
||||||
|
(contents(doc).includes(content)||contents(doc).includes(btoa(content)));
|
||||||
|
const notify=async path=>{
|
||||||
|
await manager._watchVaultRawEvents(path);
|
||||||
|
await sleep(30);
|
||||||
|
};
|
||||||
|
${body}
|
||||||
|
})()`,
|
||||||
|
session.cliEnv
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await start();
|
||||||
|
await evaluate(`
|
||||||
|
require(core.services.fileProcessing.processOptionalFileEventWithResult?.hasHandlers,
|
||||||
|
'Typed optional providers were not registered');
|
||||||
|
core.services.setting.setDeviceAndVaultName('optional-save-device');
|
||||||
|
await core.services.setting.saveSettingData();
|
||||||
|
for(const path of ['.obsidian/optional-save','.obsidian/snippets']) {
|
||||||
|
if(!(await app.vault.adapter.exists(path)))await app.vault.adapter.mkdir(path);
|
||||||
|
}
|
||||||
|
return JSON.stringify({ready:true});
|
||||||
|
`);
|
||||||
|
|
||||||
|
for (const mode of ["hidden", "config-v1", "config-v2"]) {
|
||||||
|
observations.push(
|
||||||
|
await evaluate(`
|
||||||
|
const mode=${JSON.stringify(mode)};
|
||||||
|
core.settings.usePluginSyncV2=mode!=='config-v1';
|
||||||
|
await core.services.setting.saveSettingData();
|
||||||
|
require(await core.services.fileProcessing.commitPendingFileEvents(),'Initial flush failed');
|
||||||
|
const path=mode==='hidden'?'.obsidian/optional-save/content.json':
|
||||||
|
'.obsidian/snippets/optional-save-'+mode+'.css';
|
||||||
|
const databasePath=mode==='hidden'?'i:'+path:mode==='config-v1'?
|
||||||
|
config.filenameToUnifiedKey(path):config.filenameWithUnifiedKey(path);
|
||||||
|
const content='optional save '+mode+' initial';
|
||||||
|
const original=core.localDatabase.putDBEntry.bind(core.localDatabase);
|
||||||
|
let release;
|
||||||
|
const gate=new Promise(resolve=>{release=resolve;});
|
||||||
|
let started=false;
|
||||||
|
core.localDatabase.putDBEntry=async data=>{
|
||||||
|
if(data.path===databasePath){started=true;await gate;}
|
||||||
|
return await original(data);
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
await app.vault.adapter.write(path,content);
|
||||||
|
await notify(path);
|
||||||
|
await waitFor(()=>started,'The optional database save did not start: '+mode);
|
||||||
|
require(await pending(path),'The in-flight optional save was missing from the snapshot: '+mode);
|
||||||
|
let flushed=false;
|
||||||
|
const flush=core.services.fileProcessing.commitPendingFileEvents().then(result=>{flushed=true;return result;});
|
||||||
|
await sleep(30);
|
||||||
|
require(!flushed,'Flush returned before the actual database save: '+mode);
|
||||||
|
release();
|
||||||
|
require(await flush,'Flush failed after the write was released: '+mode);
|
||||||
|
} finally {
|
||||||
|
release();
|
||||||
|
core.localDatabase.putDBEntry=original;
|
||||||
|
}
|
||||||
|
const first=await entry(databasePath);
|
||||||
|
require(represents(first,content),'The completed save did not contain the expected content: '+mode);
|
||||||
|
await notify(path);
|
||||||
|
require(await core.services.fileProcessing.commitPendingFileEvents(),'Unchanged notification failed: '+mode);
|
||||||
|
require((await entry(databasePath))._rev===first._rev,'Unchanged notification added a revision: '+mode);
|
||||||
|
const updated=content+' with a later edit';
|
||||||
|
await app.vault.adapter.write(path,updated);
|
||||||
|
await notify(path);
|
||||||
|
require(await core.services.fileProcessing.commitPendingFileEvents(),'Later edit was blocked: '+mode);
|
||||||
|
require(represents(await entry(databasePath),updated),'Later edit was not saved: '+mode);
|
||||||
|
await app.vault.adapter.remove(path);
|
||||||
|
await notify(path);
|
||||||
|
require(await core.services.fileProcessing.commitPendingFileEvents(),'Deletion failed: '+mode);
|
||||||
|
const deleted=await core.localDatabase.getDBEntryMeta(databasePath,undefined,true);
|
||||||
|
require(deleted&&deleted.deleted,'Deletion was not represented: '+mode);
|
||||||
|
await notify(path);
|
||||||
|
require(await core.services.fileProcessing.commitPendingFileEvents(),'Repeated deletion failed: '+mode);
|
||||||
|
require((await core.localDatabase.getDBEntryMeta(databasePath,undefined,true))._rev===deleted._rev,
|
||||||
|
'Repeated deletion added a revision: '+mode);
|
||||||
|
return JSON.stringify({mode,saveWaited:true,unchangedPreservedRevision:true,laterEditSaved:true,deletionReplayed:true});
|
||||||
|
`)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const replay = [
|
||||||
|
{ path: ".obsidian/optional-save/replay.json", content: "hidden save retained across restart" },
|
||||||
|
{
|
||||||
|
path: ".obsidian/snippets/optional-save-replay.css",
|
||||||
|
content: "customisation save retained across restart",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
observations.push(
|
||||||
|
await evaluate(`
|
||||||
|
const fixtures=${JSON.stringify(replay)};
|
||||||
|
core.settings.usePluginSyncV2=true;
|
||||||
|
await core.services.setting.saveSettingData();
|
||||||
|
const failures=new Set(fixtures.map(({path})=>path.endsWith('.json')?'i:'+path:config.filenameWithUnifiedKey(path)));
|
||||||
|
const original=core.localDatabase.putDBEntry.bind(core.localDatabase);
|
||||||
|
let failed=0;
|
||||||
|
core.localDatabase.putDBEntry=async data=>{
|
||||||
|
if(failures.delete(data.path)){failed++;return false;}
|
||||||
|
return await original(data);
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
for(const {path,content} of fixtures) {
|
||||||
|
await app.vault.adapter.write(path,content);
|
||||||
|
await notify(path);
|
||||||
|
}
|
||||||
|
await waitFor(()=>failed===fixtures.length,'Injected optional failures were not reached');
|
||||||
|
require(!(await core.services.fileProcessing.commitPendingFileEvents()),'Failed saves were acknowledged');
|
||||||
|
for(const {path} of fixtures)require(await pending(path),'Failed save was lost: '+path);
|
||||||
|
} finally {
|
||||||
|
core.localDatabase.putDBEntry=original;
|
||||||
|
}
|
||||||
|
return JSON.stringify({failedSavesRetained:failed});
|
||||||
|
`)
|
||||||
|
);
|
||||||
|
await session!.app.stop();
|
||||||
|
session = undefined;
|
||||||
|
await start(true);
|
||||||
|
observations.push(
|
||||||
|
await evaluate(`
|
||||||
|
const fixtures=${JSON.stringify(replay)};
|
||||||
|
for(const {path,content} of fixtures) {
|
||||||
|
const databasePath=path.endsWith('.json')?'i:'+path:config.filenameWithUnifiedKey(path);
|
||||||
|
require(represents(await entry(databasePath),content),'Start-up did not recover '+path);
|
||||||
|
require(!(await pending(path)),'Recovered save is still pending: '+path);
|
||||||
|
}
|
||||||
|
return JSON.stringify({replayedBeforeReadiness:fixtures.length,ready:core.services.appLifecycle.isReady()});
|
||||||
|
`)
|
||||||
|
);
|
||||||
|
console.log(JSON.stringify({ ok: true, bundleHash, observations }, null, 2));
|
||||||
|
} finally {
|
||||||
|
await session?.app.stop();
|
||||||
|
await vault.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
await main();
|
||||||
@@ -43,11 +43,13 @@ type TestWorkspace = {
|
|||||||
type TestLogModule = {
|
type TestLogModule = {
|
||||||
adjustStatusDivPosition?: () => void;
|
adjustStatusDivPosition?: () => void;
|
||||||
statusLog?: WritableSource<string>;
|
statusLog?: WritableSource<string>;
|
||||||
|
activeFileStatus?: WritableSource<string>;
|
||||||
|
p2pLogCollector?: { p2pReplicationLine: WritableSource<string> };
|
||||||
statusDiv?: HTMLElement;
|
statusDiv?: HTMLElement;
|
||||||
statusLine?: HTMLElement;
|
statusLine?: HTMLElement;
|
||||||
statusLineRenderer?: { render?: (parts: readonly unknown[]) => void };
|
statusLineRenderer?: { render?: (parts: readonly unknown[]) => void };
|
||||||
statusBarRenderer?: { render?: (parts: readonly unknown[]) => void };
|
statusBarRenderer?: { render?: (parts: readonly unknown[]) => void };
|
||||||
statusBarLabels?: WritableSource<{ parts: readonly unknown[] }>;
|
statusBarLabels?: WritableSource<{ parts: readonly unknown[]; message: string }>;
|
||||||
__addLog?: (message: unknown, level?: number, key?: string) => void;
|
__addLog?: (message: unknown, level?: number, key?: string) => void;
|
||||||
};
|
};
|
||||||
type TestCore = {
|
type TestCore = {
|
||||||
@@ -57,6 +59,17 @@ type TestCore = {
|
|||||||
replicator?: {
|
replicator?: {
|
||||||
replicationStatics?: WritableSource<TestReplicationStatics>;
|
replicationStatics?: WritableSource<TestReplicationStatics>;
|
||||||
};
|
};
|
||||||
|
replication?: {
|
||||||
|
replicationResultCount: WritableSource<number>;
|
||||||
|
databaseQueueCount: WritableSource<number>;
|
||||||
|
storageApplyingCount: WritableSource<number>;
|
||||||
|
};
|
||||||
|
fileProcessing?: {
|
||||||
|
processing: WritableSource<number>;
|
||||||
|
totalQueued: WritableSource<number>;
|
||||||
|
batched: WritableSource<number>;
|
||||||
|
};
|
||||||
|
conflict?: { conflictProcessQueueCount: WritableSource<number> };
|
||||||
API?: {
|
API?: {
|
||||||
requestCount?: WritableSource<number>;
|
requestCount?: WritableSource<number>;
|
||||||
responseCount?: WritableSource<number>;
|
responseCount?: WritableSource<number>;
|
||||||
@@ -120,7 +133,8 @@ async function installFixtureFile(page: Page): Promise<void> {
|
|||||||
}
|
}
|
||||||
let file = vault.getAbstractFileByPath(path);
|
let file = vault.getAbstractFileByPath(path);
|
||||||
if (!file) file = await vault.create(path, "Replication status renderer fixture.\n");
|
if (!file) file = await vault.create(path, "Replication status renderer fixture.\n");
|
||||||
const leaf = workspace.getLeaf(false);
|
// Opening a new tab emits the active-leaf transition used by the overlay placement hook.
|
||||||
|
const leaf = workspace.getLeaf("tab");
|
||||||
if (!leaf.openFile) return { ready: false, reason: "The active workspace leaf cannot open a file." };
|
if (!leaf.openFile) return { ready: false, reason: "The active workspace leaf cannot open a file." };
|
||||||
await leaf.openFile(file);
|
await leaf.openFile(file);
|
||||||
workspace.setActiveLeaf(leaf, { focus: true });
|
workspace.setActiveLeaf(leaf, { focus: true });
|
||||||
@@ -412,6 +426,9 @@ async function measureNumericWidths(page: Page): Promise<Record<(typeof numericK
|
|||||||
if (getComputedStyle(value).fontVariantNumeric !== "tabular-nums") {
|
if (getComputedStyle(value).fontVariantNumeric !== "tabular-nums") {
|
||||||
throw new Error(`The numeric status item ${key} does not use tabular figures.`);
|
throw new Error(`The numeric status item ${key} does not use tabular figures.`);
|
||||||
}
|
}
|
||||||
|
if (getComputedStyle(value).textAlign !== "right") {
|
||||||
|
throw new Error(`The numeric status item ${key} is not right-aligned.`);
|
||||||
|
}
|
||||||
result[key] = value.getBoundingClientRect().width;
|
result[key] = value.getBoundingClientRect().width;
|
||||||
}
|
}
|
||||||
return result;
|
return result;
|
||||||
@@ -707,6 +724,122 @@ async function hideMainWindow(page: Page): Promise<MainWindowControl> {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function assertLegacyStatusPaths(page: Page): Promise<void> {
|
||||||
|
await page.evaluate(async (timeout) => {
|
||||||
|
const core = (globalThis as TestWindow).app?.plugins?.plugins?.["obsidian-livesync"]?.core;
|
||||||
|
const moduleLog = core?.modules?.find((entry): entry is TestLogModule => {
|
||||||
|
if (typeof entry !== "object" || entry === null) return false;
|
||||||
|
return (entry as TestLogModule).statusBarLabels !== undefined;
|
||||||
|
});
|
||||||
|
const services = core?.services;
|
||||||
|
const settings = services?.setting?.currentSettings?.();
|
||||||
|
if (
|
||||||
|
!settings ||
|
||||||
|
!services?.replication ||
|
||||||
|
!services.fileProcessing ||
|
||||||
|
!services.conflict ||
|
||||||
|
!moduleLog?.activeFileStatus ||
|
||||||
|
!moduleLog.p2pLogCollector ||
|
||||||
|
!moduleLog.statusBarLabels
|
||||||
|
)
|
||||||
|
throw new Error("The legacy status sources are unavailable.");
|
||||||
|
const counters = [
|
||||||
|
{ key: "file-processing", mark: "⏳", source: services.fileProcessing.processing, value: 2 },
|
||||||
|
{ key: "file-pending", mark: "🛫", source: services.fileProcessing.totalQueued, value: 3 },
|
||||||
|
{ key: "file-batched", mark: "📬", source: services.fileProcessing.batched, value: 4 },
|
||||||
|
{ key: "queue-replication", mark: "📥", source: services.replication.replicationResultCount, value: 5 },
|
||||||
|
{ key: "queue-database", mark: "📄", source: services.replication.databaseQueueCount, value: 6 },
|
||||||
|
{ key: "queue-storage", mark: "💾", source: services.replication.storageApplyingCount, value: 7 },
|
||||||
|
{ key: "queue-conflicts", mark: "🔩", source: services.conflict.conflictProcessQueueCount, value: 8 },
|
||||||
|
];
|
||||||
|
const warning = moduleLog.activeFileStatus;
|
||||||
|
const p2p = moduleLog.p2pLogCollector.p2pReplicationLine;
|
||||||
|
const labels = moduleLog.statusBarLabels;
|
||||||
|
const previous = {
|
||||||
|
counters: counters.map(({ source }) => source.value),
|
||||||
|
warning: warning.value,
|
||||||
|
p2p: p2p.value,
|
||||||
|
hideFileWarningNotice: settings.hideFileWarningNotice,
|
||||||
|
};
|
||||||
|
const peerLine = "⚡ E2E peer ↑ 11/12 ↓ 13/14";
|
||||||
|
const wait = {
|
||||||
|
async until(condition: () => boolean, message: string) {
|
||||||
|
const deadline = Date.now() + timeout;
|
||||||
|
while (!condition()) {
|
||||||
|
if (Date.now() >= deadline) throw new Error(message);
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
for (const { source, value } of counters) source.value = value;
|
||||||
|
p2p.value = peerLine;
|
||||||
|
settings.hideFileWarningNotice = false;
|
||||||
|
warning.value = "E2E file warning";
|
||||||
|
await wait.until(
|
||||||
|
() =>
|
||||||
|
[".livesync-status-statusline", ".syncstatusbar"].every((root) =>
|
||||||
|
counters.every(({ key, value }) => {
|
||||||
|
const item = document.querySelector<HTMLElement>(`${root} .livesync-stat-${key}`);
|
||||||
|
return (
|
||||||
|
item !== null &&
|
||||||
|
getComputedStyle(item).display !== "none" &&
|
||||||
|
item.querySelector(".livesync-stat-value")?.textContent?.trim() === String(value)
|
||||||
|
);
|
||||||
|
})
|
||||||
|
),
|
||||||
|
"Queue values did not reach both status displays."
|
||||||
|
);
|
||||||
|
await wait.until(
|
||||||
|
() =>
|
||||||
|
document.querySelector(".livesync-status-statusline .livesync-stat-p2p")?.textContent?.trim() ===
|
||||||
|
peerLine &&
|
||||||
|
document
|
||||||
|
.querySelector(".livesync-status-messagearea")
|
||||||
|
?.textContent?.includes("⚠️ E2E file warning") === true,
|
||||||
|
"The P2P line or file warning was lost from the editor display."
|
||||||
|
);
|
||||||
|
const plain = labels.value.message.replace(/\u2007/gu, "");
|
||||||
|
if (
|
||||||
|
!counters.every(({ mark, value }) => plain.includes(` ${mark}${value}`)) ||
|
||||||
|
!plain.endsWith("\n" + peerLine)
|
||||||
|
) {
|
||||||
|
throw new Error("The plain-text status lost a queue count or its separate P2P line.");
|
||||||
|
}
|
||||||
|
if (document.querySelector(".syncstatusbar .livesync-stat-p2p:not(.livesync-stat-inactive)")) {
|
||||||
|
throw new Error("The status bar unexpectedly included the second-line P2P status.");
|
||||||
|
}
|
||||||
|
settings.hideFileWarningNotice = true;
|
||||||
|
warning.value = "E2E file warning as an icon";
|
||||||
|
await wait.until(
|
||||||
|
() =>
|
||||||
|
document
|
||||||
|
.querySelector(".livesync-status-statusline .livesync-stat-file-skip")
|
||||||
|
?.textContent?.trim() === "⛔ SKIP" &&
|
||||||
|
document
|
||||||
|
.querySelector(".livesync-status-messagearea")
|
||||||
|
?.textContent?.includes("E2E file warning") === false,
|
||||||
|
"The file warning did not switch to its existing compact icon."
|
||||||
|
);
|
||||||
|
p2p.value = "";
|
||||||
|
await wait.until(
|
||||||
|
() =>
|
||||||
|
document.querySelector(".syncstatusbar .livesync-stat-file-skip")?.textContent?.trim() ===
|
||||||
|
"⛔ SKIP" && labels.value.message.endsWith(" ⛔ SKIP"),
|
||||||
|
"The single-line status lost the compact file warning."
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
counters.forEach(({ source }, index) => {
|
||||||
|
source.value = previous.counters[index];
|
||||||
|
});
|
||||||
|
settings.hideFileWarningNotice = previous.hideFileWarningNotice;
|
||||||
|
p2p.value = previous.p2p;
|
||||||
|
warning.value = previous.warning;
|
||||||
|
}
|
||||||
|
}, uiTimeoutMs);
|
||||||
|
console.log("Queue counts, P2P text, file warnings, and plain-text status remain available.");
|
||||||
|
}
|
||||||
|
|
||||||
async function runStatusScenario(session: ObsidianLiveSyncSession): Promise<void> {
|
async function runStatusScenario(session: ObsidianLiveSyncSession): Promise<void> {
|
||||||
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||||
let popoutMarker: string | undefined;
|
let popoutMarker: string | undefined;
|
||||||
@@ -865,6 +998,7 @@ async function runStatusScenario(session: ObsidianLiveSyncSession): Promise<void
|
|||||||
const returnedLog = `E2E main-window log ${Date.now()}`;
|
const returnedLog = `E2E main-window log ${Date.now()}`;
|
||||||
await appendLiveSyncLog(page, returnedLog);
|
await appendLiveSyncLog(page, returnedLog);
|
||||||
await waitForLogMessage(page, returnedLog);
|
await waitForLogMessage(page, returnedLog);
|
||||||
|
await assertLegacyStatusPaths(page);
|
||||||
console.log("Replication status, split-pane ownership, and popout status/log checks passed.");
|
console.log("Replication status, split-pane ownership, and popout status/log checks passed.");
|
||||||
} finally {
|
} finally {
|
||||||
if (mainWindow) await mainWindow.restore();
|
if (mainWindow) await mainWindow.restore();
|
||||||
|
|||||||
@@ -44,6 +44,9 @@ const focusedScenarios = new Set([
|
|||||||
"hidden-file-snippet-sync",
|
"hidden-file-snippet-sync",
|
||||||
"hidden-file-selectors",
|
"hidden-file-selectors",
|
||||||
"customisation-sync",
|
"customisation-sync",
|
||||||
|
"customisation-catalogue-reload",
|
||||||
|
"optional-file-save-completion",
|
||||||
|
"livesync-batch",
|
||||||
"received-change-readiness",
|
"received-change-readiness",
|
||||||
"internal-metadata-doctor",
|
"internal-metadata-doctor",
|
||||||
"setting-markdown-export",
|
"setting-markdown-export",
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { mkdir } from "node:fs/promises";
|
import { mkdir, readFile } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
import { assertLocatorWithinViewport, assertNoHorizontalOverflow } from "@vrtmrz/obsidian-test-session";
|
import { assertLocatorWithinViewport, assertNoHorizontalOverflow } from "@vrtmrz/obsidian-test-session";
|
||||||
import { VER } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
import { VER } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||||
@@ -59,6 +60,10 @@ type LiveSyncTestPlugin = {
|
|||||||
useEdgeCaseMode: boolean;
|
useEdgeCaseMode: boolean;
|
||||||
hashCacheMaxCount: number;
|
hashCacheMaxCount: number;
|
||||||
customChunkSize: number;
|
customChunkSize: number;
|
||||||
|
liveSync: boolean;
|
||||||
|
periodicReplication: boolean;
|
||||||
|
batchSave: boolean;
|
||||||
|
batchSaveInLiveSync: boolean;
|
||||||
};
|
};
|
||||||
config: { getValue(key: string, defaultValue: string): Promise<string> };
|
config: { getValue(key: string, defaultValue: string): Promise<string> };
|
||||||
};
|
};
|
||||||
@@ -643,6 +648,105 @@ async function verifyConfigDoctorFollowsCompatibilityReview(): Promise<string> {
|
|||||||
return screenshot;
|
return screenshot;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function verifyBatchSettings(vaultPath: string): Promise<void> {
|
||||||
|
await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
|
||||||
|
let navigator = await openLiveSyncSettings(page, uiTimeoutMs);
|
||||||
|
let settingsPage = await navigator.openPage("Sync Settings");
|
||||||
|
const row = (name: string) =>
|
||||||
|
settingsPage.locator(".setting-item").filter({ has: page.getByText(name, { exact: true }) });
|
||||||
|
const legacyName = "Batch database update";
|
||||||
|
const liveName = "Batch database updates in LiveSync mode";
|
||||||
|
const delays = ["Minimum delay for batch database updating", "Maximum delay for batch database updating"];
|
||||||
|
|
||||||
|
async function assertState(liveSync: boolean, batchSave: boolean, batchSaveInLiveSync: boolean) {
|
||||||
|
await page.waitForFunction(
|
||||||
|
(expected) => {
|
||||||
|
const settings = (globalThis as ObsidianTestGlobal).app?.plugins?.plugins[
|
||||||
|
"obsidian-livesync"
|
||||||
|
]?.core.services.setting.currentSettings();
|
||||||
|
return (
|
||||||
|
settings?.liveSync === expected.liveSync &&
|
||||||
|
settings?.batchSave === expected.batchSave &&
|
||||||
|
settings?.batchSaveInLiveSync === expected.batchSaveInLiveSync
|
||||||
|
);
|
||||||
|
},
|
||||||
|
{ liveSync, batchSave, batchSaveInLiveSync },
|
||||||
|
{ timeout: uiTimeoutMs }
|
||||||
|
);
|
||||||
|
const visibleName = liveSync ? liveName : legacyName;
|
||||||
|
await row(visibleName).waitFor({ state: "visible", timeout: uiTimeoutMs });
|
||||||
|
await row(liveSync ? legacyName : liveName).waitFor({ state: "hidden", timeout: uiTimeoutMs });
|
||||||
|
const enabled = liveSync ? batchSaveInLiveSync : batchSave;
|
||||||
|
if ((await row(visibleName).locator(".checkbox-container.is-enabled").count()) !== Number(enabled)) {
|
||||||
|
throw new Error(`The displayed batching choice did not match ${visibleName}: ${enabled}`);
|
||||||
|
}
|
||||||
|
for (const name of delays) {
|
||||||
|
await row(name).waitFor({ state: enabled ? "visible" : "hidden", timeout: uiTimeoutMs });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function mode(value: "ONEVENTS" | "PERIODIC" | "LIVESYNC") {
|
||||||
|
await row("Sync Mode").locator("select").selectOption(value);
|
||||||
|
await page.waitForFunction(
|
||||||
|
(value) => {
|
||||||
|
const settings = (globalThis as ObsidianTestGlobal).app?.plugins?.plugins[
|
||||||
|
"obsidian-livesync"
|
||||||
|
]?.core.services.setting.currentSettings();
|
||||||
|
return (
|
||||||
|
settings?.liveSync === (value === "LIVESYNC") &&
|
||||||
|
settings?.periodicReplication === (value === "PERIODIC")
|
||||||
|
);
|
||||||
|
},
|
||||||
|
value,
|
||||||
|
{ timeout: uiTimeoutMs }
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
await assertState(false, false, false);
|
||||||
|
await row(legacyName).locator(".checkbox-container").click();
|
||||||
|
await assertState(false, true, false);
|
||||||
|
await mode("LIVESYNC");
|
||||||
|
await assertState(true, true, false);
|
||||||
|
await row(liveName).locator(".checkbox-container").click();
|
||||||
|
await assertState(true, true, true);
|
||||||
|
await mode("PERIODIC");
|
||||||
|
await assertState(false, true, true);
|
||||||
|
await row(legacyName).locator(".checkbox-container").click();
|
||||||
|
await assertState(false, false, true);
|
||||||
|
await mode("LIVESYNC");
|
||||||
|
await assertState(true, false, true);
|
||||||
|
await row(liveName).locator(".checkbox-container").click();
|
||||||
|
await assertState(true, false, false);
|
||||||
|
await mode("ONEVENTS");
|
||||||
|
await assertState(false, false, false);
|
||||||
|
await row(legacyName).locator(".checkbox-container").click();
|
||||||
|
await assertState(false, true, false);
|
||||||
|
await navigator.close();
|
||||||
|
navigator = await openLiveSyncSettings(page, uiTimeoutMs);
|
||||||
|
settingsPage = await navigator.openPage("Sync Settings");
|
||||||
|
await assertState(false, true, false);
|
||||||
|
await mode("LIVESYNC");
|
||||||
|
await assertState(true, true, false);
|
||||||
|
await navigator.close();
|
||||||
|
const saved = JSON.parse(
|
||||||
|
await readFile(join(vaultPath, ".obsidian/plugins/obsidian-livesync/data.json"), "utf8")
|
||||||
|
);
|
||||||
|
if (saved.batchSave !== true || saved.batchSaveInLiveSync !== false || saved.liveSync !== true) {
|
||||||
|
throw new Error("The independent batching choices were not saved to plug-in data.");
|
||||||
|
}
|
||||||
|
navigator = await openLiveSyncSettings(page, uiTimeoutMs);
|
||||||
|
settingsPage = await navigator.openPage("Sync Settings");
|
||||||
|
await assertState(true, true, false);
|
||||||
|
await mode("ONEVENTS");
|
||||||
|
await row(legacyName).locator(".checkbox-container").click();
|
||||||
|
await assertState(false, false, false);
|
||||||
|
await navigator.close();
|
||||||
|
console.log(
|
||||||
|
"Batching controls start disabled, show the current mode's delays, and retain independent saved choices."
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
async function verifyEffectiveSettings(): Promise<"declarative" | "imperative"> {
|
async function verifyEffectiveSettings(): Promise<"declarative" | "imperative"> {
|
||||||
return await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
|
return await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
|
||||||
const sleepPreferences = await page.evaluate(() => {
|
const sleepPreferences = await page.evaluate(() => {
|
||||||
@@ -1080,6 +1184,7 @@ async function main(): Promise<void> {
|
|||||||
console.log(`Config Doctor screenshot: ${configDoctorScreenshot}`);
|
console.log(`Config Doctor screenshot: ${configDoctorScreenshot}`);
|
||||||
}
|
}
|
||||||
settingsRenderer = await verifyEffectiveSettings();
|
settingsRenderer = await verifyEffectiveSettings();
|
||||||
|
await verifyBatchSettings(vault.path);
|
||||||
const initialisation = await verifyPendingSettingsInitialisationFlow();
|
const initialisation = await verifyPendingSettingsInitialisationFlow();
|
||||||
const p2pInitialisation = await captureP2PSettingsInitialisationDialogue();
|
const p2pInitialisation = await captureP2PSettingsInitialisationDialogue();
|
||||||
console.log(
|
console.log(
|
||||||
|
|||||||
+29
@@ -12,10 +12,39 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
|
### Customisation Sync
|
||||||
|
|
||||||
|
#### Fixed
|
||||||
|
|
||||||
|
- We can reload the customisation list and apply every file in a multi-file item on the first attempt. (#1271)
|
||||||
|
- Reloading or clearing the list also prevents earlier pending updates from restoring removed files and items.
|
||||||
|
|
||||||
|
### Setup tools
|
||||||
|
|
||||||
|
#### Fixed
|
||||||
|
|
||||||
|
- We can generate a Setup URI with prefixed connection and encryption variables, avoiding collisions with shell or system variables such as `hostname` and `USERNAME`. (#817, #1274)
|
||||||
|
- The generator reads `LIVESYNC_HOST`, `LIVESYNC_USER`, `LIVESYNC_PASS`, `LIVESYNC_DB`, and `LIVESYNC_PASSPHRASE`. The previous names remain supported when the corresponding prefixed value is unset or blank.
|
||||||
|
|
||||||
### Synchronisation
|
### Synchronisation
|
||||||
|
|
||||||
|
#### New Feature
|
||||||
|
|
||||||
- We can see CouchDB change-history comparison progressing before new documents arrive, with a ⏩ status icon and a comparison count before the upload count. An unchanged count disappears after three seconds.
|
- We can see CouchDB change-history comparison progressing before new documents arrive, with a ⏩ status icon and a comparison count before the upload count. An unchanged count disappears after three seconds.
|
||||||
- We can hide or style individual status items with CSS snippets using the `livesync-stat-` classes. Tabular, right-aligned numbers retain their width while visible, reducing movement as counts change. Wider gaps between editor status items make the counts easier to read. The `livesync-stat-inactive` class lets CSS snippets control how inactive items disappear.
|
- We can hide or style individual status items with CSS snippets using the `livesync-stat-` classes. Tabular, right-aligned numbers retain their width while visible, reducing movement as counts change. Wider gaps between editor status items make the counts easier to read. The `livesync-stat-inactive` class lets CSS snippets control how inactive items disappear.
|
||||||
|
- We can enable **Batch database updates in LiveSync mode** to combine repeated local edits before sending them to other devices.
|
||||||
|
- The choice starts disabled, including when older settings contain the previously ignored batch option. Each mode retains its own choice and uses the same minimum and maximum delays.
|
||||||
|
- Batching delays remote visibility and reduces intermediate updates. It does not establish when a template has finished or prevent genuine concurrent edits.
|
||||||
|
|
||||||
|
#### Fixed
|
||||||
|
|
||||||
|
- Local changes remain available for recovery while their database save is running, including when the application stops after a batch delay ends.
|
||||||
|
- Manual synchronisation and settings application now wait for the accepted local saves to finish and report failures. Pending events retain the existing snapshot format.
|
||||||
|
- Hidden File Sync and Customisation Sync now report completion after saving, including delayed customisation updates. Failed saves remain pending, while unchanged notifications allow later edits to proceed.
|
||||||
|
- Start-up reconciliation checks content when modification times are equal, so an edit within the same timestamp interval is not mistaken for an unchanged file.
|
||||||
|
- Existing revision provenance preserves the correct edit branch. Unreadable or continually changing content remains unresolved rather than being reported as successfully reconciled.
|
||||||
|
- We receive a clear Notice when **Automatic** requires Hidden File Sync to be enabled on this device. Our existing mode remains unchanged, and hidden-file initialisation cannot start while the feature is disabled. (#1270)
|
||||||
|
- We can initialise a whole plug-in's **MAIN** and **DATA** selections using the chosen Automatic initial action.
|
||||||
- We can run startup integrity checks with lower memory use for healthy Vault files. Files that change between the size check and content reading no longer trigger an unnecessary repair warning when their actual sizes agree and there are no conflicts. (#1283)
|
- We can run startup integrity checks with lower memory use for healthy Vault files. Files that change between the size check and content reading no longer trigger an unnecessary repair warning when their actual sizes agree and there are no conflicts. (#1283)
|
||||||
|
|
||||||
## 1.0.40
|
## 1.0.40
|
||||||
|
|||||||
+10
-8
@@ -31,21 +31,23 @@ Authentication and other non-retryable HTTP failures stop immediately. Network a
|
|||||||
|
|
||||||
The existing `flyio/generate_setupuri.ts` path remains a CouchDB-only compatibility wrapper for the Fly.io deployment script.
|
The existing `flyio/generate_setupuri.ts` path remains a CouchDB-only compatibility wrapper for the Fly.io deployment script.
|
||||||
|
|
||||||
|
The connection and Vault encryption values use a `LIVESYNC_` prefix so that they do not collide with shell or system variables such as `hostname` and `USERNAME`. The unprefixed names `hostname`, `username`, `password`, `database`, and `passphrase` are still read when the corresponding prefixed value is unset or blank.
|
||||||
|
|
||||||
The generator creates a fresh random ID key by default and includes it in the encrypted Setup URI. It prints a tagged `sls-id-v1:` recovery code. Set `id_recovery_code` to that code when generating another URI for the same Vault; a new run without it creates a different key. This restores only the ID key: reuse the original connection details too. For P2P, provide the original `p2p_room_id` and `p2p_passphrase` because omitted values are generated afresh. Set `id_mode=legacy` to generate a URI with the previous ID behaviour. `id_mode=legacy` and `id_recovery_code` cannot be combined. Keep the recovery code private and retain it if every device might be lost.
|
The generator creates a fresh random ID key by default and includes it in the encrypted Setup URI. It prints a tagged `sls-id-v1:` recovery code. Set `id_recovery_code` to that code when generating another URI for the same Vault; a new run without it creates a different key. This restores only the ID key: reuse the original connection details too. For P2P, provide the original `p2p_room_id` and `p2p_passphrase` because omitted values are generated afresh. Set `id_mode=legacy` to generate a URI with the previous ID behaviour. `id_mode=legacy` and `id_recovery_code` cannot be combined. Keep the recovery code private and retain it if every device might be lost.
|
||||||
|
|
||||||
### CouchDB
|
### CouchDB
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
export hostname=https://couch.example.com
|
export LIVESYNC_HOST=https://couch.example.com
|
||||||
export username=couchdb-admin-username
|
export LIVESYNC_USER=couchdb-admin-username
|
||||||
export password=couchdb-admin-password
|
export LIVESYNC_PASS=couchdb-admin-password
|
||||||
export database=obsidiannotes
|
export LIVESYNC_DB=obsidiannotes
|
||||||
export passphrase=a-strong-vault-encryption-passphrase
|
export LIVESYNC_PASSPHRASE=a-strong-vault-encryption-passphrase
|
||||||
export uri_passphrase=a-separate-setup-uri-passphrase # Optional
|
export uri_passphrase=a-separate-setup-uri-passphrase # Optional
|
||||||
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
|
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
|
||||||
```
|
```
|
||||||
|
|
||||||
If `uri_passphrase` is omitted, the tool generates and prints a cryptographically random one. Store the Setup URI and its passphrase separately. The `passphrase` value protects synchronised Vault data and must also be stored safely.
|
If `uri_passphrase` is omitted, the tool generates and prints a cryptographically random one. Store the Setup URI and its passphrase separately. The `LIVESYNC_PASSPHRASE` value protects synchronised Vault data and must also be stored safely.
|
||||||
|
|
||||||
The generator defaults to `uri_mode=ephemeral`. The URI opens only during the current fixed seven-day UTC window, and the tool prints its exact end time. It may have less than seven days remaining when generated. For an indefinitely reusable URI, set `uri_mode=persistent` before running the command. Persistent uses the existing encrypted format, which older clients that already support that format can read. The time condition controls opening the URI; it does not revoke settings or credentials after import.
|
The generator defaults to `uri_mode=ephemeral`. The URI opens only during the current fixed seven-day UTC window, and the tool prints its exact end time. It may have less than seven days remaining when generated. For an indefinitely reusable URI, set `uri_mode=persistent` before running the command. Persistent uses the existing encrypted format, which older clients that already support that format can read. The time condition controls opening the URI; it does not revoke settings or credentials after import.
|
||||||
|
|
||||||
@@ -59,7 +61,7 @@ export secret_key=<SECRET KEY>
|
|||||||
export bucket=vault-data
|
export bucket=vault-data
|
||||||
export region=auto
|
export region=auto
|
||||||
export bucket_prefix=team-a # Optional
|
export bucket_prefix=team-a # Optional
|
||||||
export passphrase=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||||
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
|
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -69,7 +71,7 @@ Optional Object Storage variables are `use_custom_request_handler`, `force_path_
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
export remote_type=p2p
|
export remote_type=p2p
|
||||||
export passphrase=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||||
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
|
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { fileURLToPath } from "node:url";
|
||||||
import {
|
import {
|
||||||
decodeSettingsFromSetupURI,
|
decodeSettingsFromSetupURI,
|
||||||
DEFAULT_SETTINGS,
|
DEFAULT_SETTINGS,
|
||||||
@@ -9,6 +10,76 @@ function assert(condition: unknown, message: string): asserts condition {
|
|||||||
if (!condition) throw new Error(message);
|
if (!condition) throw new Error(message);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
Deno.test("reads LIVESYNC_ variables before the unprefixed names", async () => {
|
||||||
|
const generated = await generateSetupURI({
|
||||||
|
LIVESYNC_HOST: "https://couch.example.test",
|
||||||
|
LIVESYNC_USER: "alice",
|
||||||
|
LIVESYNC_PASS: "couch-secret",
|
||||||
|
LIVESYNC_DB: "notes",
|
||||||
|
LIVESYNC_PASSPHRASE: "vault-secret",
|
||||||
|
hostname: "https://shell.example.test",
|
||||||
|
username: "shell-user",
|
||||||
|
password: "shell-secret",
|
||||||
|
database: "shell-db",
|
||||||
|
passphrase: "shell-passphrase",
|
||||||
|
uri_passphrase: "setup-secret",
|
||||||
|
});
|
||||||
|
const decoded = await decodeSettingsFromSetupURI(
|
||||||
|
generated.setupURI,
|
||||||
|
generated.setupPassphrase,
|
||||||
|
);
|
||||||
|
assert(decoded, "Commonlib could not decode the CouchDB Setup URI");
|
||||||
|
const effective = { ...DEFAULT_SETTINGS, ...decoded };
|
||||||
|
assert(
|
||||||
|
effective.couchDB_URI === "https://couch.example.test" &&
|
||||||
|
effective.couchDB_USER === "alice" &&
|
||||||
|
effective.couchDB_PASSWORD === "couch-secret" &&
|
||||||
|
effective.couchDB_DBNAME === "notes",
|
||||||
|
"the CouchDB connection did not come from the LIVESYNC_ variables",
|
||||||
|
);
|
||||||
|
assert(
|
||||||
|
effective.passphrase === "vault-secret",
|
||||||
|
"the Vault passphrase did not come from LIVESYNC_PASSPHRASE",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
Deno.test("falls back to the unprefixed names when LIVESYNC_ variables are blank", async () => {
|
||||||
|
const generated = await generateSetupURI({
|
||||||
|
LIVESYNC_HOST: " ",
|
||||||
|
hostname: "https://couch.example.test",
|
||||||
|
username: "alice",
|
||||||
|
password: "couch-secret",
|
||||||
|
database: "notes",
|
||||||
|
passphrase: "vault-secret",
|
||||||
|
uri_passphrase: "setup-secret",
|
||||||
|
});
|
||||||
|
const decoded = await decodeSettingsFromSetupURI(
|
||||||
|
generated.setupURI,
|
||||||
|
generated.setupPassphrase,
|
||||||
|
);
|
||||||
|
assert(decoded, "Commonlib could not decode the CouchDB Setup URI");
|
||||||
|
const effective = { ...DEFAULT_SETTINGS, ...decoded };
|
||||||
|
assert(
|
||||||
|
effective.couchDB_URI === "https://couch.example.test" &&
|
||||||
|
effective.couchDB_USER === "alice" &&
|
||||||
|
effective.couchDB_PASSWORD === "couch-secret" &&
|
||||||
|
effective.couchDB_DBNAME === "notes" &&
|
||||||
|
effective.passphrase === "vault-secret",
|
||||||
|
"the generator did not fall back to the unprefixed names",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
Deno.test("names both variables when a CouchDB value is missing", async () => {
|
||||||
|
let rejected = false;
|
||||||
|
try {
|
||||||
|
await generateSetupURI({ uri_passphrase: "setup-secret" });
|
||||||
|
} catch (error) {
|
||||||
|
rejected = error instanceof Error &&
|
||||||
|
error.message === "LIVESYNC_HOST (or hostname) is required";
|
||||||
|
}
|
||||||
|
assert(rejected, "the generator did not name both host variables");
|
||||||
|
});
|
||||||
|
|
||||||
Deno.test("generates an Object Storage Setup URI with a selected S3 profile", async () => {
|
Deno.test("generates an Object Storage Setup URI with a selected S3 profile", async () => {
|
||||||
const generated = await generateSetupURI({
|
const generated = await generateSetupURI({
|
||||||
remote_type: "s3",
|
remote_type: "s3",
|
||||||
@@ -259,3 +330,171 @@ for (const mode of ["ephemeral", "persistent"] as const) {
|
|||||||
assert(rejected, "legacy mode silently ignored a recovery code");
|
assert(rejected, "legacy mode silently ignored a recovery code");
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const couchDBConnection = {
|
||||||
|
hostname: "https://couch.example.test",
|
||||||
|
username: "alice",
|
||||||
|
password: "couch-secret",
|
||||||
|
database: "notes",
|
||||||
|
passphrase: "vault-secret",
|
||||||
|
};
|
||||||
|
const prefixedCouchDBConnection = {
|
||||||
|
LIVESYNC_HOST: couchDBConnection.hostname,
|
||||||
|
LIVESYNC_USER: couchDBConnection.username,
|
||||||
|
LIVESYNC_PASS: couchDBConnection.password,
|
||||||
|
LIVESYNC_DB: couchDBConnection.database,
|
||||||
|
LIVESYNC_PASSPHRASE: couchDBConnection.passphrase,
|
||||||
|
};
|
||||||
|
const commandEnvironments: { name: string; values: Record<string, string> }[] =
|
||||||
|
[
|
||||||
|
{ name: "prefixed values only", values: prefixedCouchDBConnection },
|
||||||
|
{
|
||||||
|
name: "prefixed values before conflicting legacy values",
|
||||||
|
values: {
|
||||||
|
...prefixedCouchDBConnection,
|
||||||
|
hostname: "https://shell.example.test",
|
||||||
|
username: "shell-user",
|
||||||
|
password: "shell-password",
|
||||||
|
database: "shell-database",
|
||||||
|
passphrase: "shell-passphrase",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "blank prefixed values fall back to legacy values",
|
||||||
|
values: {
|
||||||
|
...couchDBConnection,
|
||||||
|
LIVESYNC_HOST: " ",
|
||||||
|
LIVESYNC_USER: "\t",
|
||||||
|
LIVESYNC_PASS: " ",
|
||||||
|
LIVESYNC_DB: "\t",
|
||||||
|
LIVESYNC_PASSPHRASE: " ",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{ name: "compatibility: legacy values only", values: couchDBConnection },
|
||||||
|
];
|
||||||
|
|
||||||
|
async function runGeneratorCommand(
|
||||||
|
script: URL,
|
||||||
|
values: Record<string, string>,
|
||||||
|
) {
|
||||||
|
const environment = Deno.env.toObject();
|
||||||
|
const inputNames = new Set([
|
||||||
|
...Object.keys(couchDBConnection),
|
||||||
|
...Object.keys(prefixedCouchDBConnection),
|
||||||
|
"remote_type",
|
||||||
|
"uri_passphrase",
|
||||||
|
"uri_mode",
|
||||||
|
"id_mode",
|
||||||
|
"id_recovery_code",
|
||||||
|
"endpoint",
|
||||||
|
"access_key",
|
||||||
|
"secret_key",
|
||||||
|
"bucket",
|
||||||
|
"region",
|
||||||
|
"bucket_prefix",
|
||||||
|
"bucket_custom_headers",
|
||||||
|
"use_custom_request_handler",
|
||||||
|
"force_path_style",
|
||||||
|
"p2p_room_id",
|
||||||
|
"p2p_passphrase",
|
||||||
|
"p2p_relays",
|
||||||
|
"p2p_app_id",
|
||||||
|
"p2p_auto_start",
|
||||||
|
"p2p_auto_broadcast",
|
||||||
|
].map((name) => name.toLowerCase()));
|
||||||
|
for (const name of Object.keys(environment)) {
|
||||||
|
if (inputNames.has(name.toLowerCase())) delete environment[name];
|
||||||
|
}
|
||||||
|
const result = await new Deno.Command(Deno.execPath(), {
|
||||||
|
args: [
|
||||||
|
"run",
|
||||||
|
"--config",
|
||||||
|
fileURLToPath(new URL("../flyio/deno.jsonc", import.meta.url)),
|
||||||
|
"--frozen",
|
||||||
|
"--lock",
|
||||||
|
fileURLToPath(new URL("../flyio/deno.lock", import.meta.url)),
|
||||||
|
"--allow-env",
|
||||||
|
script.href,
|
||||||
|
],
|
||||||
|
clearEnv: true,
|
||||||
|
env: {
|
||||||
|
...environment,
|
||||||
|
uri_passphrase: "setup-secret",
|
||||||
|
uri_mode: "persistent",
|
||||||
|
...values,
|
||||||
|
},
|
||||||
|
stdout: "piped",
|
||||||
|
stderr: "piped",
|
||||||
|
}).output();
|
||||||
|
assert(
|
||||||
|
result.success,
|
||||||
|
`generator failed: ${new TextDecoder().decode(result.stderr)}`,
|
||||||
|
);
|
||||||
|
const output = new TextDecoder().decode(result.stdout);
|
||||||
|
const uri = output.match(/obsidian:\/\/setuplivesync\?settings=\S+/u)?.[0];
|
||||||
|
assert(uri, "the command did not print a Setup URI");
|
||||||
|
const decoded = await decodeSettingsFromSetupURI(uri, "setup-secret");
|
||||||
|
assert(decoded, "the command output could not be decoded");
|
||||||
|
return { ...DEFAULT_SETTINGS, ...decoded };
|
||||||
|
}
|
||||||
|
|
||||||
|
for (
|
||||||
|
const [name, path] of [
|
||||||
|
["Setup generator", "./generate_setup_uri.ts"],
|
||||||
|
["Fly.io wrapper", "../flyio/generate_setupuri.ts"],
|
||||||
|
]
|
||||||
|
) {
|
||||||
|
for (const scenario of commandEnvironments) {
|
||||||
|
Deno.test(`E2E ${name}: ${scenario.name}`, async () => {
|
||||||
|
const settings = await runGeneratorCommand(
|
||||||
|
new URL(path, import.meta.url),
|
||||||
|
scenario.values,
|
||||||
|
);
|
||||||
|
assert(
|
||||||
|
settings.couchDB_URI === couchDBConnection.hostname &&
|
||||||
|
settings.couchDB_USER === couchDBConnection.username &&
|
||||||
|
settings.couchDB_PASSWORD === couchDBConnection.password &&
|
||||||
|
settings.couchDB_DBNAME === couchDBConnection.database &&
|
||||||
|
settings.passphrase === couchDBConnection.passphrase,
|
||||||
|
"the command did not preserve the selected connection and Vault passphrase",
|
||||||
|
);
|
||||||
|
const profileId = settings.activeConfigurationId;
|
||||||
|
const profile = profileId
|
||||||
|
? settings.remoteConfigurations?.[profileId]
|
||||||
|
: undefined;
|
||||||
|
assert(
|
||||||
|
profile?.uri.startsWith("sls+https://"),
|
||||||
|
"the generated URI did not select its CouchDB profile",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const remoteType of ["s3", "p2p"]) {
|
||||||
|
Deno.test(`E2E Setup generator: prefixed Vault passphrase for ${remoteType}`, async () => {
|
||||||
|
const settings = await runGeneratorCommand(
|
||||||
|
new URL("./generate_setup_uri.ts", import.meta.url),
|
||||||
|
{
|
||||||
|
remote_type: remoteType,
|
||||||
|
endpoint: "https://objects.example.test",
|
||||||
|
access_key: "access-key",
|
||||||
|
secret_key: "secret-key",
|
||||||
|
bucket: "vault-data",
|
||||||
|
LIVESYNC_PASSPHRASE: "vault-secret",
|
||||||
|
passphrase: "shell-passphrase",
|
||||||
|
},
|
||||||
|
);
|
||||||
|
assert(
|
||||||
|
settings.passphrase === "vault-secret",
|
||||||
|
"the command did not prefer LIVESYNC_PASSPHRASE",
|
||||||
|
);
|
||||||
|
const profileId = settings.activeConfigurationId;
|
||||||
|
const profile = profileId
|
||||||
|
? settings.remoteConfigurations?.[profileId]
|
||||||
|
: undefined;
|
||||||
|
assert(
|
||||||
|
profile?.uri.startsWith(`sls+${remoteType}://`),
|
||||||
|
"the command did not select the requested remote profile",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -64,9 +64,17 @@ function configureIdDerivation(
|
|||||||
function requireValue(
|
function requireValue(
|
||||||
environment: SetupGeneratorEnvironment,
|
environment: SetupGeneratorEnvironment,
|
||||||
name: string,
|
name: string,
|
||||||
|
legacyName?: string,
|
||||||
): string {
|
): string {
|
||||||
const value = environment[name]?.trim();
|
const value = environment[name]?.trim() ||
|
||||||
if (!value) throw new Error(`${name} is required`);
|
(legacyName && environment[legacyName]?.trim());
|
||||||
|
if (!value) {
|
||||||
|
throw new Error(
|
||||||
|
legacyName
|
||||||
|
? `${name} (or ${legacyName}) is required`
|
||||||
|
: `${name} is required`,
|
||||||
|
);
|
||||||
|
}
|
||||||
return value;
|
return value;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -97,7 +105,7 @@ function applyEncryptedVaultSettings(
|
|||||||
Object.assign(settings, {
|
Object.assign(settings, {
|
||||||
isConfigured: true,
|
isConfigured: true,
|
||||||
encrypt: true,
|
encrypt: true,
|
||||||
passphrase: requireValue(environment, "passphrase"),
|
passphrase: requireValue(environment, "LIVESYNC_PASSPHRASE", "passphrase"),
|
||||||
usePathObfuscation: true,
|
usePathObfuscation: true,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -107,10 +115,10 @@ function createCouchDBSettings(
|
|||||||
): ObsidianLiveSyncSettings {
|
): ObsidianLiveSyncSettings {
|
||||||
const settings = createNewVaultSettings();
|
const settings = createNewVaultSettings();
|
||||||
Object.assign(settings, PREFERRED_SETTING_SELF_HOSTED, {
|
Object.assign(settings, PREFERRED_SETTING_SELF_HOSTED, {
|
||||||
couchDB_URI: requireValue(environment, "hostname"),
|
couchDB_URI: requireValue(environment, "LIVESYNC_HOST", "hostname"),
|
||||||
couchDB_USER: requireValue(environment, "username"),
|
couchDB_USER: requireValue(environment, "LIVESYNC_USER", "username"),
|
||||||
couchDB_PASSWORD: requireValue(environment, "password"),
|
couchDB_PASSWORD: requireValue(environment, "LIVESYNC_PASS", "password"),
|
||||||
couchDB_DBNAME: requireValue(environment, "database"),
|
couchDB_DBNAME: requireValue(environment, "LIVESYNC_DB", "database"),
|
||||||
batchSave: true,
|
batchSave: true,
|
||||||
periodicReplication: true,
|
periodicReplication: true,
|
||||||
syncOnStart: true,
|
syncOnStart: true,
|
||||||
|
|||||||
Reference in new issue
Block a user