mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-11-14 20:20:12 +00:00
Maintain watched file names for CLI collision checks
Share the CLI watcher with a reconciled name inventory, observe local mutations, and retain newer notifications across overlapping reads. Adopt the shared index for Obsidian and preserve fresh bulk operations.
This commit is contained in:
1 parent
78ebc2d298
commit
2a65fe0117
35 files changed
+5440
-28
No files matched your search
@@ -65,6 +65,7 @@ These records are useful for design work. Their proposed representation or behav
|
||||
|
||||
| Document | Recorded state |
|
||||
|---|---|
|
||||
| [File path index and watched CLI file names](design_docs/file_path_index.md) | Unreleased implementation design for #1346; shared name index and collision queries; Commonlib package adoption pending |
|
||||
| [Tiered Chunk Storage with Live Compaction](design_docs/tired_chunk_pack.md) | In design |
|
||||
| [Tiered Chunk Storage for Bucket Sync](design_docs/tired_chunk_pack_bucket.md) | In design |
|
||||
| [Design intent for Metadata and Chunks](design_docs/intention_of_chunks.md) | Incomplete explanation; use the data-structure reference for current representations |
|
||||
|
||||
@@ -0,0 +1,302 @@
|
||||
---
|
||||
date: 2026-11-12
|
||||
commonlib-version: "0.4.3"
|
||||
self-hosted-livesync-version: "1.0.49"
|
||||
status: implemented-unreleased
|
||||
---
|
||||
|
||||
# File path index and watched CLI file names
|
||||
|
||||
## Status and purpose
|
||||
|
||||
This is an implementation design for [#1346](https://github.com/vrtmrz/obsidian-livesync/issues/1346), reviewed against Self-hosted LiveSync `78ebc2d298c39df0ce2be3a4e62059cd74be86a0`, Commonlib `0.4.3`, and the CLI's resolved `chokidar` `6.0.0`. An unreleased implementation follows this design. Commonlib publication and adoption of its exact package version remain prerequisites for releasing the consumer change. The version fields identify the reviewed baseline, not a release containing this change.
|
||||
|
||||
The CLI should maintain its file-name inventory while running as a daemon. A shared in-memory `FilePathIndex` should group equivalent paths and accept incremental membership changes. Ordinary content edits should neither enumerate the whole Vault nor rebuild its identity map.
|
||||
|
||||
Commonlib currently caches the collision map by `totalStorageFileEventCount`, path-normalisation version, and effective case policy. A changed event count makes `targetFilter` obtain and sort all names again. `ServiceFileAccessBase.getFileNames()` obtains those names through `getFiles()`, which performs a recursive directory scan and metadata reads in the CLI. Obsidian supplies its in-memory Vault listing instead. The issue is the cost of maintaining the lookup, while collision rejection remains required.
|
||||
|
||||
A baseline experiment with 40 files and five directories, including the root, found one full enumeration and 45 adapter `stat` or `statResolved` calls after a content-only event. Repeated checks at the same event count reused the map. All four Legacy/NFC and case-policy combinations took this path. These are adapter-operation counts, not OS syscall counts or an end-to-end daemon benchmark. The issue's reported syscall reduction combines the reporter's local collision-check change with [#1347](https://github.com/vrtmrz/obsidian-livesync/pull/1347); it does not measure either change separately.
|
||||
|
||||
## Required behaviour and scope
|
||||
|
||||
1. Preserve original path spelling, the existing NFC-then-case comparison rules, and all colliding file candidates. Never choose the first alias or merge their contents.
|
||||
2. Update membership for creates, deletes, and renames. A content-only change leaves membership and identity buckets unchanged. Repeated notifications are idempotent.
|
||||
3. Keep name observation independent of whether synchronisation accepts, ignores, batches, or completes an event. A file rejected for a collision must remain visible to subsequent collision checks.
|
||||
4. Preserve the supported listing boundary, physical-path validation, and host-specific ambiguity rules. Name indexing does not grant support for previously unsupported paths or filesystem aliases.
|
||||
5. Preserve current metadata freshness for `getFiles()`, `stat()`, and individual file lookups. A name index contains no file contents or cached size/mtime claims.
|
||||
6. Preserve fresh whole-Vault collision preflight before destructive reconstruction. Watcher readiness alone is insufficient evidence for that preflight.
|
||||
7. Make initialisation, failed enumeration, stopped observation, and disposal explicit. An unavailable inventory is never an empty successful inventory.
|
||||
8. Keep the index local to one host instance and Vault root. Recreate it after restart; add no on-disk database, migration, background periodic scan, or remote protocol feature.
|
||||
|
||||
The implementation covers the Commonlib contract, CLI daemon adoption, and Obsidian adaptation. Other hosts retain their current listing implementation through a compatibility fallback. It does not expand WebApp NFC support, implement folder-rename synchronisation, change database identity settings, or fix an independently growing unfinished-operation queue. Indexing a folder rename correctly is necessary even where synchronising that rename remains unsupported.
|
||||
|
||||
## Ownership and existing components
|
||||
|
||||
| Owner | Responsibility |
|
||||
| --- | --- |
|
||||
| Commonlib `FilePathIndex` | Original file-path membership and identity-key buckets; synchronous atomic replacement and deltas; deterministic candidate lookup. |
|
||||
| Commonlib storage interfaces and base adapters | A narrow name-query contract, compatibility fallback, and target-filter integration. |
|
||||
| CLI `CLIFileNameSource` | Inventory observation, initial population, current-directory reconciliation, physical-path safeguards, and source lifecycle. |
|
||||
| CLI composition | One watcher owner, explicit injection into file access and event delivery, and shutdown ordering. |
|
||||
| Obsidian name source | Seed and update from the existing Vault API; dispose its own event subscriptions. |
|
||||
| Existing file access and resolver | Fresh metadata and content I/O, path resolution, and operation-specific read/write checks. |
|
||||
|
||||
Keep the shared class in Commonlib and export it through `/storage`. Reuse `FilePath`, `PathComparisonSettings`, and `getStoragePathComparisonKey`; effective case policy includes the host's existing case-insensitivity decision. Do not infer that policy from the operating-system name. Reuse the CLI's existing Node storage reader and strict NFC resolver. The resolver accepts NFC mode only and cannot replace Legacy traversal through several case-equivalent directories.
|
||||
|
||||
The index expresses LiveSync path identity and belongs in Commonlib under the [package-boundary ADR](../adr/2026_07_common_library_package_boundary.md). `Set` and `Map` are sufficient data structures; no new utility dependency is needed. Host instances own the index and watcher lifetime, consistent with Commonlib's [storage-path design](https://github.com/vrtmrz/livesync-commonlib/blob/0.4.3/docs/design_docs/storage_path_resolution.md). Fancy Kit's [runtime composition guide](https://github.com/vrtmrz/fancy-kit/blob/main/docs/runtime-composition.md) keeps acquisition and cleanup policy with that owner. The installed `octagonal-wheels` `0.1.56` memoisation and volatile-cache helpers do not provide incremental identity buckets or authoritative inventory membership; retain explicit `Set`/`Map` state rather than using an evictable cache for either responsibility.
|
||||
|
||||
## Shared index
|
||||
|
||||
The proposed public operations are:
|
||||
|
||||
```ts
|
||||
interface FilePathDelta {
|
||||
readonly removed: readonly FilePath[];
|
||||
readonly added: readonly FilePath[];
|
||||
}
|
||||
|
||||
declare class FilePathIndex {
|
||||
replaceAll(paths: Iterable<FilePath>): void;
|
||||
apply(delta: FilePathDelta): void;
|
||||
getFileNames(): FilePath[];
|
||||
findPathIdentityCandidates(
|
||||
path: FilePath,
|
||||
policy: PathComparisonSettings
|
||||
): readonly FilePath[];
|
||||
}
|
||||
```
|
||||
|
||||
These signatures describe the unreleased API; they are not available from the reviewed published baseline. The class performs no I/O, subscribes to no events, and owns no promises or settings service. An empty instance is a valid empty *data structure*; its host owner determines whether it is ready to answer storage queries.
|
||||
|
||||
Retain a `Set<FilePath>` of original paths. Lazily construct identity buckets for an explicit comparison policy. There are at most four supported policies: Legacy/NFC crossed with case-sensitive/case-insensitive handling. Cache only those four possibilities, and update every materialised policy map when membership changes. Reject unsupported policy values before modifying or querying state.
|
||||
|
||||
For each policy, map its storage comparison key to a set of original paths. `apply` validates its complete input against the shared path contract, removes old memberships, and adds new memberships synchronously. Host enumeration remains responsible for its supported-path rules; the index must not introduce a stricter parser or normalise original spelling. It publishes no intermediate rename state and changes nothing for duplicate adds or absent removes. Empty buckets are removed. A same-path remove/add is a no-op when the path already exists; when it is absent, the add establishes membership.
|
||||
|
||||
`replaceAll` builds a candidate state and publishes it only after validation succeeds. A failed replacement leaves the previous data structure intact, while the host can still mark that structure unavailable. Returning arrays must not expose internal mutable sets. Candidate results are sorted using the same string ordering as the current diagnostic, so logs retain every matching original path. The full inventory does not need to be sorted for each candidate query.
|
||||
|
||||
A file rename is one delta. A folder rename is expanded by its host into affected original file paths and applied as one delta. The first implementation may find those paths by a prefix scan of the in-memory set; it must use a path-segment boundary, so `notes/` does not match `notes-old/`. A trie is unnecessary unless measurements show that folder operations justify it.
|
||||
|
||||
Changing comparison settings creates or selects the appropriate policy map from the existing original paths. It never changes the paths, physical files, or Metadata document IDs. Pending settings continue to follow the existing activation rules.
|
||||
|
||||
## Storage-query boundary
|
||||
|
||||
Add a `FileNameLookup` capability at the existing file-system adapter boundary and forward it through `FileAccessBase` and `ServiceFileAccessBase`:
|
||||
|
||||
```ts
|
||||
interface FileNameReadOptions {
|
||||
readonly fresh?: boolean;
|
||||
}
|
||||
|
||||
interface FileNameLookup {
|
||||
getFileNames(options?: FileNameReadOptions): Promise<FilePath[]>;
|
||||
findPathIdentityCandidates(
|
||||
path: FilePath,
|
||||
policy: PathComparisonSettings
|
||||
): Promise<readonly FilePath[]>;
|
||||
}
|
||||
```
|
||||
|
||||
The adapter capability is optional for existing hosts. The storage-access base supplies the required `StorageAccess` methods regardless of whether a host implements the capability. Existing callers of `getFileNames()` remain valid. A direct external implementation of `StorageAccess` must implement the new candidate query when adopting the new package; document this pre-1.0 interface change and exercise a packed consumer.
|
||||
|
||||
`fresh: true` bypasses the new name index and candidate cache and invokes the host's supported listing authority again. For the CLI this is a new physical enumeration; for Obsidian it is the current Vault API. A non-indexed host retains the freshness guarantees of its existing `getFiles()` implementation: for example, this option alone does not make the WebApp FSAPI cache a fresh disk enumeration. Callers cannot use the option to bypass path validation or broaden the listing scope. The CLI and Obsidian reconstruction guarantees remain host-specific.
|
||||
|
||||
`targetFilter` computes one explicit policy snapshot and asks for candidate paths. More than one candidate retains the current collision log and returns `false`. Zero or one candidate retains the current acceptance result. Preserve `skipCaseCollisionCheck` for its existing narrowly defined callers. The filter no longer owns a full-name map or depends on the cumulative file-event count to invalidate one.
|
||||
|
||||
For a host without the new capability, `getFileNames()` continues to call the existing `getFiles()` and extract its paths. Only the fallback candidate map is cached, keyed by `(totalStorageFileEventCount, pathNormalizationVersion, effective case policy)`. Do not cache a failed computation. Place that policy in `ServiceFileAccessBase`, which already has the file-processing dependency; keep the lower `FileAccessBase` forwarding layer independent of event counters. A fresh read also invalidates the fallback candidate map. This avoids forcing new watchers into WebApp or changing its listing contract. The fallback and indexed path must return the same candidate sets for the same supported listing and policy.
|
||||
|
||||
`getFiles()` continues to perform its established metadata enumeration. `getFileNames()` can use the name source without first creating a `NodeFile` and reading its metadata. Reconstruction preflight explicitly requests `getFileNames({ fresh: true })`; it must not accidentally inherit the daemon's cached view.
|
||||
|
||||
The explicit bulk action which resolves every conflicted file by its newest revision also requests `getFileNames({ fresh: true })` before choosing its workset. A listing failure aborts before any revision deletion. Diagnostic existence hints, file choosers, and inspection candidate lists use the normal supported view; their per-entry reads establish the state needed for subsequent actions. Neither view reserves the listed paths against later external changes.
|
||||
|
||||
## CLI observation and supported names
|
||||
|
||||
`CLIFileNameSource.ts` retains the public facade, lifecycle, captured read boundaries,
|
||||
local-mutation tracking, and inventory publication queue. Its private sidecars keep
|
||||
observation and storage rules separate:
|
||||
|
||||
- `CLIFileNameSource.watch.ts` owns the native watcher and startup sync-delivery buffer.
|
||||
- `CLIFileNameSource.inventory.ts` reads supported names and validates comparison policies.
|
||||
- `CLIFileNameSource.reconciliation.ts` stages directory deltas, checks generations, and commits them.
|
||||
- `CLIFileNameSource.state.ts` defines their state contracts and pure value operations.
|
||||
|
||||
Sidecars receive narrow ports rather than the facade or Service Hub. Inventory
|
||||
operations can prepare a private candidate before the source is ready, while the
|
||||
source retains the decision to publish it. Sync callbacks run outside the inventory
|
||||
queue because they can query names again.
|
||||
|
||||
Inventory reading and reconciliation use named module-level operations with explicit
|
||||
state inputs and per-operation dependency types. Each type lists the required ports,
|
||||
including those needed by helper calls; storage and reader ports expose only the
|
||||
methods used by that operation. Factories bind these operations to the existing
|
||||
interfaces without adding private mutable state. Direct unit tests provide the
|
||||
minimal ports to exercise listing failures, policy validation, staged publication,
|
||||
and generation changes without constructing a watcher or the public source.
|
||||
|
||||
The source selects the comparison policy before entering reconciliation. Internal
|
||||
operations consume `state.policy`, so the read and its validation use one policy.
|
||||
Publication checks reject superseded transitions, acknowledgement checks preserve
|
||||
newer notifications, and the source checks the complete target branch set before
|
||||
returning candidates. These generation checks protect different boundaries.
|
||||
Strict initial enumeration belongs to the inventory reader; reconciliation updates
|
||||
existing coverage and retains failures for its owner to handle.
|
||||
|
||||
Directory reads distinguish raw Legacy entries from NFC resolver-validated
|
||||
`StorageEntry` values. Scanner and membership updates retain each validated
|
||||
`StoragePath` through index insertion. Raw directory names remain available for
|
||||
later policy validation, including invalid Legacy names excluded from membership.
|
||||
Legacy parsing stays lazy during full traversal and diagnoses only newly excluded
|
||||
file names during delta staging. This preserves diagnostic order and original
|
||||
physical spelling without adding another inventory cache.
|
||||
|
||||
Create one `CLIFileNameSource` in CLI composition and inject its read view into file access. It owns the existing chokidar instance; the event-manager watch adapter borrows event delivery from it. Constructors bind dependencies but perform no I/O. File access, the name source, and the existing resolver use the same configured Node storage boundary and root.
|
||||
|
||||
Observation has two consumers: inventory maintenance and synchronisation event delivery. Update or invalidate the inventory before applying synchronisation filters or forwarding an event to `StorageEventManager`. Content-change events can leave membership unchanged, but an unknown changed path requires reconciliation rather than assuming that its create notification was seen.
|
||||
|
||||
The current watcher's exclusion patterns are not a specification for the inventory: the watcher applies custom ignore rules and excludes paths which the current `getFiles()` listing can still inspect. Keep *enumeration exclusions* separate from *synchronisation exclusions*. The observation source must cover every directory needed by the supported name listing. Apply custom synchronisation exclusions only to synchronisation delivery. This can increase watched paths; measure watcher resources on a Vault with a large excluded subtree before acceptance.
|
||||
|
||||
Keep the existing treatment of hidden directories, hidden leaf files, invalid Legacy paths, symbolic links, and NFC aliases. Use explicit compatibility fixtures to establish that membership agrees with the old enumerator. Do not follow symlinks merely because chokidar's default does. Filesystem notifications are hints to inspect permitted storage paths, not permission to access a path.
|
||||
|
||||
The shared index contains files only. The CLI source additionally retains directory-entry information needed to preserve NFC ambiguity checks, including file/directory and parent-directory aliases. This is host validation state, not another identity implementation. Reuse the existing comparison and resolver rules. An already detected ambiguity or incomplete NFC enumeration must not become a successful query by dropping offending entries.
|
||||
|
||||
## Readiness and startup
|
||||
|
||||
Use one owner-local state with explicit `uninitialised`, `initialising`, `ready`, `failed`, and `closed` variants. Concurrent preparation calls share the same initialisation promise. Failure rejects it; disposal prevents a late enumeration from publishing into a retired source. Store the watcher as soon as it is constructed, so closing during initialisation also closes it and settles its pending `ready` wait. No query waits for synchronisation event delivery to become active.
|
||||
|
||||
Register CLI preparation with `appLifecycle.onLoaded`, after all `onSettingLoaded` handlers have finished and before `control.onReady` restores storage events and starts the offline scan. Do not initialise under a constructor-registered `onSettingLoaded` handler: later handlers can still select the Fetch path-normalisation policy. Constructors only connect dependencies. An early name query must either join preparation after configuration is final or reject as unavailable; it must not start under provisional settings. A rejected preparation propagates through loading and the existing unload path.
|
||||
|
||||
The preparation and activation sequence is:
|
||||
|
||||
1. Start inventory observation with listeners registered, retain the watcher immediately, and wait for its `ready` event. Keep synchronisation delivery gated. Assign an increasing owner-local sequence number to observed invalidations and retain eligible synchronisation notifications in arrival order. This sequence orders work; it is not an identity-map cache key.
|
||||
2. Enumerate the complete supported initial names and directory-validation state while continuing to collect observations. Build a candidate index privately.
|
||||
3. Capture the current observation number `B` once. Reconcile the finite set of directories invalidated at or before `B`, then publish the candidate state. Observations after `B` remain pending and do not extend preparation indefinitely. If a directory changes again while being read, retain its later invalidation rather than acknowledging it with the earlier result.
|
||||
4. Mark name queries ready before storage-event snapshot replay. That replay may call `isTargetFile`; it must not wait for live event activation. Startup scanning continues to obtain current metadata through `getFiles()`.
|
||||
5. At the existing `onFirstInitialise` / `beginWatch` boundary, attach the synchronisation callbacks and capture a finite delivery boundary. Place buffered notifications through that boundary into the existing delivery path in arrival order, with newer notifications queued behind them. Activation waits for this hand-off, not for every resulting file operation to finish.
|
||||
|
||||
The owner serialises inventory publication only. It must not await synchronisation callbacks, `isTargetFile`, or queue completion while holding that serialisation boundary: those operations can query the source again. Synchronisation dispatch runs separately after the inventory has been updated or invalidated. Once activated, it hands each notification directly to the existing callbacks without awaiting downstream processing; no second steady-state synchronisation queue is introduced. Keep observation-time exclusions for host-owned operations and delivery-time admission rules in their existing semantic phases, so buffering does not reinterpret an event using a later processing state.
|
||||
|
||||
Do not emit every initial file as a new synchronisation event. Retain the current `ignoreInitial` and content-stability behaviour; the initial full metadata scan handles initial contents. Preserve notifications observed between that scan and activation rather than losing them in a second watcher startup. The temporary activation buffer costs space proportional to observed events; it drains into the existing event manager after activation and is not retained as the name inventory.
|
||||
|
||||
## Changes, reconciliation, and local operations
|
||||
|
||||
| Observation | Name-source action |
|
||||
| --- | --- |
|
||||
| Known file content changes | Leave file membership and identity buckets unchanged. Preserve normal content-event delivery. |
|
||||
| File added or removed | Reconcile its parent directory and apply the resulting name delta. |
|
||||
| Directory added, removed, or renamed | Reconcile the affected subtree and replace its file memberships as one delta. |
|
||||
| Host-owned mutation is attempted | Invalidate the affected directories before I/O. On completion, a dependent name query reconciles them before returning. Later watcher notifications are idempotent. |
|
||||
| Comparison policy changes | Select or construct the in-memory identity buckets and revalidate the retained directory-entry state under the new policy. Missing validation coverage requires fresh enumeration. |
|
||||
| Enumeration fails or observation becomes unreliable | Retain diagnostic information and invalidate the affected source state; never publish a partial enumeration as complete. |
|
||||
|
||||
Coalesce inventory invalidations by directory; keep sequence numbers only to establish finite read and publication boundaries. Serialise publications by the source owner. A delayed `unlink` must not blindly remove a path which has already been recreated: inspect current directory membership before applying the delta. Likewise, a stale scan must not overwrite a newer local-operation update. Observations arriving during a read schedule a later reconciliation; they are not acknowledged by an older result. Compare the resulting original-name sets and apply only actual additions and removals, so reconciling an unchanged directory does not churn identity buckets.
|
||||
|
||||
Put local-mutation invalidation at the CLI's shared storage boundary, covering `write`, `writeBinary`, `append`, `mkdir`, `remove`, and `rename`. Both direct `FileAccessBase.adapter*` / `ensureDirectory` calls and `NodeVaultAdapter` operations must use that same observed storage instance. Hooking only Vault methods misses direct adapter writes, including non-Markdown creation. Use a CLI-owned subclass of the existing `UnicodeResolvingNodeStorageAdapter` for this hook, retaining its resolver methods and `instanceof` compatibility with Vault metadata reads. Delegate actual filesystem operations to that existing implementation; keep host callbacks outside the pure index.
|
||||
|
||||
Mark both affected parents, and any affected subtree, dirty before attempting a mutation. Advance their mutation generation at completion, including rejection, so a reconciliation which overlaps the operation cannot certify its old result. Retain dirtiness when physical I/O partially succeeds or when a later metadata lookup fails. Reconciliation must not run inside the storage operation's lock if it needs that lock itself. A subsequent name query waits for an overlapping local operation to settle and then reconciles its captured dirty scope.
|
||||
|
||||
Preserve the storage method's own success or failure result. A new index-reconciliation failure must not turn a successful filesystem write into a reported write failure or cause that write to be retried. Record the invalid inventory state and diagnostic; subsequent name queries reject until the affected scope is successfully revalidated. Existing failures from physical I/O or required metadata retrieval keep their existing meaning.
|
||||
|
||||
A normal `getFileNames()` captures the current invalidation boundary once, reconciles the finite scopes through that boundary, and returns a copy of the resulting inventory. If concurrent changes prevent safe publication for that captured boundary, the call fails explicitly and retains pending invalidations; it does not wait for the entire Vault to become quiet. It is a watched view, not a fresh physical snapshot. Current-path queries reconcile their relevant branches; discovering an unrelated new subtree schedules its population separately rather than recursively scanning it within every candidate query.
|
||||
|
||||
Chokidar currently delays `add` and `change` through `awaitWriteFinish` and combines atomic-save notifications. Preserve synchronisation timing and do not rely on a delayed public event as proof of immediate filesystem membership. `getWatched()` is useful source evidence but is neither a complete storage contract nor an efficient per-file lookup: it copies and sorts the whole watched tree on each call.
|
||||
|
||||
## Current-path checks and whole-Vault checks
|
||||
|
||||
The index is a maintained view, not an atomic filesystem snapshot. CLI candidate lookup must reconcile the current physical directory chain for the requested path before returning a collision-free result. Read directory names and entry kinds through the existing strict storage reader; do not obtain metadata for every file. In Legacy case-insensitive mode, follow every equivalent parent branch and collect every equivalent file, even if an exact spelling exists. NFC mode retains rejection of ambiguous parent components through the existing resolver.
|
||||
|
||||
Follow only parent directories admitted by the supported enumerator. A candidate query must not add descendants of an excluded hidden directory to the inventory. Hidden leaf files retain their existing listing behaviour.
|
||||
|
||||
This check also handles missing files, pending creates, delayed deletes, and a collision introduced before its watcher notification. It costs the entries in the relevant directories, not every file in unrelated subtrees. A large flat directory still costs a large directory listing; do not describe the bound as path depth alone. Existing locks and physical write validation remain necessary because a directory read cannot reserve the filesystem against an external writer.
|
||||
|
||||
A query fails on a relevant read error; it does not infer absence from a failed read. Known whole-inventory NFC ambiguity or failed coverage remains visible and cannot be bypassed by querying another path. Watch observation cannot guarantee that an unrelated, not-yet-observed directory change is immediately discovered. Whole-Vault validation belongs to explicit fresh enumeration and reconstruction preflight, not a claim attached to every in-memory lookup. This observation limit is a deliberate part of using a watched inventory and must be tested and documented separately from target-path safety.
|
||||
|
||||
Directory-validation records carry their original entry names and kinds, validation policy, and mutation generation. Retain a failed directory's previous membership and diagnostic, but do not serve it as validated. A successful read and validation of the same scope clears that failure. A fresh whole enumeration uses the same private-candidate, captured-invalidation, and atomic-publication rules as preparation before replacing all coverage records; it must not overwrite a concurrent local mutation. In NFC mode, every known ambiguity or coverage failure blocks candidate queries until repaired, including a failure outside the requested branch. A repair notification may trigger the scoped revalidation even while queries are blocked.
|
||||
|
||||
Until a fresh candidate commits, retain the earliest pending observation sequence for every scope its reconciliation can acknowledge. If the candidate is rejected, restore those observations for the still-published inventory, including readers which captured an earlier boundary. Check local-mutation revision and captured directory generations again after final-policy reconciliation: its asynchronous reads must satisfy the same publication conditions as the initial scan. This temporary record belongs to the fresh operation and adds no persistent state or automatic whole-Vault retry.
|
||||
|
||||
Policy changes revalidate retained entry sets, including directory names, under the new policy before publishing that policy as ready. File buckets alone cannot establish physical validity. Reconcile pending observations through a captured boundary first; require fresh enumeration if retained coverage is insufficient. Validation errors belong to their policy and generation: switching settings must neither discard relevant failures nor keep an old-policy failure after the new policy has been validated. Existing path exclusions and diagnosed Legacy enumeration skips remain explicit compatibility cases, rather than being silently reclassified by the common index.
|
||||
|
||||
An initial-enumeration failure leaves the source `failed`, closes its newly opened watcher, and aborts loading. A watcher error invalidates the whole source and follows the existing fatal-error policy. Scoped reconciliation errors while observing retain blocked coverage and can recover by a successful read of the same scope; they do not create an automatic whole-Vault retry loop. If observation is deliberately restarted, retire and close the old source, then prepare a new instance from scratch.
|
||||
|
||||
Rebuild/Fetch preflight uses a fresh supported listing and the existing local Metadata checks before any reset. Keep its current all-or-nothing NFC enumeration and parent-ambiguity behaviour. A preflight failure must leave both databases and all files unchanged. Full scanners retain current content/provenance checks; the name index is not proof that a file body has been processed.
|
||||
|
||||
## Non-watching commands, suspension, and shutdown
|
||||
|
||||
When `watchFiles` is false, the CLI adapter does not provide the indexed `FileNameLookup` capability. Whole-name requests use the existing fresh CLI enumeration, and candidate queries use the base fallback. Queries never start a watcher in this mode. This applies to the current `mirror`, `fetch`, and `rebuild` commands. Do not retain a name snapshot across independent CLI invocations.
|
||||
|
||||
Suspending synchronisation must keep its current effect on admission and reflection. Inventory observation performs no database write, and must not restart synchronisation, release a Scram state, or bypass recovery flags. If observation itself is stopped, invalidate its ready state and require fresh enumeration before using it again.
|
||||
|
||||
At `onBeforeUnload`, freeze source-to-manager synchronisation delivery, then use the existing manager stop/flush/snapshot sequence. Already dispatched notifications belong to that finite shutdown boundary; later observations do not extend it. There is no steady-state source queue left to feed after manager admission closes. Keep inventory access available while captured operations settle. If shutdown interrupts loading before activation, close and discard the unaccepted startup buffer; the next invocation reconstructs from storage and the existing retained snapshots. At `onUnload`, close the shared watcher and dispose the name source exactly once. Outstanding preparation/reconciliation must reject or settle without publishing after disposal. Preserve the current CLI fatal-watcher-error policy; do not introduce an unbounded in-process retry loop or silently keep serving a stale inventory.
|
||||
|
||||
## Obsidian adoption
|
||||
|
||||
Create a host-owned name source using `Vault.getFiles()` and Vault create/delete/rename events. Register inventory listeners before seeding, obtain and apply the initial snapshot synchronously within the same JavaScript turn, and update the index before sync-event filtering can query it. Subscribe directly to raw Vault folder events as well as file events: the existing synchronisation forwarder filters out folders. A folder rename expands the old prefix using the retained original paths; folder creation obtains any existing descendants from the Vault, and folder deletion removes retained descendants. Apply each membership change as one delta; duplicated child notifications remain harmless.
|
||||
|
||||
Content modification events do not rebuild the name index. Existing Vault file objects continue to supply current metadata. Preserve Obsidian's separate alias guard and its fixed case-insensitive NFC policy; it is not replaced by the configured database-identity policy. Fresh enumeration means the latest Vault listing and does not promise to reveal physical aliases hidden by Obsidian.
|
||||
|
||||
Dispose the source's listeners on unload. The source does not own Obsidian's Vault or install another disk watcher.
|
||||
|
||||
## Costs and acceptance evidence
|
||||
|
||||
Let `N` be the number of supported files, `K` the paths affected by one name change, and `C` the matching candidates. String/key work also depends on path length.
|
||||
|
||||
| Operation | Intended cost after initialisation |
|
||||
| --- | --- |
|
||||
| Shared index candidate lookup | Expected map lookup plus copying/sorting `C` candidates. |
|
||||
| Content-only edit | No full-name enumeration and no identity-map rebuild. CLI still performs relevant current-directory checks. |
|
||||
| Membership delta | Expected work proportional to `K` for each materialised policy, with at most four policies. |
|
||||
| First shared-index query for a comparison policy | One in-memory pass over `N`. CLI directory validation is separate and may require fresh enumeration if coverage is missing. |
|
||||
| Complete name list | Reconcile pending captured scopes, then copy `N` paths; an array-returning API cannot be constant-time. |
|
||||
| Folder operation | At most an in-memory prefix scan plus affected-subtree enumeration; no claim of constant-time directory rename. |
|
||||
| Fresh whole-Vault validation | Full supported enumeration, intentionally retained. |
|
||||
| Retained state | Linear in file names and host directory coverage; a temporary startup buffer is proportional to observed events. No persistent index. |
|
||||
|
||||
Compare correctness and costs against the same baseline and storage conditions. Warm the source and policy once, then perform content-only edits in a fixture with unrelated subtrees. Count full enumerations, identity-map builds, directory reads, adapter metadata calls, and applied deltas separately. Require zero new full enumerations and zero full identity-map builds for those warmed content edits. Do not claim zero filesystem calls or a particular CPU reduction.
|
||||
|
||||
Measure a large flat directory separately from a deep Vault. Repeat the issue's create/delete trace with this change alone and with #1347, and distinguish syscall measurements from adapter counts. The reported CPU usage and sustained writes remain separate hypotheses until reproduced.
|
||||
|
||||
## Verification matrix
|
||||
|
||||
Add the smallest regression test before production changes and confirm failure against the unmodified implementation for the expected full-scan cause. Fixture, watcher, and permission failures are setup failures.
|
||||
|
||||
| Boundary | Required scenarios and assertions |
|
||||
| --- | --- |
|
||||
| Pure index | All four policies; NFC/NFD aliases; original spelling; duplicate add/delete; atomic rename delta; policy switches; deterministic complete collision lists; failed replacement leaves old data intact; two independent instances. |
|
||||
| CLI startup | Initial files are indexed without becoming new sync events; changes during enumeration; snapshot replay before sync activation; final settings precede preparation; continuous events do not prevent readiness; startup failure; concurrent preparation; close before watcher readiness; no deadlock. |
|
||||
| Watch updates | Create/delete/recreate, temporary-file atomic save, delayed add, duplicate/out-of-order notification, unknown changed path, new populated directory, folder move, and ignored-but-enumerated aliases. |
|
||||
| Physical checks | An alias exists before its watcher event; exact spelling plus another alias; equivalent parent branches in Legacy mode; ambiguous parent in NFC mode; read failure versus absence; symlink rejection. |
|
||||
| Whole listing | Same supported membership as the current Legacy/NFC enumerator, including hidden and invalid-name fixtures; known global NFC failure; failed nested read; fresh preflight refuses before reset; bulk newest-revision resolution uses a fresh workset and aborts before deleting revisions when listing fails; failure clears only after scoped repair; rejected fresh candidates retain observed changes and earlier read boundaries; final-policy reconciliation rejects overlapping local I/O or changed generations; policy switches revalidate parent entries. |
|
||||
| Metadata and lifecycle | Existing `getFiles()` freshness/concurrent-list tests; direct adapter and Vault writes visible before delayed watcher delivery; partial I/O / metadata / reconciliation failure retains invalidation without replaying a successful write; restart discards the index; stopped/failed watcher cannot supply successful stale queries; non-watching commands exit. |
|
||||
| Obsidian | Existing Vault listing scope; create/delete/file and folder rename; content edits do not rebuild; fixed-policy alias guard remains; listener cleanup; no disk listing introduced. |
|
||||
| Fallback and packaging | Non-indexed host retains existing candidate sets and reuse; `skipCaseCollisionCheck`; packed `/storage` export; direct StorageAccess implementor migration; no Node/chokidar import in the common index. |
|
||||
|
||||
Run the Commonlib owner tests and packed consumer checks, then LiveSync's required `npm run check`, unit tests, and production build against that exact package. Run focused CLI real-watch/E2E and Obsidian lifecycle/rename/preflight coverage for the boundaries changed. Broad validation is sequential and uses the configured device limits. Design-only work does not establish that these implementation gates have passed.
|
||||
|
||||
## Implementation sequence and affected areas
|
||||
|
||||
1. Add the Commonlib regression test, shared `FilePathIndex`, package exports, and pure-index tests in the Commonlib repository.
|
||||
2. Add the name-query interfaces and base fallback; move the existing collision predicate to the narrow query. Change reconstruction preflight to request fresh enumeration. Verify existing hosts and packed consumers before adopting indexed behaviour.
|
||||
3. Add the CLI name source and split inventory observation from synchronisation activation. Inject the shared owner through `CLIServiceModules`, `FileAccessCLI`, the file-system adapter, and the storage-event adapter. Integrate successful local mutations without changing file-event completion semantics.
|
||||
4. Add Obsidian's Vault-backed source and preserve its alias guard and lifecycle. Keep non-indexed hosts on the tested fallback.
|
||||
5. Validate the packed Commonlib artefact and exact downstream dependency, then measure the original daemon scenario. Update the proposal's status and implementation references only when the implementation exists.
|
||||
|
||||
Primary LiveSync areas are `src/apps/cli/serviceModules/CLIServiceModules.ts`, `FileAccessCLI.ts`, `ServiceFileAccessImpl.ts`, `src/apps/cli/adapters/NodeFileSystemAdapter.ts`, `NodeVaultAdapter.ts`, the observed CLI storage subclass in `src/apps/cli/storage/ObservedNodeStorageAdapter.ts`, `src/apps/cli/managers/CLIStorageEventManagerAdapter.ts`, `StorageEventManagerCLI.ts`, CLI startup composition, and the corresponding Obsidian adapter/composition files. Commonlib owns `platform/FilePathIndex.ts` (new), storage/file-access interfaces and base classes, `serviceFeatures/targetFilter.ts`, and `serviceModules/Rebuilder.ts`. Exact method placement should follow the existing narrow dependency direction; no local Commonlib source mirror is introduced.
|
||||
|
||||
## Alternatives and limits
|
||||
|
||||
- Invalidating only on create/delete/rename is a useful reduction, but still performs a full Vault scan for every structural event and requires a complete observation source.
|
||||
- Rebuilding the map from a watched array removes repeated disk traversal but retains full-array/map work on content edits. It is insufficient for the intended hot path.
|
||||
- Current-directory lookup alone is a smaller correction and remains useful for freshness checks. It does not provide the shared inventory used by other name-list consumers.
|
||||
- Using the existing `fileCache` as the complete inventory is incorrect: it contains files discovered by selected lookups and can retain stale entries.
|
||||
- A persistent index adds recovery and migration state which this derived data does not need. Reconstruct it from the host listing on restart.
|
||||
- Treating the synchronisation queue as the file inventory excludes rejected/ignored entries and couples name correctness to sync completion. Keep those responsibilities separate.
|
||||
|
||||
The principal new risk is a stale or incomplete name view admitting an ambiguous path. Readiness gates, complete candidate sets, current-path checks, fresh reconstruction preflight, and explicit source failures address distinct parts of that risk. They are design requirements, not verified safeguards yet. A missed unrelated watcher notification can delay discovery outside the queried path; no claim of an atomic whole-filesystem snapshot is made. Do not assign a numerical post-fix risk or declare the issue fixed until the relevant behaviour and original scenario have been validated.
|
||||
|
||||
## References
|
||||
|
||||
- [Unicode path identity design](unicode_path_identity.md) and [ADR](../adr/2026_10_unicode_path_identity.md).
|
||||
- [Write and reflection paths](file_write_and_reflection_paths.md).
|
||||
- [Service feature and legacy Module boundaries](service_feature_and_legacy_module_boundaries.md).
|
||||
- [CLI database continuity](cli_database_continuity.md).
|
||||
- [Commonlib storage path resolution](https://github.com/vrtmrz/livesync-commonlib/blob/0.4.3/docs/design_docs/storage_path_resolution.md).
|
||||
- [Commonlib storage events and reflection](https://github.com/vrtmrz/livesync-commonlib/blob/0.4.3/docs/storage-events-and-reflection.md).
|
||||
@@ -4,6 +4,7 @@ import { NodePathAdapter } from "./NodePathAdapter";
|
||||
import { NodeTypeGuardAdapter } from "./NodeTypeGuardAdapter";
|
||||
import { NodeConversionAdapter } from "./NodeConversionAdapter";
|
||||
import { AmbiguousNormalisedPathError, UnicodeResolvingNodeStorageAdapter } from "./UnicodeResolvingNodeStorageAdapter";
|
||||
import { ObservedNodeStorageAdapter } from "@/apps/cli/storage/ObservedNodeStorageAdapter";
|
||||
import { NodeVaultAdapter } from "./NodeVaultAdapter";
|
||||
import type { NodeFile, NodeFolder, NodeStat } from "./NodeTypes";
|
||||
import { path } from "@vrtmrz/livesync-commonlib/node";
|
||||
@@ -15,6 +16,7 @@ import {
|
||||
parseFilePath,
|
||||
type StoragePath,
|
||||
type StoragePathResolver,
|
||||
type FileNameLookup,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
|
||||
/**
|
||||
@@ -26,7 +28,11 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
readonly conversion: NodeConversionAdapter;
|
||||
readonly storage: UnicodeResolvingNodeStorageAdapter;
|
||||
readonly vault: NodeVaultAdapter;
|
||||
/** Optional watched name capability; single-run CLI commands retain physical listing. */
|
||||
fileNameLookup?: FileNameLookup;
|
||||
|
||||
// Previously accessed objects are not a complete name inventory. Full getFiles
|
||||
// enumeration still reads current metadata rather than treating this cache as a snapshot.
|
||||
private fileCache = new Map<string, NodeFile>();
|
||||
private readonly isCaseInsensitiveSetting: () => boolean;
|
||||
|
||||
@@ -34,13 +40,19 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
private basePath: string,
|
||||
private reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||
getPathNormalizationVersion: () => number = () => 0,
|
||||
isCaseInsensitive: () => boolean = () => false
|
||||
isCaseInsensitive: () => boolean = () => false,
|
||||
onMutation?: (paths: readonly string[]) => () => void
|
||||
) {
|
||||
this.path = new NodePathAdapter();
|
||||
this.typeGuard = new NodeTypeGuardAdapter();
|
||||
this.conversion = new NodeConversionAdapter();
|
||||
this.isCaseInsensitiveSetting = isCaseInsensitive;
|
||||
this.storage = new UnicodeResolvingNodeStorageAdapter(basePath, getPathNormalizationVersion, isCaseInsensitive);
|
||||
this.storage = new ObservedNodeStorageAdapter(
|
||||
basePath,
|
||||
getPathNormalizationVersion,
|
||||
isCaseInsensitive,
|
||||
onMutation
|
||||
);
|
||||
this.vault = new NodeVaultAdapter(this.storage);
|
||||
}
|
||||
|
||||
|
||||
@@ -6,6 +6,8 @@ import { promisify } from "node:util";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||
import { UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { CLIFileNameSource } from "@/apps/cli/storage/CLIFileNameSource";
|
||||
import { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
|
||||
import { NodeTypeGuardAdapter } from "@/apps/cli/adapters/NodeTypeGuardAdapter";
|
||||
@@ -35,6 +37,18 @@ function makeHandlers(events: WatchEvent[], onRename: () => void = () => undefin
|
||||
};
|
||||
}
|
||||
|
||||
function makeIndexedAdapter(root: string, ignoreRules?: IgnoreRules) {
|
||||
const source = new CLIFileNameSource(
|
||||
new UnicodeResolvingNodeStorageAdapter(root),
|
||||
() => ({ pathNormalizationVersion: 0, handleFilenameCaseSensitive: true }),
|
||||
{ basePath: root, ignoreRules }
|
||||
);
|
||||
return {
|
||||
source,
|
||||
adapter: new CLIStorageEventManagerAdapter(root, ignoreRules, true, undefined, undefined, source),
|
||||
};
|
||||
}
|
||||
|
||||
async function waitFor(predicate: () => boolean, timeoutMs = 5_000): Promise<boolean> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
while (!predicate()) {
|
||||
@@ -69,13 +83,14 @@ describe("CLI filesystem watcher with a real filesystem", () => {
|
||||
it("reports all 400 files when another process creates directories before dot-temp renames", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "livesync-issue1328-burst-"));
|
||||
const events: WatchEvent[] = [];
|
||||
const adapter = new CLIStorageEventManagerAdapter(root, undefined, true);
|
||||
const { source, adapter } = makeIndexedAdapter(root);
|
||||
const expectedPaths = Array.from({ length: 40 }, (_, directoryIndex) =>
|
||||
Array.from({ length: 10 }, (_, fileIndex) => `book-${directoryIndex}/image-${fileIndex}.png`)
|
||||
).flat();
|
||||
|
||||
try {
|
||||
await writeFile(join(root, "present-before-watch.md"), "initial file");
|
||||
await source.prepare();
|
||||
await adapter.watch.beginWatch(makeHandlers(events));
|
||||
|
||||
await run(process.execPath, ["-e", directoryFirstWriter, root], { timeout: 15_000 });
|
||||
@@ -122,9 +137,10 @@ describe("CLI filesystem watcher with a real filesystem", () => {
|
||||
await mkdir(join(root, "private"));
|
||||
await writeFile(join(root, "present-before-watch.md"), "initial file");
|
||||
await writeFile(join(root, ".hidden-before-watch.md"), "initial hidden file");
|
||||
const adapter = new CLIStorageEventManagerAdapter(root, ignoreRules, true);
|
||||
const { source, adapter } = makeIndexedAdapter(root, ignoreRules);
|
||||
|
||||
try {
|
||||
await source.prepare();
|
||||
await adapter.watch.beginWatch(makeHandlers(events));
|
||||
await writeFile(join(root, "visible-after-watch.md"), "visible");
|
||||
await writeFile(join(root, ".hidden-after-watch.md"), "hidden");
|
||||
@@ -149,11 +165,12 @@ describe("CLI filesystem watcher with a real filesystem", () => {
|
||||
it("reports changes, deletions, rename paths, and stops after close", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "livesync-real-watch-lifecycle-"));
|
||||
const events: WatchEvent[] = [];
|
||||
const adapter = new CLIStorageEventManagerAdapter(root, undefined, true);
|
||||
const { source, adapter } = makeIndexedAdapter(root);
|
||||
let closed = false;
|
||||
let renameCalls = 0;
|
||||
|
||||
try {
|
||||
await source.prepare();
|
||||
await adapter.watch.beginWatch(makeHandlers(events, () => renameCalls++));
|
||||
|
||||
const originalPath = join(root, "original.md");
|
||||
|
||||
@@ -15,6 +15,7 @@ import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||
import { fsPromises as fs, getNodeFileCreationTime, path, type Stats } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||
import { NodeTypeGuardAdapter } from "@/apps/cli/adapters/NodeTypeGuardAdapter";
|
||||
import type { CLIFileNameSource } from "@/apps/cli/storage/CLIFileNameSource";
|
||||
import { writeFileAtomically } from "@/apps/cli/runtimeState";
|
||||
|
||||
/**
|
||||
@@ -92,7 +93,8 @@ class CLIWatchAdapter implements IStorageEventWatchAdapter {
|
||||
private basePath: string,
|
||||
private ignoreRules?: IgnoreRules,
|
||||
private watchEnabled: boolean = false,
|
||||
private reportDiagnostic: CliDiagnosticReporter = () => undefined
|
||||
private reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||
private nameSource?: CLIFileNameSource
|
||||
) {}
|
||||
|
||||
private _toNodeFile(filePath: string, stats: Stats | undefined): NodeFile {
|
||||
@@ -118,6 +120,10 @@ class CLIWatchAdapter implements IStorageEventWatchAdapter {
|
||||
|
||||
async beginWatch(handlers: IStorageEventWatchHandlers): Promise<void> {
|
||||
if (!this.watchEnabled) return;
|
||||
if (this.nameSource) {
|
||||
await this.nameSource.beginWatch(handlers);
|
||||
return;
|
||||
}
|
||||
const baseIgnored: Array<RegExp | string | ((p: string) => boolean)> = [
|
||||
/(^|[/\\])\./,
|
||||
/(^|[/\\])[^/\\]*-livesync-v2([/\\]|$)/,
|
||||
@@ -178,6 +184,9 @@ class CLIWatchAdapter implements IStorageEventWatchAdapter {
|
||||
}
|
||||
|
||||
close(): Promise<void> {
|
||||
if (this.nameSource) {
|
||||
return this.nameSource.close();
|
||||
}
|
||||
if (this._watcher) {
|
||||
return this._watcher.close();
|
||||
}
|
||||
@@ -200,11 +209,12 @@ export class CLIStorageEventManagerAdapter implements IStorageEventManagerAdapte
|
||||
ignoreRules?: IgnoreRules,
|
||||
watchEnabled: boolean = false,
|
||||
reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||
snapshotPath?: () => string
|
||||
snapshotPath?: () => string,
|
||||
nameSource?: CLIFileNameSource
|
||||
) {
|
||||
this.typeGuard = new NodeTypeGuardAdapter();
|
||||
this.persistence = new CLIPersistenceAdapter(basePath, reportDiagnostic, snapshotPath);
|
||||
this.watch = new CLIWatchAdapter(basePath, ignoreRules, watchEnabled, reportDiagnostic);
|
||||
this.watch = new CLIWatchAdapter(basePath, ignoreRules, watchEnabled, reportDiagnostic, nameSource);
|
||||
this.status = new CLIStatusAdapter();
|
||||
this.converter = new CLIConverterAdapter();
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { describe, expect, it, vi, beforeEach } from "vitest";
|
||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||
import type { CLIFileNameSource } from "@/apps/cli/storage/CLIFileNameSource";
|
||||
import type { NodeFile } from "@/apps/cli/adapters/NodeTypes";
|
||||
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { runtimeScopeDirectory } from "../runtimeState";
|
||||
@@ -86,6 +87,24 @@ describe("CLIStorageEventManagerAdapter", () => {
|
||||
expect(chokidar.watch).toHaveBeenCalledWith("/base", expect.objectContaining({ ignoreInitial: true }));
|
||||
});
|
||||
|
||||
it("borrows the prepared name source without starting a second watcher", async () => {
|
||||
const source = { beginWatch: vi.fn(), close: vi.fn() };
|
||||
const adapter = new CLIStorageEventManagerAdapter(
|
||||
"/base",
|
||||
undefined,
|
||||
true,
|
||||
undefined,
|
||||
undefined,
|
||||
source as unknown as CLIFileNameSource
|
||||
);
|
||||
const handlers = makeHandlers();
|
||||
await adapter.watch.beginWatch(handlers);
|
||||
expect(source.beginWatch).toHaveBeenCalledWith(handlers);
|
||||
expect(chokidar.watch).not.toHaveBeenCalled();
|
||||
await adapter.close();
|
||||
expect(source.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("add event produces NodeFile with correct relative path via onCreate", async () => {
|
||||
const basePath = "/vault/base";
|
||||
const adapter = new CLIStorageEventManagerAdapter(basePath, undefined, true);
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
|
||||
import type { IMinimumLiveSyncCommands, LiveSyncBaseCore } from "@/LiveSyncBaseCore";
|
||||
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
||||
import type { CLIFileNameSource } from "@/apps/cli/storage/CLIFileNameSource";
|
||||
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||
import { runtimeScopeDirectory } from "@/apps/cli/runtimeState";
|
||||
@@ -20,7 +21,8 @@ export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEv
|
||||
core: LiveSyncBaseCore<ServiceContext, IMinimumLiveSyncCommands>,
|
||||
dependencies: StorageEventManagerBaseDependencies,
|
||||
ignoreRules?: IgnoreRules,
|
||||
watchEnabled?: boolean
|
||||
watchEnabled?: boolean,
|
||||
nameSource?: CLIFileNameSource
|
||||
) {
|
||||
const context = core.services.context as NodeServiceContext;
|
||||
const adapter = new CLIStorageEventManagerAdapter(
|
||||
@@ -37,7 +39,8 @@ export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEv
|
||||
nodePath.join(
|
||||
runtimeScopeDirectory(context.databasePath, dependencies.vaultService.getVaultName(), basePath),
|
||||
"storage-events.json"
|
||||
)
|
||||
),
|
||||
nameSource
|
||||
);
|
||||
super(adapter, dependencies);
|
||||
this.core = core;
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
"buildRun": "npm run build && npm run cli --",
|
||||
"build:docker": "docker build -f Dockerfile -t livesync-cli ../../..",
|
||||
"check": "tsc -p tsconfig.json",
|
||||
"test:unit": "cd ../../.. && npx vitest run --config vitest.config.unit.ts src/apps/cli/main.unit.spec.ts src/apps/cli/main.bootstrap.unit.spec.ts src/apps/cli/settingsPersistence.unit.spec.ts src/apps/cli/runtimeState.unit.spec.ts src/apps/cli/services/NodeKeyValueDBService.unit.spec.ts src/apps/cli/services/NodeLocalStorage.unit.spec.ts src/apps/cli/services/storageCaseSensitivity.unit.spec.ts src/apps/cli/managers/CLIStorageEventManagerAdapter.unit.spec.ts src/apps/cli/managers/CLIStorageEventManagerAdapter.real-watch.unit.spec.ts src/apps/cli/commands/utils.unit.spec.ts src/apps/cli/commands/runCommand.unit.spec.ts src/apps/cli/commands/journalSkip.unit.spec.ts src/apps/cli/commands/daemonCommand.unit.spec.ts src/apps/cli/commands/p2p.unit.spec.ts src/apps/cli/deploy/install.unit.spec.ts src/apps/cli/adapters/NodeFileSystemAdapter.unit.spec.ts",
|
||||
"test:unit": "cd ../../.. && npx vitest run --config vitest.config.unit.ts src/apps/cli/main.unit.spec.ts src/apps/cli/main.bootstrap.unit.spec.ts src/apps/cli/settingsPersistence.unit.spec.ts src/apps/cli/runtimeState.unit.spec.ts src/apps/cli/services/NodeKeyValueDBService.unit.spec.ts src/apps/cli/services/NodeLocalStorage.unit.spec.ts src/apps/cli/services/storageCaseSensitivity.unit.spec.ts src/apps/cli/managers/CLIStorageEventManagerAdapter.unit.spec.ts src/apps/cli/managers/CLIStorageEventManagerAdapter.real-watch.unit.spec.ts src/apps/cli/commands/utils.unit.spec.ts src/apps/cli/commands/runCommand.unit.spec.ts src/apps/cli/commands/journalSkip.unit.spec.ts src/apps/cli/commands/daemonCommand.unit.spec.ts src/apps/cli/commands/p2p.unit.spec.ts src/apps/cli/deploy/install.unit.spec.ts src/apps/cli/adapters/NodeFileSystemAdapter.unit.spec.ts src/apps/cli/storage src/apps/cli/serviceModules/FileAccessCLI.unit.spec.ts",
|
||||
"test:e2e:two-vaults": "bash test/test-e2e-two-vaults-with-docker-linux.sh",
|
||||
"test:e2e:two-vaults:common": "bash test/test-e2e-two-vaults-common.sh",
|
||||
"test:e2e:two-vaults:matrix": "bash test/test-e2e-two-vaults-matrix.sh",
|
||||
|
||||
@@ -29,13 +29,28 @@ export function initialiseServiceModulesCLI(
|
||||
const storageAccessManager = new StorageAccessManager();
|
||||
|
||||
// CLI-specific file access using Node.js FileSystemAdapter
|
||||
const vaultAccess = new FileAccessCLI(basePath, {
|
||||
storageAccessManager: storageAccessManager,
|
||||
vaultService: services.vault,
|
||||
settingService: services.setting,
|
||||
APIService: services.API,
|
||||
pathService: services.path,
|
||||
});
|
||||
const vaultAccess = new FileAccessCLI(
|
||||
basePath,
|
||||
{
|
||||
storageAccessManager: storageAccessManager,
|
||||
vaultService: services.vault,
|
||||
settingService: services.setting,
|
||||
APIService: services.API,
|
||||
pathService: services.path,
|
||||
},
|
||||
watchEnabled ? { ignoreRules } : undefined
|
||||
);
|
||||
const nameSource = vaultAccess.fileNameSource;
|
||||
if (nameSource) {
|
||||
services.appLifecycle.onLoaded.addHandler(async () => {
|
||||
await nameSource.prepare();
|
||||
return true;
|
||||
});
|
||||
services.appLifecycle.onBeforeUnload.addHandler(async () => {
|
||||
nameSource.stopDelivery();
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
// CLI-specific storage event manager
|
||||
const storageEventManager = new StorageEventManagerCLI(
|
||||
@@ -49,7 +64,8 @@ export function initialiseServiceModulesCLI(
|
||||
APIService: services.API,
|
||||
},
|
||||
ignoreRules,
|
||||
watchEnabled
|
||||
watchEnabled,
|
||||
nameSource
|
||||
);
|
||||
|
||||
// Close the file watcher during graceful shutdown so the process can exit cleanly.
|
||||
|
||||
@@ -4,14 +4,41 @@ import {
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/FileAccessBase";
|
||||
import { NodeFileSystemAdapter } from "@/apps/cli/adapters/NodeFileSystemAdapter";
|
||||
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||
import { parseFilePath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { CLIFileNameSource } from "@/apps/cli/storage/CLIFileNameSource";
|
||||
import type { StorageMutationCompletion } from "@/apps/cli/storage/ObservedNodeStorageAdapter";
|
||||
import type { IgnoreRules } from "./IgnoreRules";
|
||||
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
|
||||
|
||||
const finishUnobservedMutation: StorageMutationCompletion = () => undefined;
|
||||
|
||||
/**
|
||||
* CLI-specific implementation of FileAccessBase
|
||||
* Uses NodeFileSystemAdapter for Node.js file operations
|
||||
* Composes CLI file access with an optional watched name inventory.
|
||||
*
|
||||
* Single-run commands retain physical enumeration. Watching commands share the
|
||||
* adapter's storage boundary with the name source, so local mutations invalidate
|
||||
* names before chokidar has delivered their notifications.
|
||||
*/
|
||||
export class FileAccessCLI extends FileAccessBase<NodeFileSystemAdapter> {
|
||||
constructor(basePath: string, dependencies: FileAccessBaseDependencies) {
|
||||
/** Present only when this CLI instance owns the shared watcher lifecycle. */
|
||||
readonly fileNameSource?: CLIFileNameSource;
|
||||
|
||||
constructor(
|
||||
basePath: string,
|
||||
dependencies: FileAccessBaseDependencies,
|
||||
watchOptions?: { ignoreRules?: IgnoreRules }
|
||||
) {
|
||||
// The adapter must exist before the source can share its storage. This
|
||||
// closure reads the source when I/O begins, after composition has finished.
|
||||
let source: CLIFileNameSource | undefined;
|
||||
const invalidateFileNames = (paths: readonly string[]): StorageMutationCompletion => {
|
||||
if (source === undefined) {
|
||||
// Single-run commands have no inventory to update; the adapter
|
||||
// still receives the completion callback its mutation hook expects.
|
||||
return finishUnobservedMutation;
|
||||
}
|
||||
return source.invalidate(paths);
|
||||
};
|
||||
const pathNormalizationVersion = () =>
|
||||
getPathNormalizationVersion(dependencies.settingService.currentSettings());
|
||||
const isCaseInsensitive = () => {
|
||||
@@ -27,14 +54,39 @@ export class FileAccessCLI extends FileAccessBase<NodeFileSystemAdapter> {
|
||||
}
|
||||
},
|
||||
pathNormalizationVersion,
|
||||
isCaseInsensitive
|
||||
isCaseInsensitive,
|
||||
invalidateFileNames
|
||||
);
|
||||
super(adapter, dependencies);
|
||||
// Only the watching composition publishes indexed lookup. An empty
|
||||
// options object still requests watching; undefined selects single-run I/O.
|
||||
if (watchOptions) {
|
||||
source = new CLIFileNameSource(
|
||||
adapter.storage,
|
||||
() => ({
|
||||
pathNormalizationVersion: getPathNormalizationVersion(
|
||||
dependencies.settingService.currentSettings()
|
||||
),
|
||||
handleFilenameCaseSensitive: !isCaseInsensitive(),
|
||||
}),
|
||||
{
|
||||
basePath,
|
||||
ignoreRules: watchOptions.ignoreRules,
|
||||
reportDiagnostic: (message, detail) => {
|
||||
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
|
||||
if (detail !== undefined) {
|
||||
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
|
||||
}
|
||||
},
|
||||
suppressEvent: (path) => dependencies.storageAccessManager.isFileProcessing(parseFilePath(path)),
|
||||
}
|
||||
);
|
||||
adapter.fileNameLookup = source;
|
||||
this.fileNameSource = source;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Expose the adapter for accessing scanDirectory
|
||||
*/
|
||||
/** Expose the storage adapter to CLI operations which require physical scans. */
|
||||
get nodeAdapter(): NodeFileSystemAdapter {
|
||||
return this.adapter;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { StorageAccessManager } from "@vrtmrz/livesync-commonlib/compat/managers/StorageProcessingManager";
|
||||
import type { FileAccessBaseDependencies } from "@vrtmrz/livesync-commonlib/compat/serviceModules/FileAccessBase";
|
||||
import { FilePathIndex, parseFilePath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { isAcceptedInFilenameDuplicationFactory } from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/targetFilter";
|
||||
import { FileAccessCLI } from "./FileAccessCLI";
|
||||
import { ServiceFileAccessCLI } from "./ServiceFileAccessImpl";
|
||||
|
||||
const roots: string[] = [];
|
||||
const accesses: FileAccessCLI[] = [];
|
||||
afterEach(async () => {
|
||||
vi.restoreAllMocks();
|
||||
await Promise.all(
|
||||
accesses.splice(0).map(async (access) => {
|
||||
await access.fileNameSource?.close();
|
||||
})
|
||||
);
|
||||
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })));
|
||||
});
|
||||
|
||||
async function fixture(version: 0 | 1, caseSensitive: boolean, watching = true) {
|
||||
const root = await mkdtemp(join(tmpdir(), "livesync-name-query-"));
|
||||
roots.push(root);
|
||||
await mkdir(join(root, "unrelated"));
|
||||
await writeFile(join(root, "note.md"), "original");
|
||||
await writeFile(join(root, "unrelated", "other.md"), "other");
|
||||
const settings = { pathNormalizationVersion: version, handleFilenameCaseSensitive: caseSensitive };
|
||||
const dependencies = {
|
||||
storageAccessManager: new StorageAccessManager(),
|
||||
vaultService: { isStorageInsensitive: () => false },
|
||||
settingService: { currentSettings: () => settings },
|
||||
APIService: { addLog: vi.fn() },
|
||||
pathService: {},
|
||||
} as unknown as FileAccessBaseDependencies;
|
||||
const access = new FileAccessCLI(root, dependencies, watching ? {} : undefined);
|
||||
accesses.push(access);
|
||||
await access.fileNameSource?.prepare();
|
||||
const on = { addHandler: vi.fn() };
|
||||
const processing = { totalStorageFileEventCount: 0, commitPendingFileEvents: on };
|
||||
const storage = new ServiceFileAccessCLI({
|
||||
API: dependencies.APIService,
|
||||
vault: dependencies.vaultService,
|
||||
setting: dependencies.settingService,
|
||||
fileProcessing: processing,
|
||||
vaultAccess: access,
|
||||
storageEventManager: {},
|
||||
appLifecycle: { onFirstInitialise: on, onBeforeUnload: on },
|
||||
} as unknown as ConstructorParameters<typeof ServiceFileAccessCLI>[0]);
|
||||
const host = {
|
||||
services: {
|
||||
setting: dependencies.settingService,
|
||||
vault: { shouldCheckCaseInsensitively: () => !caseSensitive },
|
||||
fileProcessing: processing,
|
||||
},
|
||||
serviceModules: { storageAccess: storage },
|
||||
} as unknown as Parameters<typeof isAcceptedInFilenameDuplicationFactory>[0];
|
||||
const filter = isAcceptedInFilenameDuplicationFactory(host, vi.fn());
|
||||
return { root, access, storage, processing, filter };
|
||||
}
|
||||
|
||||
describe("CLI watched names through shared storage access", () => {
|
||||
it.each([
|
||||
[0, true],
|
||||
[0, false],
|
||||
[1, true],
|
||||
[1, false],
|
||||
] as const)(
|
||||
"does not enumerate metadata or rebuild identity maps after a content edit (version %i, case %s)",
|
||||
async (version, sensitive) => {
|
||||
const { access, storage, processing, filter } = await fixture(version, sensitive);
|
||||
const path = parseFilePath("note.md");
|
||||
expect(await filter(path)).toBe(true);
|
||||
const metadata = vi.spyOn(access.nodeAdapter, "getFiles");
|
||||
const rebuild = vi.spyOn(FilePathIndex.prototype, "replaceAll");
|
||||
const directories = vi.spyOn(access.nodeAdapter.storage, "readDirectory");
|
||||
await access.adapterWrite(path, "changed content");
|
||||
processing.totalStorageFileEventCount++;
|
||||
expect(await filter(path)).toBe(true);
|
||||
expect(metadata).not.toHaveBeenCalled();
|
||||
expect(rebuild).not.toHaveBeenCalled();
|
||||
expect(directories.mock.calls.every(([directory]) => directory !== "unrelated")).toBe(true);
|
||||
expect(await storage.getFileNames()).toEqual(expect.arrayContaining(["note.md", "unrelated/other.md"]));
|
||||
const files = await storage.getFiles();
|
||||
expect(files.find((file) => file.path === path)?.stat?.size).toBe("changed content".length);
|
||||
}
|
||||
);
|
||||
|
||||
it("sees direct adapter creation and an external alias before delayed watch delivery", async () => {
|
||||
const { root, access, storage, filter } = await fixture(0, false);
|
||||
await access.adapterWrite("created.bin", new Uint8Array([1, 2, 3]));
|
||||
expect(await storage.getFileNames()).toContain("created.bin");
|
||||
await writeFile(join(root, "NOTE.MD"), "external alias");
|
||||
expect(await filter(parseFilePath("note.md"))).toBe(false);
|
||||
expect(await readFile(join(root, "note.md"), "utf8")).toBe("original");
|
||||
expect(await readFile(join(root, "NOTE.MD"), "utf8")).toBe("external alias");
|
||||
await access.adapterRemove("created.bin");
|
||||
expect(await storage.getFileNames()).not.toContain("created.bin");
|
||||
});
|
||||
|
||||
it("reflects direct append and directory creation plus Vault rename before watch delivery", async () => {
|
||||
const { root, access, storage } = await fixture(0, true);
|
||||
await access.ensureDirectory("notes/later.md");
|
||||
await access.adapterAppend("notes/later.md", "appended");
|
||||
expect(await storage.getFileNames()).toContain("notes/later.md");
|
||||
await access.ensureDirectory("archive/renamed.md");
|
||||
const file = await access.getAbstractFileByPath("notes/later.md");
|
||||
if (!access.isFile(file)) {
|
||||
throw new Error("Expected the appended file");
|
||||
}
|
||||
await access.vaultRename(file, "archive/renamed.md");
|
||||
const names = await storage.getFileNames();
|
||||
expect(names).not.toContain("notes/later.md");
|
||||
expect(names).toContain("archive/renamed.md");
|
||||
expect(await readFile(join(root, "archive/renamed.md"), "utf8")).toBe("appended");
|
||||
});
|
||||
|
||||
it("keeps non-watching commands on fresh metadata enumeration without creating a source", async () => {
|
||||
const { root, access, storage } = await fixture(0, true, false);
|
||||
expect(access.fileNameSource).toBeUndefined();
|
||||
expect(access.nodeAdapter.fileNameLookup).toBeUndefined();
|
||||
await writeFile(join(root, "later.md"), "later");
|
||||
expect(await storage.getFileNames({ fresh: true })).toContain("later.md");
|
||||
await access.adapterWrite("local.md", "local write");
|
||||
expect(await storage.getFileNames()).toContain("local.md");
|
||||
expect(await readFile(join(root, "local.md"), "utf8")).toBe("local write");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,283 @@
|
||||
import {
|
||||
createStoragePathResolver,
|
||||
FilePathIndex,
|
||||
getStoragePathComparisonKey,
|
||||
parseFilePath,
|
||||
type DirectoryEntry,
|
||||
type FilePath,
|
||||
type PathComparisonSettings,
|
||||
type StoragePath,
|
||||
type StorageEntry,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { isHiddenPath } from "@/apps/cli/cliPathFilter";
|
||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||
import {
|
||||
asError,
|
||||
getRawDirectoryEntries,
|
||||
isSupportedPolicy,
|
||||
joinRelative,
|
||||
samePolicy,
|
||||
type DirectoryListing,
|
||||
type InventoryReader,
|
||||
type InventoryState,
|
||||
} from "./CLIFileNameSource.state";
|
||||
|
||||
export type InventoryReaderDependencies = {
|
||||
/** The read-only listing capability used to build and validate inventories. */
|
||||
readonly storage: Pick<UnicodeResolvingNodeStorageAdapter, "readDirectory">;
|
||||
readonly basePath: string;
|
||||
readonly reportDiagnostic: CliDiagnosticReporter;
|
||||
readonly markDirectoryDirty: (directory: string) => void;
|
||||
readonly recordFreshFailure: (state: InventoryState, policy: PathComparisonSettings, error: unknown) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* Create an unpublished inventory candidate with an independent policy value.
|
||||
*/
|
||||
export function createEmptyState(policy: PathComparisonSettings): InventoryState {
|
||||
return {
|
||||
index: new FilePathIndex(),
|
||||
files: new Set<FilePath>(),
|
||||
directories: new Map<string, readonly DirectoryEntry[]>(),
|
||||
readErrors: new Map<string, Error>(),
|
||||
aliasErrors: new Map<string, Error>(),
|
||||
policy: { ...policy },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Bind the shared path resolver to the minimal storage read port used for sibling checks.
|
||||
*/
|
||||
export function createResolver(
|
||||
dependencies: Pick<InventoryReaderDependencies, "storage">,
|
||||
policy: PathComparisonSettings
|
||||
) {
|
||||
return createStoragePathResolver({
|
||||
readDirectory: async (directory) => await dependencies.storage.readDirectory(directory),
|
||||
policy: {
|
||||
pathNormalizationVersion: 1,
|
||||
handleFilenameCaseSensitive: policy.handleFilenameCaseSensitive,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Read original physical names in Legacy mode or resolver-validated names in NFC mode.
|
||||
*/
|
||||
export async function readDirectory(
|
||||
dependencies: Pick<InventoryReaderDependencies, "storage">,
|
||||
directory: string,
|
||||
policy: PathComparisonSettings
|
||||
): Promise<DirectoryListing | null> {
|
||||
const physical = parseFilePath(directory) as StoragePath;
|
||||
// Legacy mode reads physical spellings directly. NFC mode also validates
|
||||
// sibling identities through the shared resolver before accepting coverage.
|
||||
if (policy.pathNormalizationVersion !== 1) {
|
||||
const entries = await dependencies.storage.readDirectory(physical);
|
||||
return entries === null ? null : { kind: "legacy", entries };
|
||||
}
|
||||
const entries = await createResolver(dependencies, policy).listResolvedDirectory(physical);
|
||||
if (entries === null) {
|
||||
return null;
|
||||
}
|
||||
// Keep the resolver's path refinements through membership publication.
|
||||
return { kind: "validated", entries };
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse supported Legacy entries lazily, preserving traversal and diagnostic order.
|
||||
*
|
||||
* Host enumeration supplies physical spelling; parsing checks the path syntax.
|
||||
* Invalid entries remain in raw coverage but never enter the file-name index.
|
||||
*/
|
||||
export function* parseLegacyStorageEntries(
|
||||
dependencies: Pick<InventoryReaderDependencies, "reportDiagnostic">,
|
||||
directory: string,
|
||||
entries: readonly DirectoryEntry[]
|
||||
): Generator<StorageEntry, void> {
|
||||
for (const entry of entries) {
|
||||
const child = joinRelative(directory, entry.name);
|
||||
let childPath: StoragePath;
|
||||
try {
|
||||
childPath = parseFilePath(child) as StoragePath;
|
||||
} catch (error) {
|
||||
dependencies.reportDiagnostic("[CLIFileNameSource] Skipped invalid path '" + child + "':", error);
|
||||
continue;
|
||||
}
|
||||
yield { path: childPath, kind: entry.kind };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Add a complete directory subtree to an unpublished candidate, retaining Legacy partial reads.
|
||||
*
|
||||
* Storage supplies listings, basePath identifies an unavailable root, and diagnostics report
|
||||
* unreadable Legacy descendants or invalid physical names.
|
||||
*/
|
||||
export async function scanWholeDirectory(
|
||||
dependencies: Pick<InventoryReaderDependencies, "storage" | "basePath" | "reportDiagnostic">,
|
||||
state: InventoryState,
|
||||
directory: string,
|
||||
isRoot: boolean
|
||||
): Promise<void> {
|
||||
// Direct subtree scans and recursive descent share the same admission rule.
|
||||
if (isHiddenPath(directory)) {
|
||||
return;
|
||||
}
|
||||
let listing: DirectoryListing | null;
|
||||
try {
|
||||
listing = await readDirectory(dependencies, directory, state.policy);
|
||||
} catch (error) {
|
||||
const failure = asError(error);
|
||||
// Legacy inventories retain partial readability; NFC inventories
|
||||
// must reject unreadable scopes so their identity checks stay sound.
|
||||
if (isRoot || state.policy.pathNormalizationVersion === 1) {
|
||||
if (failure.name === "AmbiguousNormalisedPathError") {
|
||||
state.aliasErrors.set(directory, failure);
|
||||
} else {
|
||||
state.readErrors.set(directory, failure);
|
||||
}
|
||||
throw failure;
|
||||
}
|
||||
state.readErrors.set(directory, failure);
|
||||
dependencies.reportDiagnostic("[CLIFileNameSource] Skipped unreadable directory '" + directory + "':", failure);
|
||||
return;
|
||||
}
|
||||
if (listing === null) {
|
||||
// An absent descendant contributes no names. An absent root means
|
||||
// there is no listing authority, so an empty inventory would be unsafe.
|
||||
if (isRoot) {
|
||||
throw new Error("The CLI file-name source root is unavailable: " + dependencies.basePath);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
state.directories.set(directory, getRawDirectoryEntries(directory, listing));
|
||||
state.readErrors.delete(directory);
|
||||
state.aliasErrors.delete(directory);
|
||||
const entries =
|
||||
listing.kind === "validated"
|
||||
? listing.entries
|
||||
: parseLegacyStorageEntries(dependencies, directory, listing.entries);
|
||||
for (const entry of entries) {
|
||||
if (entry.kind === "file") {
|
||||
state.files.add(entry.path);
|
||||
} else if (!isHiddenPath(entry.path)) {
|
||||
// Supported listing excludes hidden subtrees, but includes hidden
|
||||
// leaf files encountered above for collision-name coverage.
|
||||
await scanWholeDirectory(dependencies, state, entry.path, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a fresh candidate; the optional owner callback retains failures from requested fresh scans.
|
||||
*/
|
||||
export async function enumerate(
|
||||
dependencies: Pick<InventoryReaderDependencies, "storage" | "basePath" | "reportDiagnostic" | "recordFreshFailure">,
|
||||
policy: PathComparisonSettings,
|
||||
retainFailure = false
|
||||
): Promise<InventoryState> {
|
||||
if (!isSupportedPolicy(policy)) {
|
||||
throw new Error("Unsupported path comparison policy: " + JSON.stringify(policy));
|
||||
}
|
||||
const state = createEmptyState(policy);
|
||||
try {
|
||||
await scanWholeDirectory(dependencies, state, "", true);
|
||||
state.index.replaceAll(state.files);
|
||||
return state;
|
||||
} catch (error) {
|
||||
if (retainFailure) {
|
||||
// The owner records the first failing scope and keeps this
|
||||
// failed fresh scan from being mistaken for an empty inventory.
|
||||
dependencies.recordFreshFailure(state, policy, error);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate cached sibling identities before publishing a policy, then dirty failed scopes for NFC rereads.
|
||||
*
|
||||
* This operation uses cached original names only; it performs no storage I/O.
|
||||
*/
|
||||
export async function selectPolicy(
|
||||
dependencies: Pick<InventoryReaderDependencies, "markDirectoryDirty">,
|
||||
state: InventoryState,
|
||||
policy: PathComparisonSettings
|
||||
): Promise<void> {
|
||||
if (!isSupportedPolicy(policy)) {
|
||||
throw new Error("Unsupported path comparison policy: " + JSON.stringify(policy));
|
||||
}
|
||||
if (samePolicy(state.policy, policy)) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Alias evidence belongs to the policy, whereas cached entries preserve
|
||||
// physical names. Recheck all covered siblings before publishing the policy.
|
||||
const nextAliasErrors = new Map<string, Error>();
|
||||
if (policy.pathNormalizationVersion === 1) {
|
||||
for (const [directory, entries] of state.directories) {
|
||||
const seen = new Map<string, string>();
|
||||
for (const entry of entries) {
|
||||
let key: string;
|
||||
try {
|
||||
key = getStoragePathComparisonKey(parseFilePath(entry.name), policy);
|
||||
} catch (error) {
|
||||
nextAliasErrors.set(directory, asError(error));
|
||||
break;
|
||||
}
|
||||
// Different physical siblings sharing this key make the NFC
|
||||
// directory ambiguous; choosing either would hide a candidate.
|
||||
const previous = seen.get(key);
|
||||
if (previous !== undefined) {
|
||||
nextAliasErrors.set(
|
||||
directory,
|
||||
new Error(
|
||||
"Ambiguous NFC entries under '" +
|
||||
directory +
|
||||
"': '" +
|
||||
previous +
|
||||
"' and '" +
|
||||
entry.name +
|
||||
"'"
|
||||
)
|
||||
);
|
||||
break;
|
||||
}
|
||||
seen.set(key, entry.name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Rebuild alias evidence from the cached physical spellings before
|
||||
// publishing the new policy; failed directories are then revalidated.
|
||||
state.policy = { ...policy };
|
||||
state.aliasErrors = nextAliasErrors;
|
||||
if (policy.pathNormalizationVersion === 1) {
|
||||
for (const directory of state.readErrors.keys()) {
|
||||
dependencies.markDirectoryDirty(directory);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Bind the inventory operations to CLIFileNameSource dependencies without owning readiness or failure state.
|
||||
*/
|
||||
export function createInventoryReader(dependencies: InventoryReaderDependencies): InventoryReader {
|
||||
const enumerationDependencies = {
|
||||
storage: dependencies.storage,
|
||||
basePath: dependencies.basePath,
|
||||
reportDiagnostic: dependencies.reportDiagnostic,
|
||||
recordFreshFailure: dependencies.recordFreshFailure,
|
||||
};
|
||||
const directoryDependencies = { storage: dependencies.storage };
|
||||
const policyDependencies = { markDirectoryDirty: dependencies.markDirectoryDirty };
|
||||
|
||||
return {
|
||||
enumerate: (policy, retainFailure) => enumerate(enumerationDependencies, policy, retainFailure),
|
||||
selectPolicy: (state, policy) => selectPolicy(policyDependencies, state, policy),
|
||||
readDirectory: (directory, policy) => readDirectory(directoryDependencies, directory, policy),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
parseFilePath,
|
||||
type DirectoryEntry,
|
||||
type PathComparisonSettings,
|
||||
type StoragePath,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import {
|
||||
createEmptyState,
|
||||
enumerate,
|
||||
readDirectory,
|
||||
scanWholeDirectory,
|
||||
selectPolicy,
|
||||
type InventoryReaderDependencies,
|
||||
} from "./CLIFileNameSource.inventory";
|
||||
import type { InventoryState } from "./CLIFileNameSource.state";
|
||||
|
||||
function policy(pathNormalizationVersion: 0 | 1, handleFilenameCaseSensitive = true): PathComparisonSettings {
|
||||
return { pathNormalizationVersion, handleFilenameCaseSensitive };
|
||||
}
|
||||
|
||||
describe("CLIFileNameSource inventory operations", () => {
|
||||
it("preserves physical spellings in Legacy listings and rejects NFC sibling aliases", async () => {
|
||||
const nfcName = "caf\u00e9.md";
|
||||
const nfdName = "cafe\u0301.md";
|
||||
const entries: readonly DirectoryEntry[] = [
|
||||
{ name: nfcName, kind: "file" },
|
||||
{ name: nfdName, kind: "file" },
|
||||
];
|
||||
const storage = {
|
||||
readDirectory: vi.fn(async (_directory: StoragePath): Promise<readonly DirectoryEntry[] | null> => entries),
|
||||
};
|
||||
const freshFailures: Array<{
|
||||
state: InventoryState;
|
||||
policy: PathComparisonSettings;
|
||||
error: unknown;
|
||||
}> = [];
|
||||
const enumerationDependencies: Pick<
|
||||
InventoryReaderDependencies,
|
||||
"storage" | "basePath" | "reportDiagnostic" | "recordFreshFailure"
|
||||
> = {
|
||||
storage,
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
recordFreshFailure: (state, selectedPolicy, error) => {
|
||||
freshFailures.push({ state, policy: selectedPolicy, error });
|
||||
},
|
||||
};
|
||||
|
||||
await expect(readDirectory({ storage }, "", policy(0))).resolves.toEqual({ kind: "legacy", entries });
|
||||
|
||||
const legacyState = await enumerate(enumerationDependencies, policy(0));
|
||||
expect(Array.from(legacyState.files)).toEqual([parseFilePath(nfcName), parseFilePath(nfdName)]);
|
||||
|
||||
await expect(readDirectory({ storage }, "", policy(1))).rejects.toThrow(/Ambiguous/);
|
||||
await expect(enumerate(enumerationDependencies, policy(1), true)).rejects.toThrow(/Ambiguous/);
|
||||
expect(freshFailures).toHaveLength(1);
|
||||
expect(freshFailures[0]?.state.aliasErrors.has("")).toBe(true);
|
||||
expect(freshFailures[0]?.policy).toEqual(policy(1));
|
||||
expect(freshFailures[0]?.error).toBeInstanceOf(Error);
|
||||
});
|
||||
|
||||
it("returns full physical paths and entry kinds for non-root NFC listings", async () => {
|
||||
const nfdName = "cafe\u0301.md";
|
||||
const entries: readonly DirectoryEntry[] = [
|
||||
{ name: nfdName, kind: "file" },
|
||||
{ name: "nested", kind: "directory" },
|
||||
];
|
||||
const storage = {
|
||||
readDirectory: vi.fn(
|
||||
async (directory: StoragePath): Promise<readonly DirectoryEntry[] | null> =>
|
||||
directory === "folder" ? entries : null
|
||||
),
|
||||
};
|
||||
|
||||
await expect(readDirectory({ storage }, "folder", policy(1))).resolves.toEqual({
|
||||
kind: "validated",
|
||||
entries: [
|
||||
{ path: "folder/" + nfdName, kind: "file" },
|
||||
{ path: "folder/nested", kind: "directory" },
|
||||
],
|
||||
});
|
||||
expect(storage.readDirectory).toHaveBeenCalledWith("folder");
|
||||
});
|
||||
|
||||
it.each([0, 1] as const)("preserves an absent listing as null in mode %i", async (version) => {
|
||||
const storage = {
|
||||
readDirectory: vi.fn(async (_directory: StoragePath): Promise<readonly DirectoryEntry[] | null> => null),
|
||||
};
|
||||
|
||||
await expect(readDirectory({ storage }, "folder", policy(version))).resolves.toBeNull();
|
||||
expect(storage.readDirectory).toHaveBeenCalledWith("folder");
|
||||
});
|
||||
|
||||
it("keeps a readable Legacy inventory after a descendant failure and retains an NFC fresh failure", async () => {
|
||||
const descendantFailure = new Error("descendant listing failed");
|
||||
const rootEntries: readonly DirectoryEntry[] = [
|
||||
{ name: "unreadable", kind: "directory" },
|
||||
{ name: "visible.md", kind: "file" },
|
||||
];
|
||||
const storage = {
|
||||
readDirectory: vi.fn(async (directory: StoragePath): Promise<readonly DirectoryEntry[] | null> => {
|
||||
if (directory === "") {
|
||||
return rootEntries;
|
||||
}
|
||||
if (directory === "unreadable") {
|
||||
throw descendantFailure;
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
};
|
||||
const reportDiagnostic = vi.fn();
|
||||
const scanDependencies: Pick<InventoryReaderDependencies, "storage" | "basePath" | "reportDiagnostic"> = {
|
||||
storage,
|
||||
basePath: "/vault",
|
||||
reportDiagnostic,
|
||||
};
|
||||
const legacyState = createEmptyState(policy(0));
|
||||
|
||||
await scanWholeDirectory(scanDependencies, legacyState, "", true);
|
||||
|
||||
expect(Array.from(legacyState.files)).toEqual([parseFilePath("visible.md")]);
|
||||
expect(legacyState.readErrors.get("unreadable")).toBe(descendantFailure);
|
||||
expect(reportDiagnostic).toHaveBeenCalledWith(
|
||||
"[CLIFileNameSource] Skipped unreadable directory 'unreadable':",
|
||||
descendantFailure
|
||||
);
|
||||
|
||||
const freshFailures: Array<{
|
||||
state: InventoryState;
|
||||
policy: PathComparisonSettings;
|
||||
error: unknown;
|
||||
}> = [];
|
||||
const enumerationDependencies: Pick<
|
||||
InventoryReaderDependencies,
|
||||
"storage" | "basePath" | "reportDiagnostic" | "recordFreshFailure"
|
||||
> = {
|
||||
...scanDependencies,
|
||||
recordFreshFailure: (state, selectedPolicy, error) => {
|
||||
freshFailures.push({ state, policy: selectedPolicy, error });
|
||||
},
|
||||
};
|
||||
|
||||
await expect(enumerate(enumerationDependencies, policy(1), true)).rejects.toThrow(/descendant listing failed/);
|
||||
expect(freshFailures).toHaveLength(1);
|
||||
expect(freshFailures[0]?.state.readErrors.has("unreadable")).toBe(true);
|
||||
expect(freshFailures[0]?.policy).toEqual(policy(1));
|
||||
expect(freshFailures[0]?.error).toBeInstanceOf(Error);
|
||||
});
|
||||
|
||||
it.each([0, 1] as const)(
|
||||
"treats a missing root as unavailable and an absent descendant as empty (version %i)",
|
||||
async (version) => {
|
||||
const rootEntries: readonly DirectoryEntry[] = [
|
||||
{ name: "absent", kind: "directory" },
|
||||
{ name: "visible.md", kind: "file" },
|
||||
];
|
||||
const descendantStorage = {
|
||||
readDirectory: vi.fn(
|
||||
async (directory: StoragePath): Promise<readonly DirectoryEntry[] | null> =>
|
||||
directory === "" ? rootEntries : null
|
||||
),
|
||||
};
|
||||
const reportDiagnostic = vi.fn();
|
||||
const descendantState = createEmptyState(policy(version));
|
||||
|
||||
await scanWholeDirectory(
|
||||
{ storage: descendantStorage, basePath: "/vault", reportDiagnostic },
|
||||
descendantState,
|
||||
"",
|
||||
true
|
||||
);
|
||||
|
||||
expect(Array.from(descendantState.files)).toEqual([parseFilePath("visible.md")]);
|
||||
expect(descendantState.directories.has("absent")).toBe(false);
|
||||
expect(reportDiagnostic).not.toHaveBeenCalled();
|
||||
|
||||
const unavailableStorage = {
|
||||
readDirectory: vi.fn(
|
||||
async (_directory: StoragePath): Promise<readonly DirectoryEntry[] | null> => null
|
||||
),
|
||||
};
|
||||
const unavailableState = createEmptyState(policy(version));
|
||||
await expect(
|
||||
scanWholeDirectory(
|
||||
{ storage: unavailableStorage, basePath: "/vault", reportDiagnostic },
|
||||
unavailableState,
|
||||
"",
|
||||
true
|
||||
)
|
||||
).rejects.toThrow("The CLI file-name source root is unavailable: /vault");
|
||||
expect(unavailableStorage.readDirectory).toHaveBeenCalledWith("");
|
||||
expect(unavailableState.files.size).toBe(0);
|
||||
}
|
||||
);
|
||||
|
||||
it("skips an invalid Legacy leaf while retaining readable names", async () => {
|
||||
const entries: readonly DirectoryEntry[] = [
|
||||
{ name: "C:unsupported.md", kind: "file" },
|
||||
{ name: "visible.md", kind: "file" },
|
||||
];
|
||||
const storage = {
|
||||
readDirectory: vi.fn(async (_directory: StoragePath): Promise<readonly DirectoryEntry[] | null> => entries),
|
||||
};
|
||||
const reportDiagnostic = vi.fn();
|
||||
const state = createEmptyState(policy(0));
|
||||
|
||||
await scanWholeDirectory({ storage, basePath: "/vault", reportDiagnostic }, state, "", true);
|
||||
|
||||
expect(Array.from(state.files)).toEqual([parseFilePath("visible.md")]);
|
||||
expect(reportDiagnostic).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Skipped invalid path 'C:unsupported.md':"),
|
||||
expect.any(Error)
|
||||
);
|
||||
|
||||
// Legacy skips remain in physical coverage and must block NFC activation.
|
||||
await selectPolicy({ markDirectoryDirty: vi.fn() }, state, policy(1));
|
||||
expect(state.aliasErrors.get("")).toBeInstanceOf(Error);
|
||||
expect(storage.readDirectory).toHaveBeenCalledTimes(1);
|
||||
expect(Array.from(state.files)).toEqual([parseFilePath("visible.md")]);
|
||||
});
|
||||
|
||||
it("rejects unsupported runtime policies before reading or publishing them", async () => {
|
||||
const unsupportedVersion = {
|
||||
pathNormalizationVersion: 2,
|
||||
handleFilenameCaseSensitive: true,
|
||||
} as unknown as PathComparisonSettings;
|
||||
const invalidCasePolicy = {
|
||||
pathNormalizationVersion: 1,
|
||||
handleFilenameCaseSensitive: 1,
|
||||
} as unknown as PathComparisonSettings;
|
||||
const storage = {
|
||||
readDirectory: vi.fn(async (_directory: StoragePath): Promise<readonly DirectoryEntry[] | null> => []),
|
||||
};
|
||||
const recordFreshFailure = vi.fn();
|
||||
const enumerationDependencies = {
|
||||
storage,
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
recordFreshFailure,
|
||||
};
|
||||
|
||||
for (const invalidPolicy of [unsupportedVersion, invalidCasePolicy]) {
|
||||
await expect(enumerate(enumerationDependencies, invalidPolicy, true)).rejects.toThrow(
|
||||
"Unsupported path comparison policy"
|
||||
);
|
||||
}
|
||||
expect(storage.readDirectory).not.toHaveBeenCalled();
|
||||
expect(recordFreshFailure).not.toHaveBeenCalled();
|
||||
|
||||
const state = createEmptyState(policy(0));
|
||||
const markDirectoryDirty = vi.fn();
|
||||
await expect(selectPolicy({ markDirectoryDirty }, state, invalidCasePolicy)).rejects.toThrow(
|
||||
"Unsupported path comparison policy"
|
||||
);
|
||||
expect(state.policy).toEqual(policy(0));
|
||||
expect(markDirectoryDirty).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("revalidates cached siblings and dirties unreadable scopes without storage access", async () => {
|
||||
const nfcName = "caf\u00e9.md";
|
||||
const nfdName = "cafe\u0301.md";
|
||||
const state = createEmptyState(policy(0));
|
||||
state.directories.set("", [
|
||||
{ name: nfcName, kind: "file" },
|
||||
{ name: nfdName, kind: "file" },
|
||||
]);
|
||||
state.readErrors.set("unreadable", new Error("prior listing failure"));
|
||||
const markDirectoryDirty = vi.fn();
|
||||
const dependencies: Pick<InventoryReaderDependencies, "markDirectoryDirty"> = { markDirectoryDirty };
|
||||
|
||||
await selectPolicy(dependencies, state, policy(1));
|
||||
|
||||
expect(state.policy).toEqual(policy(1));
|
||||
expect(state.aliasErrors.get("")).toBeInstanceOf(Error);
|
||||
expect(state.aliasErrors.get("")?.message).toContain(nfcName);
|
||||
expect(state.aliasErrors.get("")?.message).toContain(nfdName);
|
||||
expect(markDirectoryDirty).toHaveBeenCalledTimes(1);
|
||||
expect(markDirectoryDirty).toHaveBeenCalledWith("unreadable");
|
||||
|
||||
await selectPolicy(dependencies, state, policy(0));
|
||||
expect(state.aliasErrors.size).toBe(0);
|
||||
|
||||
state.directories.set("", [{ name: nfcName, kind: "file" }]);
|
||||
await selectPolicy(dependencies, state, policy(1));
|
||||
|
||||
expect(state.policy).toEqual(policy(1));
|
||||
expect(state.aliasErrors.size).toBe(0);
|
||||
expect(markDirectoryDirty).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,200 @@
|
||||
import { EventEmitter } from "node:events";
|
||||
import { fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { parseFilePath, type PathComparisonSettings } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { FSWatcher } from "chokidar";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { NodeFileSystemAdapter } from "@/apps/cli/adapters/NodeFileSystemAdapter";
|
||||
import { CLIFileNameSource } from "./CLIFileNameSource";
|
||||
import { ObservedNodeStorageAdapter } from "./ObservedNodeStorageAdapter";
|
||||
|
||||
async function withSource(
|
||||
version: 0 | 1,
|
||||
run: (fixture: {
|
||||
root: string;
|
||||
source: CLIFileNameSource;
|
||||
storage: ObservedNodeStorageAdapter;
|
||||
policy: PathComparisonSettings;
|
||||
prepare: () => Promise<void>;
|
||||
listExpected: () => Promise<string[]>;
|
||||
}) => Promise<void>
|
||||
) {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-name-membership-"));
|
||||
const policy: PathComparisonSettings = { pathNormalizationVersion: version, handleFilenameCaseSensitive: true };
|
||||
const watcher = Object.assign(new EventEmitter(), { close: async () => undefined });
|
||||
let source: CLIFileNameSource;
|
||||
const storage = new ObservedNodeStorageAdapter(
|
||||
root,
|
||||
() => version,
|
||||
() => false,
|
||||
(paths) => source.invalidate(paths)
|
||||
);
|
||||
source = new CLIFileNameSource(storage, () => policy, {
|
||||
basePath: root,
|
||||
watcherFactory: () => watcher as unknown as FSWatcher,
|
||||
});
|
||||
const expected = new NodeFileSystemAdapter(root, undefined, () => version);
|
||||
try {
|
||||
await run({
|
||||
root,
|
||||
source,
|
||||
storage,
|
||||
policy,
|
||||
prepare: async () => {
|
||||
const pending = source.prepare();
|
||||
watcher.emit("ready");
|
||||
await pending;
|
||||
},
|
||||
listExpected: async () => (await expected.getFiles()).map((file) => file.path).sort(),
|
||||
});
|
||||
} finally {
|
||||
await source.close();
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
describe("CLI watched membership and the supported physical listing", () => {
|
||||
it("skips excluded Legacy directory siblings when selecting a supported parent", async () => {
|
||||
await withSource(0, async ({ root, source, policy, prepare, listExpected }) => {
|
||||
fs.mkdirSync(path.join(root, "C:unsupported"));
|
||||
fs.mkdirSync(path.join(root, "docs"));
|
||||
fs.writeFileSync(path.join(root, "docs/note.md"), "safe");
|
||||
await prepare();
|
||||
expect(await source.getFileNames()).toEqual(await listExpected());
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("docs/note.md"), policy)).resolves.toEqual([
|
||||
"docs/note.md",
|
||||
]);
|
||||
expect(fs.readFileSync(path.join(root, "docs/note.md"), "utf8")).toBe("safe");
|
||||
});
|
||||
});
|
||||
|
||||
it.each([0, 1] as const)(
|
||||
"keeps hidden-directory mutations outside membership, with hidden leaf files retained (version %i)",
|
||||
async (version) => {
|
||||
await withSource(version, async ({ root, source, storage, prepare, listExpected }) => {
|
||||
fs.mkdirSync(path.join(root, ".hidden"));
|
||||
fs.mkdirSync(path.join(root, "notes/.private"), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, "safe.md"), "safe");
|
||||
await prepare();
|
||||
await storage.write(".hidden/internal.md", "private");
|
||||
await storage.write("notes/.private/nested.md", "nested");
|
||||
await storage.write(".visible.md", "supported leaf");
|
||||
await storage.write("notes/.leaf.md", "supported nested leaf");
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
expect((await source.getFileNames()).sort()).toEqual([".visible.md", "notes/.leaf.md", "safe.md"]);
|
||||
await storage.rename(".hidden/internal.md", "notes/moved.md");
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
await storage.rename("notes/moved.md", ".hidden/returned.md");
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
expect((await source.getFileNames({ fresh: true })).sort()).toEqual(await listExpected());
|
||||
expect(fs.readFileSync(path.join(root, ".hidden/returned.md"), "utf8")).toBe("private");
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
it.each(["write", "writeBinary", "append", "mkdir"] as const)(
|
||||
"updates an NFD physical parent after logical NFC %s before watcher delivery",
|
||||
async (operation) => {
|
||||
await withSource(1, async ({ root, source, storage, prepare, listExpected }) => {
|
||||
fs.mkdirSync(path.join(root, "cafe\u0301"));
|
||||
fs.mkdirSync(path.join(root, "unrelated"));
|
||||
fs.writeFileSync(path.join(root, "cafe\u0301/existing.md"), "old");
|
||||
fs.writeFileSync(path.join(root, "unrelated/keep.md"), "unrelated");
|
||||
await prepare();
|
||||
const reads = vi.spyOn(storage, "readDirectory");
|
||||
const target = "caf\u00e9/new/note.md";
|
||||
if (operation === "writeBinary") {
|
||||
await storage.writeBinary(target, new Uint8Array([1, 2, 3]).buffer);
|
||||
} else if (operation === "mkdir") {
|
||||
await storage.mkdir("caf\u00e9/new");
|
||||
await storage.write(target, "new");
|
||||
} else {
|
||||
await storage[operation](target, "new");
|
||||
}
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
expect(reads.mock.calls.some(([directory]) => directory === "unrelated")).toBe(false);
|
||||
expect(fs.existsSync(path.join(root, "cafe\u0301/new/note.md"))).toBe(true);
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
it("retires and adds physical memberships for logical NFC rename and removal", async () => {
|
||||
await withSource(1, async ({ root, source, storage, prepare, listExpected }) => {
|
||||
fs.mkdirSync(path.join(root, "cafe\u0301"));
|
||||
fs.mkdirSync(path.join(root, "re\u0301sume\u0301"));
|
||||
fs.writeFileSync(path.join(root, "cafe\u0301/old.md"), "preserved");
|
||||
await prepare();
|
||||
await storage.rename("caf\u00e9/old.md", "r\u00e9sum\u00e9/new.md");
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
expect(await source.getFileNames()).toEqual(["re\u0301sume\u0301/new.md"]);
|
||||
await storage.remove("r\u00e9sum\u00e9/new.md");
|
||||
expect(await source.getFileNames()).toEqual([]);
|
||||
expect(await source.getFileNames()).toEqual(await listExpected());
|
||||
});
|
||||
});
|
||||
|
||||
it("replays a logical invalidation captured before initial physical coverage was published", async () => {
|
||||
await withSource(1, async ({ root, source, storage, prepare, listExpected }) => {
|
||||
fs.mkdirSync(path.join(root, "cafe\u0301"));
|
||||
fs.writeFileSync(path.join(root, "cafe\u0301/existing.md"), "old");
|
||||
const originalRead = storage.readDirectory.bind(storage);
|
||||
let mutated = false;
|
||||
vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
const entries = await originalRead(directory);
|
||||
if (directory === "cafe\u0301" && !mutated) {
|
||||
mutated = true;
|
||||
await storage.write("caf\u00e9/added.md", "new");
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
await prepare();
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
});
|
||||
});
|
||||
it("retries a physical-parent read superseded by a logical NFC mutation", async () => {
|
||||
await withSource(1, async ({ root, source, storage, policy, prepare }) => {
|
||||
fs.mkdirSync(path.join(root, "cafe\u0301"));
|
||||
fs.writeFileSync(path.join(root, "cafe\u0301/existing.md"), "old");
|
||||
await prepare();
|
||||
const originalRead = storage.readDirectory.bind(storage);
|
||||
let mutated = false;
|
||||
vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
const entries = await originalRead(directory);
|
||||
if (directory === "cafe\u0301" && !mutated) {
|
||||
mutated = true;
|
||||
await storage.write("caf\u00e9/added.md", "new");
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
await expect(
|
||||
source.findPathIdentityCandidates(parseFilePath("caf\u00e9/added.md"), policy)
|
||||
).resolves.toEqual(["cafe\u0301/added.md"]);
|
||||
});
|
||||
});
|
||||
|
||||
it("invalidates the physical parent when NFC I/O overrides the default case rule", async () => {
|
||||
await withSource(1, async ({ root, source, storage, prepare, listExpected }) => {
|
||||
fs.mkdirSync(path.join(root, "Cafe\u0301"));
|
||||
fs.writeFileSync(path.join(root, "Cafe\u0301/existing.md"), "old");
|
||||
await prepare();
|
||||
await storage.write("CAF\u00c9/added.md", "new", undefined, true);
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
expect(await source.getFileNames()).toContain("Cafe\u0301/added.md");
|
||||
});
|
||||
});
|
||||
|
||||
it("retires whole subtrees after logical NFC directory rename and deletion", async () => {
|
||||
await withSource(1, async ({ root, source, storage, prepare, listExpected }) => {
|
||||
fs.mkdirSync(path.join(root, "cafe\u0301/sub"), { recursive: true });
|
||||
fs.mkdirSync(path.join(root, "re\u0301sume\u0301"));
|
||||
fs.writeFileSync(path.join(root, "cafe\u0301/sub/note.md"), "preserved");
|
||||
await prepare();
|
||||
await storage.rename("caf\u00e9/sub", "r\u00e9sum\u00e9/moved");
|
||||
expect((await source.getFileNames()).sort()).toEqual(await listExpected());
|
||||
expect(await source.getFileNames()).toEqual(["re\u0301sume\u0301/moved/note.md"]);
|
||||
expect(fs.readFileSync(path.join(root, "re\u0301sume\u0301/moved/note.md"), "utf8")).toBe("preserved");
|
||||
await storage.remove("r\u00e9sum\u00e9/moved");
|
||||
expect(await source.getFileNames()).toEqual([]);
|
||||
expect(await source.getFileNames()).toEqual(await listExpected());
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,77 @@
|
||||
import {
|
||||
getStoragePathComparisonKey,
|
||||
parseFilePath,
|
||||
type DirectoryEntry,
|
||||
type FilePath,
|
||||
type PathComparisonSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { isHiddenPath } from "@/apps/cli/cliPathFilter";
|
||||
import { joinRelative } from "./CLIFileNameSource.state";
|
||||
|
||||
/** Select supported physical child directories while retaining raw entries elsewhere for NFC validation. */
|
||||
export function getSupportedChildDirectories(parent: string, entries: readonly DirectoryEntry[]): FilePath[] {
|
||||
const directories: FilePath[] = [];
|
||||
for (const entry of entries) {
|
||||
if (entry.kind !== "directory") {
|
||||
continue;
|
||||
}
|
||||
const child = joinRelative(parent, entry.name);
|
||||
if (isHiddenPath(child)) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
directories.push(parseFilePath(child));
|
||||
} catch {
|
||||
// Legacy enumeration excludes these names. Their raw coverage must
|
||||
// still participate when the owner later selects an NFC policy.
|
||||
}
|
||||
}
|
||||
return directories;
|
||||
}
|
||||
|
||||
/**
|
||||
* Locate physical scopes for an invalidation using retained directory entries only.
|
||||
*
|
||||
* A missing or uncovered component selects its nearest known ancestor, whose
|
||||
* next read can discover new subtrees. NFC mutations may explicitly override
|
||||
* case handling, so invalidate every possible case-equivalent branch. Legacy
|
||||
* scopes use the effective host case policy, including insensitive storage. This is an invalidation hint,
|
||||
* not path resolution or permission to perform filesystem I/O.
|
||||
*/
|
||||
export function findInvalidatedDirectoryScopes(
|
||||
directories: ReadonlyMap<string, readonly DirectoryEntry[]>,
|
||||
requested: string,
|
||||
policy: PathComparisonSettings
|
||||
): Set<string> {
|
||||
const scopes = new Set<string>();
|
||||
if (isHiddenPath(requested)) {
|
||||
return scopes;
|
||||
}
|
||||
const comparisonPolicy = {
|
||||
...policy,
|
||||
handleFilenameCaseSensitive: policy.pathNormalizationVersion === 0 && policy.handleFilenameCaseSensitive,
|
||||
};
|
||||
let branches = new Set([""]);
|
||||
for (const segment of requested === "" ? [] : requested.split("/")) {
|
||||
const nextBranches = new Set<string>();
|
||||
for (const parent of branches) {
|
||||
const entries = directories.get(parent) ?? [];
|
||||
const key = getStoragePathComparisonKey(parseFilePath(joinRelative(parent, segment)), comparisonPolicy);
|
||||
const matches = getSupportedChildDirectories(parent, entries).filter(
|
||||
(child) => getStoragePathComparisonKey(child, comparisonPolicy) === key
|
||||
);
|
||||
if (matches.length === 0) {
|
||||
scopes.add(parent);
|
||||
} else {
|
||||
for (const child of matches) {
|
||||
nextBranches.add(child);
|
||||
}
|
||||
}
|
||||
}
|
||||
branches = nextBranches;
|
||||
}
|
||||
for (const directory of branches) {
|
||||
scopes.add(directory);
|
||||
}
|
||||
return scopes;
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { DirectoryEntry } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { findInvalidatedDirectoryScopes } from "./CLIFileNameSource.paths";
|
||||
|
||||
describe("physical directory scopes for invalidation", () => {
|
||||
it.each([true, false])(
|
||||
"retains every Legacy parent admitted by the effective case policy (sensitive: %s)",
|
||||
(sensitive) => {
|
||||
const directories = new Map<string, readonly DirectoryEntry[]>([
|
||||
[
|
||||
"",
|
||||
[
|
||||
{ name: "Docs", kind: "directory" },
|
||||
{ name: "docs", kind: "directory" },
|
||||
{ name: "other", kind: "directory" },
|
||||
],
|
||||
],
|
||||
["Docs", []],
|
||||
["docs", []],
|
||||
["other", []],
|
||||
]);
|
||||
const scopes = findInvalidatedDirectoryScopes(directories, "docs", {
|
||||
pathNormalizationVersion: 0,
|
||||
handleFilenameCaseSensitive: sensitive,
|
||||
});
|
||||
expect([...scopes].sort()).toEqual(sensitive ? ["docs"] : ["Docs", "docs"]);
|
||||
}
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,540 @@
|
||||
import {
|
||||
getStoragePathComparisonKey,
|
||||
parseFilePath,
|
||||
type DirectoryEntry,
|
||||
type FilePath,
|
||||
type PathComparisonSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { isHiddenPath } from "@/apps/cli/cliPathFilter";
|
||||
import { findInvalidatedDirectoryScopes, getSupportedChildDirectories } from "./CLIFileNameSource.paths";
|
||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||
import {
|
||||
asError,
|
||||
depth,
|
||||
joinRelative,
|
||||
getRawDirectoryEntries,
|
||||
type DirectoryListing,
|
||||
type InventoryReader,
|
||||
type InventoryReconciler,
|
||||
type InventoryState,
|
||||
type RefreshChanges,
|
||||
} from "./CLIFileNameSource.state";
|
||||
|
||||
/**
|
||||
* Full owner ports for target validation and factory composition. Target reads
|
||||
* also commit coverage under the source-selected policy, using every port.
|
||||
*/
|
||||
export type InventoryReconcilerDependencies = {
|
||||
readonly reader: Pick<InventoryReader, "readDirectory">;
|
||||
readonly basePath: string;
|
||||
readonly reportDiagnostic: CliDiagnosticReporter;
|
||||
/** Source-owned pending scopes, keyed by their first unhandled observation sequence. */
|
||||
readonly invalidatedDirectories: Map<string, number>;
|
||||
/** Latest source observation sequence per scope, used to reject superseded reads. */
|
||||
readonly directoryGenerations: Map<string, number>;
|
||||
readonly getBoundary: () => number;
|
||||
readonly markDirectoryDirty: (directory: string) => void;
|
||||
readonly assertNotClosed: () => void;
|
||||
readonly assertInventoryAvailable: (state: InventoryState, policy: PathComparisonSettings) => void;
|
||||
readonly onValidatedDirectories: (policy: PathComparisonSettings, directories: ReadonlySet<string>) => void;
|
||||
};
|
||||
|
||||
/** Ports used while reading and staging one directory; publication is separate. */
|
||||
export type RefreshDirectoryDependencies = Pick<
|
||||
InventoryReconcilerDependencies,
|
||||
"basePath" | "reportDiagnostic" | "directoryGenerations" | "markDirectoryDirty" | "getBoundary"
|
||||
> & {
|
||||
readonly reader: Pick<InventoryReader, "readDirectory">;
|
||||
};
|
||||
|
||||
/** Mutable owner bookkeeping touched only when a validated transition commits. */
|
||||
export type CommitRefreshDependencies = Pick<
|
||||
InventoryReconcilerDependencies,
|
||||
"assertNotClosed" | "onValidatedDirectories" | "invalidatedDirectories" | "directoryGenerations"
|
||||
>;
|
||||
|
||||
/** Captured-scope reconciliation reads and conditionally commits the same transition. */
|
||||
export type ReconcilePendingDependencies = RefreshDirectoryDependencies & CommitRefreshDependencies;
|
||||
|
||||
/** Invalid Legacy leaf names need diagnostics; membership itself is explicit input. */
|
||||
export type StageFileMembershipChangesDependencies = Pick<InventoryReconcilerDependencies, "reportDiagnostic">;
|
||||
|
||||
/** Create an unpublished transition which can be discarded after a superseded read. */
|
||||
export function createRefreshChanges(): RefreshChanges {
|
||||
return {
|
||||
directoryUpdates: new Map<string, readonly DirectoryEntry[]>(),
|
||||
removedDirectories: new Set<string>(),
|
||||
removedFiles: new Set<FilePath>(),
|
||||
addedFiles: new Set<FilePath>(),
|
||||
readErrors: new Map<string, Error>(),
|
||||
aliasErrors: new Map<string, Error>(),
|
||||
clearedErrors: new Set<string>(),
|
||||
checkedGenerations: new Map<string, number>(),
|
||||
};
|
||||
}
|
||||
|
||||
/** Read this transition's staged view first, then the last published listing. */
|
||||
export function currentEntries(
|
||||
state: InventoryState,
|
||||
directory: string,
|
||||
changes: RefreshChanges
|
||||
): readonly DirectoryEntry[] {
|
||||
if (changes.removedDirectories.has(directory)) {
|
||||
return [];
|
||||
}
|
||||
return changes.directoryUpdates.get(directory) ?? state.directories.get(directory) ?? [];
|
||||
}
|
||||
|
||||
/** A staged removal overrides older coverage when deciding whether a child needs population. */
|
||||
export function hasDirectoryCoverage(state: InventoryState, directory: string, changes: RefreshChanges): boolean {
|
||||
return (
|
||||
!changes.removedDirectories.has(directory) &&
|
||||
(changes.directoryUpdates.has(directory) || state.directories.has(directory))
|
||||
);
|
||||
}
|
||||
|
||||
/** Retire cached descendants without I/O; publication still waits for generation checks. */
|
||||
export function stageRemoveSubtree(
|
||||
state: InventoryState,
|
||||
directory: string,
|
||||
changes: RefreshChanges,
|
||||
visited: Set<string>
|
||||
): void {
|
||||
if (visited.has(directory)) {
|
||||
return;
|
||||
}
|
||||
visited.add(directory);
|
||||
const entries = currentEntries(state, directory, changes);
|
||||
for (const entry of entries) {
|
||||
const child = joinRelative(directory, entry.name);
|
||||
if (entry.kind === "file") {
|
||||
try {
|
||||
changes.removedFiles.add(parseFilePath(child));
|
||||
} catch {
|
||||
// Invalid legacy names were never indexed.
|
||||
}
|
||||
} else if (!entry.name.startsWith(".")) {
|
||||
stageRemoveSubtree(state, child, changes, visited);
|
||||
}
|
||||
}
|
||||
changes.directoryUpdates.delete(directory);
|
||||
changes.removedDirectories.add(directory);
|
||||
changes.clearedErrors.add(directory);
|
||||
changes.readErrors.delete(directory);
|
||||
changes.aliasErrors.delete(directory);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fence retired descendants against observations newer than the parent read.
|
||||
*
|
||||
* Generations are latest observation sequences. Only removed subtrees need this
|
||||
* check; unchanged descendants require no traversal or generation snapshot.
|
||||
*/
|
||||
export function fenceRemovedDirectories(
|
||||
directoryGenerations: ReadonlyMap<string, number>,
|
||||
changes: RefreshChanges,
|
||||
readBoundary: number
|
||||
): void {
|
||||
for (const directory of changes.removedDirectories) {
|
||||
if (changes.checkedGenerations.has(directory)) {
|
||||
continue;
|
||||
}
|
||||
const generation = directoryGenerations.get(directory) ?? 0;
|
||||
// A newer descendant cannot be retired using its parent's older listing.
|
||||
// The existing publication check rejects this sentinel and retains work.
|
||||
changes.checkedGenerations.set(directory, generation > readBoundary ? -1 : generation);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Publish a transition after its caller has checked the captured generations.
|
||||
*
|
||||
* The owner must still be open. Only invalidations within this read boundary can
|
||||
* clear; newer notifications retain their generation and pending work.
|
||||
*/
|
||||
export function commitRefresh(
|
||||
dependencies: CommitRefreshDependencies,
|
||||
state: InventoryState,
|
||||
changes: RefreshChanges,
|
||||
boundary: number
|
||||
): void {
|
||||
dependencies.assertNotClosed();
|
||||
for (const directory of changes.removedDirectories) {
|
||||
state.directories.delete(directory);
|
||||
state.readErrors.delete(directory);
|
||||
state.aliasErrors.delete(directory);
|
||||
}
|
||||
for (const [directory, entries] of changes.directoryUpdates) {
|
||||
state.directories.set(directory, entries);
|
||||
}
|
||||
for (const directory of changes.clearedErrors) {
|
||||
state.readErrors.delete(directory);
|
||||
state.aliasErrors.delete(directory);
|
||||
}
|
||||
// Clear owner-held failure evidence only after the validated scopes
|
||||
// have replaced old errors, and before this transition adds new errors.
|
||||
dependencies.onValidatedDirectories(state.policy, changes.clearedErrors);
|
||||
for (const [directory, error] of changes.readErrors) {
|
||||
state.readErrors.set(directory, error);
|
||||
state.aliasErrors.delete(directory);
|
||||
}
|
||||
for (const [directory, error] of changes.aliasErrors) {
|
||||
state.aliasErrors.set(directory, error);
|
||||
}
|
||||
|
||||
// A path removed and recreated within this transition keeps its identity
|
||||
// membership. Only actual set changes need to update the shared buckets.
|
||||
const removed = Array.from(changes.removedFiles).filter(
|
||||
(file) => !changes.addedFiles.has(file) && state.files.has(file)
|
||||
);
|
||||
const added = Array.from(changes.addedFiles).filter(
|
||||
(file) => !changes.removedFiles.has(file) && !state.files.has(file)
|
||||
);
|
||||
for (const file of removed) {
|
||||
state.files.delete(file);
|
||||
}
|
||||
for (const file of added) {
|
||||
state.files.add(file);
|
||||
}
|
||||
if (removed.length > 0 || added.length > 0) {
|
||||
state.index.apply({ removed, added });
|
||||
}
|
||||
|
||||
// Acknowledge only observations covered by this read and its caller's
|
||||
// boundary. This acknowledgement fence preserves later notifications even
|
||||
// after the caller has checked the transition's publication generations.
|
||||
for (const [directory, generation] of changes.checkedGenerations) {
|
||||
if ((dependencies.directoryGenerations.get(directory) ?? 0) !== generation) {
|
||||
continue;
|
||||
}
|
||||
const firstSequence = dependencies.invalidatedDirectories.get(directory);
|
||||
if (firstSequence !== undefined && firstSequence <= boundary) {
|
||||
dependencies.invalidatedDirectories.delete(directory);
|
||||
}
|
||||
if (changes.removedDirectories.has(directory) && (firstSequence === undefined || firstSequence <= boundary)) {
|
||||
dependencies.directoryGenerations.delete(directory);
|
||||
}
|
||||
}
|
||||
// Removing a parent also retires cached descendant generations. Keep a
|
||||
// descendant's newer invalidation if it arrived after the captured boundary.
|
||||
for (const directory of changes.removedDirectories) {
|
||||
const checkedGeneration = changes.checkedGenerations.get(directory);
|
||||
if (
|
||||
checkedGeneration !== undefined &&
|
||||
(dependencies.directoryGenerations.get(directory) ?? 0) !== checkedGeneration
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const firstSequence = dependencies.invalidatedDirectories.get(directory);
|
||||
if (firstSequence === undefined || firstSequence <= boundary) {
|
||||
dependencies.invalidatedDirectories.delete(directory);
|
||||
dependencies.directoryGenerations.delete(directory);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Compare captured reads with live counters; an unobserved scope starts at generation zero. */
|
||||
export function generationsUnchanged(
|
||||
directoryGenerations: ReadonlyMap<string, number>,
|
||||
generations: ReadonlyMap<string, number>
|
||||
): boolean {
|
||||
for (const [directory, generation] of generations) {
|
||||
if ((directoryGenerations.get(directory) ?? 0) !== generation) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stage exact-name deltas, retaining validated paths and prior Legacy exclusions.
|
||||
*
|
||||
* Only newly added raw Legacy names need parsing and diagnostics. Old invalid
|
||||
* names may remain in coverage after a policy change but were never indexed.
|
||||
*/
|
||||
export function stageFileMembershipChanges(
|
||||
dependencies: StageFileMembershipChangesDependencies,
|
||||
directory: string,
|
||||
previous: readonly DirectoryEntry[],
|
||||
listing: DirectoryListing,
|
||||
changes: RefreshChanges
|
||||
): void {
|
||||
const previousFiles = new Set(
|
||||
previous.filter((entry) => entry.kind === "file").map((entry) => joinRelative(directory, entry.name))
|
||||
);
|
||||
const currentFiles = new Set<string>(
|
||||
listing.kind === "validated"
|
||||
? listing.entries.filter((entry) => entry.kind === "file").map((entry) => entry.path)
|
||||
: listing.entries
|
||||
.filter((entry) => entry.kind === "file")
|
||||
.map((entry) => joinRelative(directory, entry.name))
|
||||
);
|
||||
for (const file of previousFiles) {
|
||||
if (!currentFiles.has(file)) {
|
||||
try {
|
||||
changes.removedFiles.add(parseFilePath(file));
|
||||
} catch {
|
||||
// Prior Legacy coverage can contain invalid names which were never indexed.
|
||||
}
|
||||
}
|
||||
}
|
||||
if (listing.kind === "validated") {
|
||||
// NFC resolution already checked these paths. Keep their original brand
|
||||
// instead of rebuilding and parsing strings from cached leaf names.
|
||||
for (const entry of listing.entries) {
|
||||
if (entry.kind === "file" && !previousFiles.has(entry.path)) {
|
||||
changes.addedFiles.add(entry.path);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
for (const file of currentFiles) {
|
||||
if (!previousFiles.has(file)) {
|
||||
try {
|
||||
changes.addedFiles.add(parseFilePath(file));
|
||||
} catch (error) {
|
||||
dependencies.reportDiagnostic("[CLIFileNameSource] Skipped invalid path '" + file + "':", error);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stage one scope: validate a listing, diff its files, then reconcile its children.
|
||||
*
|
||||
* Failed reads retain previous membership and stage an error. Missing descendants
|
||||
* retire cached subtrees; a missing root is unavailable coverage. Target-only
|
||||
* queries defer newly discovered subtrees, while full reconciliation expands them.
|
||||
* The caller publishes this transition only if all captured generations still match.
|
||||
*/
|
||||
export async function refreshDirectory(
|
||||
dependencies: RefreshDirectoryDependencies,
|
||||
state: InventoryState,
|
||||
directory: string,
|
||||
changes: RefreshChanges,
|
||||
expandNewSubtrees: boolean,
|
||||
visiting: Set<string>
|
||||
): Promise<void> {
|
||||
// Avoid re-entering a scope already on the recursion stack. Capture its
|
||||
// generation before I/O so a later observation can reject stale work.
|
||||
if (visiting.has(directory)) {
|
||||
return;
|
||||
}
|
||||
visiting.add(directory);
|
||||
const readBoundary = dependencies.getBoundary();
|
||||
changes.checkedGenerations.set(directory, dependencies.directoryGenerations.get(directory) ?? 0);
|
||||
// Direct dirty scopes obey the same listing boundary as recursive scans.
|
||||
// Retire any old coverage without reading an excluded hidden subtree.
|
||||
if (isHiddenPath(directory)) {
|
||||
stageRemoveSubtree(state, directory, changes, new Set<string>());
|
||||
fenceRemovedDirectories(dependencies.directoryGenerations, changes, readBoundary);
|
||||
visiting.delete(directory);
|
||||
return;
|
||||
}
|
||||
|
||||
// Validate current immediate children without changing the published view.
|
||||
let listing: DirectoryListing | null;
|
||||
try {
|
||||
listing = await dependencies.reader.readDirectory(directory, state.policy);
|
||||
} catch (error) {
|
||||
const failure = asError(error);
|
||||
// Identity ambiguity and unreadable coverage are retained separately:
|
||||
// policy changes can re-evaluate aliases without erasing failed I/O.
|
||||
if (failure.name === "AmbiguousNormalisedPathError") {
|
||||
changes.aliasErrors.set(directory, failure);
|
||||
} else {
|
||||
changes.readErrors.set(directory, failure);
|
||||
}
|
||||
// Retain failed coverage for the owner to reject when required by the
|
||||
// selected policy. Initial full enumeration has its own strict reader.
|
||||
dependencies.reportDiagnostic("[CLIFileNameSource] Could not validate directory '" + directory + "':", failure);
|
||||
visiting.delete(directory);
|
||||
return;
|
||||
}
|
||||
|
||||
if (listing === null) {
|
||||
// Disappearance is a removal only below the root. Losing the root
|
||||
// cannot turn a formerly populated Vault into a validated empty one.
|
||||
if (directory === "") {
|
||||
const failure = new Error("The CLI file-name source root is unavailable: " + dependencies.basePath);
|
||||
changes.readErrors.set(directory, failure);
|
||||
visiting.delete(directory);
|
||||
return;
|
||||
}
|
||||
stageRemoveSubtree(state, directory, changes, new Set<string>());
|
||||
changes.clearedErrors.add(directory);
|
||||
fenceRemovedDirectories(dependencies.directoryGenerations, changes, readBoundary);
|
||||
visiting.delete(directory);
|
||||
return;
|
||||
}
|
||||
|
||||
// This scope validated successfully; stage its listing and clear only
|
||||
// its old failures. File membership changes preserve original spellings.
|
||||
const entries = getRawDirectoryEntries(directory, listing);
|
||||
const previous = currentEntries(state, directory, changes);
|
||||
changes.directoryUpdates.set(directory, entries);
|
||||
changes.clearedErrors.add(directory);
|
||||
changes.aliasErrors.delete(directory);
|
||||
changes.readErrors.delete(directory);
|
||||
|
||||
stageFileMembershipChanges(dependencies, directory, previous, listing, changes);
|
||||
|
||||
// Retire vanished children before considering new or uncovered subtrees.
|
||||
const previousDirectories = new Set(getSupportedChildDirectories(directory, previous));
|
||||
const currentDirectories = new Set(getSupportedChildDirectories(directory, entries));
|
||||
for (const child of previousDirectories) {
|
||||
if (!currentDirectories.has(child)) {
|
||||
stageRemoveSubtree(state, child, changes, new Set<string>());
|
||||
}
|
||||
}
|
||||
for (const child of currentDirectories) {
|
||||
// Existing covered children are handled by their own invalidations.
|
||||
// A new/uncovered child needs population, but target-only queries leave
|
||||
// that recursive work pending instead of scanning unrelated descendants.
|
||||
if (!previousDirectories.has(child) || !hasDirectoryCoverage(state, child, changes)) {
|
||||
if (expandNewSubtrees) {
|
||||
await refreshDirectory(dependencies, state, child, changes, true, visiting);
|
||||
} else {
|
||||
dependencies.markDirectoryDirty(child);
|
||||
}
|
||||
}
|
||||
}
|
||||
fenceRemovedDirectories(dependencies.directoryGenerations, changes, readBoundary);
|
||||
visiting.delete(directory);
|
||||
}
|
||||
|
||||
/** Reconcile the captured scopes once, retaining those superseded during their reads. */
|
||||
export async function reconcilePending(
|
||||
dependencies: ReconcilePendingDependencies,
|
||||
state: InventoryState,
|
||||
boundary: number,
|
||||
expandNewSubtrees = true
|
||||
): Promise<Set<string>> {
|
||||
const unresolved = new Set<string>();
|
||||
// Parents can remove or repopulate descendants, so process shallower
|
||||
// scopes first and skip descendants already settled by those transitions.
|
||||
const scopes = Array.from(dependencies.invalidatedDirectories.entries())
|
||||
.filter(([, firstSequence]) => firstSequence <= boundary)
|
||||
.map(([directory]) => directory)
|
||||
.sort((left, right) => depth(left) - depth(right));
|
||||
for (const directory of scopes) {
|
||||
if (!dependencies.invalidatedDirectories.has(directory)) {
|
||||
continue;
|
||||
}
|
||||
const changes = createRefreshChanges();
|
||||
const readBoundary = dependencies.getBoundary();
|
||||
// Startup mutations can arrive before physical coverage is published.
|
||||
// Resolve their requested scopes against this candidate too, retaining
|
||||
// the original generation until every corresponding physical read commits.
|
||||
changes.checkedGenerations.set(directory, dependencies.directoryGenerations.get(directory) ?? 0);
|
||||
const physicalScopes = findInvalidatedDirectoryScopes(state.directories, directory, state.policy);
|
||||
for (const physicalDirectory of physicalScopes) {
|
||||
await refreshDirectory(
|
||||
dependencies,
|
||||
state,
|
||||
physicalDirectory,
|
||||
changes,
|
||||
expandNewSubtrees,
|
||||
new Set<string>()
|
||||
);
|
||||
}
|
||||
if (!physicalScopes.has(directory)) {
|
||||
// A logical alias or excluded scope owns no physical coverage. Its
|
||||
// pending generation must not accumulate after acknowledgement.
|
||||
stageRemoveSubtree(state, directory, changes, new Set<string>());
|
||||
}
|
||||
fenceRemovedDirectories(dependencies.directoryGenerations, changes, readBoundary);
|
||||
// An observed change during any staged read invalidates the whole
|
||||
// transition; retain its scopes for a later query or requested pass.
|
||||
if (!generationsUnchanged(dependencies.directoryGenerations, changes.checkedGenerations)) {
|
||||
unresolved.add(directory);
|
||||
continue;
|
||||
}
|
||||
commitRefresh(dependencies, state, changes, boundary);
|
||||
}
|
||||
return unresolved;
|
||||
}
|
||||
|
||||
/** Validate each equivalent physical parent without recursively expanding unrelated new subtrees. */
|
||||
export async function revalidateTargetBranches(
|
||||
dependencies: InventoryReconcilerDependencies,
|
||||
state: InventoryState,
|
||||
target: FilePath
|
||||
): Promise<Map<string, number>> {
|
||||
// The source selected this policy before entering its serialised operation.
|
||||
const policy = state.policy;
|
||||
const segments = String(target).split("/");
|
||||
const parentSegments = segments.slice(0, -1);
|
||||
let branches = [""];
|
||||
const generations = new Map<string, number>();
|
||||
|
||||
// Follow every supported physical branch matching each logical parent segment.
|
||||
// Hidden subtrees stay outside the enumerator's scope; a query must not admit
|
||||
// their descendants into the shared inventory. Keep all supported case variants.
|
||||
for (const segment of parentSegments) {
|
||||
const nextBranches = new Set<string>();
|
||||
for (const parent of branches) {
|
||||
const generation = await refreshSingleDirectory(dependencies, state, parent);
|
||||
generations.set(parent, generation);
|
||||
const entries = state.directories.get(parent) ?? [];
|
||||
const key = getStoragePathComparisonKey(parseFilePath(joinRelative(parent, segment)), policy);
|
||||
for (const child of getSupportedChildDirectories(parent, entries)) {
|
||||
if (getStoragePathComparisonKey(child, policy) === key) {
|
||||
nextBranches.add(child);
|
||||
}
|
||||
}
|
||||
}
|
||||
branches = Array.from(nextBranches);
|
||||
if (branches.length === 0) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
for (const parent of branches) {
|
||||
const generation = await refreshSingleDirectory(dependencies, state, parent);
|
||||
generations.set(parent, generation);
|
||||
}
|
||||
dependencies.assertInventoryAvailable(state, policy);
|
||||
return generations;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and publish one branch, recording a direct failure before rejecting it.
|
||||
* The owner has selected state.policy before entering this operation. A
|
||||
* superseded read returns a non-matching generation so the target query retries.
|
||||
*/
|
||||
export async function refreshSingleDirectory(
|
||||
dependencies: InventoryReconcilerDependencies,
|
||||
state: InventoryState,
|
||||
directory: string
|
||||
): Promise<number> {
|
||||
const changes = createRefreshChanges();
|
||||
await refreshDirectory(dependencies, state, directory, changes, false, new Set<string>());
|
||||
if (!generationsUnchanged(dependencies.directoryGenerations, changes.checkedGenerations)) {
|
||||
// Generations are non-negative. This sentinel makes the caller retry
|
||||
// rather than accept a branch whose validation was superseded.
|
||||
return -1;
|
||||
}
|
||||
const directError = changes.aliasErrors.get(directory) ?? changes.readErrors.get(directory);
|
||||
commitRefresh(dependencies, state, changes, dependencies.getBoundary());
|
||||
if (directError) {
|
||||
throw directError;
|
||||
}
|
||||
dependencies.assertInventoryAvailable(state, state.policy);
|
||||
return dependencies.directoryGenerations.get(directory) ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Bind the owner ports to the standalone operations without hiding their logic.
|
||||
*
|
||||
* Inventory state and staged changes remain explicit inputs. No mutable state is
|
||||
* created here, so direct operation tests can supply only the ports they exercise.
|
||||
*/
|
||||
export function createInventoryReconciler(dependencies: InventoryReconcilerDependencies): InventoryReconciler {
|
||||
return {
|
||||
reconcilePending: (state, boundary, expandNewSubtrees) =>
|
||||
reconcilePending(dependencies, state, boundary, expandNewSubtrees),
|
||||
revalidateTargetBranches: (state, target) => revalidateTargetBranches(dependencies, state, target),
|
||||
generationsUnchanged: (generations) => generationsUnchanged(dependencies.directoryGenerations, generations),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,527 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
FilePathIndex,
|
||||
parseFilePath,
|
||||
type DirectoryEntry,
|
||||
type PathComparisonSettings,
|
||||
type StoragePath,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { DirectoryListing, InventoryReader, InventoryState } from "./CLIFileNameSource.state";
|
||||
import { readDirectory as readInventoryDirectory } from "./CLIFileNameSource.inventory";
|
||||
import {
|
||||
commitRefresh,
|
||||
createRefreshChanges,
|
||||
currentEntries,
|
||||
stageRemoveSubtree,
|
||||
stageFileMembershipChanges,
|
||||
generationsUnchanged,
|
||||
reconcilePending,
|
||||
refreshDirectory,
|
||||
refreshSingleDirectory,
|
||||
revalidateTargetBranches,
|
||||
type CommitRefreshDependencies,
|
||||
type ReconcilePendingDependencies,
|
||||
type RefreshDirectoryDependencies,
|
||||
type InventoryReconcilerDependencies,
|
||||
} from "./CLIFileNameSource.reconciliation";
|
||||
|
||||
const legacyPolicy: PathComparisonSettings = {
|
||||
pathNormalizationVersion: 0,
|
||||
handleFilenameCaseSensitive: true,
|
||||
};
|
||||
const file = (name: string): DirectoryEntry => ({ name, kind: "file" });
|
||||
const directory = (name: string): DirectoryEntry => ({ name, kind: "directory" });
|
||||
|
||||
const legacyListing = (entries: readonly DirectoryEntry[]): DirectoryListing => ({ kind: "legacy", entries });
|
||||
|
||||
/** Produce validated fixture paths through the same shared resolver as the real reader. */
|
||||
function readerForListings(listings: ReadonlyMap<string, readonly DirectoryEntry[]>) {
|
||||
const storage = {
|
||||
readDirectory: (directory: StoragePath) => Promise.resolve(listings.get(directory) ?? null),
|
||||
};
|
||||
return vi.fn<InventoryReader["readDirectory"]>((directory, policy) =>
|
||||
readInventoryDirectory({ storage }, directory, policy)
|
||||
);
|
||||
}
|
||||
|
||||
function inventory(
|
||||
names: readonly string[],
|
||||
listings: readonly (readonly [string, readonly DirectoryEntry[]])[],
|
||||
policy = legacyPolicy
|
||||
): InventoryState {
|
||||
const files = new Set(names.map((name) => parseFilePath(name)));
|
||||
const index = new FilePathIndex();
|
||||
index.replaceAll(files);
|
||||
return {
|
||||
index,
|
||||
files,
|
||||
directories: new Map(listings),
|
||||
readErrors: new Map(),
|
||||
aliasErrors: new Map(),
|
||||
policy: { ...policy },
|
||||
};
|
||||
}
|
||||
|
||||
describe("CLI name reconciliation operations", () => {
|
||||
it("stages a target-only refresh without publishing or recursively reading new subtrees", async () => {
|
||||
const original = [file("old.md"), file("keep.md")];
|
||||
const state = inventory(["old.md", "keep.md"], [["", original]]);
|
||||
const entries = [file("keep.md"), file("new.md"), file(".hidden.md"), directory("later"), directory(".config")];
|
||||
const readDirectory = vi.fn<InventoryReader["readDirectory"]>().mockResolvedValue(legacyListing(entries));
|
||||
const markDirectoryDirty = vi.fn();
|
||||
const directoryGenerations = new Map([["", 4]]);
|
||||
const dependencies: RefreshDirectoryDependencies = {
|
||||
reader: { readDirectory },
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
directoryGenerations,
|
||||
getBoundary: () => Math.max(0, ...directoryGenerations.values()),
|
||||
markDirectoryDirty,
|
||||
};
|
||||
const changes = createRefreshChanges();
|
||||
|
||||
await refreshDirectory(dependencies, state, "", changes, false, new Set());
|
||||
|
||||
expect(state.index.getFileNames().sort()).toEqual(["keep.md", "old.md"]);
|
||||
expect(state.directories.get("")).toBe(original);
|
||||
expect(readDirectory).toHaveBeenCalledExactlyOnceWith("", legacyPolicy);
|
||||
expect(markDirectoryDirty).toHaveBeenCalledExactlyOnceWith("later");
|
||||
|
||||
const invalidatedDirectories = new Map([["", 1]]);
|
||||
const commitDependencies: CommitRefreshDependencies = {
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories: vi.fn(),
|
||||
invalidatedDirectories,
|
||||
directoryGenerations,
|
||||
};
|
||||
commitRefresh(commitDependencies, state, changes, 1);
|
||||
|
||||
expect(state.index.getFileNames().sort()).toEqual([".hidden.md", "keep.md", "new.md"]);
|
||||
expect([...state.files].sort()).toEqual([".hidden.md", "keep.md", "new.md"]);
|
||||
expect(state.directories.get("")).toBe(entries);
|
||||
expect(invalidatedDirectories.has("")).toBe(false);
|
||||
});
|
||||
|
||||
it("retires an absent subtree while preserving newer descendant invalidations", async () => {
|
||||
const state = inventory(
|
||||
["gone/one.md", "gone/nested/two.md", "keep.md"],
|
||||
[
|
||||
["", [directory("gone"), file("keep.md")]],
|
||||
["gone", [file("one.md"), directory("nested")]],
|
||||
["gone/nested", [file("two.md")]],
|
||||
]
|
||||
);
|
||||
const unrelatedFailure = new Error("unrelated scope remains unreadable");
|
||||
state.readErrors.set("gone/nested", new Error("obsolete descendant failure"));
|
||||
state.readErrors.set("other", unrelatedFailure);
|
||||
const directoryGenerations = new Map([
|
||||
["gone", 2],
|
||||
["gone/nested", 3],
|
||||
]);
|
||||
const readDependencies: RefreshDirectoryDependencies = {
|
||||
reader: { readDirectory: vi.fn<InventoryReader["readDirectory"]>().mockResolvedValue(null) },
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
directoryGenerations,
|
||||
getBoundary: () => Math.max(0, ...directoryGenerations.values()),
|
||||
markDirectoryDirty: vi.fn(),
|
||||
};
|
||||
const changes = createRefreshChanges();
|
||||
await refreshDirectory(readDependencies, state, "gone", changes, true, new Set());
|
||||
expect(state.index.getFileNames()).toContain("gone/nested/two.md");
|
||||
|
||||
const invalidatedDirectories = new Map([
|
||||
["gone", 5],
|
||||
["gone/nested", 9],
|
||||
]);
|
||||
commitRefresh(
|
||||
{ assertNotClosed: vi.fn(), onValidatedDirectories: vi.fn(), invalidatedDirectories, directoryGenerations },
|
||||
state,
|
||||
changes,
|
||||
5
|
||||
);
|
||||
|
||||
expect(state.index.getFileNames()).toEqual(["keep.md"]);
|
||||
expect(state.directories.has("gone")).toBe(false);
|
||||
expect(state.directories.has("gone/nested")).toBe(false);
|
||||
expect(state.readErrors.has("gone/nested")).toBe(false);
|
||||
expect(state.readErrors.get("other")).toBe(unrelatedFailure);
|
||||
expect(invalidatedDirectories.get("gone/nested")).toBe(9);
|
||||
expect(directoryGenerations.get("gone/nested")).toBe(3);
|
||||
expect(directoryGenerations.has("gone")).toBe(false);
|
||||
});
|
||||
|
||||
it("stages validated physical spellings while retiring prior Legacy exclusions", async () => {
|
||||
const originalName = "Cafe\u0301.md";
|
||||
const readDirectory = readerForListings(new Map([["", [file(originalName), file(".hidden.md")]]]));
|
||||
const policy: PathComparisonSettings = { ...legacyPolicy, pathNormalizationVersion: 1 };
|
||||
const listing = await readDirectory("", policy);
|
||||
expect(listing?.kind).toBe("validated");
|
||||
if (!listing) {
|
||||
throw new Error("Expected a validated fixture listing");
|
||||
}
|
||||
const changes = createRefreshChanges();
|
||||
const reportDiagnostic = vi.fn();
|
||||
|
||||
stageFileMembershipChanges({ reportDiagnostic }, "", [file("old.md"), file("C:excluded.md")], listing, changes);
|
||||
|
||||
expect([...changes.removedFiles]).toEqual(["old.md"]);
|
||||
expect([...changes.addedFiles].sort()).toEqual([".hidden.md", originalName]);
|
||||
expect(reportDiagnostic).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("diagnoses only newly excluded Legacy file names when staging a delta", () => {
|
||||
const changes = createRefreshChanges();
|
||||
const reportDiagnostic = vi.fn();
|
||||
stageFileMembershipChanges(
|
||||
{ reportDiagnostic },
|
||||
"",
|
||||
[file("keep.md"), file("old.md"), file("C:retained.md"), file("C:removed.md")],
|
||||
legacyListing([file("keep.md"), file("new.md"), file("C:retained.md"), file("C:added.md")]),
|
||||
changes
|
||||
);
|
||||
|
||||
expect([...changes.removedFiles]).toEqual(["old.md"]);
|
||||
expect([...changes.addedFiles]).toEqual(["new.md"]);
|
||||
expect(reportDiagnostic).toHaveBeenCalledExactlyOnceWith(
|
||||
"[CLIFileNameSource] Skipped invalid path 'C:added.md':",
|
||||
expect.any(Error)
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps subtree retirement idempotent and excludes hidden child directories", () => {
|
||||
const state = inventory(
|
||||
["gone/one.md", "gone/nested/two.md", "keep.md"],
|
||||
[
|
||||
["", [directory("gone"), file("keep.md")]],
|
||||
["gone", [file("one.md"), directory("nested"), directory(".private")]],
|
||||
["gone/nested", [file("two.md")]],
|
||||
]
|
||||
);
|
||||
const changes = createRefreshChanges();
|
||||
const visited = new Set<string>();
|
||||
stageRemoveSubtree(state, "gone", changes, visited);
|
||||
stageRemoveSubtree(state, "gone", changes, visited);
|
||||
stageRemoveSubtree(state, "gone", changes, new Set());
|
||||
|
||||
expect(currentEntries(state, "gone", changes)).toEqual([]);
|
||||
expect([...changes.removedFiles].sort()).toEqual(["gone/nested/two.md", "gone/one.md"]);
|
||||
expect([...changes.removedDirectories].sort()).toEqual(["gone", "gone/nested"]);
|
||||
expect(state.index.getFileNames().sort()).toEqual(["gone/nested/two.md", "gone/one.md", "keep.md"]);
|
||||
});
|
||||
|
||||
it("keeps the published view and pending scope when an observation supersedes a read", async () => {
|
||||
const state = inventory(["before.md"], [["", [file("before.md")]]]);
|
||||
let finishRead!: (listing: DirectoryListing) => void;
|
||||
const readDirectory = vi.fn<InventoryReader["readDirectory"]>(
|
||||
() =>
|
||||
new Promise((resolve) => {
|
||||
finishRead = resolve;
|
||||
})
|
||||
);
|
||||
const directoryGenerations = new Map([["", 1]]);
|
||||
const invalidatedDirectories = new Map([["", 7]]);
|
||||
const onValidatedDirectories = vi.fn();
|
||||
const dependencies: ReconcilePendingDependencies = {
|
||||
reader: { readDirectory },
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
directoryGenerations,
|
||||
getBoundary: () => Math.max(0, ...directoryGenerations.values()),
|
||||
invalidatedDirectories,
|
||||
markDirectoryDirty: vi.fn(),
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories,
|
||||
};
|
||||
|
||||
const pending = reconcilePending(dependencies, state, 7);
|
||||
directoryGenerations.set("", 2);
|
||||
finishRead(legacyListing([file("after.md")]));
|
||||
|
||||
await expect(pending).resolves.toEqual(new Set([""]));
|
||||
expect(state.index.getFileNames()).toEqual(["before.md"]);
|
||||
expect(state.directories.get("")).toEqual([file("before.md")]);
|
||||
expect(invalidatedDirectories.get("")).toBe(7);
|
||||
expect(onValidatedDirectories).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("clears prior error evidence before notifying the owner and then publishes new errors", () => {
|
||||
const state = inventory([], [["", []]]);
|
||||
state.readErrors.set("", new Error("previous failed read"));
|
||||
const failure = new Error("new failed read");
|
||||
const changes = createRefreshChanges();
|
||||
changes.clearedErrors.add("");
|
||||
changes.readErrors.set("", failure);
|
||||
const onValidatedDirectories = vi.fn((_policy: PathComparisonSettings, scopes: ReadonlySet<string>) => {
|
||||
expect(scopes.has("")).toBe(true);
|
||||
expect(state.readErrors.has("")).toBe(false);
|
||||
});
|
||||
const dependencies: CommitRefreshDependencies = {
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories,
|
||||
invalidatedDirectories: new Map(),
|
||||
directoryGenerations: new Map(),
|
||||
};
|
||||
|
||||
commitRefresh(dependencies, state, changes, 0);
|
||||
|
||||
expect(onValidatedDirectories).toHaveBeenCalledOnce();
|
||||
expect(state.readErrors.get("")).toBe(failure);
|
||||
});
|
||||
|
||||
it("retains an observation whose generation is newer than the staged read", () => {
|
||||
const state = inventory(["keep.md"], [["", [file("keep.md")]]]);
|
||||
const changes = createRefreshChanges();
|
||||
changes.checkedGenerations.set("", 1);
|
||||
const invalidatedDirectories = new Map([["", 5]]);
|
||||
const directoryGenerations = new Map([["", 2]]);
|
||||
|
||||
commitRefresh(
|
||||
{
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories: vi.fn(),
|
||||
invalidatedDirectories,
|
||||
directoryGenerations,
|
||||
},
|
||||
state,
|
||||
changes,
|
||||
5
|
||||
);
|
||||
|
||||
// Its first sequence is captured, but the newer observation remains pending.
|
||||
expect(invalidatedDirectories.get("")).toBe(5);
|
||||
expect(directoryGenerations.get("")).toBe(2);
|
||||
expect(state.index.getFileNames()).toEqual(["keep.md"]);
|
||||
});
|
||||
|
||||
it.each([0, 1] as const)(
|
||||
"retires a parent disappearing during target traversal without treating absence as a read error (version %i)",
|
||||
async (version) => {
|
||||
const policy: PathComparisonSettings = { ...legacyPolicy, pathNormalizationVersion: version };
|
||||
const rootEntries = [directory("docs"), file("keep.md")];
|
||||
const state = inventory(
|
||||
["docs/nested/note.md", "keep.md"],
|
||||
[
|
||||
["", rootEntries],
|
||||
["docs", [directory("nested")]],
|
||||
["docs/nested", [file("note.md")]],
|
||||
],
|
||||
policy
|
||||
);
|
||||
// The root listing precedes deletion; the next physical read sees absence.
|
||||
const readDirectory = readerForListings(new Map([["", rootEntries]]));
|
||||
const directoryGenerations = new Map([
|
||||
["", 1],
|
||||
["docs", 2],
|
||||
["docs/nested", 3],
|
||||
]);
|
||||
const dependencies: InventoryReconcilerDependencies = {
|
||||
reader: {
|
||||
readDirectory,
|
||||
},
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
invalidatedDirectories: new Map(),
|
||||
directoryGenerations,
|
||||
markDirectoryDirty: vi.fn(),
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories: vi.fn(),
|
||||
getBoundary: () => 3,
|
||||
assertInventoryAvailable: vi.fn(),
|
||||
};
|
||||
const target = parseFilePath("docs/nested/note.md");
|
||||
|
||||
const generations = await revalidateTargetBranches(dependencies, state, target);
|
||||
|
||||
expect(readDirectory.mock.calls.map(([path]) => path)).toEqual(["", "docs"]);
|
||||
expect(state.directories.has("docs")).toBe(false);
|
||||
expect(state.directories.has("docs/nested")).toBe(false);
|
||||
expect(state.index.getFileNames()).toEqual(["keep.md"]);
|
||||
expect(state.index.findPathIdentityCandidates(target, policy)).toEqual([]);
|
||||
expect(state.readErrors.size).toBe(0);
|
||||
expect(dependencies.reportDiagnostic).not.toHaveBeenCalled();
|
||||
expect(generationsUnchanged(directoryGenerations, generations)).toBe(true);
|
||||
}
|
||||
);
|
||||
|
||||
it("retains last known membership and the read failure before rejecting a target branch", async () => {
|
||||
const policy: PathComparisonSettings = { pathNormalizationVersion: 1, handleFilenameCaseSensitive: true };
|
||||
const state = inventory(["retained.md"], [["", [file("retained.md")]]], policy);
|
||||
const failure = new Error("directory unreadable");
|
||||
const dependencies: InventoryReconcilerDependencies = {
|
||||
reader: {
|
||||
readDirectory: vi.fn<InventoryReader["readDirectory"]>().mockRejectedValue(failure),
|
||||
},
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
invalidatedDirectories: new Map([["", 1]]),
|
||||
directoryGenerations: new Map([["", 1]]),
|
||||
markDirectoryDirty: vi.fn(),
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories: vi.fn(),
|
||||
getBoundary: () => 1,
|
||||
assertInventoryAvailable: vi.fn(),
|
||||
};
|
||||
|
||||
await expect(refreshSingleDirectory(dependencies, state, "")).rejects.toBe(failure);
|
||||
|
||||
expect(state.index.getFileNames()).toEqual(["retained.md"]);
|
||||
expect(state.readErrors.get("")).toBe(failure);
|
||||
});
|
||||
|
||||
it("validates every equivalent physical parent before returning target generations", async () => {
|
||||
const policy: PathComparisonSettings = { pathNormalizationVersion: 0, handleFilenameCaseSensitive: false };
|
||||
const state = inventory([], [["", []]], policy);
|
||||
const listings = new Map<string, readonly DirectoryEntry[]>([
|
||||
["", [directory("Docs"), directory("docs")]],
|
||||
["Docs", [file("Note.md")]],
|
||||
["docs", [file("note.md")]],
|
||||
]);
|
||||
const readDirectory = readerForListings(listings);
|
||||
const invalidatedDirectories = new Map<string, number>();
|
||||
const directoryGenerations = new Map([["", 1]]);
|
||||
let boundary = 1;
|
||||
const dependencies: InventoryReconcilerDependencies = {
|
||||
reader: {
|
||||
readDirectory,
|
||||
},
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
invalidatedDirectories,
|
||||
directoryGenerations,
|
||||
markDirectoryDirty: (path) => {
|
||||
invalidatedDirectories.set(path, ++boundary);
|
||||
directoryGenerations.set(path, (directoryGenerations.get(path) ?? 0) + 1);
|
||||
},
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories: vi.fn(),
|
||||
getBoundary: () => boundary,
|
||||
assertInventoryAvailable: vi.fn(),
|
||||
};
|
||||
|
||||
const generations = await revalidateTargetBranches(dependencies, state, parseFilePath("docs/note.md"));
|
||||
|
||||
expect(readDirectory.mock.calls.map(([path]) => path)).toEqual(["", "Docs", "docs"]);
|
||||
expect(state.index.findPathIdentityCandidates(parseFilePath("docs/note.md"), policy)).toEqual([
|
||||
"Docs/Note.md",
|
||||
"docs/note.md",
|
||||
]);
|
||||
expect(generationsUnchanged(directoryGenerations, generations)).toBe(true);
|
||||
directoryGenerations.set("Docs", 2);
|
||||
expect(generationsUnchanged(directoryGenerations, generations)).toBe(false);
|
||||
});
|
||||
it.each([
|
||||
[0, ".private/secret.md", [""]],
|
||||
[1, "visible/.private/secret.md", ["", "visible"]],
|
||||
] as const)(
|
||||
"keeps hidden subtrees outside current-path validation (version %i, path %s)",
|
||||
async (version, target, expectedReads) => {
|
||||
const policy: PathComparisonSettings = { ...legacyPolicy, pathNormalizationVersion: version };
|
||||
const state = inventory([], [["", []]], policy);
|
||||
const listings = new Map<string, readonly DirectoryEntry[]>([
|
||||
["", [directory(".private"), directory("visible")]],
|
||||
["visible", [directory(".private")]],
|
||||
[".private", [file("secret.md")]],
|
||||
["visible/.private", [file("secret.md")]],
|
||||
]);
|
||||
const readDirectory = readerForListings(listings);
|
||||
const dependencies: InventoryReconcilerDependencies = {
|
||||
reader: {
|
||||
readDirectory,
|
||||
},
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
invalidatedDirectories: new Map(),
|
||||
directoryGenerations: new Map(),
|
||||
markDirectoryDirty: vi.fn(),
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories: vi.fn(),
|
||||
getBoundary: () => 0,
|
||||
assertInventoryAvailable: vi.fn(),
|
||||
};
|
||||
|
||||
await revalidateTargetBranches(dependencies, state, parseFilePath(target));
|
||||
|
||||
expect(readDirectory.mock.calls.map(([path]) => path)).toEqual(expectedReads);
|
||||
expect(state.index.getFileNames()).toEqual([]);
|
||||
expect(state.index.findPathIdentityCandidates(parseFilePath(target), policy)).toEqual([]);
|
||||
}
|
||||
);
|
||||
it("rejects subtree retirement when a descendant changes during its parent read", async () => {
|
||||
const state = inventory(
|
||||
["gone/nested/old.md"],
|
||||
[
|
||||
["", [directory("gone")]],
|
||||
["gone", [directory("nested")]],
|
||||
["gone/nested", [file("old.md")]],
|
||||
]
|
||||
);
|
||||
let sequence = 2;
|
||||
const directoryGenerations = new Map([
|
||||
["gone", 1],
|
||||
["gone/nested", 2],
|
||||
]);
|
||||
const readDirectory = vi.fn<InventoryReader["readDirectory"]>(async () => {
|
||||
// The parent snapshot misses a child recreated before this read
|
||||
// completes. Its earlier pending sequence is deliberately retained.
|
||||
directoryGenerations.set("gone/nested", ++sequence);
|
||||
return legacyListing([]);
|
||||
});
|
||||
const dependencies = {
|
||||
reader: { readDirectory },
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
directoryGenerations,
|
||||
getBoundary: () => sequence,
|
||||
markDirectoryDirty: vi.fn(),
|
||||
};
|
||||
const changes = createRefreshChanges();
|
||||
await refreshDirectory(dependencies, state, "gone", changes, true, new Set());
|
||||
|
||||
expect(generationsUnchanged(directoryGenerations, changes.checkedGenerations)).toBe(false);
|
||||
expect(state.index.getFileNames()).toEqual(["gone/nested/old.md"]);
|
||||
});
|
||||
it("reconciles a recreated descendant instead of losing its older pending observation", async () => {
|
||||
const state = inventory(
|
||||
["gone/nested/old.md"],
|
||||
[
|
||||
["", [directory("gone")]],
|
||||
["gone", [directory("nested")]],
|
||||
["gone/nested", [file("old.md")]],
|
||||
]
|
||||
);
|
||||
let sequence = 2;
|
||||
const directoryGenerations = new Map([
|
||||
["gone", 1],
|
||||
["gone/nested", 2],
|
||||
]);
|
||||
const invalidatedDirectories = new Map([
|
||||
["gone", 1],
|
||||
["gone/nested", 1],
|
||||
]);
|
||||
const readDirectory = vi.fn<InventoryReader["readDirectory"]>(async (path) => {
|
||||
if (path === "gone") {
|
||||
directoryGenerations.set("gone/nested", ++sequence);
|
||||
return legacyListing([]);
|
||||
}
|
||||
return legacyListing([file("new.md")]);
|
||||
});
|
||||
const dependencies: ReconcilePendingDependencies = {
|
||||
reader: { readDirectory },
|
||||
basePath: "/vault",
|
||||
reportDiagnostic: vi.fn(),
|
||||
directoryGenerations,
|
||||
invalidatedDirectories,
|
||||
getBoundary: () => sequence,
|
||||
markDirectoryDirty: vi.fn(),
|
||||
assertNotClosed: vi.fn(),
|
||||
onValidatedDirectories: vi.fn(),
|
||||
};
|
||||
const unresolved = await reconcilePending(dependencies, state, 2);
|
||||
|
||||
expect(unresolved).toEqual(new Set(["gone"]));
|
||||
expect(invalidatedDirectories.get("gone")).toBe(1);
|
||||
expect(readDirectory.mock.calls.map(([path]) => path)).toEqual(["gone", "gone/nested"]);
|
||||
expect(state.index.getFileNames()).toEqual(["gone/nested/new.md"]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,151 @@
|
||||
import type {
|
||||
FilePathIndex,
|
||||
FilePath,
|
||||
PathComparisonSettings,
|
||||
DirectoryEntry,
|
||||
StorageEntry,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { Stats } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { ChokidarOptions, FSWatcher } from "chokidar";
|
||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||
import type { IgnoreRules } from "@/apps/cli/serviceModules/IgnoreRules";
|
||||
|
||||
/**
|
||||
* One physical listing under the owner's selected policy.
|
||||
*
|
||||
* Legacy entries retain raw names, including diagnosed exclusions needed for
|
||||
* later policy validation. NFC listings retain the resolver's branded physical
|
||||
* paths. The owner still checks generations before trusting the read's freshness.
|
||||
*/
|
||||
export type DirectoryListing =
|
||||
| { readonly kind: "legacy"; readonly entries: readonly DirectoryEntry[] }
|
||||
| { readonly kind: "validated"; readonly entries: readonly StorageEntry[] };
|
||||
|
||||
/**
|
||||
* Private contracts shared by the CLI name-source modules.
|
||||
*
|
||||
* Directory strings use original spellings relative to one storage root; the
|
||||
* empty string denotes that root. Validated entries retain physical names,
|
||||
* while invalidations may include requested scopes awaiting validation. The
|
||||
* source owns readiness and publication order.
|
||||
*/
|
||||
export type InventoryState = {
|
||||
/** Candidate buckets derived from the published original file names. */
|
||||
readonly index: FilePathIndex;
|
||||
/** Published membership used to discard repeated add/remove observations. */
|
||||
readonly files: Set<FilePath>;
|
||||
/** Last accepted immediate-child listing of each covered directory, including the root. */
|
||||
readonly directories: Map<string, readonly DirectoryEntry[]>;
|
||||
/** Failed scope reads retain earlier membership, which NFC queries must not trust. */
|
||||
readonly readErrors: Map<string, Error>;
|
||||
/** Identity-validation failures per directory under the selected policy. */
|
||||
aliasErrors: Map<string, Error>;
|
||||
/** Policy used to validate listings and retained identity failures. */
|
||||
policy: PathComparisonSettings;
|
||||
};
|
||||
|
||||
/** A directory transition staged privately until every observed generation still matches. */
|
||||
export type RefreshChanges = {
|
||||
/** Replacement listings and removed coverage, applied before membership changes. */
|
||||
readonly directoryUpdates: Map<string, readonly DirectoryEntry[]>;
|
||||
readonly removedDirectories: Set<string>;
|
||||
/** A path staged in both sets has unchanged membership when the transition commits. */
|
||||
readonly removedFiles: Set<FilePath>;
|
||||
readonly addedFiles: Set<FilePath>;
|
||||
/** New failures replace old diagnostics only when this transition can be published. */
|
||||
readonly readErrors: Map<string, Error>;
|
||||
readonly aliasErrors: Map<string, Error>;
|
||||
/** Scopes validated or removed by this transition, allowing old failures to clear. */
|
||||
readonly clearedErrors: Set<string>;
|
||||
/** Generations accepted for reads and retired scopes; a mismatch rejects the transition. */
|
||||
readonly checkedGenerations: Map<string, number>;
|
||||
};
|
||||
|
||||
/** A failed fresh scan blocks reuse under the same policy until its scope is validated again. */
|
||||
export type FreshValidationFailure = {
|
||||
readonly policy: PathComparisonSettings;
|
||||
readonly directory: string;
|
||||
readonly error: Error;
|
||||
};
|
||||
|
||||
export type WatchEventType = "add" | "change" | "unlink" | "addDir" | "unlinkDir";
|
||||
|
||||
/** Eligible synchronisation notifications retained in arrival order before activation. */
|
||||
export type QueuedWatchEvent = {
|
||||
readonly type: WatchEventType;
|
||||
readonly path: FilePath;
|
||||
readonly stats?: Stats;
|
||||
};
|
||||
|
||||
/** Host-selected root, sync-only exclusions, diagnostics, and the watcher construction port. */
|
||||
export type CLIFileNameSourceOptions = {
|
||||
readonly basePath: string;
|
||||
/** Filters sync delivery after inventory observation; it does not narrow name coverage. */
|
||||
readonly ignoreRules?: IgnoreRules;
|
||||
readonly reportDiagnostic?: CliDiagnosticReporter;
|
||||
/** Capture operation-time suppression when observing, before startup buffering changes timing. */
|
||||
readonly suppressEvent?: (path: string) => boolean;
|
||||
/** Tests can control readiness and event order without installing another native watcher. */
|
||||
readonly watcherFactory?: (basePath: string, options: ChokidarOptions) => FSWatcher;
|
||||
};
|
||||
|
||||
/** Storage reads and policy checks; callers decide when a candidate inventory can become ready. */
|
||||
export interface InventoryReader {
|
||||
enumerate(policy: PathComparisonSettings, retainFailure?: boolean): Promise<InventoryState>;
|
||||
selectPolicy(state: InventoryState, policy: PathComparisonSettings): Promise<void>;
|
||||
readDirectory(directory: string, policy: PathComparisonSettings): Promise<DirectoryListing | null>;
|
||||
}
|
||||
|
||||
/** Staged reconciliation uses the source's live generation maps but never owns its lifecycle. */
|
||||
export interface InventoryReconciler {
|
||||
/** Reconcile captured scopes; return those invalidated again while being read. */
|
||||
reconcilePending(state: InventoryState, boundary: number, expandNewSubtrees?: boolean): Promise<Set<string>>;
|
||||
/** Follow physical parents under state.policy, already selected by the serialised owner. */
|
||||
revalidateTargetBranches(state: InventoryState, target: FilePath): Promise<Map<string, number>>;
|
||||
generationsUnchanged(generations: ReadonlyMap<string, number>): boolean;
|
||||
}
|
||||
|
||||
/** Retain an existing error's identity and diagnostics when a host callback throws. */
|
||||
export function asError(error: unknown): Error {
|
||||
return error instanceof Error ? error : new Error(String(error));
|
||||
}
|
||||
|
||||
export function joinRelative(directory: string, leaf: string): string {
|
||||
return directory === "" ? leaf : directory + "/" + leaf;
|
||||
}
|
||||
|
||||
export function childLeaf(parent: string, child: string): string {
|
||||
return parent === "" ? child : child.slice(parent.length + 1);
|
||||
}
|
||||
|
||||
/** Derive cache names for future policy checks while the caller retains the typed listing. */
|
||||
export function getRawDirectoryEntries(directory: string, listing: DirectoryListing): readonly DirectoryEntry[] {
|
||||
if (listing.kind === "legacy") {
|
||||
return listing.entries;
|
||||
}
|
||||
return listing.entries.map((entry) => ({ name: childLeaf(directory, entry.path), kind: entry.kind }));
|
||||
}
|
||||
|
||||
export function parentPath(value: string): string {
|
||||
const separator = value.lastIndexOf("/");
|
||||
return separator < 0 ? "" : value.slice(0, separator);
|
||||
}
|
||||
|
||||
export function depth(value: string): number {
|
||||
return value === "" ? 0 : value.split("/").length;
|
||||
}
|
||||
|
||||
/** Compare identity policy values rather than callback/object identity. */
|
||||
export function samePolicy(left: PathComparisonSettings, right: PathComparisonSettings): boolean {
|
||||
return (
|
||||
left.pathNormalizationVersion === right.pathNormalizationVersion &&
|
||||
left.handleFilenameCaseSensitive === right.handleFilenameCaseSensitive
|
||||
);
|
||||
}
|
||||
|
||||
export function isSupportedPolicy(policy: PathComparisonSettings): boolean {
|
||||
return (
|
||||
(policy.pathNormalizationVersion === 0 || policy.pathNormalizationVersion === 1) &&
|
||||
typeof policy.handleFilenameCaseSensitive === "boolean"
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,642 @@
|
||||
import {
|
||||
parseFilePath,
|
||||
type FilePath,
|
||||
type FileNameLookup,
|
||||
type FileNameReadOptions,
|
||||
type PathComparisonSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { path, type UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||
import { findInvalidatedDirectoryScopes } from "./CLIFileNameSource.paths";
|
||||
import { createInventoryReader } from "./CLIFileNameSource.inventory";
|
||||
import { createInventoryReconciler } from "./CLIFileNameSource.reconciliation";
|
||||
import { CLIFileNameSourceWatch } from "./CLIFileNameSource.watch";
|
||||
import {
|
||||
asError,
|
||||
parentPath,
|
||||
samePolicy,
|
||||
type CLIFileNameSourceOptions,
|
||||
type FreshValidationFailure,
|
||||
type InventoryReader,
|
||||
type InventoryReconciler,
|
||||
type InventoryState,
|
||||
type WatchEventType,
|
||||
} from "./CLIFileNameSource.state";
|
||||
|
||||
export type { CLIFileNameSourceOptions } from "./CLIFileNameSource.state";
|
||||
|
||||
/** Only ready owns a published inventory; failed and closed sources cannot serve names. */
|
||||
type Lifecycle =
|
||||
| { readonly kind: "uninitialised" }
|
||||
| { readonly kind: "initialising" }
|
||||
| { readonly kind: "ready"; readonly inventory: InventoryState }
|
||||
| { readonly kind: "failed"; readonly error: Error }
|
||||
| { readonly kind: "closed" };
|
||||
|
||||
const noDiagnostic: CliDiagnosticReporter = () => undefined;
|
||||
|
||||
/**
|
||||
* Coordinates the CLI's watched name inventory without owning file metadata.
|
||||
*
|
||||
* The lifecycle and publication queue stay here. Sidecars own native observation,
|
||||
* supported enumeration, and staged directory reconciliation. Name queries become
|
||||
* ready before retained sync operations are restored; live sync delivery starts
|
||||
* later, so a collision query never waits for its own event-manager activation.
|
||||
*/
|
||||
export class CLIFileNameSource implements FileNameLookup {
|
||||
private lifecycle: Lifecycle = { kind: "uninitialised" };
|
||||
// Retain the shared attempt so concurrent preparation cannot create a second watcher.
|
||||
private preparePromise: Promise<void> | undefined;
|
||||
private get state(): InventoryState | undefined {
|
||||
return this.lifecycle.kind === "ready" ? this.lifecycle.inventory : undefined;
|
||||
}
|
||||
// A failed replacement scan leaves the previous inventory intact, but its
|
||||
// known failing scope must validate again before queries can reuse that view.
|
||||
private freshValidationFailure: FreshValidationFailure | undefined;
|
||||
|
||||
// Sequence bounds captured work; directory generations reject stale reads of
|
||||
// individual scopes. Neither is an identity-map cache invalidation counter.
|
||||
private sequence = 0;
|
||||
// Both ends of host-owned I/O advance this revision, fencing fresh full scans.
|
||||
private mutationRevision = 0;
|
||||
// Earliest unhandled observation per scope: later notifications do not extend
|
||||
// a caller's captured work boundary, but do advance that scope's generation.
|
||||
private readonly invalidatedDirectories = new Map<string, number>();
|
||||
private readonly directoryGenerations = new Map<string, number>();
|
||||
// Local I/O may still be changing names after invalidation. Captured readers
|
||||
// wait for these completion tokens before they validate the affected scopes.
|
||||
private readonly pendingMutations = new Map<
|
||||
symbol,
|
||||
{ readonly sequence: number; readonly completion: Promise<void> }
|
||||
>();
|
||||
// Coalesce background passes. A request during a pass permits one later pass;
|
||||
// unresolved coverage alone does not cause a busy retry loop.
|
||||
private reconciliationScheduled = false;
|
||||
private reconciliationRequested = false;
|
||||
|
||||
// Only inventory operations enter this queue. Awaiting sync callbacks here
|
||||
// would deadlock when their collision checks query the same source again.
|
||||
private operationTail: Promise<void> = Promise.resolve();
|
||||
private readonly basePath: string;
|
||||
private readonly reportDiagnostic: CliDiagnosticReporter;
|
||||
private readonly reader: InventoryReader;
|
||||
private readonly reconciler: InventoryReconciler;
|
||||
private readonly watch: CLIFileNameSourceWatch;
|
||||
|
||||
/**
|
||||
* Wire the same storage boundary used by file I/O; constructors perform no I/O.
|
||||
*
|
||||
* @param storage - Host storage with existing path and symlink safeguards.
|
||||
* @param getPolicy - Active settings, resolved after configuration has settled.
|
||||
* @param options - Root, delivery exclusions, diagnostic, and watcher ports.
|
||||
*/
|
||||
constructor(
|
||||
private readonly storage: UnicodeResolvingNodeStorageAdapter,
|
||||
private readonly getPolicy: () => PathComparisonSettings,
|
||||
options: CLIFileNameSourceOptions
|
||||
) {
|
||||
this.basePath = path.resolve(options.basePath);
|
||||
this.reportDiagnostic = options.reportDiagnostic ?? noDiagnostic;
|
||||
this.reader = createInventoryReader({
|
||||
storage: this.storage,
|
||||
basePath: this.basePath,
|
||||
reportDiagnostic: this.reportDiagnostic,
|
||||
markDirectoryDirty: (directory) => this.markDirectoryDirty(directory),
|
||||
recordFreshFailure: (state, policy, error) => this.recordFreshFailure(state, policy, error),
|
||||
});
|
||||
this.reconciler = createInventoryReconciler({
|
||||
reader: this.reader,
|
||||
basePath: this.basePath,
|
||||
reportDiagnostic: this.reportDiagnostic,
|
||||
invalidatedDirectories: this.invalidatedDirectories,
|
||||
directoryGenerations: this.directoryGenerations,
|
||||
getBoundary: () => this.sequence,
|
||||
markDirectoryDirty: (directory) => this.markDirectoryDirty(directory),
|
||||
assertNotClosed: () => this.assertNotClosed(),
|
||||
assertInventoryAvailable: (state, policy) => this.assertNfcInventoryAvailable(state, policy),
|
||||
onValidatedDirectories: (policy, directories) => {
|
||||
if (
|
||||
this.freshValidationFailure &&
|
||||
samePolicy(this.freshValidationFailure.policy, policy) &&
|
||||
directories.has(this.freshValidationFailure.directory)
|
||||
) {
|
||||
this.freshValidationFailure = undefined;
|
||||
}
|
||||
},
|
||||
});
|
||||
this.watch = new CLIFileNameSourceWatch({
|
||||
basePath: this.basePath,
|
||||
options,
|
||||
reportDiagnostic: this.reportDiagnostic,
|
||||
canObserve: () => this.lifecycle.kind !== "closed" && this.lifecycle.kind !== "failed",
|
||||
canDeliver: () => this.lifecycle.kind === "ready",
|
||||
onObservation: (type, filePath) => this.observeChange(type, filePath),
|
||||
onInvalidPath: () => this.markPathsDirty([""]),
|
||||
onFatalError: (error) => {
|
||||
this.lifecycle = { kind: "failed", error };
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Start observation and prepare names once, without activating sync delivery.
|
||||
*
|
||||
* Concurrent calls share preparation. The host calls this after settings are
|
||||
* final and before replaying retained operations which can query names.
|
||||
*/
|
||||
prepare(): Promise<void> {
|
||||
if (this.lifecycle.kind === "closed") {
|
||||
return Promise.reject(new Error("The CLI file-name source is closed"));
|
||||
}
|
||||
if (this.lifecycle.kind === "failed") {
|
||||
return Promise.reject(this.lifecycle.error);
|
||||
}
|
||||
if (this.lifecycle.kind === "ready") {
|
||||
return Promise.resolve();
|
||||
}
|
||||
if (this.preparePromise) {
|
||||
return this.preparePromise;
|
||||
}
|
||||
|
||||
this.lifecycle = { kind: "initialising" };
|
||||
this.preparePromise = this.prepareInternal();
|
||||
return this.preparePromise;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read names through one captured observation boundary, or perform a fresh scan.
|
||||
*
|
||||
* Later events remain pending. If a captured scope cannot be published safely,
|
||||
* the request fails instead of extending its boundary until the Vault is quiet.
|
||||
*/
|
||||
async getFileNames(options: FileNameReadOptions = {}): Promise<FilePath[]> {
|
||||
await this.ensureReady();
|
||||
// Reconstruction preflight needs a new listing authority read. Ordinary
|
||||
// calls settle only observations captured below and keep watched semantics.
|
||||
if (options.fresh) {
|
||||
return await this.getFreshFileNames();
|
||||
}
|
||||
|
||||
const boundary = this.sequence;
|
||||
await this.waitForMutations(boundary);
|
||||
return await this.withOperationLock(async () => {
|
||||
if (this.hasPendingMutationThrough(boundary)) {
|
||||
throw new Error("A local mutation did not settle at the captured file-name boundary");
|
||||
}
|
||||
const state = this.requireState();
|
||||
const policy = this.getPolicy();
|
||||
await this.reader.selectPolicy(state, policy);
|
||||
const unresolved = await this.reconciler.reconcilePending(state, boundary);
|
||||
// A captured scope changed during validation or still needs work.
|
||||
// Reject this view instead of silently presenting incomplete coverage.
|
||||
if (unresolved.size > 0 || this.hasInvalidationThrough(boundary)) {
|
||||
throw new Error("File-name observations changed while reconciling the captured boundary");
|
||||
}
|
||||
this.assertNfcInventoryAvailable(state, policy);
|
||||
this.scheduleReconciliation();
|
||||
return state.index.getFileNames();
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the requested physical branches before selecting identity candidates.
|
||||
*
|
||||
* Local mutations can require another attempt. Unrelated pending subtrees are
|
||||
* reconciled separately; a target query does not recursively scan the Vault.
|
||||
*/
|
||||
async findPathIdentityCandidates(
|
||||
pathValue: FilePath,
|
||||
policy: PathComparisonSettings
|
||||
): Promise<readonly FilePath[]> {
|
||||
await this.ensureReady();
|
||||
const target = parseFilePath(pathValue);
|
||||
|
||||
while (true) {
|
||||
const mutationBoundary = this.sequence;
|
||||
await this.waitForMutations(mutationBoundary);
|
||||
const result = await this.withOperationLock(async () => {
|
||||
if (this.pendingMutations.size > 0) {
|
||||
return undefined;
|
||||
}
|
||||
const state = this.requireState();
|
||||
// The source selects the policy before reconciliation consumes it
|
||||
// from state. A new policy can expose failures in cached spellings;
|
||||
// settle those scopes before choosing candidates for this path.
|
||||
const policyChanged = !samePolicy(state.policy, policy);
|
||||
await this.reader.selectPolicy(state, policy);
|
||||
if (policyChanged) {
|
||||
const boundary = this.sequence;
|
||||
const unresolved = await this.reconciler.reconcilePending(state, boundary, false);
|
||||
if (unresolved.size > 0) {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
if (this.pendingMutations.size > 0) {
|
||||
return undefined;
|
||||
}
|
||||
this.assertNfcInventoryAvailable(state, policy);
|
||||
|
||||
const generations = await this.reconciler.revalidateTargetBranches(state, target);
|
||||
// This final cross-branch fence also catches an earlier parent
|
||||
// changing while a later parent is read. Retry instead of returning
|
||||
// candidates assembled from different observation generations.
|
||||
if (!this.reconciler.generationsUnchanged(generations)) {
|
||||
return undefined;
|
||||
}
|
||||
this.assertNfcInventoryAvailable(state, policy);
|
||||
return state.index.findPathIdentityCandidates(target, policy);
|
||||
});
|
||||
if (result !== undefined) {
|
||||
this.scheduleReconciliation();
|
||||
return result;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark original paths before host-owned I/O and return its completion callback.
|
||||
*
|
||||
* The storage adapter calls the idempotent callback in finally, including on
|
||||
* partial I/O failure. Bookkeeping failure blocks name queries while preserving
|
||||
* the filesystem operation's own success or failure result.
|
||||
*/
|
||||
invalidate(paths: readonly string[]): () => void {
|
||||
const token = Symbol("storage-mutation");
|
||||
let settle!: () => void;
|
||||
const pending = new Promise<void>((resolve) => {
|
||||
settle = resolve;
|
||||
});
|
||||
this.mutationRevision++;
|
||||
try {
|
||||
this.markPathsDirty(paths);
|
||||
} catch (error) {
|
||||
this.markInventoryUnavailable(error);
|
||||
}
|
||||
this.pendingMutations.set(token, { sequence: this.sequence, completion: pending });
|
||||
|
||||
let completed = false;
|
||||
return () => {
|
||||
if (completed) {
|
||||
return;
|
||||
}
|
||||
completed = true;
|
||||
this.mutationRevision++;
|
||||
try {
|
||||
this.markPathsDirty(paths);
|
||||
} catch (error) {
|
||||
this.markInventoryUnavailable(error);
|
||||
} finally {
|
||||
this.pendingMutations.delete(token);
|
||||
settle();
|
||||
this.scheduleReconciliation();
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/** Hand off observed startup events only after retained sync operations have been restored. */
|
||||
async beginWatch(handlers: IStorageEventWatchHandlers): Promise<void> {
|
||||
await this.ensureReady();
|
||||
this.watch.beginDelivery(handlers);
|
||||
}
|
||||
|
||||
/** Freeze sync admission while keeping inventory observation available for shutdown work. */
|
||||
stopDelivery(): void {
|
||||
this.watch.stopDelivery();
|
||||
}
|
||||
|
||||
/** Retire observation before waiting for inventory work, preventing late publication. */
|
||||
async close(): Promise<void> {
|
||||
if (this.lifecycle.kind === "closed") {
|
||||
return;
|
||||
}
|
||||
this.lifecycle = { kind: "closed" };
|
||||
this.watch.stopDelivery();
|
||||
await this.watch.close();
|
||||
await this.operationTail;
|
||||
if (this.preparePromise) {
|
||||
await this.preparePromise.catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
private async prepareInternal(): Promise<void> {
|
||||
try {
|
||||
// Observe first: changes during seeding must reach both name invalidation
|
||||
// and the finite startup delivery buffer.
|
||||
await this.watch.startObservation();
|
||||
this.assertNotClosed();
|
||||
const mutationBoundary = this.sequence;
|
||||
await this.waitForMutations(mutationBoundary);
|
||||
const policy = this.getPolicy();
|
||||
const candidate = await this.reader.enumerate(policy);
|
||||
const boundary = this.sequence;
|
||||
await this.reconciler.reconcilePending(candidate, boundary);
|
||||
const finalPolicy = this.getPolicy();
|
||||
await this.reader.selectPolicy(candidate, finalPolicy);
|
||||
await this.reconciler.reconcilePending(candidate, this.sequence);
|
||||
this.assertNfcInventoryAvailable(candidate, finalPolicy);
|
||||
this.assertNotClosed();
|
||||
this.lifecycle = { kind: "ready", inventory: candidate };
|
||||
this.scheduleReconciliation();
|
||||
} catch (error) {
|
||||
const failure = asError(error);
|
||||
if (this.lifecycle.kind !== "closed") {
|
||||
this.lifecycle = { kind: "failed", error: failure };
|
||||
}
|
||||
await this.watch.close().catch(() => undefined);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
|
||||
/** This synchronous port runs before the watcher applies any sync-only exclusion. */
|
||||
private observeChange(type: WatchEventType, filePath: FilePath): void {
|
||||
this.markPathsDirty([filePath]);
|
||||
if (type === "addDir" || type === "unlinkDir") {
|
||||
this.markDirectoryDirty(String(filePath));
|
||||
}
|
||||
this.scheduleReconciliation();
|
||||
}
|
||||
|
||||
private async ensureReady(): Promise<void> {
|
||||
if (this.lifecycle.kind === "ready") {
|
||||
return;
|
||||
}
|
||||
if (this.lifecycle.kind === "initialising" && this.preparePromise) {
|
||||
await this.preparePromise;
|
||||
return;
|
||||
}
|
||||
if (this.lifecycle.kind === "failed") {
|
||||
throw this.lifecycle.error;
|
||||
}
|
||||
if (this.lifecycle.kind === "closed") {
|
||||
throw new Error("The CLI file-name source is closed");
|
||||
}
|
||||
throw new Error("The CLI file-name source has not been prepared");
|
||||
}
|
||||
|
||||
private requireState(): InventoryState {
|
||||
if (!this.state || this.lifecycle.kind !== "ready") {
|
||||
if (this.lifecycle.kind === "failed") {
|
||||
throw this.lifecycle.error;
|
||||
}
|
||||
throw new Error("The CLI file-name inventory is unavailable");
|
||||
}
|
||||
return this.state;
|
||||
}
|
||||
|
||||
private assertNotClosed(): void {
|
||||
if (this.lifecycle.kind === "closed") {
|
||||
throw new Error("The CLI file-name source is closed");
|
||||
}
|
||||
}
|
||||
|
||||
private async waitForMutations(boundary = Number.POSITIVE_INFINITY): Promise<void> {
|
||||
while (true) {
|
||||
const pending = Array.from(this.pendingMutations.values())
|
||||
.filter((mutation) => mutation.sequence <= boundary)
|
||||
.map((mutation) => mutation.completion);
|
||||
if (pending.length === 0) {
|
||||
return;
|
||||
}
|
||||
await Promise.all(pending);
|
||||
}
|
||||
}
|
||||
|
||||
private hasPendingMutationThrough(boundary: number): boolean {
|
||||
return Array.from(this.pendingMutations.values()).some((mutation) => mutation.sequence <= boundary);
|
||||
}
|
||||
|
||||
/** Serialise inventory work and always release the queue after a failed read. */
|
||||
private async withOperationLock<T>(operation: () => Promise<T>): Promise<T> {
|
||||
const previous = this.operationTail;
|
||||
let release!: () => void;
|
||||
this.operationTail = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
await previous;
|
||||
try {
|
||||
return await operation();
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
}
|
||||
|
||||
/** Build privately, fence overlapping local I/O, and replace only a validated full view. */
|
||||
private async getFreshFileNames(): Promise<FilePath[]> {
|
||||
const boundary = this.sequence;
|
||||
await this.waitForMutations(boundary);
|
||||
const startRevision = this.mutationRevision;
|
||||
return await this.withOperationLock(async () => {
|
||||
if (this.hasPendingMutationThrough(boundary)) {
|
||||
throw new Error("A local mutation did not settle at the captured fresh file-name boundary");
|
||||
}
|
||||
this.assertNotClosed();
|
||||
// Waiting for the queue may have overlapped a completed local write.
|
||||
// Its changed revision prevents this fresh scan from replacing newer work.
|
||||
this.assertFreshReadUnchanged(startRevision);
|
||||
// Candidate reconciliation can acknowledge shared observations before
|
||||
// publication. Retain their earliest boundary until the candidate commits,
|
||||
// so a later rejection cannot hide work from the still-published inventory.
|
||||
const retainedObservations = new Map<string, number>();
|
||||
this.retainPendingObservations(retainedObservations);
|
||||
try {
|
||||
const policy = this.getPolicy();
|
||||
const candidate = await this.reader.enumerate(policy, true);
|
||||
this.assertFreshReadUnchanged(startRevision);
|
||||
const scanBoundary = this.sequence;
|
||||
this.retainPendingObservations(retainedObservations);
|
||||
const unresolved = await this.reconciler.reconcilePending(candidate, scanBoundary);
|
||||
// Host-owned I/O at either end of enumeration invalidates this candidate,
|
||||
// even if delayed watcher notifications have not advanced any scope yet.
|
||||
this.assertFreshReadUnchanged(startRevision);
|
||||
if (unresolved.size > 0 || this.hasInvalidationThrough(scanBoundary)) {
|
||||
throw new Error("File-name observations changed during fresh enumeration");
|
||||
}
|
||||
const finalPolicy = this.getPolicy();
|
||||
await this.reader.selectPolicy(candidate, finalPolicy);
|
||||
const finalBoundary = this.sequence;
|
||||
this.retainPendingObservations(retainedObservations);
|
||||
const finalUnresolved = await this.reconciler.reconcilePending(candidate, finalBoundary);
|
||||
// Policy selection can await directory reads too. Apply the same
|
||||
// publication fences after its final pass, before exposing names.
|
||||
this.assertFreshReadUnchanged(startRevision);
|
||||
if (finalUnresolved.size > 0 || this.hasInvalidationThrough(finalBoundary)) {
|
||||
throw new Error("File-name observations changed during fresh enumeration");
|
||||
}
|
||||
this.freshValidationFailure = undefined;
|
||||
try {
|
||||
this.assertNfcInventoryAvailable(candidate, finalPolicy);
|
||||
} catch (error) {
|
||||
this.recordFreshFailure(candidate, finalPolicy, error);
|
||||
throw error;
|
||||
}
|
||||
this.assertNotClosed();
|
||||
this.lifecycle = { kind: "ready", inventory: candidate };
|
||||
return candidate.index.getFileNames();
|
||||
} catch (error) {
|
||||
this.restorePendingObservations(retainedObservations);
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** Host I/O which starts or completes during a fresh read prevents publication. */
|
||||
private assertFreshReadUnchanged(startRevision: number): void {
|
||||
if (this.mutationRevision !== startRevision || this.pendingMutations.size > 0) {
|
||||
throw new Error("A local storage mutation overlapped fresh file-name enumeration");
|
||||
}
|
||||
}
|
||||
|
||||
/** Remember each scope's first pending sequence before a private reconciliation pass. */
|
||||
private retainPendingObservations(retained: Map<string, number>): void {
|
||||
for (const [directory, sequence] of this.invalidatedDirectories) {
|
||||
if (!retained.has(directory)) {
|
||||
retained.set(directory, sequence);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** A rejected candidate must leave its acknowledged work pending for the published view. */
|
||||
private restorePendingObservations(retained: ReadonlyMap<string, number>): void {
|
||||
for (const [directory, sequence] of retained) {
|
||||
const pendingSequence = this.invalidatedDirectories.get(directory);
|
||||
if (pendingSequence === undefined || sequence < pendingSequence) {
|
||||
this.markDirectoryDirty(directory, sequence);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Run one captured background pass; only newly requested work schedules another pass. */
|
||||
private scheduleReconciliation(): void {
|
||||
if (this.reconciliationScheduled) {
|
||||
this.reconciliationRequested = true;
|
||||
return;
|
||||
}
|
||||
if (this.lifecycle.kind !== "ready" || this.invalidatedDirectories.size === 0) {
|
||||
return;
|
||||
}
|
||||
this.reconciliationScheduled = true;
|
||||
const boundary = this.sequence;
|
||||
void Promise.resolve()
|
||||
.then(async () => {
|
||||
await this.waitForMutations(boundary);
|
||||
await this.withOperationLock(async () => {
|
||||
if (this.lifecycle.kind !== "ready" || this.pendingMutations.size > 0) {
|
||||
return;
|
||||
}
|
||||
const state = this.requireState();
|
||||
await this.reconciler.reconcilePending(state, boundary);
|
||||
});
|
||||
})
|
||||
.catch((error: unknown) => {
|
||||
this.reportDiagnostic("[CLIFileNameSource] Background name reconciliation failed:", error);
|
||||
})
|
||||
.finally(() => {
|
||||
this.reconciliationScheduled = false;
|
||||
const requested = this.reconciliationRequested;
|
||||
this.reconciliationRequested = false;
|
||||
if (requested && this.lifecycle.kind === "ready" && this.invalidatedDirectories.size > 0) {
|
||||
this.scheduleReconciliation();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** Known failed coverage stays visible until its scope and policy validate again. */
|
||||
private assertNfcInventoryAvailable(state: InventoryState, policy: PathComparisonSettings): void {
|
||||
// A failed fresh scan is stronger evidence than the previous successful view.
|
||||
if (this.freshValidationFailure && samePolicy(this.freshValidationFailure.policy, policy)) {
|
||||
throw this.freshValidationFailure.error;
|
||||
}
|
||||
const rootError = state.readErrors.get("");
|
||||
if (rootError) {
|
||||
throw rootError;
|
||||
}
|
||||
// Legacy mode tolerates unreadable non-root scopes. NFC identity checks
|
||||
// require all known scopes to be unambiguous and readable, even off-path.
|
||||
if (policy.pathNormalizationVersion !== 1) {
|
||||
return;
|
||||
}
|
||||
const error = state.aliasErrors.values().next().value ?? state.readErrors.values().next().value;
|
||||
if (error) {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
private hasInvalidationThrough(boundary: number): boolean {
|
||||
return Array.from(this.invalidatedDirectories.values()).some((sequence) => sequence <= boundary);
|
||||
}
|
||||
|
||||
/** Keep the earliest pending sequence, while advancing every affected directory's generation. */
|
||||
private markPathsDirty(paths: readonly string[]): void {
|
||||
const sequence = ++this.sequence;
|
||||
const directories = new Set<string>();
|
||||
for (const rawPath of paths) {
|
||||
const trimmed = rawPath.replace(/\/+$/u, "");
|
||||
let parsed: FilePath;
|
||||
try {
|
||||
parsed = parseFilePath(trimmed);
|
||||
} catch {
|
||||
directories.add("");
|
||||
continue;
|
||||
}
|
||||
const value = String(parsed);
|
||||
const state = this.state;
|
||||
const parent = parentPath(value);
|
||||
if (value !== "" && state?.directories.has(value)) {
|
||||
directories.add(value);
|
||||
}
|
||||
let current = parent;
|
||||
while (current !== "" && !state?.directories.has(current)) {
|
||||
directories.add(current);
|
||||
current = parentPath(current);
|
||||
}
|
||||
directories.add(current);
|
||||
}
|
||||
const state = this.state;
|
||||
if (state) {
|
||||
// I/O accepts logical NFC spellings, while coverage and generations
|
||||
// use physical names. Fence known equivalent parents synchronously,
|
||||
// before an in-flight read could publish their old membership.
|
||||
const policy = this.getPolicy();
|
||||
for (const directory of [...directories]) {
|
||||
for (const physicalDirectory of findInvalidatedDirectoryScopes(state.directories, directory, policy)) {
|
||||
directories.add(physicalDirectory);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const directory of directories) {
|
||||
this.markDirectoryDirty(directory, sequence);
|
||||
}
|
||||
}
|
||||
|
||||
private markInventoryUnavailable(error: unknown): void {
|
||||
const failure = asError(error);
|
||||
if (this.lifecycle.kind !== "closed") {
|
||||
this.lifecycle = { kind: "failed", error: failure };
|
||||
}
|
||||
this.reportDiagnostic(
|
||||
"[CLIFileNameSource] Inventory invalidation failed; name queries are unavailable:",
|
||||
failure
|
||||
);
|
||||
}
|
||||
|
||||
private markDirectoryDirty(directory: string, sequence = this.sequence + 1): void {
|
||||
// Keep the latest observation comparable with a read boundary, while
|
||||
// the pending entry retains its earliest unhandled observation.
|
||||
const generation = ++this.sequence;
|
||||
const pendingSequence = this.invalidatedDirectories.get(directory);
|
||||
// Failed private candidates can restore an earlier observation than a
|
||||
// notification received after that candidate acknowledged the scope.
|
||||
if (pendingSequence === undefined || sequence < pendingSequence) {
|
||||
this.invalidatedDirectories.set(directory, sequence);
|
||||
}
|
||||
this.directoryGenerations.set(directory, generation);
|
||||
}
|
||||
|
||||
/** Record a failed fresh scan without replacing the last validated inventory. */
|
||||
private recordFreshFailure(state: InventoryState, policy: PathComparisonSettings, error: unknown): void {
|
||||
const directory = state.aliasErrors.keys().next().value ?? state.readErrors.keys().next().value ?? "";
|
||||
this.freshValidationFailure = { policy: { ...policy }, directory, error: asError(error) };
|
||||
this.reportDiagnostic("[CLIFileNameSource] Fresh inventory validation failed:", error);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,889 @@
|
||||
import { EventEmitter } from "node:events";
|
||||
import { fs, os, path, UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import {
|
||||
FilePathIndex,
|
||||
parseFilePath,
|
||||
type PathComparisonSettings,
|
||||
type StoragePath,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { ChokidarOptions, FSWatcher } from "chokidar";
|
||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { CLIFileNameSource } from "./CLIFileNameSource";
|
||||
import { ObservedNodeStorageAdapter } from "./ObservedNodeStorageAdapter";
|
||||
|
||||
type TestWatcher = {
|
||||
readonly eventEmitter: EventEmitter;
|
||||
readonly watcher: FSWatcher;
|
||||
readonly close: ReturnType<typeof vi.fn>;
|
||||
};
|
||||
|
||||
const roots: string[] = [];
|
||||
|
||||
function deferred<T>() {
|
||||
let resolve!: (value: T | PromiseLike<T>) => void;
|
||||
const promise = new Promise<T>((accept) => {
|
||||
resolve = accept;
|
||||
});
|
||||
return { promise, resolve };
|
||||
}
|
||||
|
||||
function makeWatcher(autoReady = true): TestWatcher {
|
||||
const eventEmitter = new EventEmitter();
|
||||
const close = vi.fn(async () => undefined);
|
||||
const watcher = Object.assign(eventEmitter, { close }) as unknown as FSWatcher;
|
||||
if (autoReady) {
|
||||
queueMicrotask(() => eventEmitter.emit("ready"));
|
||||
}
|
||||
return { eventEmitter, watcher, close };
|
||||
}
|
||||
|
||||
function makeRoot(): string {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "livesync-name-source-"));
|
||||
roots.push(root);
|
||||
return root;
|
||||
}
|
||||
|
||||
function policy(version = 0, caseSensitive = true): PathComparisonSettings {
|
||||
return {
|
||||
pathNormalizationVersion: version as 0 | 1,
|
||||
handleFilenameCaseSensitive: caseSensitive,
|
||||
};
|
||||
}
|
||||
|
||||
function makeSource(
|
||||
root: string,
|
||||
getPolicy: () => PathComparisonSettings,
|
||||
watcher: TestWatcher,
|
||||
storage: UnicodeResolvingNodeStorageAdapter = new UnicodeResolvingNodeStorageAdapter(
|
||||
root,
|
||||
() => getPolicy().pathNormalizationVersion,
|
||||
() => !getPolicy().handleFilenameCaseSensitive
|
||||
)
|
||||
): CLIFileNameSource {
|
||||
const watcherFactory = (_basePath: string, _options: ChokidarOptions): FSWatcher => watcher.watcher;
|
||||
return new CLIFileNameSource(storage, getPolicy, { basePath: root, watcherFactory });
|
||||
}
|
||||
|
||||
async function prepare(source: CLIFileNameSource, watcher: TestWatcher): Promise<void> {
|
||||
const preparation = source.prepare();
|
||||
await Promise.resolve();
|
||||
watcher.eventEmitter.emit("ready");
|
||||
await preparation;
|
||||
}
|
||||
|
||||
function emit(watcher: TestWatcher, type: string, root: string, relativePath: string): void {
|
||||
watcher.eventEmitter.emit(type, path.join(root, relativePath));
|
||||
}
|
||||
|
||||
async function closeAndRemove(source: CLIFileNameSource, root: string): Promise<void> {
|
||||
await source.close();
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
for (const root of roots.splice(0)) {
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe("CLIFileNameSource", () => {
|
||||
it("updates one changed directory without another full inventory or index replacement", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "sub"), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, "sub", "note.md"), "initial");
|
||||
const watcher = makeWatcher();
|
||||
let currentPolicy = policy();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const readDirectory = vi.spyOn(storage, "readDirectory");
|
||||
const replaceAll = vi.spyOn(FilePathIndex.prototype, "replaceAll");
|
||||
const source = makeSource(root, () => currentPolicy, watcher, storage);
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
expect(replaceAll).toHaveBeenCalledTimes(1);
|
||||
const initialReads = readDirectory.mock.calls.length;
|
||||
|
||||
emit(watcher, "change", root, "sub/note.md");
|
||||
expect(await source.getFileNames()).toEqual([parseFilePath("sub/note.md")]);
|
||||
|
||||
expect(readDirectory.mock.calls.length - initialReads).toBe(1);
|
||||
expect(replaceAll).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("finds an NFC alias created before its watcher notification", async () => {
|
||||
const root = makeRoot();
|
||||
let currentPolicy = policy();
|
||||
const watcher = makeWatcher();
|
||||
const source = makeSource(root, () => currentPolicy, watcher);
|
||||
const nfc = "caf\u00e9.md";
|
||||
const nfd = "cafe\u0301.md";
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
fs.writeFileSync(path.join(root, nfc), "one");
|
||||
fs.writeFileSync(path.join(root, nfd), "two");
|
||||
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath(nfc), policy(1, true))).rejects.toThrow(
|
||||
"Ambiguous path after Unicode normalisation"
|
||||
);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("retains a failed fresh NFC validation until a fresh scan repairs the unrelated scope", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "unrelated"), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, "safe.md"), "safe");
|
||||
const currentPolicy = policy(1, true);
|
||||
const watcher = makeWatcher();
|
||||
const source = makeSource(root, () => currentPolicy, watcher);
|
||||
const nfc = "café.md";
|
||||
const nfd = "café.md";
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
fs.writeFileSync(path.join(root, "unrelated", nfc), "one");
|
||||
fs.writeFileSync(path.join(root, "unrelated", nfd), "two");
|
||||
|
||||
await expect(source.getFileNames({ fresh: true })).rejects.toThrow(/Ambiguous/);
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("safe.md"), currentPolicy)).rejects.toThrow(
|
||||
/Ambiguous/
|
||||
);
|
||||
|
||||
fs.rmSync(path.join(root, "unrelated", nfd));
|
||||
expect(fs.existsSync(path.join(root, "unrelated", nfd))).toBe(false);
|
||||
expect(fs.readdirSync(path.join(root, "unrelated"))).toEqual([nfc]);
|
||||
const internalStorage = (source as unknown as { storage: UnicodeResolvingNodeStorageAdapter }).storage;
|
||||
await expect(internalStorage.readDirectory(parseFilePath("unrelated") as StoragePath)).resolves.toEqual([
|
||||
{ name: nfc, kind: "file" },
|
||||
]);
|
||||
await expect(source.getFileNames({ fresh: true })).resolves.toEqual(
|
||||
expect.arrayContaining([parseFilePath("safe.md"), parseFilePath("unrelated/" + nfc)])
|
||||
);
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("safe.md"), currentPolicy)).resolves.toEqual([
|
||||
parseFilePath("safe.md"),
|
||||
]);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("collects equivalent parent branches and every matching file in Legacy case-insensitive mode", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "Docs"), { recursive: true });
|
||||
fs.mkdirSync(path.join(root, "docs"), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, "Docs", "Note.md"), "one");
|
||||
fs.writeFileSync(path.join(root, "docs", "note.md"), "two");
|
||||
const currentPolicy = policy(0, false);
|
||||
const watcher = makeWatcher();
|
||||
const source = makeSource(root, () => currentPolicy, watcher);
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
await expect(
|
||||
source.findPathIdentityCandidates(parseFilePath("docs/note.md"), currentPolicy)
|
||||
).resolves.toEqual([parseFilePath("Docs/Note.md"), parseFilePath("docs/note.md")]);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("blocks all NFC lookups on a known unrelated alias and clears it after that directory is repaired", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "unrelated"), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, "safe.md"), "safe");
|
||||
let currentPolicy = policy();
|
||||
const watcher = makeWatcher();
|
||||
const source = makeSource(root, () => currentPolicy, watcher);
|
||||
const nfc = "caf\u00e9.md";
|
||||
const nfd = "cafe\u0301.md";
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
currentPolicy = policy(1, true);
|
||||
fs.writeFileSync(path.join(root, "unrelated", nfc), "one");
|
||||
fs.writeFileSync(path.join(root, "unrelated", nfd), "two");
|
||||
emit(watcher, "add", root, "unrelated/" + nfc);
|
||||
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("safe.md"), currentPolicy)).rejects.toThrow(
|
||||
/Ambiguous/
|
||||
);
|
||||
|
||||
fs.rmSync(path.join(root, "unrelated", nfd));
|
||||
emit(watcher, "unlink", root, "unrelated/" + nfd);
|
||||
await expect(source.getFileNames()).resolves.toEqual([
|
||||
parseFilePath("safe.md"),
|
||||
parseFilePath("unrelated/" + nfc),
|
||||
]);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("revalidates parent aliases when the active Unicode or case policy changes", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "Notes"));
|
||||
fs.mkdirSync(path.join(root, "notes"));
|
||||
fs.writeFileSync(path.join(root, "Notes/a.md"), "upper parent");
|
||||
fs.writeFileSync(path.join(root, "notes/b.md"), "lower parent");
|
||||
const watcher = makeWatcher();
|
||||
let activePolicy = policy(0, false);
|
||||
const source = makeSource(root, () => activePolicy, watcher);
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("Notes/a.md"), activePolicy)).resolves.toEqual(
|
||||
["Notes/a.md"]
|
||||
);
|
||||
activePolicy = policy(1, false);
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("Notes/a.md"), activePolicy)).rejects.toThrow(
|
||||
/Ambiguous/
|
||||
);
|
||||
activePolicy = policy(1, true);
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("Notes/a.md"), activePolicy)).resolves.toEqual(
|
||||
["Notes/a.md"]
|
||||
);
|
||||
activePolicy = policy(0, true);
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("notes/b.md"), activePolicy)).resolves.toEqual(
|
||||
["notes/b.md"]
|
||||
);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("settles preparation and closes the watcher when closed before ready", async () => {
|
||||
const root = makeRoot();
|
||||
const watcher = makeWatcher(false);
|
||||
const source = makeSource(root, () => policy(), watcher);
|
||||
const preparation = source.prepare();
|
||||
const rejected = expect(preparation).rejects.toThrow("closed before watcher readiness");
|
||||
|
||||
await source.close();
|
||||
await rejected;
|
||||
await expect(source.prepare()).rejects.toThrow("closed");
|
||||
expect(watcher.close).toHaveBeenCalledTimes(1);
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("preserves a synchronous watcher-close failure during failed preparation", async () => {
|
||||
const root = makeRoot();
|
||||
const watcher = makeWatcher(false);
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const readFailure = new Error("initial directory read failed");
|
||||
const closeFailure = new Error("synchronous watcher close failed");
|
||||
vi.spyOn(storage, "readDirectory").mockRejectedValue(readFailure);
|
||||
watcher.close.mockImplementation(() => {
|
||||
throw closeFailure;
|
||||
});
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
try {
|
||||
const preparation = source.prepare();
|
||||
const rejected = expect(preparation).rejects.toBe(closeFailure);
|
||||
await Promise.resolve();
|
||||
watcher.eventEmitter.emit("ready");
|
||||
await rejected;
|
||||
expect(watcher.close).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("buffers only observed startup events and hands them off in arrival order", async () => {
|
||||
const root = makeRoot();
|
||||
fs.writeFileSync(path.join(root, "seed.md"), "seed");
|
||||
const watcher = makeWatcher();
|
||||
const currentPolicy = policy();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const originalReadDirectory = storage.readDirectory.bind(storage);
|
||||
const readDirectory = vi.spyOn(storage, "readDirectory");
|
||||
let emitted = false;
|
||||
readDirectory.mockImplementation(async (directory) => {
|
||||
const entries = await originalReadDirectory(directory);
|
||||
if (directory === "" && !emitted) {
|
||||
emitted = true;
|
||||
fs.writeFileSync(path.join(root, "during-startup.md"), "new");
|
||||
emit(watcher, "add", root, "during-startup.md");
|
||||
emit(watcher, "change", root, "during-startup.md");
|
||||
fs.writeFileSync(path.join(root, ".hidden-during-startup.md"), "hidden");
|
||||
emit(watcher, "add", root, ".hidden-during-startup.md");
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
const source = makeSource(root, () => currentPolicy, watcher, storage);
|
||||
const delivered: string[] = [];
|
||||
const handlers = {
|
||||
onCreate: vi.fn((file: { path: string }) => delivered.push("create:" + file.path)),
|
||||
onChange: vi.fn((file: { path: string }) => delivered.push("change:" + file.path)),
|
||||
onDelete: vi.fn(),
|
||||
onRename: vi.fn(),
|
||||
} as unknown as IStorageEventWatchHandlers;
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
await expect(source.getFileNames()).resolves.toEqual(
|
||||
expect.arrayContaining([parseFilePath("during-startup.md"), parseFilePath("seed.md")])
|
||||
);
|
||||
expect(delivered).toEqual([]);
|
||||
const buffered = Reflect.get(Reflect.get(source, "watch"), "bufferedEvents") as unknown[];
|
||||
expect(buffered).toHaveLength(2);
|
||||
expect(await source.getFileNames()).toContain(".hidden-during-startup.md");
|
||||
|
||||
await source.beginWatch(handlers);
|
||||
expect(delivered).toEqual(["create:during-startup.md", "change:during-startup.md"]);
|
||||
expect(handlers.onCreate).toHaveBeenCalledTimes(1);
|
||||
expect(handlers.onChange).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps one finite name-list boundary when a watched directory changes during reconciliation", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "sub"), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, "sub", "note.md"), "initial");
|
||||
const watcher = makeWatcher();
|
||||
const currentPolicy = policy();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const source = makeSource(root, () => currentPolicy, watcher, storage);
|
||||
const internal = source as unknown as {
|
||||
markPathsDirty(paths: readonly string[]): void;
|
||||
invalidatedDirectories: Map<string, number>;
|
||||
};
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
internal.markPathsDirty(["sub/note.md"]);
|
||||
const originalReadDirectory = storage.readDirectory.bind(storage);
|
||||
const readDirectory = vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
internal.markPathsDirty(["sub/note.md"]);
|
||||
return await originalReadDirectory(directory);
|
||||
});
|
||||
|
||||
await expect(source.getFileNames()).rejects.toThrow("captured boundary");
|
||||
expect(readDirectory).toHaveBeenCalledTimes(1);
|
||||
expect(internal.invalidatedDirectories.has("sub")).toBe(true);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("prunes directory generations after repeated folder creation and deletion", async () => {
|
||||
const root = makeRoot();
|
||||
const watcher = makeWatcher();
|
||||
const source = makeSource(root, () => policy(), watcher);
|
||||
const generations = (source as unknown as { directoryGenerations: Map<string, number> }).directoryGenerations;
|
||||
const invalidations = (source as unknown as { invalidatedDirectories: Map<string, number> })
|
||||
.invalidatedDirectories;
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
for (let index = 0; index < 8; index++) {
|
||||
const folder = "temporary-" + index;
|
||||
fs.mkdirSync(path.join(root, folder));
|
||||
fs.writeFileSync(path.join(root, folder, "note.md"), "content");
|
||||
emit(watcher, "addDir", root, folder);
|
||||
emit(watcher, "add", root, folder + "/note.md");
|
||||
await source.getFileNames();
|
||||
|
||||
fs.rmSync(path.join(root, folder), { recursive: true, force: true });
|
||||
emit(watcher, "unlinkDir", root, folder);
|
||||
await source.getFileNames();
|
||||
expect(Array.from(generations.keys()).every((directory) => directory === "")).toBe(true);
|
||||
expect(invalidations.size).toBe(0);
|
||||
}
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("reconciles a partial write failure after always completing mutation invalidation", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "notes"), { recursive: true });
|
||||
const currentPolicy = policy();
|
||||
const watcher = makeWatcher();
|
||||
let source: CLIFileNameSource;
|
||||
const storage = new ObservedNodeStorageAdapter(
|
||||
root,
|
||||
() => currentPolicy.pathNormalizationVersion,
|
||||
() => !currentPolicy.handleFilenameCaseSensitive,
|
||||
(paths) => source.invalidate(paths)
|
||||
);
|
||||
source = makeSource(root, () => currentPolicy, watcher, storage);
|
||||
const write = vi
|
||||
.spyOn(UnicodeResolvingNodeStorageAdapter.prototype, "write")
|
||||
.mockImplementation(async (relativePath: string) => {
|
||||
fs.writeFileSync(path.join(root, relativePath), "partially written");
|
||||
throw new Error("injected write failure after physical change");
|
||||
});
|
||||
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
await expect(storage.write("notes/partial.md", "body")).rejects.toThrow(
|
||||
"injected write failure after physical change"
|
||||
);
|
||||
await expect(source.getFileNames()).resolves.toEqual([parseFilePath("notes/partial.md")]);
|
||||
} finally {
|
||||
write.mockRestore();
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
it("retains direct mutation observations when a fresh scan is rejected", async () => {
|
||||
const root = makeRoot();
|
||||
fs.writeFileSync(path.join(root, "before.md"), "before");
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
const started = deferred<void>();
|
||||
const release = deferred<void>();
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
vi.spyOn(storage, "readDirectory").mockImplementationOnce(async (directory) => {
|
||||
const entries = await read(directory);
|
||||
started.resolve(undefined);
|
||||
await release.promise;
|
||||
return entries;
|
||||
});
|
||||
|
||||
const fresh = source.getFileNames({ fresh: true });
|
||||
const rejected = expect(fresh).rejects.toThrow("local storage mutation overlapped");
|
||||
await started.promise;
|
||||
const complete = source.invalidate(["added.md"]);
|
||||
try {
|
||||
fs.writeFileSync(path.join(root, "added.md"), "retained local write");
|
||||
} finally {
|
||||
complete();
|
||||
}
|
||||
release.resolve(undefined);
|
||||
await rejected;
|
||||
|
||||
await expect(source.getFileNames()).resolves.toEqual(expect.arrayContaining(["before.md", "added.md"]));
|
||||
expect(fs.readFileSync(path.join(root, "added.md"), "utf8")).toBe("retained local write");
|
||||
} finally {
|
||||
release.resolve(undefined);
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("retains watched scopes when unrelated local I/O rejects a fresh candidate", async () => {
|
||||
const root = makeRoot();
|
||||
for (const directory of ["a", "b"]) {
|
||||
fs.mkdirSync(path.join(root, directory));
|
||||
fs.writeFileSync(path.join(root, directory, "before.md"), "before");
|
||||
}
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
const started = deferred<void>();
|
||||
const release = deferred<void>();
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
let observed = false;
|
||||
let readsOfA = 0;
|
||||
vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
const entries = await read(directory);
|
||||
if (directory === "" && !observed) {
|
||||
observed = true;
|
||||
fs.writeFileSync(path.join(root, "a/observed.md"), "observed");
|
||||
emit(watcher, "add", root, "a/observed.md");
|
||||
}
|
||||
if (directory === "a" && ++readsOfA === 2) {
|
||||
started.resolve(undefined);
|
||||
await release.promise;
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
|
||||
const fresh = source.getFileNames({ fresh: true });
|
||||
const rejected = expect(fresh).rejects.toThrow("local storage mutation overlapped");
|
||||
await started.promise;
|
||||
const complete = source.invalidate(["b/local.md"]);
|
||||
try {
|
||||
fs.writeFileSync(path.join(root, "b/local.md"), "local");
|
||||
} finally {
|
||||
complete();
|
||||
}
|
||||
release.resolve(undefined);
|
||||
await rejected;
|
||||
|
||||
await expect(source.getFileNames()).resolves.toEqual(
|
||||
expect.arrayContaining(["a/observed.md", "b/local.md"])
|
||||
);
|
||||
} finally {
|
||||
release.resolve(undefined);
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("retains successful scope observations when a fresh reconciliation is superseded", async () => {
|
||||
const root = makeRoot();
|
||||
for (const directory of ["a", "b"]) {
|
||||
fs.mkdirSync(path.join(root, directory));
|
||||
fs.writeFileSync(path.join(root, directory, "before.md"), "before");
|
||||
}
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
const started = deferred<void>();
|
||||
const release = deferred<void>();
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
let observed = false;
|
||||
let readsOfA = 0;
|
||||
vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
const entries = await read(directory);
|
||||
if (directory === "" && !observed) {
|
||||
observed = true;
|
||||
for (const child of ["a/observed.md", "b/observed.md"]) {
|
||||
fs.writeFileSync(path.join(root, child), "observed");
|
||||
emit(watcher, "add", root, child);
|
||||
}
|
||||
}
|
||||
if (directory === "a" && ++readsOfA === 2) {
|
||||
started.resolve(undefined);
|
||||
await release.promise;
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
|
||||
const fresh = source.getFileNames({ fresh: true });
|
||||
const rejected = expect(fresh).rejects.toThrow("observations changed during fresh enumeration");
|
||||
await started.promise;
|
||||
emit(watcher, "change", root, "a/observed.md");
|
||||
release.resolve(undefined);
|
||||
await rejected;
|
||||
|
||||
await expect(source.getFileNames()).resolves.toEqual(
|
||||
expect.arrayContaining(["a/observed.md", "b/observed.md"])
|
||||
);
|
||||
} finally {
|
||||
release.resolve(undefined);
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
it("rejects name reads before preparation and after close", async () => {
|
||||
const root = makeRoot();
|
||||
const watcher = makeWatcher(false);
|
||||
const source = makeSource(root, () => policy(), watcher);
|
||||
try {
|
||||
await expect(source.getFileNames()).rejects.toThrow("has not been prepared");
|
||||
await source.close();
|
||||
await expect(source.getFileNames()).rejects.toThrow("closed");
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("shares preparation and lets a waiting reader use the completed inventory", async () => {
|
||||
const root = makeRoot();
|
||||
fs.writeFileSync(path.join(root, "ready.md"), "ready");
|
||||
const watcher = makeWatcher(false);
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const read = vi.spyOn(storage, "readDirectory");
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
try {
|
||||
const first = source.prepare();
|
||||
expect(source.prepare()).toBe(first);
|
||||
const names = source.getFileNames();
|
||||
watcher.eventEmitter.emit("ready");
|
||||
await first;
|
||||
await source.prepare();
|
||||
await expect(names).resolves.toEqual(["ready.md"]);
|
||||
expect(read).toHaveBeenCalledOnce();
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps failed preparation unavailable for later reads and prepare calls", async () => {
|
||||
const root = makeRoot();
|
||||
const watcher = makeWatcher(false);
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const failure = new Error("root listing failed");
|
||||
vi.spyOn(storage, "readDirectory").mockRejectedValue(failure);
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
try {
|
||||
const preparation = source.prepare();
|
||||
const rejected = expect(preparation).rejects.toBe(failure);
|
||||
watcher.eventEmitter.emit("ready");
|
||||
await rejected;
|
||||
await expect(source.prepare()).rejects.toBe(failure);
|
||||
await expect(source.getFileNames()).rejects.toBe(failure);
|
||||
expect(watcher.close).toHaveBeenCalledOnce();
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("waits for host I/O completion and tolerates a duplicate completion", async () => {
|
||||
const root = makeRoot();
|
||||
const watcher = makeWatcher();
|
||||
const source = makeSource(root, () => policy(), watcher);
|
||||
let complete: () => void = () => undefined;
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
complete = source.invalidate(["pending.md"]);
|
||||
fs.writeFileSync(path.join(root, "pending.md"), "pending");
|
||||
let settled = false;
|
||||
const names = source.getFileNames().then((result) => {
|
||||
settled = true;
|
||||
return result;
|
||||
});
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
expect(settled).toBe(false);
|
||||
complete();
|
||||
complete();
|
||||
await expect(names).resolves.toEqual(["pending.md"]);
|
||||
} finally {
|
||||
complete();
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects fresh and queued reads when close retires their inventory", async () => {
|
||||
const root = makeRoot();
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
const started = deferred<void>();
|
||||
const release = deferred<void>();
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
vi.spyOn(storage, "readDirectory").mockImplementationOnce(async (directory) => {
|
||||
const entries = await read(directory);
|
||||
started.resolve(undefined);
|
||||
await release.promise;
|
||||
return entries;
|
||||
});
|
||||
const fresh = source.getFileNames({ fresh: true });
|
||||
const rejectedFresh = expect(fresh).rejects.toThrow("closed");
|
||||
await started.promise;
|
||||
const queued = source.getFileNames();
|
||||
const rejectedQueued = expect(queued).rejects.toThrow("inventory is unavailable");
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
const closing = source.close();
|
||||
release.resolve(undefined);
|
||||
await rejectedFresh;
|
||||
await rejectedQueued;
|
||||
await closing;
|
||||
await expect(source.getFileNames()).rejects.toThrow("closed");
|
||||
} finally {
|
||||
release.resolve(undefined);
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("restores observations through a reader's earlier captured boundary", async () => {
|
||||
const root = makeRoot();
|
||||
for (const directory of ["a", "b"]) {
|
||||
fs.mkdirSync(path.join(root, directory));
|
||||
fs.writeFileSync(path.join(root, directory, "before.md"), "before");
|
||||
}
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const source = makeSource(root, () => policy(), watcher, storage);
|
||||
const started = deferred<void>();
|
||||
const release = deferred<void>();
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
let observed = false;
|
||||
let readsOfB = 0;
|
||||
vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
const entries = await read(directory);
|
||||
if (directory === "" && !observed) {
|
||||
observed = true;
|
||||
for (const child of ["a/observed.md", "b/observed.md"]) {
|
||||
fs.writeFileSync(path.join(root, child), "observed");
|
||||
emit(watcher, "add", root, child);
|
||||
}
|
||||
}
|
||||
if (directory === "b" && ++readsOfB === 2) {
|
||||
started.resolve(undefined);
|
||||
await release.promise;
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
const fresh = source.getFileNames({ fresh: true });
|
||||
const rejected = expect(fresh).rejects.toThrow("observations changed during fresh enumeration");
|
||||
await started.promise;
|
||||
const capturedRead = source.getFileNames();
|
||||
// Let this request capture its boundary before the later native-event turn.
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
emit(watcher, "change", root, "a/observed.md");
|
||||
emit(watcher, "change", root, "b/observed.md");
|
||||
release.resolve(undefined);
|
||||
await rejected;
|
||||
await expect(capturedRead).resolves.toEqual(expect.arrayContaining(["a/observed.md", "b/observed.md"]));
|
||||
} finally {
|
||||
release.resolve(undefined);
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects an unavailable root and recovers after that scope is recreated", async () => {
|
||||
const root = makeRoot();
|
||||
fs.writeFileSync(path.join(root, "before.md"), "before");
|
||||
const watcher = makeWatcher();
|
||||
const source = makeSource(root, () => policy(0), watcher);
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const complete = source.invalidate([""]);
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
complete();
|
||||
await expect(source.getFileNames()).rejects.toThrow("root is unavailable");
|
||||
|
||||
fs.mkdirSync(root);
|
||||
fs.writeFileSync(path.join(root, "after.md"), "after");
|
||||
emit(watcher, "add", root, "after.md");
|
||||
await expect(source.getFileNames()).resolves.toEqual(["after.md"]);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("clears a known fresh failure only after the affected scope validates", async () => {
|
||||
const root = makeRoot();
|
||||
fs.mkdirSync(path.join(root, "blocked"));
|
||||
fs.writeFileSync(path.join(root, "visible.md"), "visible");
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const selectedPolicy = policy(1);
|
||||
const source = makeSource(root, () => selectedPolicy, watcher, storage);
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
const failure = new Error("blocked scope cannot be read");
|
||||
const listing = vi.spyOn(storage, "readDirectory").mockImplementation((directory) => {
|
||||
if (directory === "blocked") {
|
||||
return Promise.reject(failure);
|
||||
}
|
||||
return read(directory);
|
||||
});
|
||||
await expect(source.getFileNames({ fresh: true })).rejects.toBe(failure);
|
||||
await expect(source.findPathIdentityCandidates(parseFilePath("visible.md"), selectedPolicy)).rejects.toBe(
|
||||
failure
|
||||
);
|
||||
|
||||
listing.mockRestore();
|
||||
fs.writeFileSync(path.join(root, "blocked/restored.md"), "restored");
|
||||
emit(watcher, "add", root, "blocked/restored.md");
|
||||
await expect(source.getFileNames()).resolves.toEqual(
|
||||
expect.arrayContaining(["visible.md", "blocked/restored.md"])
|
||||
);
|
||||
await expect(
|
||||
source.findPathIdentityCandidates(parseFilePath("visible.md"), selectedPolicy)
|
||||
).resolves.toEqual(["visible.md"]);
|
||||
} finally {
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("retries a target read superseded by a watched alias before choosing candidates", async () => {
|
||||
const root = makeRoot();
|
||||
fs.writeFileSync(path.join(root, "note.md"), "original");
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
const selectedPolicy = policy(0, false);
|
||||
const source = makeSource(root, () => selectedPolicy, watcher, storage);
|
||||
const started = deferred<void>();
|
||||
const release = deferred<void>();
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
let firstRootRead = true;
|
||||
const listing = vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
const entries = await read(directory);
|
||||
if (directory === "" && firstRootRead) {
|
||||
firstRootRead = false;
|
||||
started.resolve(undefined);
|
||||
await release.promise;
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
const candidates = source.findPathIdentityCandidates(parseFilePath("note.md"), selectedPolicy);
|
||||
await started.promise;
|
||||
fs.writeFileSync(path.join(root, "NOTE.MD"), "external alias");
|
||||
emit(watcher, "add", root, "NOTE.MD");
|
||||
release.resolve(undefined);
|
||||
|
||||
await expect(candidates).resolves.toEqual(["NOTE.MD", "note.md"]);
|
||||
expect(listing.mock.calls.filter(([directory]) => directory === "").length).toBeGreaterThan(1);
|
||||
expect(fs.readFileSync(path.join(root, "note.md"), "utf8")).toBe("original");
|
||||
expect(fs.readFileSync(path.join(root, "NOTE.MD"), "utf8")).toBe("external alias");
|
||||
} finally {
|
||||
release.resolve(undefined);
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["local mutation", "watched change"] as const)(
|
||||
"rejects an overlapping %s during final-policy reconciliation",
|
||||
async (change) => {
|
||||
const root = makeRoot();
|
||||
for (const directory of ["a", "b"]) {
|
||||
fs.mkdirSync(path.join(root, directory));
|
||||
fs.writeFileSync(path.join(root, directory, "before.md"), "before");
|
||||
}
|
||||
const watcher = makeWatcher();
|
||||
const storage = new UnicodeResolvingNodeStorageAdapter(root);
|
||||
let currentPolicy = policy(0);
|
||||
const source = makeSource(root, () => currentPolicy, watcher, storage);
|
||||
const legacyReadStarted = deferred<void>();
|
||||
const releaseLegacyRead = deferred<void>();
|
||||
const finalReadStarted = deferred<void>();
|
||||
const releaseFinalRead = deferred<void>();
|
||||
let complete: (() => void) | undefined;
|
||||
try {
|
||||
await prepare(source, watcher);
|
||||
const read = storage.readDirectory.bind(storage);
|
||||
let readsOfA = 0;
|
||||
vi.spyOn(storage, "readDirectory").mockImplementation(async (directory) => {
|
||||
if (directory === "a" && ++readsOfA === 1) {
|
||||
legacyReadStarted.resolve(undefined);
|
||||
await releaseLegacyRead.promise;
|
||||
throw new Error("transient Legacy read failure");
|
||||
}
|
||||
const entries = await read(directory);
|
||||
if (directory === "a" && readsOfA === 2) {
|
||||
finalReadStarted.resolve(undefined);
|
||||
await releaseFinalRead.promise;
|
||||
}
|
||||
return entries;
|
||||
});
|
||||
const fresh = source.getFileNames({ fresh: true });
|
||||
const expectedError =
|
||||
change === "local mutation"
|
||||
? "local storage mutation overlapped"
|
||||
: "observations changed during fresh enumeration";
|
||||
const rejected = expect(fresh).rejects.toThrow(expectedError);
|
||||
await legacyReadStarted.promise;
|
||||
currentPolicy = policy(1);
|
||||
releaseLegacyRead.resolve(undefined);
|
||||
await finalReadStarted.promise;
|
||||
if (change === "local mutation") {
|
||||
complete = source.invalidate(["b/pending.md"]);
|
||||
fs.writeFileSync(path.join(root, "b/pending.md"), "pending");
|
||||
} else {
|
||||
fs.writeFileSync(path.join(root, "a/observed.md"), "observed");
|
||||
emit(watcher, "add", root, "a/observed.md");
|
||||
}
|
||||
releaseFinalRead.resolve(undefined);
|
||||
await rejected;
|
||||
} finally {
|
||||
complete?.();
|
||||
releaseLegacyRead.resolve(undefined);
|
||||
releaseFinalRead.resolve(undefined);
|
||||
await closeAndRemove(source, root);
|
||||
}
|
||||
}
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,288 @@
|
||||
import { watch as chokidarWatch, type ChokidarOptions, type FSWatcher } from "chokidar";
|
||||
import { parseFilePath, type FilePath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { getNodeFileCreationTime, path, type Stats } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||
import type { NodeFile, NodeFolder } from "@/apps/cli/adapters/NodeTypes";
|
||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||
import { isHiddenPath } from "@/apps/cli/cliPathFilter";
|
||||
import {
|
||||
asError,
|
||||
type CLIFileNameSourceOptions,
|
||||
type QueuedWatchEvent,
|
||||
type WatchEventType,
|
||||
} from "./CLIFileNameSource.state";
|
||||
|
||||
interface FileNameWatchDependencies {
|
||||
readonly basePath: string;
|
||||
readonly options: CLIFileNameSourceOptions;
|
||||
readonly reportDiagnostic: CliDiagnosticReporter;
|
||||
readonly canObserve: () => boolean;
|
||||
readonly canDeliver: () => boolean;
|
||||
readonly onObservation: (type: WatchEventType, path: FilePath) => void;
|
||||
readonly onInvalidPath: () => void;
|
||||
readonly onFatalError: (error: Error) => void;
|
||||
}
|
||||
|
||||
const ignoredSyncSuffix = /(^|[/\\])[^/\\]*-livesync-v2([/\\]|$)/;
|
||||
|
||||
/**
|
||||
* Owns the source's one native watcher and its synchronisation-delivery buffer.
|
||||
*
|
||||
* Observation begins before enumeration, whereas delivery starts after startup
|
||||
* recovery. Inventory invalidation always runs before sync-only exclusions, so
|
||||
* an ignored or rejected event cannot hide a name from collision checks.
|
||||
*/
|
||||
export class CLIFileNameSourceWatch {
|
||||
// Acquire ownership before awaiting readiness, so close can release a watcher
|
||||
// whose initial scan has not completed or whose ready event will never arrive.
|
||||
private watcher: FSWatcher | undefined;
|
||||
private cancelReadyWait: ((error: Error) => void) | undefined;
|
||||
private closed = false;
|
||||
// Only eligible startup notifications are retained; live events go straight
|
||||
// to handlers after activation, without another long-lived delivery queue.
|
||||
private readonly bufferedEvents: QueuedWatchEvent[] = [];
|
||||
private handlers: IStorageEventWatchHandlers | undefined;
|
||||
// Activation happens once after recovery. Stopping is terminal for delivery,
|
||||
// while observation can remain open until accepted shutdown work has drained.
|
||||
private deliveryActive = false;
|
||||
private deliveryStopped = false;
|
||||
private readonly basePath: string;
|
||||
private readonly options: CLIFileNameSourceOptions;
|
||||
private readonly reportDiagnostic: CliDiagnosticReporter;
|
||||
|
||||
constructor(private readonly dependencies: FileNameWatchDependencies) {
|
||||
this.basePath = dependencies.basePath;
|
||||
this.options = dependencies.options;
|
||||
this.reportDiagnostic = dependencies.reportDiagnostic;
|
||||
}
|
||||
|
||||
/** Acquire the watcher immediately so shutdown can cancel even its ready wait. */
|
||||
async startObservation(): Promise<void> {
|
||||
if (this.closed) {
|
||||
throw new Error("The CLI file-name source is closed");
|
||||
}
|
||||
const watcher = (this.options.watcherFactory ?? chokidarWatch)(this.basePath, this.createWatcherOptions());
|
||||
this.watcher = watcher;
|
||||
this.attachWatcher(watcher);
|
||||
await this.waitForWatcherReady(watcher);
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop delivery and release observation; this method owns no inventory queue.
|
||||
*
|
||||
* Return the native close promise directly so synchronous close failures keep
|
||||
* their original propagation through preparation cleanup.
|
||||
*/
|
||||
close(): Promise<void> {
|
||||
if (this.closed) {
|
||||
return Promise.resolve();
|
||||
}
|
||||
this.closed = true;
|
||||
this.stopDelivery();
|
||||
this.cancelReadyWait?.(new Error("The CLI file-name source closed before watcher readiness"));
|
||||
this.cancelReadyWait = undefined;
|
||||
const watcher = this.watcher;
|
||||
this.watcher = undefined;
|
||||
return watcher ? watcher.close() : Promise.resolve();
|
||||
}
|
||||
|
||||
/** Hand off the finite startup buffer synchronously; downstream processing is not awaited. */
|
||||
beginDelivery(handlers: IStorageEventWatchHandlers): void {
|
||||
if (this.deliveryStopped) {
|
||||
throw new Error("Synchronisation delivery has been stopped");
|
||||
}
|
||||
if (this.deliveryActive) {
|
||||
throw new Error("Synchronisation delivery has already started");
|
||||
}
|
||||
|
||||
this.handlers = handlers;
|
||||
this.deliveryActive = true;
|
||||
const buffered = this.bufferedEvents.splice(0);
|
||||
for (const event of buffered) {
|
||||
if (this.dependencies.canDeliver() && !this.deliveryStopped && event.path !== "") {
|
||||
this.deliver(event);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Inventory observation continues while accepted sync work is flushed on shutdown. */
|
||||
stopDelivery(): void {
|
||||
this.deliveryStopped = true;
|
||||
this.handlers = undefined;
|
||||
this.bufferedEvents.length = 0;
|
||||
}
|
||||
|
||||
/** Preserve content-stability timing while observing the complete supported listing scope. */
|
||||
private createWatcherOptions(): ChokidarOptions {
|
||||
return {
|
||||
ignored: (candidatePath, stats) => this.isIgnoredDirectory(candidatePath, stats),
|
||||
ignoreInitial: true,
|
||||
persistent: true,
|
||||
followSymlinks: false,
|
||||
awaitWriteFinish: {
|
||||
stabilityThreshold: 500,
|
||||
pollInterval: 100,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Hidden directories are excluded from enumeration; hidden leaf files still need name coverage. */
|
||||
private isIgnoredDirectory(candidatePath: string, stats?: Stats): boolean {
|
||||
const relative = this.relativePath(candidatePath);
|
||||
if (relative === null || relative === "") {
|
||||
return false;
|
||||
}
|
||||
const segments = relative.split("/");
|
||||
if (segments.slice(0, -1).some((segment) => segment.startsWith("."))) {
|
||||
return true;
|
||||
}
|
||||
return Boolean(stats?.isDirectory() && segments[segments.length - 1]?.startsWith("."));
|
||||
}
|
||||
|
||||
private attachWatcher(watcher: FSWatcher): void {
|
||||
watcher.on("add", (filePath, stats) => this.observe("add", filePath, stats));
|
||||
watcher.on("change", (filePath, stats) => this.observe("change", filePath, stats));
|
||||
watcher.on("unlink", (filePath) => this.observe("unlink", filePath));
|
||||
watcher.on("addDir", (dirPath) => this.observe("addDir", dirPath));
|
||||
watcher.on("unlinkDir", (dirPath) => this.observe("unlinkDir", dirPath));
|
||||
watcher.on("error", (error) => this.handleWatcherError(watcher, error));
|
||||
}
|
||||
|
||||
private waitForWatcherReady(watcher: FSWatcher): Promise<void> {
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
let settled = false;
|
||||
const finish = (error?: Error): void => {
|
||||
if (settled) {
|
||||
return;
|
||||
}
|
||||
settled = true;
|
||||
this.cancelReadyWait = undefined;
|
||||
if (error) {
|
||||
reject(error);
|
||||
} else if (this.closed) {
|
||||
reject(new Error("The CLI file-name source closed before watcher readiness"));
|
||||
} else {
|
||||
resolve();
|
||||
}
|
||||
};
|
||||
this.cancelReadyWait = (error) => finish(error);
|
||||
watcher.once("ready", () => finish());
|
||||
});
|
||||
}
|
||||
|
||||
/** Preserve the CLI's fatal-watcher policy: mark names unavailable before process exit. */
|
||||
private handleWatcherError(watcher: FSWatcher, error: unknown): void {
|
||||
const failure = asError(error);
|
||||
this.dependencies.onFatalError(failure);
|
||||
this.reportDiagnostic("[CLIFileNameSource] Fatal watcher error — file watching stopped:", failure);
|
||||
this.reportDiagnostic("[CLIFileNameSource] Exiting for systemd restart.");
|
||||
this.cancelReadyWait?.(failure);
|
||||
this.cancelReadyWait = undefined;
|
||||
if (this.watcher === watcher) {
|
||||
this.watcher = undefined;
|
||||
}
|
||||
void watcher.close();
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
private observe(type: WatchEventType, absolutePath: string, stats?: Stats): void {
|
||||
if (!this.dependencies.canObserve()) {
|
||||
return;
|
||||
}
|
||||
const relative = this.relativePath(absolutePath);
|
||||
if (relative === null || relative === "") {
|
||||
return;
|
||||
}
|
||||
|
||||
let filePath: FilePath;
|
||||
try {
|
||||
filePath = parseFilePath(relative);
|
||||
} catch (error) {
|
||||
this.reportDiagnostic("[CLIFileNameSource] Ignoring invalid watcher path '" + relative + "':", error);
|
||||
this.dependencies.onInvalidPath();
|
||||
return;
|
||||
}
|
||||
|
||||
// Name coverage is independent of whether synchronisation accepts this event.
|
||||
this.dependencies.onObservation(type, filePath);
|
||||
// Delivery may be stopped or excluded while name observation remains valid.
|
||||
if (this.deliveryStopped || !this.isEligibleForDelivery(filePath)) {
|
||||
return;
|
||||
}
|
||||
const event: QueuedWatchEvent = { type, path: filePath, stats };
|
||||
if (!this.deliveryActive) {
|
||||
this.bufferedEvents.push(event);
|
||||
return;
|
||||
}
|
||||
this.deliver(event);
|
||||
}
|
||||
|
||||
/** These exclusions apply only after inventory invalidation, including operation-time suppression. */
|
||||
private isEligibleForDelivery(filePath: FilePath): boolean {
|
||||
const relative = String(filePath);
|
||||
if (isHiddenPath(relative) || ignoredSyncSuffix.test(relative)) {
|
||||
return false;
|
||||
}
|
||||
if (this.options.ignoreRules?.shouldIgnore(relative)) {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
return !(this.options.suppressEvent?.(relative) ?? false);
|
||||
} catch (error) {
|
||||
this.reportDiagnostic(
|
||||
"[CLIFileNameSource] Could not evaluate event suppression for '" + relative + "':",
|
||||
error
|
||||
);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private deliver(event: QueuedWatchEvent): void {
|
||||
const handlers = this.handlers;
|
||||
if (!handlers || this.deliveryStopped) {
|
||||
return;
|
||||
}
|
||||
if (event.type === "add" || event.type === "change" || event.type === "unlink") {
|
||||
const file = this.toNodeFile(event.path, event.stats);
|
||||
if (event.type === "add") {
|
||||
handlers.onCreate(file);
|
||||
} else if (event.type === "change") {
|
||||
handlers.onChange(file);
|
||||
} else {
|
||||
handlers.onDelete(file);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const folder: NodeFolder = { path: event.path, isFolder: true };
|
||||
if (event.type === "addDir") {
|
||||
handlers.onCreate(folder);
|
||||
} else {
|
||||
handlers.onDelete(folder);
|
||||
}
|
||||
}
|
||||
|
||||
private toNodeFile(filePath: FilePath, stats?: Stats): NodeFile {
|
||||
const now = Date.now();
|
||||
return {
|
||||
path: filePath,
|
||||
stat: {
|
||||
ctime: stats ? getNodeFileCreationTime(stats) : now,
|
||||
mtime: Math.floor(stats?.mtimeMs ?? now),
|
||||
size: stats?.size ?? 0,
|
||||
type: "file",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
private relativePath(candidatePath: string): string | null {
|
||||
const absolute = path.isAbsolute(candidatePath)
|
||||
? path.resolve(candidatePath)
|
||||
: path.resolve(this.basePath, candidatePath);
|
||||
const relative = path.relative(this.basePath, absolute).replace(/\\/g, "/");
|
||||
if (relative === ".." || relative.startsWith("../") || path.isAbsolute(relative)) {
|
||||
return null;
|
||||
}
|
||||
return relative;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,313 @@
|
||||
import { EventEmitter } from "node:events";
|
||||
import { os, path, type Stats } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { ChokidarOptions, FSWatcher } from "chokidar";
|
||||
import type { IStorageEventWatchHandlers } from "@vrtmrz/livesync-commonlib/compat/managers/adapters";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import type { NodeFile, NodeFolder } from "@/apps/cli/adapters/NodeTypes";
|
||||
import { CLIFileNameSourceWatch } from "./CLIFileNameSource.watch";
|
||||
|
||||
type WatchDependencies = ConstructorParameters<typeof CLIFileNameSourceWatch>[0];
|
||||
type WatchOverrides = Partial<Omit<WatchDependencies, "basePath" | "options">> & {
|
||||
options?: Partial<WatchDependencies["options"]>;
|
||||
};
|
||||
type WatchItem = NodeFile | NodeFolder;
|
||||
|
||||
type TestWatcher = {
|
||||
readonly eventEmitter: EventEmitter;
|
||||
readonly watcher: FSWatcher;
|
||||
readonly close: ReturnType<typeof vi.fn>;
|
||||
};
|
||||
|
||||
type TestWatch = {
|
||||
readonly source: CLIFileNameSourceWatch;
|
||||
readonly watcher: TestWatcher;
|
||||
readonly dependencies: WatchDependencies;
|
||||
readonly watcherFactory: ReturnType<typeof vi.fn>;
|
||||
};
|
||||
|
||||
function makeWatcher(): TestWatcher {
|
||||
const eventEmitter = new EventEmitter();
|
||||
const close = vi.fn(async () => undefined);
|
||||
const watcher = Object.assign(eventEmitter, { close }) as unknown as FSWatcher;
|
||||
return { eventEmitter, watcher, close };
|
||||
}
|
||||
|
||||
function makeWatch(basePath: string, overrides: WatchOverrides = {}): TestWatch {
|
||||
const watcher = makeWatcher();
|
||||
const watcherFactory = vi.fn((_basePath: string, _options: ChokidarOptions) => watcher.watcher);
|
||||
const { options: optionOverrides, ...dependencyOverrides } = overrides;
|
||||
const dependencies: WatchDependencies = {
|
||||
basePath,
|
||||
options: { basePath, watcherFactory, ...optionOverrides },
|
||||
reportDiagnostic: vi.fn(),
|
||||
canObserve: vi.fn(() => true),
|
||||
canDeliver: vi.fn(() => true),
|
||||
onObservation: vi.fn(),
|
||||
onInvalidPath: vi.fn(),
|
||||
onFatalError: vi.fn(),
|
||||
...dependencyOverrides,
|
||||
};
|
||||
return { source: new CLIFileNameSourceWatch(dependencies), watcher, dependencies, watcherFactory };
|
||||
}
|
||||
|
||||
async function startObservation(testWatch: TestWatch): Promise<void> {
|
||||
const started = testWatch.source.startObservation();
|
||||
testWatch.watcher.eventEmitter.emit("ready");
|
||||
await started;
|
||||
}
|
||||
|
||||
function emit(testWatch: TestWatch, type: string, candidatePath: string, stats?: Stats): void {
|
||||
testWatch.watcher.eventEmitter.emit(type, candidatePath, stats);
|
||||
}
|
||||
|
||||
function makeHandlers(events: string[] = []) {
|
||||
const recordDelivery = (action: string, item: WatchItem): void => {
|
||||
const kind = "isFolder" in item ? "folder" : "file";
|
||||
events.push(`${action}:${item.path}:${kind}`);
|
||||
};
|
||||
const onCreate = vi.fn((item: WatchItem) => recordDelivery("create", item));
|
||||
const onChange = vi.fn((item: WatchItem) => recordDelivery("change", item));
|
||||
const onDelete = vi.fn((item: WatchItem) => recordDelivery("delete", item));
|
||||
const handlers = {
|
||||
onCreate,
|
||||
onChange,
|
||||
onDelete,
|
||||
onRename: vi.fn(),
|
||||
onRaw: vi.fn(),
|
||||
} as unknown as IStorageEventWatchHandlers;
|
||||
return { handlers, onCreate, onChange, onDelete };
|
||||
}
|
||||
|
||||
const basePath = path.join(os.tmpdir(), "livesync-cli-file-watch-unit-root");
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
describe("CLIFileNameSourceWatch", () => {
|
||||
it("rejects observation started after close without acquiring a watcher", async () => {
|
||||
const testWatch = makeWatch(basePath);
|
||||
await testWatch.source.close();
|
||||
|
||||
await expect(testWatch.source.startObservation()).rejects.toThrow("source is closed");
|
||||
expect(testWatch.watcherFactory).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("cancels a delayed ready wait and ignores readiness delivered afterwards", async () => {
|
||||
const testWatch = makeWatch(basePath);
|
||||
const started = testWatch.source.startObservation();
|
||||
const rejected = expect(started).rejects.toThrow("closed before watcher readiness");
|
||||
|
||||
await testWatch.source.close();
|
||||
testWatch.watcher.eventEmitter.emit("ready");
|
||||
await rejected;
|
||||
await testWatch.source.close();
|
||||
|
||||
expect(testWatch.watcher.close).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("rejects repeated delivery activation while preserving the first handlers", async () => {
|
||||
const testWatch = makeWatch(basePath);
|
||||
await startObservation(testWatch);
|
||||
const first = makeHandlers();
|
||||
const second = makeHandlers();
|
||||
testWatch.source.beginDelivery(first.handlers);
|
||||
|
||||
expect(() => testWatch.source.beginDelivery(second.handlers)).toThrow("already started");
|
||||
emit(testWatch, "add", path.join(basePath, "note.md"));
|
||||
|
||||
expect(first.onCreate).toHaveBeenCalledOnce();
|
||||
expect(second.onCreate).not.toHaveBeenCalled();
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("keeps observing after delivery stops and rejects later activation", async () => {
|
||||
const testWatch = makeWatch(basePath);
|
||||
await startObservation(testWatch);
|
||||
const handlers = makeHandlers();
|
||||
testWatch.source.beginDelivery(handlers.handlers);
|
||||
testWatch.source.stopDelivery();
|
||||
|
||||
emit(testWatch, "add", path.join(basePath, "after-stop.md"));
|
||||
|
||||
expect(testWatch.dependencies.onObservation).toHaveBeenCalledOnce();
|
||||
expect(handlers.onCreate).not.toHaveBeenCalled();
|
||||
expect(() => testWatch.source.beginDelivery(makeHandlers().handlers)).toThrow("delivery has been stopped");
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("invalidates before sync-only exclusions and delivers accepted events in arrival order", async () => {
|
||||
const testWatch = makeWatch(basePath);
|
||||
await startObservation(testWatch);
|
||||
const events: string[] = [];
|
||||
const handlers = makeHandlers(events);
|
||||
const addPath = path.join(basePath, "note.md");
|
||||
const folderPath = path.join(basePath, "folder");
|
||||
|
||||
emit(testWatch, "add", addPath);
|
||||
emit(testWatch, "change", addPath);
|
||||
emit(testWatch, "unlink", addPath);
|
||||
emit(testWatch, "addDir", folderPath);
|
||||
emit(testWatch, "unlinkDir", folderPath);
|
||||
emit(testWatch, "add", path.join(basePath, ".hidden.md"));
|
||||
emit(testWatch, "add", path.join(basePath, "internal-livesync-v2", "record.md"));
|
||||
|
||||
expect(
|
||||
vi.mocked(testWatch.dependencies.onObservation).mock.calls.map(([type, filePath]) => `${type}:${filePath}`)
|
||||
).toEqual([
|
||||
"add:note.md",
|
||||
"change:note.md",
|
||||
"unlink:note.md",
|
||||
"addDir:folder",
|
||||
"unlinkDir:folder",
|
||||
"add:.hidden.md",
|
||||
"add:internal-livesync-v2/record.md",
|
||||
]);
|
||||
|
||||
const beforeDelivery = Date.now();
|
||||
testWatch.source.beginDelivery(handlers.handlers);
|
||||
const afterDelivery = Date.now();
|
||||
|
||||
expect(events).toEqual([
|
||||
"create:note.md:file",
|
||||
"change:note.md:file",
|
||||
"delete:note.md:file",
|
||||
"create:folder:folder",
|
||||
"delete:folder:folder",
|
||||
]);
|
||||
const created = handlers.onCreate.mock.calls[0]?.[0];
|
||||
expect(created).toBeDefined();
|
||||
if (!created || !("stat" in created)) {
|
||||
throw new Error("Expected a file notification with fallback metadata");
|
||||
}
|
||||
expect(created.stat.size).toBe(0);
|
||||
expect(created.stat.type).toBe("file");
|
||||
expect(created.stat.mtime).toBeGreaterThanOrEqual(beforeDelivery);
|
||||
expect(created.stat.mtime).toBeLessThanOrEqual(afterDelivery);
|
||||
expect(created.stat.ctime).toBe(created.stat.mtime);
|
||||
|
||||
const deletedFolder = handlers.onDelete.mock.calls[1]?.[0];
|
||||
expect(deletedFolder).toEqual({ path: "folder", isFolder: true });
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("drops buffered delivery while the caller disallows delivery", async () => {
|
||||
const testWatch = makeWatch(basePath, { canDeliver: () => false });
|
||||
await startObservation(testWatch);
|
||||
emit(testWatch, "add", path.join(basePath, "buffered.md"));
|
||||
const handlers = makeHandlers();
|
||||
|
||||
testWatch.source.beginDelivery(handlers.handlers);
|
||||
|
||||
expect(testWatch.dependencies.onObservation).toHaveBeenCalledOnce();
|
||||
expect(handlers.onCreate).not.toHaveBeenCalled();
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("ignores root and outside paths, observes hidden and sync-only paths, and reports invalid names", async () => {
|
||||
const testWatch = makeWatch(basePath);
|
||||
await startObservation(testWatch);
|
||||
const handlers = makeHandlers();
|
||||
testWatch.source.beginDelivery(handlers.handlers);
|
||||
|
||||
emit(testWatch, "add", basePath);
|
||||
emit(testWatch, "add", path.resolve(basePath, "..", "outside.md"));
|
||||
emit(testWatch, "add", path.join(basePath, ".hidden.md"));
|
||||
emit(testWatch, "add", path.join(basePath, "nested-livesync-v2", "entry.md"));
|
||||
emit(testWatch, "add", path.join(basePath, "C:\\invalid.md"));
|
||||
|
||||
expect(
|
||||
vi.mocked(testWatch.dependencies.onObservation).mock.calls.map(([type, filePath]) => `${type}:${filePath}`)
|
||||
).toEqual(["add:.hidden.md", "add:nested-livesync-v2/entry.md"]);
|
||||
expect(testWatch.dependencies.onInvalidPath).toHaveBeenCalledOnce();
|
||||
expect(testWatch.dependencies.reportDiagnostic).toHaveBeenCalledOnce();
|
||||
expect(handlers.onCreate).not.toHaveBeenCalled();
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("skips path validation and invalidation while observation is disabled", async () => {
|
||||
const testWatch = makeWatch(basePath, { canObserve: () => false });
|
||||
await startObservation(testWatch);
|
||||
|
||||
emit(testWatch, "add", path.join(basePath, "C:\\invalid.md"));
|
||||
|
||||
expect(testWatch.dependencies.onObservation).not.toHaveBeenCalled();
|
||||
expect(testWatch.dependencies.onInvalidPath).not.toHaveBeenCalled();
|
||||
expect(testWatch.dependencies.reportDiagnostic).not.toHaveBeenCalled();
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("filters hidden directories while retaining hidden leaf files in the watcher scope", async () => {
|
||||
const testWatch = makeWatch(basePath);
|
||||
await startObservation(testWatch);
|
||||
const options = testWatch.watcherFactory.mock.calls[0]?.[1];
|
||||
const ignored = options?.ignored;
|
||||
if (typeof ignored !== "function") {
|
||||
throw new Error("Expected the watcher factory to receive a directory filter");
|
||||
}
|
||||
const directoryStats = { isDirectory: () => true } as Stats;
|
||||
const fileStats = { isDirectory: () => false } as Stats;
|
||||
|
||||
expect(ignored(basePath)).toBe(false);
|
||||
expect(ignored(path.join(basePath, ".private"), directoryStats)).toBe(true);
|
||||
expect(ignored(path.join(basePath, ".private", "visible.md"), fileStats)).toBe(true);
|
||||
expect(ignored(path.join(basePath, ".hidden.md"), fileStats)).toBe(false);
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("reports suppression-hook failures while retaining name observation", async () => {
|
||||
const failure = new Error("suppression failed");
|
||||
const testWatch = makeWatch(basePath, {
|
||||
options: {
|
||||
suppressEvent: () => {
|
||||
throw failure;
|
||||
},
|
||||
},
|
||||
});
|
||||
await startObservation(testWatch);
|
||||
const handlers = makeHandlers();
|
||||
testWatch.source.beginDelivery(handlers.handlers);
|
||||
|
||||
emit(testWatch, "add", path.join(basePath, "note.md"));
|
||||
|
||||
expect(testWatch.dependencies.onObservation).toHaveBeenCalledOnce();
|
||||
expect(testWatch.dependencies.reportDiagnostic).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Could not evaluate event suppression"),
|
||||
failure
|
||||
);
|
||||
expect(handlers.onCreate).not.toHaveBeenCalled();
|
||||
await testWatch.source.close();
|
||||
});
|
||||
|
||||
it("marks names unavailable, closes, and requests restart on a fatal watcher error", async () => {
|
||||
const failure = new Error("watcher failed");
|
||||
const exitSignal = new Error("intercepted process exit");
|
||||
const testWatch = makeWatch(basePath);
|
||||
const started = testWatch.source.startObservation();
|
||||
const rejected = expect(started).rejects.toBe(failure);
|
||||
const exit = vi.spyOn(process, "exit").mockImplementation(() => {
|
||||
throw exitSignal;
|
||||
});
|
||||
|
||||
try {
|
||||
expect(() => testWatch.watcher.eventEmitter.emit("error", failure)).toThrow(exitSignal);
|
||||
await rejected;
|
||||
|
||||
expect(testWatch.dependencies.onFatalError).toHaveBeenCalledWith(failure);
|
||||
expect(testWatch.dependencies.reportDiagnostic).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
expect.stringContaining("Fatal watcher error"),
|
||||
failure
|
||||
);
|
||||
expect(testWatch.dependencies.reportDiagnostic).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
expect.stringContaining("Exiting for systemd restart")
|
||||
);
|
||||
expect(testWatch.watcher.close).toHaveBeenCalledOnce();
|
||||
expect(exit).toHaveBeenCalledWith(1);
|
||||
} finally {
|
||||
exit.mockRestore();
|
||||
await testWatch.source.close();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,87 @@
|
||||
import { UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { UXDataWriteOptions } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
|
||||
/** Ends the invalidation fence even if the filesystem operation fails partially. */
|
||||
export type StorageMutationCompletion = () => void;
|
||||
/** Begin observation before I/O and optionally return its matching completion callback. */
|
||||
export type StorageMutationInvalidator = (paths: readonly string[]) => StorageMutationCompletion | void;
|
||||
|
||||
/** Adds inventory invalidation around the shared CLI storage mutation boundary. */
|
||||
export class ObservedNodeStorageAdapter extends UnicodeResolvingNodeStorageAdapter {
|
||||
constructor(
|
||||
basePath: string,
|
||||
getPathNormalizationVersion: () => number = () => 0,
|
||||
isCaseInsensitive: () => boolean = () => false,
|
||||
private readonly onMutation?: StorageMutationInvalidator
|
||||
) {
|
||||
super(basePath, getPathNormalizationVersion, isCaseInsensitive);
|
||||
}
|
||||
|
||||
private async runMutation<T>(paths: readonly string[], operation: () => Promise<T>): Promise<T> {
|
||||
// Capture one completion for this operation. Nested storage methods can
|
||||
// create their own tokens; the source waits for all tokens in its boundary.
|
||||
let complete: StorageMutationCompletion | void;
|
||||
try {
|
||||
complete = this.onMutation?.(paths);
|
||||
} catch {
|
||||
// A failed observation hook must not suppress the requested filesystem I/O.
|
||||
complete = undefined;
|
||||
}
|
||||
|
||||
try {
|
||||
return await operation();
|
||||
} finally {
|
||||
// Failed I/O may still have changed names, so completion always runs.
|
||||
try {
|
||||
complete?.();
|
||||
} catch {
|
||||
// Inventory bookkeeping must not change the storage operation's result.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
override async write(
|
||||
path: string,
|
||||
data: string,
|
||||
options?: UXDataWriteOptions,
|
||||
caseInsensitive?: boolean
|
||||
): Promise<void> {
|
||||
return await this.runMutation([path], async () => await super.write(path, data, options, caseInsensitive));
|
||||
}
|
||||
|
||||
override async writeBinary(
|
||||
path: string,
|
||||
data: ArrayBuffer,
|
||||
options?: UXDataWriteOptions,
|
||||
caseInsensitive?: boolean
|
||||
): Promise<void> {
|
||||
return await this.runMutation(
|
||||
[path],
|
||||
async () => await super.writeBinary(path, data, options, caseInsensitive)
|
||||
);
|
||||
}
|
||||
|
||||
override async append(
|
||||
path: string,
|
||||
data: string,
|
||||
options?: UXDataWriteOptions,
|
||||
caseInsensitive?: boolean
|
||||
): Promise<void> {
|
||||
return await this.runMutation([path], async () => await super.append(path, data, options, caseInsensitive));
|
||||
}
|
||||
|
||||
override async mkdir(path: string, caseInsensitive?: boolean): Promise<void> {
|
||||
return await this.runMutation([path], async () => await super.mkdir(path, caseInsensitive));
|
||||
}
|
||||
|
||||
override async remove(path: string, caseInsensitive?: boolean): Promise<void> {
|
||||
return await this.runMutation([path], async () => await super.remove(path, caseInsensitive));
|
||||
}
|
||||
|
||||
override async rename(sourcePath: string, targetPath: string, caseInsensitive?: boolean): Promise<void> {
|
||||
return await this.runMutation(
|
||||
[sourcePath, targetPath],
|
||||
async () => await super.rename(sourcePath, targetPath, caseInsensitive)
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
import { fs, os, path, UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { ObservedNodeStorageAdapter } from "./ObservedNodeStorageAdapter";
|
||||
|
||||
function makeRoot(): string {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), "livesync-observed-storage-"));
|
||||
}
|
||||
|
||||
function removeRoot(root: string): void {
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
describe("ObservedNodeStorageAdapter", () => {
|
||||
it("writes using the default Legacy comparison callbacks", async () => {
|
||||
const root = makeRoot();
|
||||
const events: string[] = [];
|
||||
const storage = new ObservedNodeStorageAdapter(root, undefined, undefined, (paths) => {
|
||||
events.push(`start:${paths.join(",")}`);
|
||||
return () => events.push("complete");
|
||||
});
|
||||
|
||||
try {
|
||||
await storage.write("Legacy.md", "legacy contents");
|
||||
|
||||
expect(fs.readFileSync(path.join(root, "Legacy.md"), "utf8")).toBe("legacy contents");
|
||||
expect(events).toEqual(["start:Legacy.md", "complete"]);
|
||||
} finally {
|
||||
removeRoot(root);
|
||||
}
|
||||
});
|
||||
|
||||
it("writes an NFC name with version 1 and the default case comparison callback", async () => {
|
||||
const root = makeRoot();
|
||||
const nfcPath = "caf\u00e9.md";
|
||||
const events: string[] = [];
|
||||
const storage = new ObservedNodeStorageAdapter(
|
||||
root,
|
||||
() => 1,
|
||||
undefined,
|
||||
(paths) => {
|
||||
events.push(`start:${paths.join(",")}`);
|
||||
return () => events.push("complete");
|
||||
}
|
||||
);
|
||||
|
||||
try {
|
||||
await storage.write(nfcPath, "nfc contents");
|
||||
|
||||
expect(fs.readFileSync(path.join(root, nfcPath), "utf8")).toBe("nfc contents");
|
||||
expect(events).toEqual([`start:${nfcPath}`, "complete"]);
|
||||
} finally {
|
||||
removeRoot(root);
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves successful filesystem writes when observation setup throws", async () => {
|
||||
const root = makeRoot();
|
||||
const failure = new Error("observation setup failed");
|
||||
const onMutation = vi.fn(() => {
|
||||
throw failure;
|
||||
});
|
||||
const storage = new ObservedNodeStorageAdapter(root, undefined, undefined, onMutation);
|
||||
|
||||
try {
|
||||
await expect(
|
||||
storage.write("observer-start.md", "written despite observer failure")
|
||||
).resolves.toBeUndefined();
|
||||
|
||||
expect(onMutation).toHaveBeenCalledWith(["observer-start.md"]);
|
||||
expect(fs.readFileSync(path.join(root, "observer-start.md"), "utf8")).toBe(
|
||||
"written despite observer failure"
|
||||
);
|
||||
} finally {
|
||||
removeRoot(root);
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves successful filesystem writes when invalidation completion throws", async () => {
|
||||
const root = makeRoot();
|
||||
const failure = new Error("invalidation completion failed");
|
||||
const complete = vi.fn(() => {
|
||||
throw failure;
|
||||
});
|
||||
const onMutation = vi.fn(() => complete);
|
||||
const storage = new ObservedNodeStorageAdapter(root, undefined, undefined, onMutation);
|
||||
|
||||
try {
|
||||
await expect(
|
||||
storage.write("observer-complete.md", "written despite completion failure")
|
||||
).resolves.toBeUndefined();
|
||||
|
||||
expect(onMutation).toHaveBeenCalledWith(["observer-complete.md"]);
|
||||
expect(complete).toHaveBeenCalledOnce();
|
||||
expect(fs.readFileSync(path.join(root, "observer-complete.md"), "utf8")).toBe(
|
||||
"written despite completion failure"
|
||||
);
|
||||
} finally {
|
||||
removeRoot(root);
|
||||
}
|
||||
});
|
||||
|
||||
it("runs invalidation completion and preserves the underlying I/O error", async () => {
|
||||
const root = makeRoot();
|
||||
const failure = new Error("underlying write failed");
|
||||
const complete = vi.fn();
|
||||
const onMutation = vi.fn(() => complete);
|
||||
const storage = new ObservedNodeStorageAdapter(root, undefined, undefined, onMutation);
|
||||
const write = vi.spyOn(UnicodeResolvingNodeStorageAdapter.prototype, "write").mockRejectedValue(failure);
|
||||
|
||||
try {
|
||||
await expect(storage.write("failed-write.md", "contents")).rejects.toBe(failure);
|
||||
|
||||
expect(onMutation).toHaveBeenCalledWith(["failed-write.md"]);
|
||||
expect(complete).toHaveBeenCalledOnce();
|
||||
expect(fs.existsSync(path.join(root, "failed-write.md"))).toBe(false);
|
||||
} finally {
|
||||
write.mockRestore();
|
||||
removeRoot(root);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -77,6 +77,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
|
||||
APIService: services.API,
|
||||
pathService: services.path,
|
||||
});
|
||||
services.appLifecycle.onUnload.addHandler(() => {
|
||||
vaultAccess.dispose();
|
||||
return Promise.resolve(true);
|
||||
});
|
||||
const storageEventManager = new StorageEventManagerObsidian(this, core, {
|
||||
fileProcessing: services.fileProcessing,
|
||||
setting: services.setting,
|
||||
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { EVENT_CONFLICT_CANCELLED, EVENT_PLUGIN_UNLOADED } from "@vrtmrz/livesync-commonlib/compat/events/coreEvents";
|
||||
import { NO_INTERACTION } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import type { FileNameReadOptions } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { IAPIService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
||||
import {
|
||||
createInstanceLogFunction,
|
||||
@@ -58,7 +59,9 @@ function createHarness(options: HarnessOptions = {}) {
|
||||
};
|
||||
const addLog = vi.fn();
|
||||
const storageAccess = {
|
||||
getFileNames: vi.fn(async () => options.files ?? []),
|
||||
getFileNames: vi.fn<(readOptions?: FileNameReadOptions) => Promise<FilePathWithPrefix[]>>(
|
||||
async () => options.files ?? []
|
||||
),
|
||||
};
|
||||
const activeFile = vi.fn(() => options.activeFile);
|
||||
const services = {
|
||||
@@ -262,6 +265,41 @@ describe("conflict resolution serviceFeature", () => {
|
||||
expect(API.addLog).toHaveBeenLastCalledWith("Done!", LOG_LEVEL_NOTICE, "resolveAllConflictedFilesByNewerOnes");
|
||||
});
|
||||
|
||||
it("uses a fresh supported workset before bulk conflict revision deletion", async () => {
|
||||
const harness = createHarness();
|
||||
const currentPath = "current.md" as FilePathWithPrefix;
|
||||
const removedPath = "removed-before-watch-delivery.md" as FilePathWithPrefix;
|
||||
harness.storageAccess.getFileNames.mockImplementation((options) =>
|
||||
Promise.resolve(options?.fresh ? [currentPath] : [removedPath])
|
||||
);
|
||||
harness.databaseFileAccess.fetchEntryMeta.mockImplementation((path: FilePathWithPrefix, rev?: string) =>
|
||||
Promise.resolve(metadata(path, rev ?? "2-current", rev ? 1000 : 2000))
|
||||
);
|
||||
harness.databaseFileAccess.getConflictedRevs.mockResolvedValue(["1-older"]);
|
||||
|
||||
await harness.conflict.resolveAllConflictedFilesByNewerOnes();
|
||||
|
||||
expect(harness.fileHandler.deleteRevisionFromDB).toHaveBeenCalledExactlyOnceWith(currentPath, "1-older");
|
||||
expect(harness.databaseFileAccess.fetchEntryMeta.mock.calls.every(([path]) => path === currentPath)).toBe(true);
|
||||
expect(harness.storageAccess.getFileNames).toHaveBeenCalledExactlyOnceWith({ fresh: true });
|
||||
});
|
||||
|
||||
it("does not delete conflict revisions when fresh bulk enumeration fails", async () => {
|
||||
const harness = createHarness();
|
||||
const failure = new Error("fresh listing could not validate its scope");
|
||||
harness.storageAccess.getFileNames.mockImplementation((options) => {
|
||||
if (options?.fresh) {
|
||||
return Promise.reject(failure);
|
||||
}
|
||||
return Promise.resolve(["cached.md" as FilePathWithPrefix]);
|
||||
});
|
||||
|
||||
await expect(harness.conflict.resolveAllConflictedFilesByNewerOnes()).rejects.toBe(failure);
|
||||
|
||||
expect(harness.databaseFileAccess.fetchEntryMeta).not.toHaveBeenCalled();
|
||||
expect(harness.fileHandler.deleteRevisionFromDB).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("keeps resolver operations on narrow collaborators and extension seams", async () => {
|
||||
const harness = createOperationsHarness();
|
||||
const path = "same.md" as FilePathWithPrefix;
|
||||
|
||||
@@ -294,7 +294,10 @@ export function createConflictResolutionOperations(
|
||||
const resolveAllConflictedFilesByNewerOnes = async (): Promise<void> => {
|
||||
dependencies.log(`Resolving conflicts by newer ones`, LOG_LEVEL_NOTICE);
|
||||
|
||||
const files = await dependencies.storageAccess.getFileNames();
|
||||
// This explicit bulk action deletes older revisions. Read the host's
|
||||
// supported listing anew before choosing its workset; a listing failure
|
||||
// must abort before any revision is removed.
|
||||
const files = await dependencies.storageAccess.getFileNames({ fresh: true });
|
||||
|
||||
let i = 0;
|
||||
for (const file of files) {
|
||||
|
||||
@@ -81,6 +81,8 @@ export async function findDiagnosticFiles(
|
||||
reportProgress();
|
||||
return cancelledResult(processed, candidates);
|
||||
}
|
||||
// The watched listing supplies a diagnostic existence hint. Membership here
|
||||
// authorises no file or revision mutation, so a fresh full scan is unnecessary.
|
||||
const vaultPaths = new Set(await core.storageAccess.getFileNames());
|
||||
const matchingMetadataPaths = new Set<string>();
|
||||
if (options.signal?.aborted) {
|
||||
|
||||
@@ -463,6 +463,8 @@ export async function copyFileDatabaseInfo(core: FileDatabaseInfoCore, path: str
|
||||
export async function collectFileDatabaseInfoPaths(core: FileDatabaseInfoCore): Promise<string[]> {
|
||||
const ignorePatterns = getFileRegExp(core.settings, "syncInternalFilesIgnorePatterns");
|
||||
const targetPatterns = getFileRegExp(core.settings, "syncInternalFilesTargetPatterns");
|
||||
// These names seed a chooser or inspection; per-entry reads establish current
|
||||
// state before offering actions. Hidden-file mode keeps its separate listing scope.
|
||||
const storagePaths = core.settings.syncInternalFiles
|
||||
? await core.storageAccess.getFilesIncludeHidden("/", targetPatterns, ignorePatterns)
|
||||
: await core.storageAccess.getFileNames();
|
||||
|
||||
@@ -11,4 +11,8 @@ export class FileAccessObsidian extends FileAccessBase<ObsidianFileSystemAdapter
|
||||
const adapter = new ObsidianFileSystemAdapter(app);
|
||||
super(adapter, dependencies);
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
this.adapter.dispose();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
import {
|
||||
FilePathIndex,
|
||||
type FileNameLookup,
|
||||
type FileNameReadOptions,
|
||||
type FilePath,
|
||||
type PathComparisonSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { EventRef, TAbstractFile, TFolder, Vault } from "obsidian";
|
||||
|
||||
function isFolder(file: TAbstractFile): file is TFolder {
|
||||
return "children" in file;
|
||||
}
|
||||
|
||||
function isInsideFolder(path: FilePath, folderPath: string): boolean {
|
||||
return path.startsWith(`${folderPath}/`);
|
||||
}
|
||||
|
||||
/** Maintains file-name membership from Obsidian's in-memory Vault listing and events. */
|
||||
export class ObsidianFileNameSource implements FileNameLookup {
|
||||
private readonly index = new FilePathIndex();
|
||||
private readonly eventRefs: EventRef[] = [];
|
||||
private disposed = false;
|
||||
|
||||
constructor(private readonly vault: Vault) {
|
||||
try {
|
||||
this.eventRefs.push(this.vault.on("create", this.onCreate));
|
||||
this.eventRefs.push(this.vault.on("delete", this.onDelete));
|
||||
this.eventRefs.push(this.vault.on("rename", this.onRename));
|
||||
this.index.replaceAll(this.getVaultFileNames());
|
||||
} catch (error) {
|
||||
this.dispose();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async getFileNames(options?: FileNameReadOptions): Promise<FilePath[]> {
|
||||
this.assertActive();
|
||||
if (options?.fresh) {
|
||||
const fileNames = this.getVaultFileNames();
|
||||
this.index.replaceAll(fileNames);
|
||||
return await Promise.resolve(fileNames);
|
||||
}
|
||||
return await Promise.resolve(this.index.getFileNames());
|
||||
}
|
||||
|
||||
async findPathIdentityCandidates(
|
||||
path: FilePath,
|
||||
policy: PathComparisonSettings
|
||||
): Promise<readonly FilePath[]> {
|
||||
this.assertActive();
|
||||
return await Promise.resolve(this.index.findPathIdentityCandidates(path, policy));
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
if (this.disposed) {
|
||||
return;
|
||||
}
|
||||
this.disposed = true;
|
||||
for (const eventRef of this.eventRefs.splice(0)) {
|
||||
this.vault.offref(eventRef);
|
||||
}
|
||||
}
|
||||
|
||||
private readonly onCreate = (file: TAbstractFile): void => {
|
||||
if (this.disposed) {
|
||||
return;
|
||||
}
|
||||
if (isFolder(file)) {
|
||||
const prefix = `${file.path}/`;
|
||||
const added = this.getVaultFileNames().filter((path) => path.startsWith(prefix));
|
||||
this.index.apply({ removed: [], added });
|
||||
return;
|
||||
}
|
||||
this.index.apply({ removed: [], added: [file.path as FilePath] });
|
||||
};
|
||||
|
||||
private readonly onDelete = (file: TAbstractFile): void => {
|
||||
if (this.disposed) {
|
||||
return;
|
||||
}
|
||||
if (isFolder(file)) {
|
||||
const removed = this.index.getFileNames().filter((path) => isInsideFolder(path, file.path));
|
||||
this.index.apply({ removed, added: [] });
|
||||
return;
|
||||
}
|
||||
this.index.apply({ removed: [file.path as FilePath], added: [] });
|
||||
};
|
||||
|
||||
private readonly onRename = (file: TAbstractFile, oldPath: string): void => {
|
||||
if (this.disposed) {
|
||||
return;
|
||||
}
|
||||
if (isFolder(file)) {
|
||||
const oldPrefix = `${oldPath}/`;
|
||||
const newPrefix = `${file.path}/`;
|
||||
const removed = this.index.getFileNames().filter((path) => path.startsWith(oldPrefix));
|
||||
const added = removed.map((path) => `${newPrefix}${path.slice(oldPrefix.length)}` as FilePath);
|
||||
this.index.apply({ removed, added });
|
||||
return;
|
||||
}
|
||||
this.index.apply({ removed: [oldPath as FilePath], added: [file.path as FilePath] });
|
||||
};
|
||||
|
||||
private getVaultFileNames(): FilePath[] {
|
||||
return this.vault.getFiles().map((file) => file.path as FilePath);
|
||||
}
|
||||
|
||||
private assertActive(): void {
|
||||
if (this.disposed) {
|
||||
throw new Error("Obsidian file-name source has been disposed");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -7,12 +7,14 @@ import type {
|
||||
IStorageAdapter,
|
||||
IVaultAdapter,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/adapters";
|
||||
import type { FileNameLookup } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import type { TAbstractFile, TFile, TFolder, Stat, App } from "obsidian";
|
||||
import { ObsidianConversionAdapter } from "./ObsidianConversionAdapter";
|
||||
import { ObsidianPathAdapter } from "./ObsidianPathAdapter";
|
||||
import { ObsidianStorageAdapter } from "./ObsidianStorageAdapter";
|
||||
import { ObsidianTypeGuardAdapter } from "./ObsidianTypeGuardAdapter";
|
||||
import { ObsidianVaultAdapter } from "./ObsidianVaultAdapter";
|
||||
import { ObsidianFileNameSource } from "./ObsidianFileNameSource";
|
||||
|
||||
declare module "obsidian" {
|
||||
interface Vault {
|
||||
@@ -33,6 +35,8 @@ export class ObsidianFileSystemAdapter implements IFileSystemAdapter<TAbstractFi
|
||||
readonly conversion: IConversionAdapter<TFile, TFolder>;
|
||||
readonly storage: IStorageAdapter<Stat>;
|
||||
readonly vault: IVaultAdapter<TFile>;
|
||||
readonly fileNameLookup: FileNameLookup;
|
||||
private readonly fileNameSource: ObsidianFileNameSource;
|
||||
|
||||
constructor(private app: App) {
|
||||
this.path = new ObsidianPathAdapter();
|
||||
@@ -40,6 +44,8 @@ export class ObsidianFileSystemAdapter implements IFileSystemAdapter<TAbstractFi
|
||||
this.conversion = new ObsidianConversionAdapter();
|
||||
this.storage = new ObsidianStorageAdapter(app);
|
||||
this.vault = new ObsidianVaultAdapter(app);
|
||||
this.fileNameSource = new ObsidianFileNameSource(app.vault);
|
||||
this.fileNameLookup = this.fileNameSource;
|
||||
}
|
||||
|
||||
getAbstractFileByPath(path: FilePath | string): Promise<TAbstractFile | null> {
|
||||
@@ -66,4 +72,8 @@ export class ObsidianFileSystemAdapter implements IFileSystemAdapter<TAbstractFi
|
||||
async reconcileInternalFile(path: string): Promise<void> {
|
||||
return await Promise.resolve(this.app.vault.adapter.reconcileInternalFile?.(path));
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
this.fileNameSource.dispose();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,227 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import type { App, TAbstractFile, TFile, TFolder } from "obsidian";
|
||||
import { FilePathIndex, type FilePath, type PathComparisonSettings } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { ObsidianFileSystemAdapter } from "./ObsidianFileSystemAdapter";
|
||||
|
||||
vi.mock("./ObsidianConversionAdapter", () => ({ ObsidianConversionAdapter: class {} }));
|
||||
vi.mock("./ObsidianPathAdapter", () => ({ ObsidianPathAdapter: class {} }));
|
||||
vi.mock("./ObsidianStorageAdapter", () => ({ ObsidianStorageAdapter: class {} }));
|
||||
vi.mock("./ObsidianTypeGuardAdapter", () => ({ ObsidianTypeGuardAdapter: class {} }));
|
||||
vi.mock("./ObsidianVaultAdapter", () => ({ ObsidianVaultAdapter: class {} }));
|
||||
afterEach(() => vi.restoreAllMocks());
|
||||
|
||||
type VaultEvent = "create" | "delete" | "rename" | "modify";
|
||||
type VaultEventHandler = (file: TAbstractFile, oldPath?: string) => void;
|
||||
type TestFileNameLookup = {
|
||||
getFileNames(options?: { fresh?: boolean }): Promise<FilePath[]>;
|
||||
findPathIdentityCandidates(path: FilePath, policy: PathComparisonSettings): Promise<readonly FilePath[]>;
|
||||
};
|
||||
|
||||
function makeFile(path: string): TFile {
|
||||
return { path } as TFile;
|
||||
}
|
||||
|
||||
function makeFolder(path: string, children: TAbstractFile[] = []): TFolder {
|
||||
return { path, children } as TFolder;
|
||||
}
|
||||
|
||||
function getLookup(adapter: ObsidianFileSystemAdapter): TestFileNameLookup | undefined {
|
||||
return (adapter as unknown as { fileNameLookup?: TestFileNameLookup }).fileNameLookup;
|
||||
}
|
||||
|
||||
async function sortedFileNames(lookup: TestFileNameLookup): Promise<FilePath[]> {
|
||||
return (await lookup.getFileNames()).sort();
|
||||
}
|
||||
|
||||
function makeApp(initialFiles: TFile[]) {
|
||||
const handlers = new Map<VaultEvent, VaultEventHandler[]>();
|
||||
const refs: object[] = [];
|
||||
const files = [...initialFiles];
|
||||
const on = vi.fn((event: VaultEvent, handler: VaultEventHandler) => {
|
||||
const eventHandlers = handlers.get(event) ?? [];
|
||||
eventHandlers.push(handler);
|
||||
handlers.set(event, eventHandlers);
|
||||
const ref = {};
|
||||
refs.push(ref);
|
||||
return ref;
|
||||
});
|
||||
const getFiles = vi.fn(() => [...files]);
|
||||
const offref = vi.fn();
|
||||
const app = {
|
||||
vault: {
|
||||
getFiles,
|
||||
on,
|
||||
offref,
|
||||
},
|
||||
} as unknown as App;
|
||||
|
||||
return {
|
||||
app,
|
||||
files,
|
||||
handlers,
|
||||
on,
|
||||
offref,
|
||||
refs,
|
||||
getFiles,
|
||||
emit(event: VaultEvent, file: TAbstractFile, oldPath?: string) {
|
||||
for (const handler of handlers.get(event) ?? []) {
|
||||
handler(file, oldPath);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe("ObsidianFileSystemAdapter file-name lookup", () => {
|
||||
it("seeds the lookup from the Vault after registering direct membership listeners", async () => {
|
||||
const note = makeFile("notes/first.md");
|
||||
const mock = makeApp([note]);
|
||||
const adapter = new ObsidianFileSystemAdapter(mock.app);
|
||||
const lookup = getLookup(adapter);
|
||||
|
||||
expect(lookup).toBeDefined();
|
||||
expect(mock.on.mock.calls.map(([event]) => event)).toEqual(["create", "delete", "rename"]);
|
||||
expect(mock.getFiles).toHaveBeenCalledTimes(1);
|
||||
expect(Math.max(...mock.on.mock.invocationCallOrder)).toBeLessThan(mock.getFiles.mock.invocationCallOrder[0]);
|
||||
expect(await sortedFileNames(lookup!)).toEqual(["notes/first.md"]);
|
||||
});
|
||||
|
||||
it("updates file and folder membership before any synchronisation filtering, without rebuilding on content edits", async () => {
|
||||
const apply = vi.spyOn(FilePathIndex.prototype, "apply");
|
||||
const replaceAll = vi.spyOn(FilePathIndex.prototype, "replaceAll");
|
||||
const mock = makeApp([makeFile("notes/first.md"), makeFile("notes-old/keep.md")]);
|
||||
const adapter = new ObsidianFileSystemAdapter(mock.app);
|
||||
const lookup = getLookup(adapter);
|
||||
|
||||
expect(lookup).toBeDefined();
|
||||
const initialEnumerationCount = mock.getFiles.mock.calls.length;
|
||||
expect(replaceAll).toHaveBeenCalledTimes(1);
|
||||
const added = makeFile("notes/second.md");
|
||||
mock.files.push(added);
|
||||
mock.emit("create", added);
|
||||
mock.emit("create", added);
|
||||
expect(await sortedFileNames(lookup!)).toEqual(["notes-old/keep.md", "notes/first.md", "notes/second.md"]);
|
||||
mock.emit("delete", added);
|
||||
expect(await sortedFileNames(lookup!)).toEqual(["notes-old/keep.md", "notes/first.md"]);
|
||||
mock.emit("create", added);
|
||||
mock.emit("create", added);
|
||||
expect(await sortedFileNames(lookup!)).toEqual(["notes-old/keep.md", "notes/first.md", "notes/second.md"]);
|
||||
added.path = "notes/renamed.md";
|
||||
mock.emit("rename", added, "notes/second.md");
|
||||
expect(await sortedFileNames(lookup!)).toEqual(["notes-old/keep.md", "notes/first.md", "notes/renamed.md"]);
|
||||
|
||||
const applyCount = apply.mock.calls.length;
|
||||
const replaceAllCount = replaceAll.mock.calls.length;
|
||||
mock.emit("modify", added);
|
||||
expect(mock.getFiles).toHaveBeenCalledTimes(initialEnumerationCount);
|
||||
expect(apply).toHaveBeenCalledTimes(applyCount);
|
||||
expect(replaceAll).toHaveBeenCalledTimes(replaceAllCount);
|
||||
|
||||
mock.files.push(makeFile("new-folder/nested.md"));
|
||||
mock.emit("create", makeFolder("new-folder"));
|
||||
expect(await sortedFileNames(lookup!)).toEqual([
|
||||
"new-folder/nested.md",
|
||||
"notes-old/keep.md",
|
||||
"notes/first.md",
|
||||
"notes/renamed.md",
|
||||
]);
|
||||
|
||||
const movedFolder = makeFolder("archive");
|
||||
mock.emit("rename", movedFolder, "notes");
|
||||
expect(await sortedFileNames(lookup!)).toEqual([
|
||||
"archive/first.md",
|
||||
"archive/renamed.md",
|
||||
"new-folder/nested.md",
|
||||
"notes-old/keep.md",
|
||||
]);
|
||||
|
||||
mock.emit("delete", makeFolder("archive"));
|
||||
expect(await sortedFileNames(lookup!)).toEqual(["new-folder/nested.md", "notes-old/keep.md"]);
|
||||
});
|
||||
|
||||
it("uses the latest Vault listing for a fresh read and disposes only its own listeners", async () => {
|
||||
const mock = makeApp([makeFile("notes/first.md")]);
|
||||
const adapter = new ObsidianFileSystemAdapter(mock.app);
|
||||
const lookup = getLookup(adapter);
|
||||
|
||||
expect(lookup).toBeDefined();
|
||||
mock.files.push(makeFile("notes/second.md"));
|
||||
expect([...(await lookup!.getFileNames({ fresh: true }))].sort()).toEqual([
|
||||
"notes/first.md",
|
||||
"notes/second.md",
|
||||
]);
|
||||
expect(mock.getFiles).toHaveBeenCalledTimes(2);
|
||||
expect(await sortedFileNames(lookup!)).toEqual(["notes/first.md", "notes/second.md"]);
|
||||
|
||||
const sourceRefs = mock.refs.slice();
|
||||
const unrelatedRef = mock.app.vault.on("modify", vi.fn());
|
||||
const offref = mock.offref;
|
||||
(adapter as unknown as { dispose: () => void }).dispose();
|
||||
const disposedRefs = offref.mock.calls.map(([ref]) => ref);
|
||||
expect(disposedRefs).toHaveLength(sourceRefs.length);
|
||||
expect(disposedRefs.every((ref, index) => ref === sourceRefs[index])).toBe(true);
|
||||
(adapter as unknown as { dispose: () => void }).dispose();
|
||||
expect(offref).toHaveBeenCalledTimes(sourceRefs.length);
|
||||
expect(disposedRefs.some((ref) => ref === unrelatedRef)).toBe(false);
|
||||
await expect(lookup!.getFileNames()).rejects.toThrow("disposed");
|
||||
});
|
||||
|
||||
it("ignores late membership callbacks and rejects queries after disposal", async () => {
|
||||
const mock = makeApp([makeFile("note.md")]);
|
||||
const adapter = new ObsidianFileSystemAdapter(mock.app);
|
||||
const lookup = getLookup(adapter)!;
|
||||
const apply = vi.spyOn(FilePathIndex.prototype, "apply");
|
||||
adapter.dispose();
|
||||
|
||||
// The host can already have captured a callback when listeners are removed.
|
||||
mock.emit("create", makeFolder("later"));
|
||||
mock.emit("delete", makeFile("note.md"));
|
||||
mock.emit("rename", makeFile("renamed.md"), "note.md");
|
||||
expect(apply).not.toHaveBeenCalled();
|
||||
expect(mock.getFiles).toHaveBeenCalledTimes(1);
|
||||
await expect(
|
||||
lookup.findPathIdentityCandidates("note.md" as FilePath, {
|
||||
pathNormalizationVersion: 0,
|
||||
handleFilenameCaseSensitive: true,
|
||||
})
|
||||
).rejects.toThrow("disposed");
|
||||
});
|
||||
|
||||
it("returns every equivalent original path under each supported comparison policy", async () => {
|
||||
const mock = makeApp([
|
||||
makeFile("notes/Café.md"),
|
||||
makeFile("notes/café.md"),
|
||||
makeFile("notes/Cafe\u0301.md"),
|
||||
makeFile("notes/cafe\u0301.md"),
|
||||
]);
|
||||
const adapter = new ObsidianFileSystemAdapter(mock.app);
|
||||
const lookup = getLookup(adapter)!;
|
||||
const path = "notes/Café.md" as FilePath;
|
||||
const policies: [PathComparisonSettings, readonly string[]][] = [
|
||||
[{ pathNormalizationVersion: 0, handleFilenameCaseSensitive: true }, ["notes/Café.md"]],
|
||||
[{ pathNormalizationVersion: 0, handleFilenameCaseSensitive: false }, ["notes/Café.md", "notes/café.md"]],
|
||||
[
|
||||
{ pathNormalizationVersion: 1, handleFilenameCaseSensitive: true },
|
||||
["notes/Café.md", "notes/Cafe\u0301.md"],
|
||||
],
|
||||
[
|
||||
{ pathNormalizationVersion: 1, handleFilenameCaseSensitive: false },
|
||||
["notes/Café.md", "notes/café.md", "notes/Cafe\u0301.md", "notes/cafe\u0301.md"],
|
||||
],
|
||||
];
|
||||
|
||||
for (const [policy, expected] of policies) {
|
||||
const matches = await lookup.findPathIdentityCandidates(path, policy);
|
||||
expect([...matches].sort()).toEqual([...expected].sort());
|
||||
}
|
||||
});
|
||||
|
||||
it("removes its listeners if the initial Vault listing fails", () => {
|
||||
const mock = makeApp([]);
|
||||
mock.getFiles.mockImplementation(() => {
|
||||
throw new Error("Vault listing failed");
|
||||
});
|
||||
|
||||
expect(() => new ObsidianFileSystemAdapter(mock.app)).toThrow("Vault listing failed");
|
||||
expect(mock.offref.mock.calls.map(([ref]) => ref)).toEqual(mock.refs);
|
||||
});
|
||||
});
|
||||
@@ -12,6 +12,15 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Fixed
|
||||
|
||||
- File-name collision checks no longer rebuild the whole path map after content-only edits.
|
||||
|
||||
#### CLI
|
||||
|
||||
- CLI daemon collision checks no longer enumerate every file and its metadata after each content edit. (#1346)
|
||||
- The daemon maintains file names from filesystem observations and checks current directory names before accepting a path.
|
||||
|
||||
## 1.0.49
|
||||
|
||||
11th November, 2026
|
||||
|
||||
Reference in new issue
Block a user