mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-11-05 15:58:19 +00:00
Merge main into CouchDB cache compatibility changes
This commit is contained in:
commit
c306146dcd
78 files changed
+6177
-929
No files matched your search
@@ -126,6 +126,16 @@ Shared synchronisation code is compiled and typed by the `@vrtmrz/livesync-commo
|
||||
|
||||
Changes spanning both repositories must first produce a packed Commonlib artefact which passes its standalone package checks. Install that exact artefact in LiveSync, then run the LiveSync type checks, unit tests, application builds, CLI E2E, and any focused real-Obsidian E2E required by the changed boundary. Replace the temporary artefact reference with the reviewed immutable package version before release.
|
||||
|
||||
When a change requires a new Commonlib entry point or behaviour, publish and
|
||||
validate that immutable Commonlib version before releasing a dependent
|
||||
application. Follow the [Commonlib release sequence](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/releasing.md#coordinating-package-and-application-releases),
|
||||
then update this repository's exact `package.json` dependency and lockfile
|
||||
integrity from the registry. The plug-in, CLI, WebApp, and WebPeer use that root
|
||||
dependency; validate their affected builds and runtime boundaries against the
|
||||
same artefact. Bridge has its own import map and lockfile and must be pinned
|
||||
separately. Local tarball overlays are preparation evidence, not clean registry
|
||||
installation evidence.
|
||||
|
||||
## Architecture
|
||||
|
||||
The [Project glossary](docs/glossary.md#developer-and-design-terms) defines the
|
||||
@@ -264,9 +274,10 @@ Commonlib owns the typed English fallback for messages requested by its services
|
||||
|
||||
### File Path Handling
|
||||
|
||||
- Use tagged types from `types.ts`: `FilePath`, `FilePathWithPrefix`, `DocumentID`
|
||||
- Prefix constants: `CHeader` (chunks), `ICHeader`/`ICHeaderEnd` (internal data)
|
||||
- Path utilities are supplied by the focused Commonlib compatibility path `@vrtmrz/livesync-commonlib/compat/string_and_binary/path`
|
||||
- New path boundaries use `FilePath`, `DocumentPath`, `StoragePath`, and domain-specific `PathComparisonKey` from `@vrtmrz/livesync-commonlib/storage`. Original paths retain spelling; comparison keys are never converted back into filenames.
|
||||
- Parse original document input and route its namespace before storage access. `splitDocumentPath` and `joinDocumentPath` retain one recognised namespace; `DocumentPath` excludes legacy comparison keys. Existing `FilePathWithPrefix`, `FilePathWithPrefixLC`, and `DocumentID` remain at compatibility boundaries.
|
||||
- The CLI uses Commonlib's `UnicodeResolvingNodeStorageAdapter` and one strict resolver per NFC directory scan. Only absence becomes a missing file; permission and enumeration failures propagate. Obsidian keeps its own Vault APIs and alias guard, whose fixed case-insensitive NFC policy is independent of the selected database identity mode.
|
||||
- The [Commonlib storage-path design](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/design_docs/storage_path_resolution.md) and [ADR](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/adr/2026_11_shared_storage_path_resolution.md) define the contract published in Commonlib `0.3.1`. LiveSync's CLI resolver and Obsidian comparison-helper adoption remain unreleased. WebApp and Bridge NFC enablement remain separate work.
|
||||
|
||||
### Logging & Debugging
|
||||
|
||||
|
||||
@@ -13,6 +13,10 @@ Accepted on 2026-10-17. Unicode NFC path identity was released in Self-hosted
|
||||
LiveSync 1.0.38 with Commonlib 0.1.42. Local validation evidence and platform
|
||||
limits are recorded in the design document.
|
||||
|
||||
The maintained design now also describes an unreleased Journal and P2P
|
||||
extension and the move to General Settings. The decision below retains the
|
||||
initial CouchDB support boundary and original acceptance date.
|
||||
|
||||
## Context
|
||||
|
||||
Self-hosted LiveSync derives normal-file Metadata document IDs from Vault paths.
|
||||
@@ -102,9 +106,19 @@ preflight uses the supported host listing and local Metadata. It refuses
|
||||
ambiguous file or parent-directory candidates before database reconstruction;
|
||||
it does not claim to detect physical aliases which a host does not expose.
|
||||
|
||||
## Unreleased extension: defaults for new environments
|
||||
|
||||
The later Journal and P2P extension uses NFC for newly generated CLI settings
|
||||
and Obsidian manual onboarding. It retains legacy defaults for stored settings
|
||||
and old Setup URIs, and reads an existing central remote's declaration before
|
||||
first participation. This refines the original default decision without
|
||||
implicitly changing existing document IDs. Shared Commonlib defaults and other
|
||||
hosts remain unchanged. See the maintained [design](../design_docs/unicode_path_identity.md#initial-settings-and-existing-environments)
|
||||
for the initial-setup and failure rules.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Use NFC by default:** rejected because existing clients and databases must
|
||||
- **Use NFC by default for all settings:** rejected because existing clients and databases must
|
||||
retain their current IDs until their owners deliberately migrate them.
|
||||
- **Normalise only lookups:** rejected because new Metadata writes and deletes
|
||||
could continue using legacy IDs and create duplicate documents.
|
||||
|
||||
@@ -34,10 +34,15 @@ admission checks and supported transports.
|
||||
|
||||
The central replication gate, mismatch dialogues, and RedFlag Fetch preparation
|
||||
consume the same effective differences. The P2P transport retains its separate
|
||||
policy: ordinary representation differences warn without rejecting transfer,
|
||||
while its existing passphrase and peer checks still apply. NFC path identity
|
||||
is outside that warning-only policy: P2P refuses version `1`. Sharing assessment
|
||||
does not make the central replication policy appropriate for every transport.
|
||||
policy: ordinary representation and case differences warn without rejecting
|
||||
transfer, while its existing passphrase and peer checks still apply. Path
|
||||
identity has a separate required check: both peers must use the same version,
|
||||
and a version mismatch rejects before replication. Commonlib `0.3.0` includes
|
||||
NFC protocol support for Journal and P2P, but the matching CLI and Obsidian host
|
||||
adoption remains an unreleased extension tracked by
|
||||
[the Unicode path identity design](unicode_path_identity.md). Sharing
|
||||
assessment does not make the central replication policy appropriate for every
|
||||
transport.
|
||||
|
||||
## Host responsibilities
|
||||
|
||||
@@ -96,7 +101,8 @@ failed transfer into a completed transfer.
|
||||
## Verification boundaries
|
||||
|
||||
Commonlib tests protect missing-value interpretation, representation differences,
|
||||
directional consequences, immutable results, central admission, and P2P policy.
|
||||
directional consequences, immutable results, central admission, and P2P policy,
|
||||
including the distinction between ordinary warnings and a path-version mismatch.
|
||||
LiveSync unit tests protect ordinary versus reconstruction choices, trial-setting
|
||||
selection, partial-setting adoption, and invalidated decisions. RedFlag tests
|
||||
protect the distinction between adopting settings for Fetch and retaining local
|
||||
|
||||
@@ -22,14 +22,33 @@ and is included in current Commonlib `0.2.2`, used by Self-hosted LiveSync
|
||||
`1.0.42`. This later handling is distinct from the initial NFC identity
|
||||
release.
|
||||
|
||||
Commonlib `0.3.0` publishes the Journal and P2P path-identity protocol
|
||||
extension. The matching Self-hosted LiveSync CLI and Obsidian host adoption,
|
||||
including moving the setting to General Settings and defaulting new
|
||||
synchronisation environments to NFC, remains an unreleased candidate tracked
|
||||
by [pull request #1309](https://github.com/vrtmrz/obsidian-livesync/pull/1309).
|
||||
The original dates and reviewed versions above remain the historical baseline.
|
||||
|
||||
## Unreleased shared storage boundary
|
||||
|
||||
Commonlib `0.3.1` publishes the shared storage-path contract, including
|
||||
its Unicode resolver and typed physical-I/O ports. The LiveSync candidate
|
||||
adopts the shared resolver and typed physical-I/O ports in the CLI. Obsidian
|
||||
adopts shared namespace and comparison helpers while retaining its Vault APIs
|
||||
and existing alias-guard policy. Each NFC scan retains one comparison-policy
|
||||
snapshot; directory and permission errors propagate instead of producing a
|
||||
successful partial scan. LiveSync's host adoption remains unreleased. The
|
||||
[developer guide](../../devs.md#file-path-handling) links the Commonlib design
|
||||
and ADR.
|
||||
|
||||
## Scope
|
||||
|
||||
| Area | Contract |
|
||||
| --- | --- |
|
||||
| Logical identity | Version `1` uses NFC for Metadata ID and path comparison keys. Existing case handling remains a separate rule. Missing and `0` retain legacy behaviour. |
|
||||
| Stored paths and content | Keep the spelling in `Metadata.path` and existing physical Vault names. NFC does not change Chunk IDs or content. |
|
||||
| Initial hosts and remote | Support CLI and Obsidian with CouchDB. |
|
||||
| Other transports | Journal and P2P reject version `1`; Direct file access refuses a remote declaring the NFC feature. Other host adapters remain outside the support matrix. |
|
||||
| Maintained hosts and transports | CLI and Obsidian with CouchDB are released; their Journal and P2P host adoption remains unreleased. See the support matrix below. |
|
||||
| Other consumers | Direct-file access and other host adapters are covered separately below; shared path primitives alone do not enable NFC in an application. |
|
||||
| Migration | Rebuild both databases from one complete authoritative Vault, then Fetch each remaining client. Do not rewrite IDs or merge histories in place. |
|
||||
|
||||
## Path identity contract
|
||||
@@ -63,6 +82,31 @@ under legacy identity cannot be resumed under version `1`, or vice versa. This
|
||||
reuses the existing checkpoint identity and does not add a separate recovery
|
||||
store.
|
||||
|
||||
## Initial settings and existing environments
|
||||
|
||||
The unreleased extension uses version `1` for newly generated CLI settings and
|
||||
Obsidian manual onboarding. Stored settings and old Setup URIs with no path
|
||||
version retain `0`. Commonlib's shared settings defaults remain `0`, so this
|
||||
host policy does not extend to WebApp or WebPeer. An absent settings file alone
|
||||
does not establish that an existing local database is new.
|
||||
|
||||
Before saving initial manual CouchDB or Object Storage setup, the host reads
|
||||
the remote declaration through its owned connection resource. A configured
|
||||
remote supplies `0` or `1`; only a confirmed empty remote retains the new
|
||||
settings' selected version. Read failures and unsupported or contradictory
|
||||
declarations stop setup without saving a guessed value. The CLI makes the same
|
||||
read before opening the local database for an unconfigured client's first
|
||||
explicit Fetch. Ordinary configured startup and source Rebuild do not adopt a
|
||||
remote's mode automatically.
|
||||
|
||||
CouchDB inspection reads the version document without preparing encryption or
|
||||
creating a database. Without a version document, a non-empty database remains
|
||||
legacy. Journal inspection reads its existing milestone ranges and preferred
|
||||
Tweaks without registering a node, preparing a Security Seed, or resetting a
|
||||
checkpoint. P2P Setup URIs carry the selected version. Manual P2P configuration
|
||||
requires the owner to match the source; the signalling probe does not establish
|
||||
the remote database's identity, and replication still rejects a mismatch.
|
||||
|
||||
## Admission and database compatibility
|
||||
|
||||
Clients validate local and remote database identity before accepting ordinary
|
||||
@@ -133,8 +177,8 @@ check until a remote deletion has been replicated.
|
||||
## Reconstruction and client operations
|
||||
|
||||
Mode changes are never applied by an ordinary Metadata write. In Obsidian, the
|
||||
Advanced page offers **Legacy path identity** (`0`) and **Unicode NFC path
|
||||
identity** (`1`) for CouchDB. A changed `pathNormalizationVersion` remains
|
||||
General Settings → **File identity** offers **Legacy path identity** (`0`)
|
||||
and **Unicode NFC path identity** (`1`) independently of the selected transport. A changed `pathNormalizationVersion` remains
|
||||
pending through ordinary settings saves, the settings-only Apply fallback, and
|
||||
reload. It becomes active only after the existing Rebuild or Fetch operation
|
||||
has been successfully reserved; cancellation or failed reservation leaves it
|
||||
@@ -144,7 +188,8 @@ the settings-only fallback is not offered.
|
||||
The [user migration procedure](../recovery.md#migrate-to-unicode-nfc-path-identity)
|
||||
owns the operational sequence: updated and stopped clients, a complete backed-up
|
||||
authoritative Vault, source Rebuild, then Fetch on each remaining client. The
|
||||
source operation reconstructs both databases; a receiving client's Fetch
|
||||
source operation reconstructs both databases for CouchDB and Journal. With
|
||||
P2P, it reconstructs only the source database. A receiving client's Fetch
|
||||
replaces only its local database. The [CLI guide](../../src/apps/cli/README.md)
|
||||
provides the equivalent `rebuild` and `fetch` commands.
|
||||
|
||||
@@ -187,15 +232,53 @@ updates and deletions.
|
||||
|
||||
## Compatibility matrix
|
||||
|
||||
| Client and remote | Version `0` | Version `1` |
|
||||
| Host or API | Published baseline | Unreleased or host-specific status |
|
||||
| --- | --- | --- |
|
||||
| CLI with CouchDB | Supported with legacy identity. | Supported with NFC path resolution. |
|
||||
| Obsidian with CouchDB | Supported with legacy identity. | Supported with NFC path resolution. |
|
||||
| Journal or P2P | Existing behaviour. | Rejected. |
|
||||
| Direct file access | Existing behaviour. | Refuses a remote requiring `nfc-path-identity-v1`. |
|
||||
| Other host adapters, including WebApp | Existing behaviour, subject to their existing support. | Outside the initial support matrix until physical-path handling is verified. |
|
||||
| CLI and Obsidian with CouchDB | NFC identity is supported from Self-hosted LiveSync `1.0.38` with Commonlib `0.1.42`. | Commonlib `0.3.1` publishes the shared storage-path contract. LiveSync's CLI adoption of the shared resolver and typed physical-I/O ports, and its Obsidian adoption of shared namespace and comparison helpers, remain unreleased. |
|
||||
| CLI and Obsidian with Journal | Commonlib `0.3.0` publishes the NFC protocol support. | Matching LiveSync host adoption remains an unreleased candidate tracked by [pull request #1309](https://github.com/vrtmrz/obsidian-livesync/pull/1309). Update and migrate every client before using version `1`. |
|
||||
| CLI and Obsidian with P2P | Commonlib `0.3.0` publishes the NFC protocol support. | Matching LiveSync host adoption remains an unreleased candidate. All peers must use the same path version; P2P warns and continues for ordinary representation or case differences. |
|
||||
| Bridge | The published baseline uses Legacy version `0`. | The Bridge integration is an unreleased candidate with explicit per-peer opt-in to version `1`; every peer in a group must use the same version. Its ordinary case mismatch check remains enabled unless the Bridge-wide `disableCheckingConfigMismatch: true` override disables that case check. The override does not bypass a path-version mismatch. |
|
||||
| `DirectFileManipulator` | Commonlib `0.3.1` accepts explicit `pathNormalizationVersion` values `0` or `1`, defaults to Legacy (`0`), and rejects a populated database in the opposite mode. | This admission setting does not resolve physical paths; callers own physical path resolution and migration. |
|
||||
| WebApp | Legacy version `0` only. | WebApp remains Legacy-only and rejects an explicit version `1` at startup. The browser resolver primitive does not enable NFC in WebApp. |
|
||||
| WebPeer | A vaultless P2P utility, not a filesystem host. | No NFC filesystem support is implied by its P2P role. |
|
||||
| Older client without this feature | Can use a legacy database. | Cannot join a database declaring `nfc-path-identity-v1`. |
|
||||
|
||||
## Journal and P2P extension
|
||||
|
||||
The transport moves the Metadata IDs already generated in the local database;
|
||||
it does not derive a second remote ID. Journal uses its existing compatibility
|
||||
range with NFC value `4`, plus a required path-version comparison. A legacy
|
||||
client with ordinary version checking cannot join the rebuilt Journal. The
|
||||
explicit version override in older clients remains outside this guarantee.
|
||||
Journal refuses to initialise NFC compatibility when journal objects exist
|
||||
without their milestone.
|
||||
|
||||
Journal shares one read-only identity assessment across ordinary admission,
|
||||
Fetch before local reset, and ordinary milestone updates. It checks stored
|
||||
node ranges even when a Rebuild lock has retired the previous accepted nodes.
|
||||
Rebuild locks the existing milestone for its source node while retaining an
|
||||
existing compatibility range, and declares the replacement identity only after
|
||||
the remote journals have
|
||||
been deleted. If the milestone is missing, the pre-reset lock does not create
|
||||
one over the existing journals. A failed or interrupted reset therefore keeps
|
||||
the old identity or the missing declaration.
|
||||
|
||||
P2P compares the effective path version through its existing Tweak exchange,
|
||||
checks local database identity, and reads the peer version document before
|
||||
replication. Fresh control-only databases can declare the existing NFC feature.
|
||||
Every participating peer must be updated and migrated; this is not a new
|
||||
authentication protocol for already deployed clients.
|
||||
|
||||
The existing Rebuild and Fetch paths own migration and state reset. Journal
|
||||
reconstructs the local database and remote objects; P2P Rebuild reconstructs
|
||||
only the source database. Other devices Fetch the source. Local reset clears
|
||||
Journal KV checkpoints, while P2P uses its existing source mark and target
|
||||
checkpoint. No new migration store, checkpoint format, or persistent admission
|
||||
state is added. An explicit Journal bucket reset also clears the existing
|
||||
transfer checkpoint when the bucket is already empty, so a remote-only Rebuild
|
||||
resends the complete local database. P2P retains its existing peer selection
|
||||
during Fetch.
|
||||
|
||||
## Verification contract and limits
|
||||
|
||||
The implementation acceptance matrix covers:
|
||||
@@ -212,8 +295,11 @@ The implementation acceptance matrix covers:
|
||||
- refusal to write when an existing legacy database has Metadata, refusal of
|
||||
version-`0` clients against an NFC database, and enforcement when ordinary
|
||||
mismatch warnings are disabled;
|
||||
- URI round trips, checkpoint invalidation across path versions, and transport
|
||||
rejection for Journal, P2P, and Direct file access;
|
||||
- URI round trips, checkpoint invalidation across path versions, and
|
||||
explicit Legacy/NFC admission by `DirectFileManipulator` in Commonlib `0.3.1`,
|
||||
including rejection of a populated database in the opposite mode. Journal and
|
||||
P2P protocol support is published in Commonlib `0.3.0`, while matching CLI and
|
||||
Obsidian host adoption and end-to-end checks remain unreleased;
|
||||
- refusal for ambiguous file or parent-directory candidates before local or
|
||||
remote reset, preserving both databases on rejection, within the paths
|
||||
returned by each host's supported listing; and
|
||||
@@ -252,6 +338,81 @@ Streaming Fast Fetch. Fast Fetch removed the obsolete Metadata and retained
|
||||
the current NFC document, its active state, and its stored path. The existing
|
||||
content, creation-time, physical-name, update, and deletion checks also passed.
|
||||
|
||||
|
||||
### Unreleased Journal and P2P validation
|
||||
|
||||
The extension was initially checked on Linux with a locally packed Commonlib
|
||||
`0.2.3` candidate and real Obsidian `1.12.7`. The Self-hosted LiveSync extension
|
||||
remains unreleased. The initial checks covered:
|
||||
|
||||
- real S3-compatible Journal transfer with and without E2EE and Path
|
||||
Obfuscation, including restart, logical rename and deletion, both conflict
|
||||
leaves and their revision ancestry, and a full resend after bucket reset;
|
||||
- real Obsidian and CLI Journal round trips with plain IDs and independent
|
||||
encrypted IDs: Standard Fetch, identical Metadata IDs and Chunk content,
|
||||
retention of a pre-existing NFD physical name, edits after CLI process
|
||||
restart, and deletion reflected back into Obsidian;
|
||||
- real CLI P2P peers in legacy mode and NFC mode, with and without encryption
|
||||
in NFC mode: canonical path reads, incompatible-peer refusal, reverse edits
|
||||
after restart, retained original revision content, logical rename, and
|
||||
deletion. Existing automatic-host, adapter-replacement, and explicit
|
||||
disconnect scenarios also passed; and
|
||||
- two real Obsidian `1.12.7` processes in an isolated Compose project, with NFC
|
||||
identity in plain and E2EE/Path Obfuscation modes using an independent ID key:
|
||||
initial Fetch, identical Metadata IDs and Chunk-backed content, reverse edits,
|
||||
retained original revision bodies, a full application restart and subsequent
|
||||
transfer, and explicit path-version refusal after ordinary peer permission.
|
||||
The refused peer retained its sentinel ID, revision, and content and imported
|
||||
no source Metadata or file. These cases passed on 2026-11-02 with the updated
|
||||
locally packed candidate after integrating the current main branches.
|
||||
|
||||
The Journal migration scenarios also start with existing legacy CLI databases.
|
||||
The source Rebuild changes its Metadata ID and declares compatibility range
|
||||
`4`. A receiving client's legacy Fetch is refused while preserving its local
|
||||
Metadata ID and revision and the remote milestone. After selecting NFC, its
|
||||
Fetch converges to the source ID and content while preserving the NFD physical
|
||||
path. Both plain and independently encrypted ID scenarios passed.
|
||||
|
||||
On 2026-11-02, the exact published Commonlib `0.3.0` package passed the
|
||||
consumer source checks, unit tests, and plug-in, CLI, WebApp, and WebPeer builds.
|
||||
The real Obsidian P2P cases above were repeated successfully with that package.
|
||||
Real Obsidian and CLI Journal round trips and the legacy-to-NFC migration were
|
||||
also repeated with plain IDs and independently encrypted IDs, retaining the
|
||||
physical NFD names and verifying edits after restart and deletion propagation.
|
||||
The CLI Compose P2P suite also passed with the published package, including
|
||||
legacy and NFC identity, plain and encrypted NFC transfer, logical rename and
|
||||
deletion, automatic hosting, adapter replacement, and relay disconnection.
|
||||
Additional real Obsidian cases with the published package passed legacy-to-NFC
|
||||
P2P migration with plain IDs and independently encrypted IDs. They seed legacy
|
||||
NFD Metadata IDs using real Chunks, since Obsidian's file cache normalises
|
||||
names. The checks covered source Rebuild and restart, Standard Fetch on the
|
||||
receiving device, removal of obsolete active IDs, exact content, and retention
|
||||
of physical NFD names. Both cases also passed transfer of a Unicode note,
|
||||
logical rename with matching new IDs, and deletion after restart. An
|
||||
incompatible P2P Fetch reset the local database before a peer was selected.
|
||||
Its physical note retained its exact bytes, and a Fetch with matching settings
|
||||
completed recovery. This check does not establish preservation of the old
|
||||
database's revision history.
|
||||
|
||||
Real Obsidian Journal cases also passed remote-only Rebuild after deleting
|
||||
their isolated Object Storage prefix while leaving the source database and
|
||||
populated transfer checkpoint intact. With both plain IDs and independently
|
||||
encrypted IDs, a fresh receiving database and its subsequent restart retained
|
||||
the exact bodies of both conflict leaves, Chunk references, revision IDs, and
|
||||
each leaf's `_revisions`. The source retained the readable body of its common
|
||||
ancestor. Fresh receivers retained the ancestor ID without its historical
|
||||
body; ancestry IDs alone do not establish that an older body is readable. The
|
||||
[conflict policy](../specs_conflict_resolution.md#file-saving-and-reflection-guarantees)
|
||||
continues to require an available shared base for conservative automatic merge.
|
||||
|
||||
The settings, first-join identity inspection, legacy Setup URI defaults, and
|
||||
initial CLI Fetch persistence are also covered by unit tests. The additional
|
||||
migration cases call the running plug-in's production Rebuild and Fetch
|
||||
methods with real peer selection; reservation through the settings dialogue
|
||||
has separate coverage.
|
||||
These checks do not extend NFC support to other hosts or establish native iOS
|
||||
or macOS filesystem behaviour.
|
||||
|
||||
## Non-goals
|
||||
|
||||
This design does not:
|
||||
@@ -262,9 +423,9 @@ This design does not:
|
||||
- migrate remote document IDs or revision histories in place;
|
||||
- choose among ambiguous NFC-equivalent files or parents, or merge their
|
||||
content;
|
||||
- change the default away from legacy path identity;
|
||||
- enable version `1` for Journal, P2P, Direct file access, WebApp, or an
|
||||
unverified host adapter; or
|
||||
- automatically change existing settings or database identity;
|
||||
- automatically enable version `1` for WebApp or an unverified host adapter;
|
||||
or
|
||||
- claim native iOS or macOS filesystem compatibility.
|
||||
|
||||
## References
|
||||
|
||||
+3
-3
@@ -51,13 +51,13 @@ Compare the recovered files with the preserved originals, including the affected
|
||||
|
||||
## Migrate to Unicode NFC path identity
|
||||
|
||||
Use this coordinated migration when CLI and Obsidian clients share a CouchDB database and composed and decomposed Unicode spellings should identify the same path. Version `0` remains the default and preserves legacy identity. Version `1` is supported with CouchDB only; it is unsupported with Object Storage and P2P.
|
||||
Use this coordinated migration when CLI and Obsidian clients should identify composed and decomposed Unicode spellings as the same path. Existing settings and old Setup URIs retain version `0`, which preserves legacy identity. The unreleased extension defaults new CLI settings and Obsidian manual onboarding to version `1`; this does not migrate an existing database. CouchDB support is released; Object Storage and P2P support belong to the unreleased extension described in the [Unicode path identity design](design_docs/unicode_path_identity.md).
|
||||
|
||||
This migration reconstructs the databases from one authoritative Vault. It does not rewrite document IDs in place or rename files.
|
||||
|
||||
1. Update every participating CLI and Obsidian client to a version which supports Unicode NFC path identity. Back up a complete authoritative Vault and the CouchDB database. Stop synchronisation and editing on every device until the migration is complete.
|
||||
1. Update every participating CLI and Obsidian client to a version which supports Unicode NFC path identity. Back up a complete authoritative Vault and the selected remote database or Object Storage bucket. For P2P, preserve the participating devices because there is no central remote backup. Stop synchronisation and editing on every device until the migration is complete.
|
||||
2. Choose the complete Vault which contains the intended files. Review any composed and decomposed names which would map to the same path identity, and resolve them before reconstruction. If Rebuild or Fetch reports an ambiguous path or Metadata ID, stop and resolve that collision before retrying; LiveSync does not choose or merge the files for you.
|
||||
3. On the source client, select **Unicode NFC path identity** (`pathNormalizationVersion: 1`). In Obsidian, choose **Overwrite Server Data with This Device's Files**. In the CLI, set the same value in its settings and run `rebuild` with `--force --write-settings`.
|
||||
3. On the source client, open **General Settings** → **File identity** and select **Unicode NFC path identity** (`pathNormalizationVersion: 1`). In Obsidian, choose **Overwrite Server Data with This Device's Files**. In the CLI, set the same value in its settings and run `rebuild` with `--force --write-settings`. For Object Storage, this replaces the remote journals as well as the local database. For P2P, it prepares only the source device's local database; keep that device online for the other devices to Fetch.
|
||||
4. On each remaining client, select **Unicode NFC path identity** (`pathNormalizationVersion: 1`) before fetching. In Obsidian, choose **Reset Synchronisation on This Device**. In the CLI, set the same value in its settings and run `fetch` with `--force --write-settings`.
|
||||
5. Confirm that every client uses version `1` and that the expected files are present before resuming synchronisation. Do not reconnect a version `0` client to the migrated database.
|
||||
|
||||
|
||||
+16
-12
@@ -236,6 +236,21 @@ Setting key: showVerboseLog
|
||||
|
||||
Show verbose log. Please enable when you report the logs
|
||||
|
||||
### 3. File identity
|
||||
|
||||
#### Unicode path identity
|
||||
|
||||
Setting key: pathNormalizationVersion
|
||||
|
||||
Version `0` preserves legacy file identity. Existing settings and Setup URIs which omit this field retain version `0`. New CLI settings and Obsidian manual onboarding use version `1` in the unreleased extension. On first manual setup or an unconfigured CLI's initial Fetch, an existing CouchDB or Object Storage declaration takes precedence; an unavailable or unsupported declaration stops setup. P2P Setup URIs carry the selected version; manual P2P setup requires matching settings on every peer. Version `1` uses Unicode NFC when deriving document IDs, so equivalent composed and decomposed paths identify the same file across the CLI and Obsidian. Existing physical file names are retained. CouchDB support is released. The unreleased extension also supports Object Storage and P2P for the maintained CLI and Obsidian hosts. This is a common local database setting, independently of the active transport.
|
||||
|
||||
Version `1` requires a coordinated migration: set it on the source client before Rebuild, then set it on each remaining client before Fetch. Changing the setting alone does not migrate existing Metadata. Follow [Migrate to Unicode NFC path identity](recovery.md#migrate-to-unicode-nfc-path-identity) for the complete procedure. Do not resume a version `0` client against the migrated remote.
|
||||
|
||||
The selected value stays pending through ordinary settings saves and reloads. It becomes active only when the Rebuild or Fetch operation is successfully reserved; cancellation leaves it pending.
|
||||
|
||||
Distinct files or retained Metadata documents which derive the same identity stop reconstruction before either database is reset. When replacing the local database, Rebuild and Fetch exclude logical deletions without unresolved conflicts; a deleted winner with unresolved conflicts still participates in the check. This check uses the host's supported file listing; Obsidian checks the paths exposed by its Vault APIs and cannot detect physical aliases hidden by its index. Resolve the duplicate names or synchronisation scope first; Self-hosted LiveSync does not merge them automatically. Case-insensitive handling also applies the existing letter-case rule.
|
||||
|
||||
|
||||
## 3. Remote Configuration
|
||||
|
||||
### 1. Connection settings
|
||||
@@ -1176,17 +1191,6 @@ When this setting is disabled, changing only the letter case of a file name with
|
||||
|
||||
New Vaults use case-insensitive handling for cross-platform compatibility. Existing settings with an explicit value preserve that choice. Earlier releases also followed the case-insensitive branch when this value was absent, so 1.0 saves a missing legacy value as `false` without requiring a compatibility review or database rebuild.
|
||||
|
||||
#### Unicode path identity
|
||||
|
||||
Setting key: pathNormalizationVersion
|
||||
|
||||
Version `0` preserves legacy file identity and is the default for existing and new settings. Version `1` uses Unicode NFC when deriving document IDs, so equivalent composed and decomposed paths identify the same file across the CLI and Obsidian. Existing physical file names are retained. This mode currently supports CouchDB only.
|
||||
|
||||
Version `1` requires a coordinated migration: set it on the source client before Rebuild, then set it on each remaining client before Fetch. Changing the setting alone does not migrate existing Metadata. Follow [Migrate to Unicode NFC path identity](recovery.md#migrate-to-unicode-nfc-path-identity) for the complete procedure. Do not resume a version `0` client against the migrated remote.
|
||||
|
||||
The selected value stays pending through ordinary settings saves and reloads. It becomes active only when the Rebuild or Fetch operation is successfully reserved; cancellation leaves it pending.
|
||||
|
||||
Distinct files or retained Metadata documents which derive the same identity stop reconstruction before either database is reset. When replacing the local database, Rebuild and Fetch exclude logical deletions without unresolved conflicts; a deleted winner with unresolved conflicts still participates in the check. This check uses the host's supported file listing; Obsidian checks the paths exposed by its Vault APIs and cannot detect physical aliases hidden by its index. Resolve the duplicate names or synchronisation scope first; Self-hosted LiveSync does not merge them automatically. Case-insensitive handling also applies the existing letter-case rule.
|
||||
|
||||
### 4. Compatibility (Internal API Usage)
|
||||
|
||||
@@ -1234,7 +1238,7 @@ Enable this setting to process files with size mismatches, which can sometimes b
|
||||
#### Do not split chunks in the background
|
||||
|
||||
Setting key: disableWorkerForGeneratingChunks
|
||||
If disabled(toggled), chunks will be split on the UI thread (Previous behaviour).
|
||||
If enabled, chunks will be split on the UI thread (previous behaviour).
|
||||
|
||||
#### Process small files in the foreground
|
||||
|
||||
|
||||
Generated
+15
-140
@@ -30,7 +30,7 @@
|
||||
"diff-match-patch": "^1.0.5",
|
||||
"idb": "^8.0.3",
|
||||
"markdown-it": "^14.3.3",
|
||||
"minimatch": "^10.2.5",
|
||||
"minimatch": "^11.0.3",
|
||||
"obsidian": "^1.13.1",
|
||||
"octagonal-wheels": "0.1.56",
|
||||
"qrcode-generator": "^1.4.4",
|
||||
@@ -1560,37 +1560,6 @@
|
||||
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/config-array/node_modules/balanced-match": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
|
||||
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@eslint/config-array/node_modules/brace-expansion": {
|
||||
"version": "1.1.21",
|
||||
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
|
||||
"integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"balanced-match": "^1.0.0",
|
||||
"concat-map": "0.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/config-array/node_modules/minimatch": {
|
||||
"version": "3.1.5",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
|
||||
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"brace-expansion": "^1.1.7"
|
||||
},
|
||||
"engines": {
|
||||
"node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/config-helpers": {
|
||||
"version": "0.4.2",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/config-helpers/-/config-helpers-0.4.2.tgz",
|
||||
@@ -1641,37 +1610,6 @@
|
||||
"url": "https://opencollective.com/eslint"
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/eslintrc/node_modules/balanced-match": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
|
||||
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@eslint/eslintrc/node_modules/brace-expansion": {
|
||||
"version": "1.1.21",
|
||||
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
|
||||
"integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"balanced-match": "^1.0.0",
|
||||
"concat-map": "0.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/eslintrc/node_modules/minimatch": {
|
||||
"version": "3.1.5",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
|
||||
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"brace-expansion": "^1.1.7"
|
||||
},
|
||||
"engines": {
|
||||
"node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/js": {
|
||||
"version": "9.39.4",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.39.4.tgz",
|
||||
@@ -5936,23 +5874,6 @@
|
||||
"eslint": ">=8.23.0"
|
||||
}
|
||||
},
|
||||
"node_modules/eslint-plugin-n/node_modules/balanced-match": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
|
||||
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/eslint-plugin-n/node_modules/brace-expansion": {
|
||||
"version": "2.1.7",
|
||||
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.7.tgz",
|
||||
"integrity": "sha512-uZbew1NqdmPDTMJ8ah1y+b+9QEJrfkXFk3RcTQw3X0jW/xRUvFKsg1CfQdSYGdTbXZWExtU3J3ccxtnfw1Fi0g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"balanced-match": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/eslint-plugin-n/node_modules/globals": {
|
||||
"version": "15.15.0",
|
||||
"resolved": "https://registry.npmjs.org/globals/-/globals-15.15.0.tgz",
|
||||
@@ -5966,22 +5887,6 @@
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/eslint-plugin-n/node_modules/minimatch": {
|
||||
"version": "9.0.9",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-9.0.9.tgz",
|
||||
"integrity": "sha512-OBwBN9AL4dqmETlpS2zasx+vTeWclWzkblfZk7KTA5j3jeOONz/tRCnZomUyvNg83wL5Zv9Ss6HMJXAgL8R2Yg==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"brace-expansion": "^2.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=16 || 14 >=14.17"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/eslint-plugin-no-unsanitized": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/eslint-plugin-no-unsanitized/-/eslint-plugin-no-unsanitized-4.1.5.tgz",
|
||||
@@ -6315,24 +6220,6 @@
|
||||
"url": "https://opencollective.com/eslint"
|
||||
}
|
||||
},
|
||||
"node_modules/eslint/node_modules/balanced-match": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
|
||||
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/eslint/node_modules/brace-expansion": {
|
||||
"version": "1.1.21",
|
||||
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
|
||||
"integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"balanced-match": "^1.0.0",
|
||||
"concat-map": "0.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/eslint/node_modules/eslint-visitor-keys": {
|
||||
"version": "4.2.1",
|
||||
"resolved": "https://registry.npmjs.org/eslint-visitor-keys/-/eslint-visitor-keys-4.2.1.tgz",
|
||||
@@ -6346,19 +6233,6 @@
|
||||
"url": "https://opencollective.com/eslint"
|
||||
}
|
||||
},
|
||||
"node_modules/eslint/node_modules/minimatch": {
|
||||
"version": "3.1.5",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
|
||||
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"brace-expansion": "^1.1.7"
|
||||
},
|
||||
"engines": {
|
||||
"node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/esm-env": {
|
||||
"version": "1.2.2",
|
||||
"resolved": "https://registry.npmjs.org/esm-env/-/esm-env-1.2.2.tgz",
|
||||
@@ -6858,15 +6732,15 @@
|
||||
}
|
||||
},
|
||||
"node_modules/glob": {
|
||||
"version": "13.0.6",
|
||||
"resolved": "https://registry.npmjs.org/glob/-/glob-13.0.6.tgz",
|
||||
"integrity": "sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==",
|
||||
"version": "14.0.3",
|
||||
"resolved": "https://registry.npmjs.org/glob/-/glob-14.0.3.tgz",
|
||||
"integrity": "sha512-A1vjIwC3qpdCso45Iw127gnp7sPhqgY8lUQhDrArIs+GbmfaaVPFuoyy7WyLT9EPZCRZ7j7aLdwUhQmHWesEow==",
|
||||
"dev": true,
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
"minimatch": "^10.2.2",
|
||||
"minimatch": "^11.0.2",
|
||||
"minipass": "^7.1.3",
|
||||
"path-scurry": "^2.0.2"
|
||||
"path-scurry": "^3.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": "18 || 20 || >=22"
|
||||
@@ -8552,12 +8426,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/minimatch": {
|
||||
"version": "10.2.6",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.6.tgz",
|
||||
"integrity": "sha512-vpLQEs+VLCr1nU0BXS07maYoFwlDAH0gngQuuttxIwutDFEMHq2blX+8vpgxDdK3J1PwjCJiep77OitTZ4Ll1A==",
|
||||
"version": "11.0.3",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-11.0.3.tgz",
|
||||
"integrity": "sha512-iny/40PEGkNJQ5/Pv6z3nm6FbRAby32GeZRjCUJk2WnNDbgPSTcdI+DDvLqsgBdD4cD5sMtMheqPPnehCCa8Nw==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
"brace-expansion": "^5.0.8"
|
||||
"@isaacs/nonrecursive-glob-match": "^1.0.3",
|
||||
"brace-expansion": "^5.0.12"
|
||||
},
|
||||
"engines": {
|
||||
"node": "18 || 20 || >=22"
|
||||
@@ -9014,9 +8889,9 @@
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/path-scurry": {
|
||||
"version": "2.0.2",
|
||||
"resolved": "https://registry.npmjs.org/path-scurry/-/path-scurry-2.0.2.tgz",
|
||||
"integrity": "sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==",
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/path-scurry/-/path-scurry-3.0.0.tgz",
|
||||
"integrity": "sha512-1pTbQqH1wIwMi/eleQ5leFTTCkzFsPcgXNjwOA8YGGW/W4/kHT3dssE7Z6NH+zuqNLOzOB0uaD61yl7cAcVhgg==",
|
||||
"dev": true,
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
@@ -11789,7 +11664,7 @@
|
||||
"dependencies": {
|
||||
"@aws-crypto/sha256-js": "5.2.0",
|
||||
"chokidar": "^4.0.0",
|
||||
"minimatch": "^10.2.5",
|
||||
"minimatch": "^11.0.3",
|
||||
"octagonal-wheels": "0.1.56",
|
||||
"pouchdb-adapter-http": "^9.0.0",
|
||||
"pouchdb-adapter-leveldb": "^9.0.0",
|
||||
|
||||
+23
-2
@@ -79,6 +79,7 @@
|
||||
"test:e2e:obsidian:couchdb-manual-setup-workflow": "tsx test/e2e-obsidian/scripts/couchdb-manual-setup-workflow.ts",
|
||||
"test:e2e:obsidian:cli-to-obsidian-sync": "tsx test/e2e-obsidian/scripts/cli-to-obsidian-sync.ts",
|
||||
"test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts",
|
||||
"test:e2e:obsidian:journal-nfc-resend": "tsx test/e2e-obsidian/scripts/journal-nfc-resend.ts",
|
||||
"test:e2e:obsidian:object-storage-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts",
|
||||
"test:e2e:obsidian:object-storage-compatible-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --compatible",
|
||||
"test:e2e:obsidian:object-storage-qr-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --qr",
|
||||
@@ -135,7 +136,8 @@
|
||||
"test:e2e:obsidian:third-party-chunks": "tsx test/e2e-obsidian/scripts/third-party-chunks.ts",
|
||||
"test:e2e:obsidian:upgrade-livesync-batching": "tsx test/e2e-obsidian/scripts/upgrade-livesync-batching.ts",
|
||||
"pretest:browser-ui-lifecycle": "svelte-check --tsconfig test/browser-apps/ui-lifecycle/tsconfig.json --fail-on-warnings && vite build --config test/browser-apps/ui-lifecycle/vite.config.ts",
|
||||
"test:browser-ui-lifecycle": "deno test -A --no-check --frozen --config test/browser-apps/deno.json --lock test/browser-apps/deno.lock test/browser-apps/ui-lifecycle/browser-ui-lifecycle.test.ts"
|
||||
"test:browser-ui-lifecycle": "deno test -A --no-check --frozen --config test/browser-apps/deno.json --lock test/browser-apps/deno.lock test/browser-apps/ui-lifecycle/browser-ui-lifecycle.test.ts",
|
||||
"test:e2e:obsidian:p2p-nfc": "tsx test/e2e-obsidian/scripts/run-compose-p2p-nfc.ts"
|
||||
},
|
||||
"keywords": [],
|
||||
"author": "vorotamoroz",
|
||||
@@ -216,7 +218,7 @@
|
||||
"diff-match-patch": "^1.0.5",
|
||||
"idb": "^8.0.3",
|
||||
"markdown-it": "^14.3.3",
|
||||
"minimatch": "^10.2.5",
|
||||
"minimatch": "^11.0.3",
|
||||
"obsidian": "^1.13.1",
|
||||
"octagonal-wheels": "0.1.56",
|
||||
"qrcode-generator": "^1.4.4",
|
||||
@@ -234,6 +236,25 @@
|
||||
},
|
||||
"pouchdb-utils": {
|
||||
"uuid": "11.1.1"
|
||||
},
|
||||
"@typescript-eslint/typescript-estree": {
|
||||
"minimatch": "$minimatch"
|
||||
},
|
||||
"eslint-plugin-n": {
|
||||
"minimatch": "$minimatch"
|
||||
},
|
||||
"test-exclude": {
|
||||
"glob": "14.0.3",
|
||||
"minimatch": "$minimatch"
|
||||
},
|
||||
"eslint": {
|
||||
"minimatch": "$minimatch"
|
||||
},
|
||||
"@eslint/config-array": {
|
||||
"minimatch": "$minimatch"
|
||||
},
|
||||
"@eslint/eslintrc": {
|
||||
"minimatch": "$minimatch"
|
||||
}
|
||||
},
|
||||
"workspaces": [
|
||||
|
||||
@@ -418,7 +418,8 @@ The CLI uses the same settings format as the Obsidian plug-in. Create a `.livesy
|
||||
#### Unicode path identity
|
||||
|
||||
The `pathNormalizationVersion` setting controls how file paths identify
|
||||
Metadata documents when using CouchDB. An absent value or `0` keeps the legacy
|
||||
Metadata documents. CouchDB support is released; the unreleased extension also
|
||||
supports Object Storage and P2P. An absent value or `0` keeps the legacy
|
||||
identity. Version `1` uses Unicode Normalization Form C (NFC) for Metadata IDs
|
||||
and path comparison keys. It does not rewrite `Metadata.path`, rename existing
|
||||
files, or change Chunks or file content. `handleFilenameCaseSensitive` remains
|
||||
@@ -442,8 +443,11 @@ with `--force --write-settings`. Set the same value on each remaining CLI
|
||||
client, then run `fetch` with `--force --write-settings`. Use **Overwrite
|
||||
Server Data with This Device's Files** and **Reset Synchronisation on This
|
||||
Device** for the corresponding Obsidian Rebuild and Fetch operations. Resolve
|
||||
any ambiguous names before reconstruction. Version `1` is supported with
|
||||
CouchDB only; do not resume a legacy client against the rebuilt database.
|
||||
any ambiguous names before reconstruction. Object Storage Rebuild replaces the
|
||||
remote journals as well as the local database. P2P Rebuild prepares only the
|
||||
source local database; keep that peer online for the other devices to Fetch.
|
||||
Every P2P participant must be updated and migrated. Do not resume a legacy
|
||||
client against the rebuilt database.
|
||||
|
||||
### Command-line Reference
|
||||
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { NodeStorageAdapter, fsPromises, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { StandardIo } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
import { createDefaultCliSettings } from "@/apps/cli/cliSettingsDefaults";
|
||||
import { runCommand } from "@/apps/cli/commands/runCommand";
|
||||
import type { CLICommand, CLICommandContext } from "@/apps/cli/commands/types";
|
||||
import { main, type CliCommandRunner } from "@/apps/cli/main";
|
||||
|
||||
type StoredFileSnapshot = {
|
||||
id: string;
|
||||
revision: string;
|
||||
path: string;
|
||||
size: number;
|
||||
children: string[];
|
||||
content: string;
|
||||
};
|
||||
|
||||
describe("CLI NFC scan failure recovery", () => {
|
||||
const originalArgv = process.argv.slice();
|
||||
const originalExitCode = process.exitCode;
|
||||
const signalHandlers = new Map<"SIGINT" | "SIGTERM", Set<Parameters<typeof process.removeListener>[1]>>();
|
||||
let directory: string;
|
||||
let vaultPath: string;
|
||||
let settingsPath: string;
|
||||
let standardIo: StandardIo;
|
||||
|
||||
afterEach(async () => {
|
||||
for (const [signal, originalHandlers] of signalHandlers) {
|
||||
for (const handler of process.listeners(signal)) {
|
||||
if (!originalHandlers.has(handler)) process.removeListener(signal, handler);
|
||||
}
|
||||
}
|
||||
signalHandlers.clear();
|
||||
process.argv = originalArgv.slice();
|
||||
process.exitCode = originalExitCode;
|
||||
vi.restoreAllMocks();
|
||||
if (directory) await fsPromises.rm(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
async function start(command: CLICommand, runner: CliCommandRunner, exitCode: number) {
|
||||
process.exitCode = undefined;
|
||||
process.argv = [
|
||||
"node",
|
||||
"livesync-cli",
|
||||
path.join(directory, "db"),
|
||||
"--vault",
|
||||
vaultPath,
|
||||
"--settings",
|
||||
settingsPath,
|
||||
command,
|
||||
];
|
||||
await expect(main(standardIo, runner)).rejects.toThrow(`__EXIT__:${exitCode}`);
|
||||
}
|
||||
|
||||
async function snapshotFile(context: CLICommandContext, targetPath: string): Promise<StoredFileSnapshot> {
|
||||
for await (const doc of context.core.services.database.localDatabase.findAllNormalDocs({ conflicts: true }) || []) {
|
||||
if (stripAllPrefixes(doc.path) !== targetPath) continue;
|
||||
const file = await context.core.serviceModules.databaseFileAccess.fetch(targetPath as FilePathWithPrefix);
|
||||
if (!file || file.deleted) throw new Error(`Expected readable database content for ${targetPath}`);
|
||||
const body = file.body;
|
||||
if (body.type !== "text/plain") throw new Error(`Expected text database content for ${targetPath}`);
|
||||
return {
|
||||
id: doc._id,
|
||||
revision: doc._rev ?? "",
|
||||
path: doc.path,
|
||||
size: doc.size,
|
||||
children: [...doc.children],
|
||||
content: await body.text(),
|
||||
};
|
||||
}
|
||||
throw new Error(`Expected a stored Metadata document for ${targetPath}`);
|
||||
}
|
||||
|
||||
it("does not accept a partially enumerated Vault as complete and keeps its PouchDB Metadata", async () => {
|
||||
directory = await fsPromises.mkdtemp(path.join(os.tmpdir(), "livesync-cli-nfc-scan-integration-"));
|
||||
vaultPath = path.join(directory, "vault");
|
||||
settingsPath = path.join(directory, "settings.json");
|
||||
const databasePath = path.join(directory, "db");
|
||||
const knownPath = "notes/known.md";
|
||||
const blockedPath = "notes/blocked/new.md";
|
||||
const knownContent = "stored before the failed inventory\n";
|
||||
const blockedContent = "available after recovery\n";
|
||||
|
||||
await fsPromises.mkdir(path.join(vaultPath, "notes"), { recursive: true });
|
||||
await fsPromises.mkdir(databasePath);
|
||||
await fsPromises.writeFile(path.join(vaultPath, knownPath), knownContent);
|
||||
await fsPromises.writeFile(
|
||||
settingsPath,
|
||||
JSON.stringify({
|
||||
...createDefaultCliSettings(),
|
||||
isConfigured: true,
|
||||
pathNormalizationVersion: 1,
|
||||
})
|
||||
);
|
||||
standardIo = {
|
||||
readStdin: vi.fn(async () => ""),
|
||||
prompt: vi.fn(async () => ""),
|
||||
writeStdout: vi.fn(),
|
||||
writeStderr: vi.fn(),
|
||||
};
|
||||
for (const signal of ["SIGINT", "SIGTERM"] as const) {
|
||||
signalHandlers.set(signal, new Set(process.listeners(signal)));
|
||||
}
|
||||
vi.spyOn(process, "exit").mockImplementation((code) => {
|
||||
throw new Error(`__EXIT__:${code ?? 0}`);
|
||||
});
|
||||
|
||||
let seeded: StoredFileSnapshot | undefined;
|
||||
const seedMirror = vi.fn<CliCommandRunner>(async (options, context) => {
|
||||
const result = await runCommand(options, context);
|
||||
seeded = await snapshotFile(context, knownPath);
|
||||
return result;
|
||||
});
|
||||
await start("mirror", seedMirror, 0);
|
||||
expect(seedMirror).toHaveBeenCalledOnce();
|
||||
expect(seeded).toMatchObject({ path: knownPath, content: knownContent });
|
||||
|
||||
await fsPromises.mkdir(path.join(vaultPath, "notes", "blocked"));
|
||||
await fsPromises.writeFile(path.join(vaultPath, blockedPath), blockedContent);
|
||||
|
||||
const scanFailure = Object.assign(new Error("Storage permission denied"), { code: "EACCES" });
|
||||
const directoryReads: string[] = [];
|
||||
const parentDirectoryEntries: string[] = [];
|
||||
// Retain the prototype implementation before the spy replaces it; invoke it with each real adapter below.
|
||||
const originalReadDirectory = Reflect.get(
|
||||
NodeStorageAdapter.prototype,
|
||||
"readDirectory"
|
||||
);
|
||||
const readDirectory = vi.spyOn(NodeStorageAdapter.prototype, "readDirectory").mockImplementation(
|
||||
async function (this: NodeStorageAdapter, relativePath) {
|
||||
directoryReads.push(relativePath);
|
||||
if (relativePath === "notes/blocked") throw scanFailure;
|
||||
const entries = await originalReadDirectory.call(this, relativePath);
|
||||
if (relativePath === "notes") {
|
||||
parentDirectoryEntries.push(...(entries ?? []).map((entry) => entry.name));
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
);
|
||||
const failedMirror = vi.fn<CliCommandRunner>(async (options, context) => await runCommand(options, context));
|
||||
try {
|
||||
await start("mirror", failedMirror, 1);
|
||||
} finally {
|
||||
readDirectory.mockRestore();
|
||||
}
|
||||
|
||||
expect(failedMirror).not.toHaveBeenCalled();
|
||||
expect(directoryReads).toContain("");
|
||||
expect(directoryReads).toContain("notes");
|
||||
expect(directoryReads).toContain("notes/blocked");
|
||||
expect(parentDirectoryEntries).toContain("known.md");
|
||||
expect(parentDirectoryEntries).toContain("blocked");
|
||||
|
||||
let afterFailure: StoredFileSnapshot | undefined;
|
||||
const inspectAfterFailure = vi.fn<CliCommandRunner>(async (options, context) => {
|
||||
const result = await runCommand(options, context);
|
||||
afterFailure = await snapshotFile(context, knownPath);
|
||||
return result;
|
||||
});
|
||||
await start("ls", inspectAfterFailure, 0);
|
||||
expect(afterFailure).toEqual(seeded);
|
||||
expect(await fsPromises.readFile(path.join(vaultPath, knownPath), "utf8")).toBe(knownContent);
|
||||
expect(await fsPromises.readFile(path.join(vaultPath, blockedPath), "utf8")).toBe(blockedContent);
|
||||
|
||||
let recoveredKnown: StoredFileSnapshot | undefined;
|
||||
let recoveredNewFile: StoredFileSnapshot | undefined;
|
||||
const recoverMirror = vi.fn<CliCommandRunner>(async (options, context) => {
|
||||
const result = await runCommand(options, context);
|
||||
recoveredKnown = await snapshotFile(context, knownPath);
|
||||
recoveredNewFile = await snapshotFile(context, blockedPath);
|
||||
return result;
|
||||
});
|
||||
await start("mirror", recoverMirror, 0);
|
||||
expect(recoverMirror).toHaveBeenCalledOnce();
|
||||
expect(recoveredKnown).toEqual(seeded);
|
||||
expect(recoveredNewFile).toMatchObject({ path: blockedPath, content: blockedContent });
|
||||
});
|
||||
});
|
||||
@@ -9,6 +9,12 @@ import type { NodeFile, NodeFolder, NodeStat } from "./NodeTypes";
|
||||
import { path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { CliDiagnosticReporter } from "@/apps/cli/cliOutput";
|
||||
import { findCaseInsensitiveFile } from "@/apps/caseInsensitiveFileLookup";
|
||||
import {
|
||||
createStoragePathResolver,
|
||||
parseFilePath,
|
||||
type StoragePath,
|
||||
type StoragePathResolver,
|
||||
} from "@vrtmrz/livesync-commonlib/storage";
|
||||
|
||||
/**
|
||||
* Complete file system adapter implementation for Node.js
|
||||
@@ -21,6 +27,7 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
readonly vault: NodeVaultAdapter;
|
||||
|
||||
private fileCache = new Map<string, NodeFile>();
|
||||
private readonly isCaseInsensitiveSetting: () => boolean;
|
||||
|
||||
constructor(
|
||||
private basePath: string,
|
||||
@@ -31,6 +38,7 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
this.path = new NodePathAdapter();
|
||||
this.typeGuard = new NodeTypeGuardAdapter();
|
||||
this.conversion = new NodeConversionAdapter();
|
||||
this.isCaseInsensitiveSetting = isCaseInsensitive;
|
||||
this.storage = new UnicodeResolvingNodeStorageAdapter(basePath, getPathNormalizationVersion, isCaseInsensitive);
|
||||
this.vault = new NodeVaultAdapter(this.storage);
|
||||
}
|
||||
@@ -39,8 +47,8 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
return path.join(this.basePath, p);
|
||||
}
|
||||
|
||||
private normalisePath(p: FilePath | string): string {
|
||||
return this.path.normalisePath(p);
|
||||
private normalisePath(p: FilePath | string): FilePath {
|
||||
return parseFilePath(this.path.normalisePath(p));
|
||||
}
|
||||
|
||||
private async hasExactPathCase(pathStr: string): Promise<boolean> {
|
||||
@@ -122,15 +130,28 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
|
||||
async refreshFile(p: string, caseInsensitive = false): Promise<NodeFile | null> {
|
||||
const pathStr = this.normalisePath(p);
|
||||
if (this.storage.isUnicodePathNormalizationEnabled) {
|
||||
try {
|
||||
const resolvedPath = await this.storage.resolveLogicalPath(pathStr, caseInsensitive, false);
|
||||
if (resolvedPath === null) {
|
||||
this.fileCache.delete(pathStr);
|
||||
return null;
|
||||
}
|
||||
return await this.refreshResolvedFile(resolvedPath);
|
||||
} catch (error) {
|
||||
this.fileCache.delete(pathStr);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
try {
|
||||
const resolvedPath = this.storage.isUnicodePathNormalizationEnabled
|
||||
? await this.storage.resolveLogicalPath(pathStr, caseInsensitive, false)
|
||||
: pathStr;
|
||||
if (resolvedPath === null) {
|
||||
const stat = await this.storage.stat(pathStr, caseInsensitive);
|
||||
if (stat?.type !== "file") {
|
||||
this.fileCache.delete(pathStr);
|
||||
return null;
|
||||
}
|
||||
return await this.refreshResolvedFile(resolvedPath);
|
||||
const file: NodeFile = { path: pathStr, stat };
|
||||
this.fileCache.set(pathStr, file);
|
||||
return file;
|
||||
} catch (error) {
|
||||
if (error instanceof AmbiguousNormalisedPathError) throw error;
|
||||
this.fileCache.delete(pathStr);
|
||||
@@ -138,16 +159,16 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
}
|
||||
}
|
||||
|
||||
private async refreshResolvedFile(resolvedPath: string): Promise<NodeFile | null> {
|
||||
private async refreshResolvedFile(resolvedPath: StoragePath): Promise<NodeFile | null> {
|
||||
try {
|
||||
const stat = await this.storage.statExact(resolvedPath);
|
||||
const stat = await this.storage.statResolved(resolvedPath);
|
||||
if (stat?.type !== "file") {
|
||||
this.fileCache.delete(resolvedPath);
|
||||
return null;
|
||||
}
|
||||
|
||||
const file: NodeFile = {
|
||||
path: resolvedPath as FilePath,
|
||||
path: resolvedPath,
|
||||
stat: {
|
||||
size: stat.size,
|
||||
mtime: stat.mtime,
|
||||
@@ -158,10 +179,8 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
this.fileCache.set(resolvedPath, file);
|
||||
return file;
|
||||
} catch (error) {
|
||||
if (error instanceof AmbiguousNormalisedPathError) throw error;
|
||||
// Evict so a deleted file is not returned by subsequent cache scans.
|
||||
this.fileCache.delete(resolvedPath);
|
||||
return null;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -169,34 +188,83 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
* Helper method to recursively scan directory and populate file cache
|
||||
*/
|
||||
async scanDirectory(relativePath: string = ""): Promise<void> {
|
||||
const resolvedPath = await this.storage.resolveLogicalPath(relativePath, false, false);
|
||||
if (resolvedPath !== null) await this.scanDirectoryInto(resolvedPath, this.fileCache);
|
||||
const files = new Map<string, NodeFile>();
|
||||
await this.scanDirectoryInto(relativePath, files);
|
||||
for (const [path, file] of files) this.fileCache.set(path, file);
|
||||
}
|
||||
|
||||
private async scanDirectoryInto(relativePath: string, files: Map<string, NodeFile>): Promise<void> {
|
||||
const resolver = this.createScanResolver();
|
||||
const filePath = parseFilePath(relativePath);
|
||||
if (resolver) {
|
||||
const resolvedPath =
|
||||
filePath === "" ? await resolver.resolveForWrite(filePath) : await resolver.resolveExisting(filePath);
|
||||
if (resolvedPath !== null) await this.scanResolvedDirectoryInto(resolver, resolvedPath, files);
|
||||
return;
|
||||
}
|
||||
await this.scanLegacyDirectoryInto(filePath, files);
|
||||
}
|
||||
|
||||
private createScanResolver(): StoragePathResolver | null {
|
||||
if (!this.storage.isUnicodePathNormalizationEnabled) return null;
|
||||
const caseInsensitive = this.isCaseInsensitiveSetting();
|
||||
return createStoragePathResolver({
|
||||
readDirectory: async (physicalDirectory) => await this.storage.readDirectory(physicalDirectory),
|
||||
policy: {
|
||||
pathNormalizationVersion: 1,
|
||||
handleFilenameCaseSensitive: !caseInsensitive,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
private async scanResolvedDirectoryInto(
|
||||
resolver: StoragePathResolver,
|
||||
relativePath: StoragePath,
|
||||
files: Map<string, NodeFile>
|
||||
): Promise<void> {
|
||||
const directoryStat = await this.storage.statResolved(relativePath);
|
||||
if (directoryStat === null) return;
|
||||
if (directoryStat.type !== "folder") {
|
||||
throw new Error(`Path is not a directory: ${this.resolvePath(relativePath)}`);
|
||||
}
|
||||
|
||||
const entries = await resolver.listResolvedDirectory(relativePath);
|
||||
if (entries === null) return;
|
||||
for (const entry of entries) {
|
||||
if (entry.kind === "directory") {
|
||||
await this.scanResolvedDirectoryInto(resolver, entry.path, files);
|
||||
continue;
|
||||
}
|
||||
|
||||
const stat = await this.storage.statResolved(entry.path);
|
||||
if (stat?.type !== "file") continue;
|
||||
const file: NodeFile = { path: entry.path, stat };
|
||||
files.set(entry.path, file);
|
||||
}
|
||||
}
|
||||
|
||||
private async scanLegacyDirectoryInto(relativePath: string, files: Map<string, NodeFile>): Promise<void> {
|
||||
const fullPath = this.resolvePath(relativePath);
|
||||
try {
|
||||
const directoryStat = this.storage.isUnicodePathNormalizationEnabled
|
||||
? await this.storage.statExact(relativePath)
|
||||
: await this.storage.stat(relativePath, false);
|
||||
const directoryStat = await this.storage.stat(relativePath, false);
|
||||
if (directoryStat?.type !== "folder") throw new Error(`Directory does not exist: ${fullPath}`);
|
||||
const entries = await this.storage.listExact(relativePath);
|
||||
const entries = await this.storage.list(relativePath, false);
|
||||
|
||||
for (const entryPath of entries.files) {
|
||||
const stat = await this.storage.statExact(entryPath);
|
||||
const stat = await this.storage.stat(entryPath, false);
|
||||
if (stat?.type !== "file") continue;
|
||||
const filePath = parseFilePath(entryPath);
|
||||
const file: NodeFile = {
|
||||
path: entryPath as FilePath,
|
||||
path: filePath,
|
||||
stat,
|
||||
};
|
||||
files.set(entryPath, file);
|
||||
files.set(filePath, file);
|
||||
}
|
||||
for (const entryPath of entries.folders) {
|
||||
await this.scanDirectoryInto(entryPath, files);
|
||||
await this.scanLegacyDirectoryInto(entryPath, files);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof AmbiguousNormalisedPathError) throw error;
|
||||
// Directory doesn't exist or is not readable
|
||||
this.reportDiagnostic(`Error scanning directory ${fullPath}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { fsPromises as fs, NodeStorageAdapter, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { DEFAULT_SETTINGS, type FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { parseFilePath, type StoragePath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import {
|
||||
ServiceRebuilder,
|
||||
type ServiceRebuilderDependencies,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/Rebuilder";
|
||||
import { PathServiceCompat } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectablePathService";
|
||||
import { ServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
|
||||
import { AmbiguousNormalisedPathError } from "./UnicodeResolvingNodeStorageAdapter";
|
||||
import { NodeFileSystemAdapter } from "./NodeFileSystemAdapter";
|
||||
|
||||
describe("NodeFileSystemAdapter file enumeration", () => {
|
||||
@@ -49,6 +51,17 @@ describe("NodeFileSystemAdapter file enumeration", () => {
|
||||
expect((await adapter.getFiles()).map((file) => file.path).sort()).toEqual(paths);
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")(
|
||||
"compatibility: lists other files in Legacy mode when one root name is unsupported",
|
||||
async () => {
|
||||
const { directory, adapter } = await createVault();
|
||||
await fs.writeFile(path.join(directory, "a:unrelated.md"), "unrelated local bytes");
|
||||
|
||||
expect((await adapter.getFiles()).map((file) => file.path).sort()).toEqual(paths);
|
||||
expect(await fs.readFile(path.join(directory, "a:unrelated.md"), "utf8")).toBe("unrelated local bytes");
|
||||
}
|
||||
);
|
||||
|
||||
it("excludes a deleted file after its cache entry is refreshed", async () => {
|
||||
const { directory, adapter } = await createVault();
|
||||
await adapter.getFiles();
|
||||
@@ -219,6 +232,120 @@ describe("NodeFileSystemAdapter Unicode path resolution", () => {
|
||||
);
|
||||
}
|
||||
|
||||
it("propagates path-resolution failures when refreshing a file", async () => {
|
||||
const directory = await createVault();
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
const error = Object.assign(new Error("Storage permission denied"), { code: "EACCES" });
|
||||
vi.spyOn(adapter.storage, "resolveLogicalPath").mockRejectedValue(error);
|
||||
|
||||
await expect(adapter.refreshFile("a.md")).rejects.toBe(error);
|
||||
});
|
||||
|
||||
it("propagates strict metadata failures when refreshing a resolved file", async () => {
|
||||
const directory = await createVault();
|
||||
await fs.writeFile(path.join(directory, "a.md"), "content");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
const error = Object.assign(new Error("Storage permission denied"), { code: "EACCES" });
|
||||
vi.spyOn(adapter.storage, "statResolved").mockRejectedValue(error);
|
||||
|
||||
await expect(adapter.getAbstractFileByPath("a.md")).rejects.toBe(error);
|
||||
});
|
||||
|
||||
it("keeps resolved metadata reads strict when the identity mode changes during lookup", async () => {
|
||||
const directory = await createVault();
|
||||
await fs.writeFile(path.join(directory, "a.md"), "content");
|
||||
let version = 1;
|
||||
const adapter = new NodeFileSystemAdapter(directory, undefined, () => version);
|
||||
const error = Object.assign(new Error("Storage permission denied"), { code: "EACCES" });
|
||||
const resolve = adapter.storage.resolveLogicalPath.bind(adapter.storage);
|
||||
vi.spyOn(adapter.storage, "resolveLogicalPath").mockImplementation(async (...args) => {
|
||||
const result = await resolve(...args);
|
||||
version = 0;
|
||||
return result;
|
||||
});
|
||||
vi.spyOn(adapter.storage, "statResolved").mockRejectedValue(error);
|
||||
|
||||
await expect(adapter.getAbstractFileByPath("a.md")).rejects.toBe(error);
|
||||
});
|
||||
|
||||
it("propagates strict directory-read failures without reporting an incomplete scan", async () => {
|
||||
const directory = await createVault();
|
||||
const report = vi.fn();
|
||||
const adapter = new NodeFileSystemAdapter(directory, report, () => 1);
|
||||
const error = Object.assign(new Error("Storage permission denied"), { code: "EACCES" });
|
||||
vi.spyOn(adapter.storage, "readDirectory").mockRejectedValue(error);
|
||||
|
||||
await expect(adapter.getFiles()).rejects.toBe(error);
|
||||
expect(report).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not return or cache a partial NFC listing after a nested directory read fails", async () => {
|
||||
const directory = await createVault();
|
||||
const knownPath = "Known.md";
|
||||
const initialContent = "initial content";
|
||||
await fs.writeFile(path.join(directory, knownPath), initialContent);
|
||||
let normalizationVersion = 1;
|
||||
const adapter = new NodeFileSystemAdapter(directory, undefined, () => normalizationVersion, () => true);
|
||||
await adapter.scanDirectory();
|
||||
|
||||
const changedContent = "changed content after cache";
|
||||
await fs.writeFile(path.join(directory, knownPath), changedContent);
|
||||
await fs.mkdir(path.join(directory, "late-directory"));
|
||||
await fs.writeFile(path.join(directory, "late-directory", "later.md"), "later content");
|
||||
|
||||
const error = Object.assign(new Error("Storage permission denied"), { code: "EACCES" });
|
||||
const readDirectory = adapter.storage.readDirectory.bind(adapter.storage);
|
||||
const readDirectorySpy = vi.spyOn(adapter.storage, "readDirectory").mockImplementation(async (relativePath) => {
|
||||
if (relativePath === "late-directory") throw error;
|
||||
const entries = await readDirectory(relativePath);
|
||||
return relativePath === "" && entries !== null
|
||||
? [...entries].sort((left, right) => (left.name < right.name ? -1 : left.name > right.name ? 1 : 0))
|
||||
: entries;
|
||||
});
|
||||
|
||||
try {
|
||||
await expect(adapter.getFiles()).rejects.toBe(error);
|
||||
await expect(adapter.scanDirectory()).rejects.toBe(error);
|
||||
} finally {
|
||||
readDirectorySpy.mockRestore();
|
||||
}
|
||||
|
||||
// Compatibility-mode case-insensitive lookup exposes the cache without
|
||||
// triggering another scan, so it can verify that the failed NFC scan did
|
||||
// not publish its already-enumerated file metadata.
|
||||
normalizationVersion = 0;
|
||||
const scanDirectory = vi.spyOn(adapter, "scanDirectory");
|
||||
await expect(adapter.getAbstractFileByPathInsensitive("known.md")).resolves.toMatchObject({
|
||||
path: knownPath,
|
||||
stat: { size: initialContent.length },
|
||||
});
|
||||
expect(scanDirectory).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("uses one comparison policy snapshot for the whole NFC scan", async () => {
|
||||
const directory = await createVault();
|
||||
await fs.mkdir(path.join(directory, "Folder"));
|
||||
await fs.writeFile(path.join(directory, "Folder/Note.md"), "upper");
|
||||
await fs.writeFile(path.join(directory, "Folder/note.md"), "lower");
|
||||
let policyReads = 0;
|
||||
let normalizationReads = 0;
|
||||
const adapter = new NodeFileSystemAdapter(
|
||||
directory,
|
||||
undefined,
|
||||
() => {
|
||||
normalizationReads++;
|
||||
return 1;
|
||||
},
|
||||
() => ++policyReads > 1
|
||||
);
|
||||
|
||||
const files = await adapter.getFiles();
|
||||
|
||||
expect(files.map((file) => file.path).sort()).toEqual(["Folder/Note.md", "Folder/note.md"]);
|
||||
expect(normalizationReads).toBe(1);
|
||||
expect(policyReads).toBe(1);
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(tempDirs.splice(0).map((directory) => fs.rm(directory, { recursive: true, force: true })));
|
||||
});
|
||||
@@ -253,13 +380,13 @@ describe("NodeFileSystemAdapter Unicode path resolution", () => {
|
||||
);
|
||||
for (const name of names) await fs.writeFile(path.join(directory, child, name), "content");
|
||||
const adapter = createNormalisingAdapter(directory, caseInsensitive);
|
||||
const list = vi.spyOn(NodeStorageAdapter.prototype, "list");
|
||||
const readDirectory = vi.spyOn(NodeStorageAdapter.prototype, "readDirectory");
|
||||
try {
|
||||
const files = await adapter.getFiles();
|
||||
expect(files.map((file) => file.path).sort()).toEqual(names.map((name) => `${child}/${name}`).sort());
|
||||
expect(list).toHaveBeenCalledTimes(3);
|
||||
expect(readDirectory).toHaveBeenCalledTimes(3);
|
||||
} finally {
|
||||
list.mockRestore();
|
||||
readDirectory.mockRestore();
|
||||
}
|
||||
}
|
||||
);
|
||||
@@ -306,12 +433,12 @@ describe("NodeFileSystemAdapter Unicode path resolution", () => {
|
||||
await fs.writeFile(path.join(directory, `${index}.md`), "content");
|
||||
}
|
||||
const adapter = createNormalisingAdapter(directory, caseInsensitive);
|
||||
const list = vi.spyOn(NodeStorageAdapter.prototype, "list");
|
||||
const readDirectory = vi.spyOn(NodeStorageAdapter.prototype, "readDirectory");
|
||||
try {
|
||||
expect(await adapter.getFiles()).toHaveLength(12);
|
||||
expect(list).toHaveBeenCalledTimes(1);
|
||||
expect(readDirectory).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
list.mockRestore();
|
||||
readDirectory.mockRestore();
|
||||
}
|
||||
}
|
||||
);
|
||||
@@ -379,11 +506,11 @@ describe("NodeFileSystemAdapter Unicode path resolution", () => {
|
||||
await fs.writeFile(path.join(directory, nfdName), "NFD", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
const exactFile = {
|
||||
path: nfcName as FilePath,
|
||||
path: parseFilePath(nfcName),
|
||||
stat: { ctime: 1, mtime: 2, size: 3, type: "file" as const },
|
||||
};
|
||||
|
||||
await expect(adapter.getAbstractFileByPath(nfcName)).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.getAbstractFileByPath(nfcName)).rejects.toBeInstanceOf(AmbiguousNormalisedPathError);
|
||||
await expect(adapter.vault.modify(exactFile, "changed")).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.vault.delete(exactFile)).rejects.toThrow(/ambiguous/i);
|
||||
|
||||
@@ -545,7 +672,8 @@ describe("NodeFileSystemAdapter Unicode path resolution", () => {
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
await expect(adapter.getFiles()).resolves.toEqual([]);
|
||||
await expect(adapter.storage.statExact("linked/note.md")).rejects.toThrow(/symbolic links/i);
|
||||
await expect(adapter.storage.resolveLogicalPath(parseFilePath("linked/note.md"))).rejects.toThrow(/symbolic links/i);
|
||||
await expect(adapter.storage.statExact("linked/note.md" as StoragePath)).rejects.toThrow(/symbolic links/i);
|
||||
await expect(adapter.storage.write("linked/note.md", "changed")).rejects.toThrow(/symbolic links/i);
|
||||
await expect(fs.readFile(path.join(outside, "note.md"), "utf8")).resolves.toBe("outside");
|
||||
});
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import type { FilePath, UXDataWriteOptions } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { UXDataWriteOptions } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { IVaultAdapter } from "@vrtmrz/livesync-commonlib/compat/serviceModules/adapters";
|
||||
import type { NodeFile, NodeFolder } from "./NodeTypes";
|
||||
import { NodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { UnicodeResolvingNodeStorageAdapter } from "./UnicodeResolvingNodeStorageAdapter";
|
||||
import { NodeStorageAdapter, UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { parseFilePath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
|
||||
/**
|
||||
* Vault adapter implementation for Node.js
|
||||
@@ -56,7 +56,7 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
|
||||
|
||||
async rename(file: NodeFile, newPath: string): Promise<void> {
|
||||
await this.storage.rename(file.path, newPath);
|
||||
file.path = newPath as FilePath;
|
||||
file.path = parseFilePath(newPath);
|
||||
}
|
||||
|
||||
async delete(file: NodeFile | NodeFolder, force = false): Promise<void> {
|
||||
@@ -74,16 +74,17 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
|
||||
}
|
||||
|
||||
private async toNodeFile(path: string): Promise<NodeFile> {
|
||||
const resolvedPath =
|
||||
this.storage instanceof UnicodeResolvingNodeStorageAdapter
|
||||
? await this.storage.resolveLogicalPath(path, undefined, false)
|
||||
: path;
|
||||
if (resolvedPath === null) throw new Error(`Could not read created file metadata: ${path}`);
|
||||
const stat =
|
||||
this.storage instanceof UnicodeResolvingNodeStorageAdapter
|
||||
? await this.storage.statExact(resolvedPath)
|
||||
: await this.storage.stat(resolvedPath);
|
||||
const filePath = parseFilePath(path);
|
||||
if (this.storage instanceof UnicodeResolvingNodeStorageAdapter) {
|
||||
const resolvedPath = await this.storage.resolveLogicalPath(filePath, undefined, false);
|
||||
if (resolvedPath === null) throw new Error(`Could not read created file metadata: ${path}`);
|
||||
const stat = await this.storage.statExact(resolvedPath);
|
||||
if (stat?.type !== "file") throw new Error(`Could not read created file metadata: ${path}`);
|
||||
return { path: resolvedPath, stat };
|
||||
}
|
||||
|
||||
const stat = await this.storage.stat(filePath);
|
||||
if (stat?.type !== "file") throw new Error(`Could not read created file metadata: ${path}`);
|
||||
return { path: resolvedPath as FilePath, stat };
|
||||
return { path: filePath, stat };
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
import { fsPromises, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { parseFilePath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { NodeFileSystemAdapter } from "./NodeFileSystemAdapter";
|
||||
import { NodeVaultAdapter } from "./NodeVaultAdapter";
|
||||
|
||||
@@ -11,7 +11,7 @@ describe("NodeVaultAdapter.rename", () => {
|
||||
await fsPromises.writeFile(path.join(directory, "Calculus.md"), "content", "utf8");
|
||||
const adapter = new NodeVaultAdapter(directory);
|
||||
const file = {
|
||||
path: "Calculus.md" as FilePath,
|
||||
path: parseFilePath("Calculus.md"),
|
||||
stat: { ctime: 1, mtime: 2, size: 7, type: "file" as const },
|
||||
};
|
||||
|
||||
@@ -37,7 +37,7 @@ describe("NodeVaultAdapter.rename", () => {
|
||||
);
|
||||
const adapter = new NodeVaultAdapter(directory);
|
||||
const file = {
|
||||
path: "source.md" as FilePath,
|
||||
path: parseFilePath("source.md"),
|
||||
stat: { ctime: 1, mtime: 2, size: 7, type: "file" as const },
|
||||
};
|
||||
|
||||
@@ -65,7 +65,7 @@ describe("NodeVaultAdapter.rename", () => {
|
||||
);
|
||||
const adapter = new NodeVaultAdapter(directory);
|
||||
const file = {
|
||||
path: "linked/victim.md" as FilePath,
|
||||
path: parseFilePath("linked/victim.md"),
|
||||
stat: { ctime: 1, mtime: 2, size: 6, type: "file" as const },
|
||||
};
|
||||
|
||||
|
||||
@@ -1,205 +1,2 @@
|
||||
import { NodeStorageAdapter, validateStoragePath } from "@vrtmrz/livesync-commonlib/node";
|
||||
|
||||
export class AmbiguousNormalisedPathError extends Error {
|
||||
constructor(path: string) {
|
||||
super(`Ambiguous path after Unicode normalisation: ${path}`);
|
||||
this.name = "AmbiguousNormalisedPathError";
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolves NFC-equivalent logical paths against current Node storage listings. */
|
||||
export class UnicodeResolvingNodeStorageAdapter extends NodeStorageAdapter {
|
||||
constructor(
|
||||
rootPath: string,
|
||||
private readonly getPathNormalizationVersion: () => number = () => 0,
|
||||
private readonly isCaseInsensitive: () => boolean = () => false
|
||||
) {
|
||||
super(rootPath);
|
||||
}
|
||||
|
||||
get isUnicodePathNormalizationEnabled(): boolean {
|
||||
return this.getPathNormalizationVersion() === 1;
|
||||
}
|
||||
|
||||
/** Read metadata for an entry already resolved or obtained from a storage listing. */
|
||||
async statExact(path: string) {
|
||||
return await super.trystat(path);
|
||||
}
|
||||
|
||||
/** Inspect a physical directory once before descending into its enumerated entries. */
|
||||
async listExact(path: string) {
|
||||
const entries = await super.list(path);
|
||||
if (this.isUnicodePathNormalizationEnabled) {
|
||||
const caseInsensitive = this.isCaseInsensitive();
|
||||
const names = new Map<string, string>();
|
||||
for (const entry of [...entries.files, ...entries.folders]) {
|
||||
const key = this.comparisonKey(entry, caseInsensitive);
|
||||
if (names.has(key)) throw new AmbiguousNormalisedPathError(entry);
|
||||
names.set(key, entry);
|
||||
}
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
async resolveLogicalPath(
|
||||
path: string,
|
||||
caseInsensitive = this.isCaseInsensitive(),
|
||||
allowMissing = true
|
||||
): Promise<string | null> {
|
||||
if (!this.isUnicodePathNormalizationEnabled) return path;
|
||||
|
||||
validateStoragePath(path, true);
|
||||
if (path === "") return "";
|
||||
|
||||
const segments = path.split("/");
|
||||
let resolvedParent = "";
|
||||
|
||||
for (let index = 0; index < segments.length; index++) {
|
||||
const segment = segments[index];
|
||||
const entries = await super.list(resolvedParent);
|
||||
const candidates = [...entries.files, ...entries.folders];
|
||||
const requestedKey = this.comparisonKey(segment, caseInsensitive);
|
||||
const matches = candidates.filter((entryPath) => {
|
||||
const entryName = entryPath.slice(resolvedParent === "" ? 0 : resolvedParent.length + 1);
|
||||
return this.comparisonKey(entryName, caseInsensitive) === requestedKey;
|
||||
});
|
||||
|
||||
if (matches.length > 1) throw new AmbiguousNormalisedPathError(path);
|
||||
if (matches.length === 1) {
|
||||
resolvedParent = matches[0];
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!allowMissing) return null;
|
||||
const remainingPath = segments.slice(index).join("/");
|
||||
return resolvedParent === "" ? remainingPath : `${resolvedParent}/${remainingPath}`;
|
||||
}
|
||||
|
||||
return resolvedParent;
|
||||
}
|
||||
|
||||
override async exists(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<boolean> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
if (resolvedPath === null) return false;
|
||||
return await super.exists(resolvedPath);
|
||||
}
|
||||
|
||||
override async trystat(path: string, caseInsensitive = this.isCaseInsensitive()) {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
if (resolvedPath === null) return null;
|
||||
return await super.trystat(resolvedPath);
|
||||
}
|
||||
|
||||
override async stat(path: string, caseInsensitive = this.isCaseInsensitive()) {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
if (resolvedPath === null) return null;
|
||||
return await super.trystat(resolvedPath);
|
||||
}
|
||||
|
||||
override async mkdir(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.mkdir(resolvedPath);
|
||||
}
|
||||
|
||||
override async remove(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
await super.remove(resolvedPath ?? path);
|
||||
}
|
||||
|
||||
override async read(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<string> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
return await super.read(resolvedPath ?? path);
|
||||
}
|
||||
|
||||
override async readBinary(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<ArrayBuffer> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
return await super.readBinary(resolvedPath ?? path);
|
||||
}
|
||||
|
||||
override async write(
|
||||
path: string,
|
||||
data: string,
|
||||
options?: Parameters<NodeStorageAdapter["write"]>[2],
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.write(resolvedPath, data, options);
|
||||
}
|
||||
|
||||
override async writeBinary(
|
||||
path: string,
|
||||
data: ArrayBuffer,
|
||||
options?: Parameters<NodeStorageAdapter["writeBinary"]>[2],
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.writeBinary(resolvedPath, data, options);
|
||||
}
|
||||
|
||||
override async append(
|
||||
path: string,
|
||||
data: string,
|
||||
options?: Parameters<NodeStorageAdapter["append"]>[2],
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.append(resolvedPath, data, options);
|
||||
}
|
||||
|
||||
override async rename(
|
||||
sourcePath: string,
|
||||
targetPath: string,
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedSource = await this.resolveLogicalPath(sourcePath, caseInsensitive, false);
|
||||
if (resolvedSource === null) return await super.rename(sourcePath, targetPath);
|
||||
|
||||
const resolvedTarget = await this.resolveLogicalPath(targetPath, caseInsensitive, true);
|
||||
if (resolvedTarget === null) throw new Error(`Could not resolve path: ${targetPath}`);
|
||||
|
||||
// Preserve explicit case-only or Unicode-spelling renames when the target resolves to the source.
|
||||
const sourceKey = this.comparisonKey(resolvedSource, caseInsensitive);
|
||||
const targetKey = this.comparisonKey(resolvedTarget, caseInsensitive);
|
||||
const target =
|
||||
sourceKey === targetKey && resolvedTarget === resolvedSource && targetPath !== resolvedSource
|
||||
? await this.resolveRenameTargetSpelling(targetPath, caseInsensitive)
|
||||
: resolvedTarget;
|
||||
|
||||
await super.rename(resolvedSource, target);
|
||||
}
|
||||
|
||||
override async list(basePath: string, caseInsensitive = this.isCaseInsensitive()) {
|
||||
const resolvedPath = await this.resolveLogicalPath(basePath, caseInsensitive, false);
|
||||
if (resolvedPath === null) return { files: [], folders: [] };
|
||||
return await super.list(resolvedPath);
|
||||
}
|
||||
|
||||
private async resolveRenameTargetSpelling(path: string, caseInsensitive: boolean): Promise<string> {
|
||||
validateStoragePath(path, false);
|
||||
const segments = path.split("/");
|
||||
const leaf = segments.pop();
|
||||
if (leaf === undefined) throw new Error(`Could not resolve path: ${path}`);
|
||||
const parentPath = segments.join("/");
|
||||
const resolvedParent = await this.resolveLogicalPath(parentPath, caseInsensitive, true);
|
||||
if (resolvedParent === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
|
||||
const entries = await super.list(resolvedParent);
|
||||
const requestedKey = this.comparisonKey(leaf, caseInsensitive);
|
||||
const matches = [...entries.files, ...entries.folders].filter((entryPath) => {
|
||||
const entryName = entryPath.slice(resolvedParent === "" ? 0 : resolvedParent.length + 1);
|
||||
return this.comparisonKey(entryName, caseInsensitive) === requestedKey;
|
||||
});
|
||||
if (matches.length > 1) throw new AmbiguousNormalisedPathError(path);
|
||||
|
||||
return resolvedParent === "" ? leaf : `${resolvedParent}/${leaf}`;
|
||||
}
|
||||
|
||||
private comparisonKey(segment: string, caseInsensitive: boolean): string {
|
||||
const normalisedSegment = segment.normalize("NFC");
|
||||
return caseInsensitive ? normalisedSegment.toLowerCase() : normalisedSegment;
|
||||
}
|
||||
}
|
||||
export { UnicodeResolvingNodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
export { AmbiguousNormalisedPathError } from "@vrtmrz/livesync-commonlib/storage";
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { createNewVaultSettings } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { createNewVaultSettings } from "@/common/newVaultSettings.ts";
|
||||
|
||||
export function createDefaultCliSettings(): ObsidianLiveSyncSettings {
|
||||
return {
|
||||
|
||||
@@ -7,10 +7,14 @@ describe("createDefaultCliSettings", () => {
|
||||
const settings = createDefaultCliSettings();
|
||||
const recommended = createNewVaultSettings();
|
||||
|
||||
expect(settings.pathNormalizationVersion).toBe(1);
|
||||
expect(settings).toEqual({
|
||||
...recommended,
|
||||
pathNormalizationVersion: 1,
|
||||
useIndexedDBAdapter: false,
|
||||
isConfigured: false,
|
||||
});
|
||||
expect(recommended.pathNormalizationVersion).toBe(0);
|
||||
expect(settings.remoteConfigurations).not.toBe(recommended.remoteConfigurations);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,20 @@
|
||||
import {
|
||||
REMOTE_COUCHDB,
|
||||
REMOTE_MINIO,
|
||||
type ObsidianLiveSyncSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { RemoteConnectionProbe } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { resolveInitialPathIdentity } from "@/common/initialPathIdentity";
|
||||
|
||||
/** Adopt an existing central remote only for the first explicit Fetch. */
|
||||
export async function resolveInitialFetchPathIdentity(
|
||||
settings: ObsidianLiveSyncSettings,
|
||||
createProbe: () => Promise<RemoteConnectionProbe | undefined>
|
||||
): Promise<0 | 1 | undefined> {
|
||||
if (settings.isConfigured) return undefined;
|
||||
const hasRemote =
|
||||
(settings.remoteType === REMOTE_COUCHDB && !!settings.couchDB_URI && !!settings.couchDB_DBNAME) ||
|
||||
(settings.remoteType === REMOTE_MINIO && !!settings.endpoint && !!settings.bucket);
|
||||
if (!hasRemote) return undefined;
|
||||
return await resolveInitialPathIdentity(await createProbe(), settings.pathNormalizationVersion);
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { REMOTE_COUCHDB, REMOTE_MINIO, REMOTE_P2P } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { createDefaultCliSettings } from "./cliSettingsDefaults";
|
||||
import { resolveInitialFetchPathIdentity } from "./initialFetchPathIdentity";
|
||||
|
||||
function fixture() {
|
||||
const settings = {
|
||||
...createDefaultCliSettings(),
|
||||
couchDB_URI: "https://db.test",
|
||||
couchDB_DBNAME: "vault",
|
||||
endpoint: "https://objects.test",
|
||||
bucket: "vault",
|
||||
};
|
||||
const probe = {
|
||||
check: vi.fn(),
|
||||
getStatus: vi.fn(),
|
||||
inspectPathIdentity: vi.fn(async () => ({
|
||||
status: "configured" as const,
|
||||
pathNormalizationVersion: 0 as const,
|
||||
})),
|
||||
dispose: vi.fn(async () => undefined),
|
||||
};
|
||||
return { settings, probe, createProbe: vi.fn(async () => probe) };
|
||||
}
|
||||
|
||||
describe("initial CLI Fetch path identity", () => {
|
||||
it.each([REMOTE_COUCHDB, REMOTE_MINIO])("adopts a legacy %s remote before the first Fetch", async (remoteType) => {
|
||||
const { settings, probe, createProbe } = fixture();
|
||||
settings.remoteType = remoteType;
|
||||
await expect(resolveInitialFetchPathIdentity(settings, createProbe)).resolves.toBe(0);
|
||||
expect(settings.pathNormalizationVersion).toBe(1);
|
||||
expect(probe.dispose).toHaveBeenCalledOnce();
|
||||
});
|
||||
it("leaves an already configured device unchanged", async () => {
|
||||
const { settings, createProbe } = fixture();
|
||||
settings.isConfigured = true;
|
||||
await expect(resolveInitialFetchPathIdentity(settings, createProbe)).resolves.toBeUndefined();
|
||||
expect(createProbe).not.toHaveBeenCalled();
|
||||
});
|
||||
it.each(["unconfigured", "p2p"])("does not infer identity for %s settings", async (kind) => {
|
||||
const { settings, createProbe } = fixture();
|
||||
if (kind === "p2p") settings.remoteType = REMOTE_P2P;
|
||||
else settings.couchDB_URI = "";
|
||||
await expect(resolveInitialFetchPathIdentity(settings, createProbe)).resolves.toBeUndefined();
|
||||
expect(createProbe).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -5,6 +5,7 @@ import { join } from "node:path";
|
||||
import * as chokidar from "chokidar";
|
||||
import { path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import * as offlineScanner from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
import { ReplicatorService } from "@vrtmrz/livesync-commonlib/compat/services/base/ReplicatorService";
|
||||
import { ControlService } from "@vrtmrz/livesync-commonlib/compat/services/base/ControlService";
|
||||
import {
|
||||
AutoAccepting,
|
||||
@@ -333,6 +334,45 @@ describe("CLI database preparation", () => {
|
||||
expect(watch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("retains the adopted initial Fetch identity without --write-settings", async () => {
|
||||
await writeFile(
|
||||
settingsPath,
|
||||
JSON.stringify({
|
||||
...createDefaultCliSettings(),
|
||||
isConfigured: false,
|
||||
couchDB_URI: "https://example.test",
|
||||
couchDB_DBNAME: "vault",
|
||||
couchDB_USER: "user",
|
||||
couchDB_PASSWORD: "pass",
|
||||
})
|
||||
);
|
||||
const dispose = vi.fn(async () => undefined);
|
||||
const inspect = vi.spyOn(ReplicatorService.prototype, "createRemoteResource").mockResolvedValue({
|
||||
check: vi.fn(),
|
||||
getStatus: vi.fn(),
|
||||
inspectPathIdentity: vi.fn(async () => ({ status: "configured", pathNormalizationVersion: 0 }) as const),
|
||||
dispose,
|
||||
});
|
||||
await start(
|
||||
"fetch",
|
||||
async (_options, { core }) => {
|
||||
expect(core.services.setting.currentSettings().pathNormalizationVersion).toBe(0);
|
||||
return true;
|
||||
},
|
||||
0,
|
||||
true
|
||||
);
|
||||
const persisted = JSON.parse(await readFile(settingsPath, "utf-8"));
|
||||
expect(persisted.pathNormalizationVersion).toBe(0);
|
||||
expect(dispose).toHaveBeenCalledOnce();
|
||||
expect(inspect).toHaveBeenCalledOnce();
|
||||
await start("ls", async (_options, { core }) => {
|
||||
expect(core.services.setting.currentSettings().pathNormalizationVersion).toBe(0);
|
||||
return false;
|
||||
});
|
||||
expect(inspect).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it.each(["rebuild", "fetch"] as const)("does not scan or mirror legacy Vault files before %s", async (command) => {
|
||||
const storedPaths: string[] = [];
|
||||
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
||||
|
||||
@@ -30,6 +30,8 @@ import type { ReplicationSchedulingControl } from "@/serviceFeatures/replication
|
||||
import { createNodeStandardIo, fsPromises as fs, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { StandardIo } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { writeStderrLine, writeStdoutLine } from "./cliOutput";
|
||||
import { REMOTE_RESOURCE_KINDS } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { resolveInitialFetchPathIdentity } from "./initialFetchPathIdentity";
|
||||
import { createDefaultCliSettings } from "./cliSettingsDefaults";
|
||||
import {
|
||||
applyStoredSetting,
|
||||
@@ -601,6 +603,22 @@ export async function main(
|
||||
throw new Error("Replication scheduling was not provided during core feature composition.");
|
||||
}
|
||||
|
||||
let initialFetchPathIdentity: 0 | 1 | undefined;
|
||||
if (options.command === "fetch") {
|
||||
core.services.appLifecycle.onSettingLoaded.addHandler(async () => {
|
||||
const settings = core.services.setting.currentSettings();
|
||||
const version = await resolveInitialFetchPathIdentity(settings, () =>
|
||||
core.services.replicator.createRemoteResource(REMOTE_RESOURCE_KINDS.CONNECTION, settings)
|
||||
);
|
||||
if (version !== undefined) {
|
||||
initialFetchPathIdentity = version;
|
||||
await core.services.setting.applyPartial({ pathNormalizationVersion: version });
|
||||
await core.services.setting.saveSettingData();
|
||||
}
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
// Setup signal handlers for graceful shutdown
|
||||
const shutdown = async (signal: string) => {
|
||||
writeStdoutLine(standardIo);
|
||||
@@ -802,6 +820,7 @@ export async function main(
|
||||
: (JSON.parse(originalSettingsText) as ObsidianLiveSyncSettings);
|
||||
durable.additionalSuffixOfDatabaseName =
|
||||
core.services.setting.currentSettings().additionalSuffixOfDatabaseName;
|
||||
if (initialFetchPathIdentity !== undefined) durable.pathNormalizationVersion = initialFetchPathIdentity;
|
||||
settingsTextToCommit = JSON.stringify(durable, null, 2);
|
||||
}
|
||||
settingsTextToCommit ??= originalSettingsText;
|
||||
|
||||
@@ -39,7 +39,7 @@
|
||||
"dependencies": {
|
||||
"@aws-crypto/sha256-js": "5.2.0",
|
||||
"chokidar": "^4.0.0",
|
||||
"minimatch": "^10.2.5",
|
||||
"minimatch": "^11.0.3",
|
||||
"octagonal-wheels": "0.1.56",
|
||||
"pouchdb-adapter-http": "^9.0.0",
|
||||
"pouchdb-adapter-leveldb": "^9.0.0",
|
||||
|
||||
@@ -26,6 +26,17 @@ WebApp requires:
|
||||
|
||||
Automated browser coverage uses Chromium. Other browsers, background execution, and remote types other than CouchDB remain experimental. Use feature detection rather than relying on a fixed browser-version list.
|
||||
|
||||
## Path identity compatibility
|
||||
|
||||
WebApp currently supports **Legacy path identity** (`pathNormalizationVersion: 0`) only. **Unicode NFC path identity** (`1`) is not yet supported. WebApp rejects NFC settings before creating its services or opening its local database; it reports the reason and leaves the Vault files and settings unchanged. Use legacy identity in WebApp's `.livesync/settings.json` and on every client sharing its CouchDB database or P2P group, including new CLI and Obsidian environments which default to NFC.
|
||||
|
||||
The File System Access API can supply the directory listings and handles needed
|
||||
for NFC resolution. Commonlib provides that adapter, but WebApp has not yet
|
||||
integrated it with its file cache, inventory, and watcher behaviour. API-level
|
||||
support does not enable this application boundary automatically.
|
||||
|
||||
Changing the setting alone does not migrate an existing NFC database. Migration back to legacy identity requires a coordinated Rebuild and Fetch from a complete, authoritative Vault. Back up the Vault and database, and stop synchronisation on all clients before migrating. Keep WebApp disconnected until the migration is complete. See [Recovery and flag files](../../../docs/recovery.md) before rebuilding.
|
||||
|
||||
## Use
|
||||
|
||||
Serve the built files over HTTPS, or from `localhost`, then open `webapp.html`.
|
||||
@@ -44,6 +55,7 @@ For example, a minimal manually configured CouchDB connection includes:
|
||||
"couchDB_DBNAME": "vault",
|
||||
"isConfigured": true,
|
||||
"liveSync": true,
|
||||
"pathNormalizationVersion": 0,
|
||||
"syncOnSave": true
|
||||
}
|
||||
```
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
||||
/** Browser runtime for Self-hosted LiveSync over the File System Access API. */
|
||||
|
||||
@@ -55,6 +56,14 @@ const DEFAULT_SETTINGS: Partial<ObsidianLiveSyncSettings> = {
|
||||
autoSweepPluginsPeriodic: false,
|
||||
};
|
||||
|
||||
function assertWebAppPathIdentity(settings: Partial<ObsidianLiveSyncSettings> | null): void {
|
||||
if (getPathNormalizationVersion(settings ?? {}) === 1) {
|
||||
throw new Error(
|
||||
"WebApp supports Legacy path identity (pathNormalizationVersion: 0) only. Unicode NFC path identity is not supported."
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export type WebAppRuntimeStatusKind = "info" | "warning" | "error" | "success";
|
||||
|
||||
export interface WebAppRuntimeOptions {
|
||||
@@ -100,6 +109,7 @@ export class WebAppRuntime {
|
||||
getSystemVaultName: () => this.rootHandle.name || "livesync-webapp",
|
||||
settings: {
|
||||
save: async (data) => {
|
||||
assertWebAppPathIdentity(data);
|
||||
try {
|
||||
await this.saveSettingsToFile(data);
|
||||
this.addLog("Saved to .livesync/settings.json", LOG_LEVEL_VERBOSE, "settings");
|
||||
@@ -108,15 +118,17 @@ export class WebAppRuntime {
|
||||
}
|
||||
},
|
||||
load: async () => {
|
||||
let data: Partial<ObsidianLiveSyncSettings> | null = null;
|
||||
try {
|
||||
const data = await this.loadSettingsFromFile();
|
||||
if (data) {
|
||||
this.addLog("Loaded from .livesync/settings.json", LOG_LEVEL_VERBOSE, "settings");
|
||||
return { ...DEFAULT_SETTINGS, ...data } as ObsidianLiveSyncSettings;
|
||||
}
|
||||
data = await this.loadSettingsFromFile();
|
||||
} catch {
|
||||
this.addLog("Failed to load settings; using defaults", LOG_LEVEL_NOTICE, "settings");
|
||||
}
|
||||
assertWebAppPathIdentity(data);
|
||||
if (data) {
|
||||
this.addLog("Loaded from .livesync/settings.json", LOG_LEVEL_VERBOSE, "settings");
|
||||
return { ...DEFAULT_SETTINGS, ...data } as ObsidianLiveSyncSettings;
|
||||
}
|
||||
return DEFAULT_SETTINGS as ObsidianLiveSyncSettings;
|
||||
},
|
||||
},
|
||||
@@ -205,6 +217,13 @@ export class WebAppRuntime {
|
||||
throw new Error("The WebApp runtime has already been started");
|
||||
}
|
||||
|
||||
try {
|
||||
assertWebAppPathIdentity(await this.loadSettingsFromFile());
|
||||
} catch (error) {
|
||||
this.showError(`Failed to start: ${String(error)}`);
|
||||
throw error;
|
||||
}
|
||||
|
||||
// Create service context and hub
|
||||
this.serviceHub = createLiveSyncBrowserServiceHub<ServiceContext>(this.createServiceOptions());
|
||||
this.addLog("Self-hosted LiveSync WebApp", LOG_LEVEL_INFO, "initialise");
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
import type { RemoteConnectionProbe } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { withOwnedRemoteResource } from "./ownedRemoteResource";
|
||||
|
||||
/** Select initial settings from a read-only declaration before opening the local database. */
|
||||
export async function resolveInitialPathIdentity(
|
||||
probe: RemoteConnectionProbe | undefined,
|
||||
newEnvironmentVersion: number
|
||||
): Promise<0 | 1> {
|
||||
if (!probe) throw new Error("The remote path identity cannot be inspected.");
|
||||
return await withOwnedRemoteResource(probe, async (ownedProbe) => {
|
||||
if (!ownedProbe.inspectPathIdentity) throw new Error("The remote path identity cannot be inspected.");
|
||||
const result = await ownedProbe.inspectPathIdentity();
|
||||
return result.status === "empty" ? (newEnvironmentVersion === 1 ? 1 : 0) : result.pathNormalizationVersion;
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import type { PathIdentityInspection, RemoteConnectionProbe } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { resolveInitialPathIdentity } from "./initialPathIdentity";
|
||||
|
||||
function makeProbe(result: PathIdentityInspection) {
|
||||
return {
|
||||
check: vi.fn(),
|
||||
getStatus: vi.fn(),
|
||||
inspectPathIdentity: vi.fn(async () => result),
|
||||
dispose: vi.fn(async () => undefined),
|
||||
} satisfies RemoteConnectionProbe;
|
||||
}
|
||||
|
||||
describe("resolveInitialPathIdentity", () => {
|
||||
it.each([0, 1] as const)("adopts existing version %s instead of the new-environment default", async (version) => {
|
||||
const probe = makeProbe({ status: "configured", pathNormalizationVersion: version });
|
||||
await expect(resolveInitialPathIdentity(probe, 1 - version)).resolves.toBe(version);
|
||||
expect(probe.dispose).toHaveBeenCalledOnce();
|
||||
});
|
||||
it.each([0, 1] as const)("retains version %s only for a confirmed empty remote", async (version) => {
|
||||
const probe = makeProbe({ status: "empty" });
|
||||
await expect(resolveInitialPathIdentity(probe, version)).resolves.toBe(version);
|
||||
expect(probe.dispose).toHaveBeenCalledOnce();
|
||||
});
|
||||
it("preserves inspection failures and disposes the probe", async () => {
|
||||
const probe = makeProbe({ status: "empty" });
|
||||
const error = new Error("remote unavailable");
|
||||
probe.inspectPathIdentity.mockRejectedValue(error);
|
||||
await expect(resolveInitialPathIdentity(probe, 1)).rejects.toBe(error);
|
||||
expect(probe.dispose).toHaveBeenCalledOnce();
|
||||
});
|
||||
it("refuses an unavailable inspection capability without guessing", async () => {
|
||||
const probe = makeProbe({ status: "empty" });
|
||||
const { inspectPathIdentity: _inspect, ...unsupported } = probe;
|
||||
await expect(resolveInitialPathIdentity(unsupported, 1)).rejects.toThrow("cannot be inspected");
|
||||
expect(probe.dispose).toHaveBeenCalledOnce();
|
||||
await expect(resolveInitialPathIdentity(undefined, 1)).rejects.toThrow("cannot be inspected");
|
||||
});
|
||||
});
|
||||
@@ -7,6 +7,9 @@
|
||||
* remove it from this map in the same change.
|
||||
*/
|
||||
export const liveSyncProvisionalEnglishMessages = {
|
||||
"File identity": "File identity",
|
||||
"Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild from a complete authoritative Vault and Fetch on the other devices. With P2P, Rebuild prepares only the source device's local database. Resolve colliding file names first.":
|
||||
"Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild from a complete authoritative Vault and Fetch on the other devices. With P2P, Rebuild prepares only the source device's local database. Resolve colliding file names first.",
|
||||
"Synchronisation protection": "Synchronisation protection",
|
||||
"Synchronising your Vault": "Synchronising your Vault",
|
||||
"Receiving changes from the server…": "Receiving changes from the server…",
|
||||
|
||||
@@ -3167,7 +3167,7 @@ export const allMessages: Readonly<Record<string, Readonly<Record<string, string
|
||||
"zh-tw": "若啟用「自動啟動 P2P 連線」,外掛啟動時會自動建立 P2P 連線。",
|
||||
},
|
||||
"If disabled(toggled), chunks will be split on the UI thread (Previous behaviour).": {
|
||||
def: "If disabled(toggled), chunks will be split on the UI thread (Previous behaviour).",
|
||||
def: "When enabled, chunks are split on the UI thread (the previous behaviour).",
|
||||
es: "Si se desactiva, los chunks se dividen en el hilo de UI (comportamiento anterior)",
|
||||
fr: "Si désactivé, les fragments seront découpés sur le thread UI (comportement précédent).",
|
||||
he: "אם מכובה, נתחים יפוצלו בשרשור ממשק המשתמש (התנהגות קודמת).",
|
||||
|
||||
@@ -368,7 +368,7 @@
|
||||
"I understand the risks and will proceed without a backup.": "I understand the risks and will proceed without a backup.",
|
||||
"I Understand, Overwrite Server": "I Understand, Overwrite Server",
|
||||
"If \"Auto Start P2P Connection\" is enabled, the P2P connection will be started automatically when the plug-in launches.": "If \"Auto Start P2P Connection\" is enabled, the P2P connection will be started automatically when the plug-in launches.",
|
||||
"If disabled(toggled), chunks will be split on the UI thread (Previous behaviour).": "If disabled(toggled), chunks will be split on the UI thread (Previous behaviour).",
|
||||
"If disabled(toggled), chunks will be split on the UI thread (Previous behaviour).": "When enabled, chunks are split on the UI thread (the previous behaviour).",
|
||||
"If enabled per-filed efficient customization sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost a compatibility with old versions.": "If enabled per-filed efficient customization sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost a compatibility with old versions.",
|
||||
"If enabled, changes will be automatically broadcasted to all connected peers. Notified peers will start fetching the changes.": "If enabled, changes will be automatically broadcasted to all connected peers. Notified peers will start fetching the changes.",
|
||||
"If enabled, chunks will be split into no more than 100 items. However, dedupe is slightly weaker.": "If enabled, chunks will be split into no more than 100 items. However, dedupe is slightly weaker.",
|
||||
|
||||
@@ -437,8 +437,7 @@ If "Auto Start P2P Connection" is enabled, the P2P connection will be started au
|
||||
If "Auto Start P2P Connection" is enabled, the P2P connection will be
|
||||
started automatically when the plug-in launches.
|
||||
If disabled(toggled), chunks will be split on the UI thread (Previous behaviour).:
|
||||
If disabled(toggled), chunks will be split on the UI thread (Previous
|
||||
behaviour).
|
||||
When enabled, chunks are split on the UI thread (the previous behaviour).
|
||||
If enabled per-filed efficient customization sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost a compatibility with old versions.:
|
||||
If enabled per-filed efficient customization sync will be used. We need a
|
||||
small migration when enabling this. And all devices should be updated to
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
import { createNewVaultSettings as createCommonlibNewVaultSettings } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
|
||||
/** Create settings for a new LiveSync environment. */
|
||||
export function createNewVaultSettings(): ObsidianLiveSyncSettings {
|
||||
return {
|
||||
...createCommonlibNewVaultSettings(),
|
||||
pathNormalizationVersion: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
NEW_VAULT_SETTINGS,
|
||||
SETTINGS_SCHEMA_DEFAULTS,
|
||||
createNewVaultSettings as createCommonlibNewVaultSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { createNewVaultSettings } from "./newVaultSettings.ts";
|
||||
|
||||
describe("createNewVaultSettings", () => {
|
||||
it("selects NFC path identity only for new host settings", () => {
|
||||
expect(SETTINGS_SCHEMA_DEFAULTS.pathNormalizationVersion).toBe(0);
|
||||
expect(NEW_VAULT_SETTINGS.pathNormalizationVersion).toBe(0);
|
||||
expect(createCommonlibNewVaultSettings().pathNormalizationVersion).toBe(0);
|
||||
expect(createNewVaultSettings().pathNormalizationVersion).toBe(1);
|
||||
});
|
||||
|
||||
it("keeps remote configurations independent of shared and other new-Vault settings", () => {
|
||||
const first = createNewVaultSettings();
|
||||
const second = createNewVaultSettings();
|
||||
const shared = createCommonlibNewVaultSettings();
|
||||
|
||||
expect(first.remoteConfigurations).not.toBe(second.remoteConfigurations);
|
||||
expect(first.remoteConfigurations).not.toBe(shared.remoteConfigurations);
|
||||
expect(first.pluginSyncExtendedSetting).not.toBe(second.pluginSyncExtendedSetting);
|
||||
expect(first.pluginSyncExtendedSetting).not.toBe(shared.pluginSyncExtendedSetting);
|
||||
|
||||
first.remoteConfigurations.example = {
|
||||
id: "example",
|
||||
name: "Example",
|
||||
uri: "sls+http://localhost/",
|
||||
isEncrypted: false,
|
||||
};
|
||||
|
||||
expect(second.remoteConfigurations).toEqual({});
|
||||
expect(shared.remoteConfigurations).toEqual({});
|
||||
expect(NEW_VAULT_SETTINGS.remoteConfigurations).toEqual({});
|
||||
expect(NEW_VAULT_SETTINGS.pluginSyncExtendedSetting).toEqual({});
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,5 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { LOG_LEVEL_NOTICE, REMOTE_COUCHDB, REMOTE_MINIO } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { LOG_LEVEL_NOTICE, REMOTE_COUCHDB, REMOTE_MINIO, VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { createNewVaultSettings } from "@vrtmrz/livesync-commonlib/settings";
|
||||
|
||||
@@ -21,6 +21,7 @@ const mocks = vi.hoisted(() => ({
|
||||
objectStorage: [] as Array<{
|
||||
host: unknown;
|
||||
tryConnectRemote: ReturnType<typeof vi.fn>;
|
||||
inspectRemotePathIdentity: ReturnType<typeof vi.fn>;
|
||||
getRemoteStatus: ReturnType<typeof vi.fn>;
|
||||
getRemotePreferredTweakValues: ReturnType<typeof vi.fn>;
|
||||
getReplicationPBKDF2Salt: ReturnType<typeof vi.fn>;
|
||||
@@ -64,6 +65,7 @@ vi.mock("@vrtmrz/livesync-commonlib/compat/replication/journal/LiveSyncJournalRe
|
||||
LiveSyncJournalReplicator: class {
|
||||
host: unknown;
|
||||
tryConnectRemote = vi.fn();
|
||||
inspectRemotePathIdentity = vi.fn();
|
||||
getRemoteStatus = vi.fn();
|
||||
getRemotePreferredTweakValues = vi.fn();
|
||||
getReplicationPBKDF2Salt = vi.fn();
|
||||
@@ -135,6 +137,45 @@ describe("replicator probe factories", () => {
|
||||
expect(databaseClose).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("inspects CouchDB identity without encryption setup or remote writes and closes on failure", async () => {
|
||||
const source = { ...createSettings(), encrypt: true };
|
||||
const probe = await createCouchDBConnectionProbeFactory({} as never)(source);
|
||||
const replicator = mocks.couchDB[0];
|
||||
const get = vi.fn(async () => ({ _id: VERSIONING_DOCID, type: "versioninfo", version: 13, used_features: ["nfc-path-identity-v1"] }));
|
||||
const close = vi.fn(async () => undefined);
|
||||
replicator.connectRemoteCouchDBWithSetting.mockResolvedValue({ db: { get }, close });
|
||||
await expect(probe.inspectPathIdentity!()).resolves.toEqual({
|
||||
status: "configured",
|
||||
pathNormalizationVersion: 1,
|
||||
});
|
||||
expect(replicator.connectRemoteCouchDBWithSetting).toHaveBeenCalledWith(
|
||||
{ ...source, encrypt: false },
|
||||
false,
|
||||
false,
|
||||
false
|
||||
);
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
get.mockRejectedValueOnce(new Error("read unavailable"));
|
||||
await expect(probe.inspectPathIdentity!()).rejects.toThrow("read unavailable");
|
||||
expect(close).toHaveBeenCalledTimes(2);
|
||||
expect(replicator.getReplicationPBKDF2Salt).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("delegates Journal identity inspection to the owned settings snapshot", async () => {
|
||||
const source = createSettings();
|
||||
const snapshot = { ...source };
|
||||
const probe = await createObjectStorageConnectionProbeFactory({} as never)(source);
|
||||
const replicator = mocks.objectStorage[0];
|
||||
replicator.inspectRemotePathIdentity.mockResolvedValue({ status: "empty" });
|
||||
source.bucket = "changed";
|
||||
await expect(probe.inspectPathIdentity!()).resolves.toEqual({ status: "empty" });
|
||||
expect(replicator.inspectRemotePathIdentity).toHaveBeenCalledWith(snapshot);
|
||||
expect(replicator.tryConnectRemote).not.toHaveBeenCalled();
|
||||
expect(replicator.getReplicationPBKDF2Salt).not.toHaveBeenCalled();
|
||||
await probe.dispose();
|
||||
expect(replicator.closeReplication).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("maps a CouchDB connection error string and delegates status to the same snapshot", async () => {
|
||||
const source = createSettings();
|
||||
const snapshot = { ...source };
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
LiveSyncCouchDBReplicator,
|
||||
type LiveSyncCouchDBReplicatorEnv,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/replication/couchdb/LiveSyncReplicator";
|
||||
import { inspectDatabasePathIdentity } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { LOG_LEVEL_NOTICE, Logger } from "@vrtmrz/livesync-commonlib/compat/common/logger";
|
||||
import { LiveSyncJournalReplicator } from "@vrtmrz/livesync-commonlib/compat/replication/journal/LiveSyncJournalReplicator";
|
||||
import { createReplicatorDisposer, snapshotRemoteSettings } from "./shared";
|
||||
@@ -51,6 +52,20 @@ function createCouchDBConnectionProbe(
|
||||
await connection.close();
|
||||
}
|
||||
},
|
||||
inspectPathIdentity: async () => {
|
||||
const connection = await replicator.connectRemoteCouchDBWithSetting(
|
||||
{ ...snapshot, encrypt: false },
|
||||
replicator.isMobile(),
|
||||
false,
|
||||
false
|
||||
);
|
||||
if (typeof connection === "string") throw new Error(connection);
|
||||
try {
|
||||
return await inspectDatabasePathIdentity(connection.db);
|
||||
} finally {
|
||||
await connection.close();
|
||||
}
|
||||
},
|
||||
getStatus: () => replicator.getRemoteStatus(snapshot),
|
||||
dispose,
|
||||
};
|
||||
@@ -70,6 +85,7 @@ function createObjectStorageConnectionProbe(
|
||||
return { ok: false, reason: error };
|
||||
}
|
||||
},
|
||||
inspectPathIdentity: () => replicator.inspectRemotePathIdentity(snapshot),
|
||||
getStatus: () => replicator.getRemoteStatus(snapshot),
|
||||
dispose,
|
||||
};
|
||||
|
||||
@@ -81,6 +81,10 @@ function createBufferedLog(entries: ReactiveSource<LogEntry[]>) {
|
||||
};
|
||||
}
|
||||
|
||||
function logToConsole(message: unknown, level?: number, key?: string) {
|
||||
console.log({ message, level, key });
|
||||
}
|
||||
|
||||
// Legacy Logger calls use this shared buffer; service logs use their owner's buffer.
|
||||
const recentGlobalLogEntries = reactiveSource<LogEntry[]>([]);
|
||||
setGlobalLogFunction(createBufferedLog(recentGlobalLogEntries));
|
||||
@@ -126,7 +130,7 @@ function redactLog(log: string) {
|
||||
const showDebugLog = false;
|
||||
export const MARK_DONE = "\u{2009}\u{2009}";
|
||||
export class ModuleLog extends AbstractObsidianModule {
|
||||
/** The host releases this connection after the complete asynchronous unload. */
|
||||
/** The host releases log display subscriptions after the complete asynchronous unload. */
|
||||
logResource!: OwnedResource<LogFunction>;
|
||||
private readonly recentLogEntries = reactiveSource<LogEntry[]>([]);
|
||||
private readonly displayLogEntries = () => {
|
||||
@@ -615,8 +619,10 @@ ${stringifyYaml(info)}
|
||||
}
|
||||
}
|
||||
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
|
||||
const log = createBufferedLog(this.recentLogEntries);
|
||||
const unregisterLog = services.API.addLog.setHandler(log);
|
||||
const bufferedLog = createBufferedLog(this.recentLogEntries);
|
||||
let logTarget: LogFunction = bufferedLog;
|
||||
const log: LogFunction = (message, level, key) => logTarget(message, level, key);
|
||||
services.API.addLog.setHandler(log);
|
||||
const collectGlobalLogs = () => {
|
||||
const entries = recentGlobalLogEntries.value;
|
||||
if (entries.length === 0) return;
|
||||
@@ -624,7 +630,7 @@ ${stringifyYaml(info)}
|
||||
this.recentLogEntries.value = [...this.recentLogEntries.value, ...entries];
|
||||
};
|
||||
this.logResource = createOwnedResource(log, () => {
|
||||
unregisterLog();
|
||||
logTarget = logToConsole;
|
||||
recentGlobalLogEntries.offChanged(collectGlobalLogs);
|
||||
this.recentLogEntries.offChanged(this.displayLogEntries);
|
||||
this.recentLogEntries.value = [];
|
||||
|
||||
@@ -107,25 +107,92 @@ describe("ModuleLog service connection", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("detaches both service registration and legacy subscription on disposal", async () => {
|
||||
it("keeps service logging callable after disposal and sends late logs to the console", async () => {
|
||||
const { services, module, sink } = await createHarness();
|
||||
await services.appLifecycle.onSettingLoaded();
|
||||
sink.mockClear();
|
||||
const consoleLog = vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
|
||||
await module.logResource.dispose();
|
||||
try {
|
||||
expect(() => services.API.addLog("late-service", 32, "late-key")).not.toThrow();
|
||||
expect(consoleLog).toHaveBeenCalledExactlyOnceWith({
|
||||
message: "late-service",
|
||||
level: 32,
|
||||
key: "late-key",
|
||||
});
|
||||
const cause = new Error("root cause");
|
||||
const error = Object.assign(new Error("late failure"), { cause });
|
||||
services.API.addLog(error, 128, "late-error");
|
||||
const errorEntry = consoleLog.mock.calls[1][0] as {
|
||||
message: Error & { cause?: unknown };
|
||||
level?: number;
|
||||
key?: string;
|
||||
};
|
||||
expect(errorEntry.message).toBe(error);
|
||||
expect(errorEntry.message.cause).toBe(cause);
|
||||
expect(errorEntry.level).toBe(128);
|
||||
expect(errorEntry.key).toBe("late-error");
|
||||
expect(sink).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
consoleLog.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("sends late circular service messages directly to the console", async () => {
|
||||
const { services, module, sink } = await createHarness();
|
||||
await services.appLifecycle.onSettingLoaded();
|
||||
sink.mockClear();
|
||||
const consoleLog = vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
const circular: { self?: unknown } = {};
|
||||
circular.self = circular;
|
||||
|
||||
try {
|
||||
await module.logResource.dispose();
|
||||
expect(() => services.API.addLog(circular, 32, "circular")).not.toThrow();
|
||||
const consoleEntry = consoleLog.mock.calls[0][0] as {
|
||||
message: unknown;
|
||||
level?: number;
|
||||
key?: string;
|
||||
};
|
||||
expect(consoleEntry.message).toBe(circular);
|
||||
expect(consoleEntry.level).toBe(32);
|
||||
expect(consoleEntry.key).toBe("circular");
|
||||
expect(sink).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
consoleLog.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("releases legacy and UI subscriptions while retaining late service console logging", async () => {
|
||||
const first = await createHarness();
|
||||
const second = await createHarness();
|
||||
await first.services.appLifecycle.onSettingLoaded();
|
||||
await second.services.appLifecycle.onSettingLoaded();
|
||||
first.sink.mockClear();
|
||||
second.sink.mockClear();
|
||||
const consoleLog = vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
|
||||
await first.module.logResource.dispose();
|
||||
await first.module.logResource.dispose();
|
||||
expect(() => first.services.API.addLog("retired")).toThrow("not assigned");
|
||||
second.services.API.addLog("active", 32, "service");
|
||||
first.Logger("legacy-after-retirement", 32, "legacy");
|
||||
expect(first.sink).not.toHaveBeenCalled();
|
||||
expect(second.sink.mock.calls).toEqual([
|
||||
["active", 32, "service"],
|
||||
["legacy-after-retirement", 32, "legacy"],
|
||||
]);
|
||||
await second.module.logResource.dispose();
|
||||
try {
|
||||
await first.module.logResource.dispose();
|
||||
await first.module.logResource.dispose();
|
||||
expect(() => first.services.API.addLog("retired", 32, "retired-key")).not.toThrow();
|
||||
expect(consoleLog).toHaveBeenCalledWith({
|
||||
message: "retired",
|
||||
level: 32,
|
||||
key: "retired-key",
|
||||
});
|
||||
second.services.API.addLog("active", 32, "service");
|
||||
first.Logger("legacy-after-retirement", 32, "legacy");
|
||||
expect(first.sink).not.toHaveBeenCalled();
|
||||
expect(second.sink.mock.calls).toEqual([
|
||||
["active", 32, "service"],
|
||||
["legacy-after-retirement", 32, "legacy"],
|
||||
]);
|
||||
await second.module.logResource.dispose();
|
||||
} finally {
|
||||
consoleLog.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("leaves a replacement service handler registered when the old owner is disposed", async () => {
|
||||
@@ -143,20 +210,31 @@ describe("ModuleLog service connection", () => {
|
||||
expect(sink).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("releases a connection which never reached the settings-loaded phase", async () => {
|
||||
it("retires a connection which never reached the settings-loaded phase", async () => {
|
||||
const { services, module, sink, Logger } = await createHarness();
|
||||
services.API.addLog("pending");
|
||||
await module.logResource.dispose();
|
||||
Logger("after-disposal");
|
||||
expect(sink).not.toHaveBeenCalled();
|
||||
expect(() => services.API.addLog("retired")).toThrow("not assigned");
|
||||
const consoleLog = vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
try {
|
||||
await module.logResource.dispose();
|
||||
Logger("after-disposal");
|
||||
expect(() => services.API.addLog("retired", 32, "retired-key")).not.toThrow();
|
||||
expect(consoleLog).toHaveBeenCalledWith({
|
||||
message: "retired",
|
||||
level: 32,
|
||||
key: "retired-key",
|
||||
});
|
||||
expect(sink).not.toHaveBeenCalled();
|
||||
|
||||
const replacement = await createHarness();
|
||||
await replacement.services.appLifecycle.onSettingLoaded();
|
||||
const messages = replacement.sink.mock.calls.map(([message]) => message);
|
||||
expect(messages).toContain("after-disposal");
|
||||
expect(messages).not.toContain("pending");
|
||||
await replacement.module.logResource.dispose();
|
||||
const replacement = await createHarness();
|
||||
await replacement.services.appLifecycle.onSettingLoaded();
|
||||
const messages = replacement.sink.mock.calls.map(([message]) => message);
|
||||
expect(messages).toContain("after-disposal");
|
||||
expect(messages).not.toContain("pending");
|
||||
expect(messages).not.toContain("retired");
|
||||
await replacement.module.logResource.dispose();
|
||||
} finally {
|
||||
consoleLog.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["complete", "fail"] as const)(
|
||||
@@ -219,10 +297,20 @@ describe("ModuleLog service connection", () => {
|
||||
"moduleLiveSyncMain.logUnloadingPlugin"
|
||||
);
|
||||
}
|
||||
expect(() => services.API.addLog("retired")).toThrow("not assigned");
|
||||
sink.mockClear();
|
||||
Logger("after-unload");
|
||||
expect(sink).not.toHaveBeenCalled();
|
||||
const consoleLog = vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
try {
|
||||
expect(() => services.API.addLog("late-after-close", 32, "late-after-close")).not.toThrow();
|
||||
expect(consoleLog).toHaveBeenCalledWith({
|
||||
message: "late-after-close",
|
||||
level: 32,
|
||||
key: "late-after-close",
|
||||
});
|
||||
sink.mockClear();
|
||||
Logger("after-unload");
|
||||
expect(sink).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
consoleLog.mockRestore();
|
||||
}
|
||||
}
|
||||
);
|
||||
});
|
||||
@@ -1,5 +1,4 @@
|
||||
import { ChunkAlgorithmNames } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { $msg } from "@/common/translation";
|
||||
import type { SettingSpecGroup } from "./SettingSpec.ts";
|
||||
|
||||
export type AdvancedSettingSpecContext = {
|
||||
@@ -24,18 +23,6 @@ export function createAdvancedSettingSpecGroups({
|
||||
},
|
||||
{ key: "customChunkSize", control: { type: "number", min: 0, allowZero: true } },
|
||||
{ key: "strictContentCheckOnStartup", control: { type: "toggle" } },
|
||||
{
|
||||
key: "pathNormalizationVersion",
|
||||
holdValue: true,
|
||||
control: {
|
||||
type: "numeric-dropdown",
|
||||
options: () => ({
|
||||
"0": $msg("Legacy path identity"),
|
||||
"1": $msg("Unicode NFC path identity"),
|
||||
}),
|
||||
},
|
||||
visible: isCouchDB,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -8,7 +8,7 @@ export type GeneralSettingSpecContext = {
|
||||
showVerboseLog: () => boolean;
|
||||
};
|
||||
|
||||
/** Build the shared Appearance and Logging controls. */
|
||||
/** Build the shared General Settings controls. */
|
||||
export function createGeneralSettingSpecGroups({
|
||||
showEditorStatusDetails,
|
||||
showVerboseLog,
|
||||
@@ -61,6 +61,22 @@ export function createGeneralSettingSpecGroups({
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
heading: "File identity",
|
||||
items: [
|
||||
{
|
||||
key: "pathNormalizationVersion",
|
||||
holdValue: true,
|
||||
control: {
|
||||
type: "numeric-dropdown",
|
||||
options: () => ({
|
||||
"0": $msg("Legacy path identity"),
|
||||
"1": $msg("Unicode NFC path identity"),
|
||||
}),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
|
||||
+15
-3
@@ -319,7 +319,7 @@ describe("ObsidianLiveSyncSettingTab native page lifecycle", () => {
|
||||
).toEqual(["Copy the current settings to a Setup URI", "Show QR code"]);
|
||||
});
|
||||
|
||||
it("keeps Appearance, Logging, and Extra menus inside General Settings", () => {
|
||||
it("keeps Appearance, Logging, File identity, and Extra menus inside General Settings", () => {
|
||||
const tab = createSettingsTab();
|
||||
const definitions = tab.getSettingDefinitions();
|
||||
const general = definitions.find(
|
||||
@@ -332,7 +332,19 @@ describe("ObsidianLiveSyncSettingTab native page lifecycle", () => {
|
||||
(item): item is SettingDefinitionPage => isPage(item) && item.name === "🎚️ Extra menus"
|
||||
);
|
||||
|
||||
expect(generalPages?.map(({ name }) => name)).toEqual(["🎨 Appearance", "📝 Logging", "🎚️ Extra menus"]);
|
||||
expect(generalPages?.map(({ name }) => name)).toEqual([
|
||||
"🎨 Appearance",
|
||||
"📝 Logging",
|
||||
"🗂️ File identity",
|
||||
"🎚️ Extra menus",
|
||||
]);
|
||||
const fileIdentity = generalPages?.find(({ name }) => name === "🗂️ File identity");
|
||||
expect(
|
||||
fileIdentity?.items?.flatMap((item) => ("control" in item && item.control ? [item.control.key] : []))
|
||||
).toEqual(["pathNormalizationVersion"]);
|
||||
expect(
|
||||
fileIdentity?.items?.filter((item) => "action" in item && typeof item.action === "function")
|
||||
).toHaveLength(1);
|
||||
expect(
|
||||
appearance?.items?.flatMap((item) => ("control" in item && item.control ? [item.control.key] : []))
|
||||
).toEqual([
|
||||
@@ -390,7 +402,7 @@ describe("ObsidianLiveSyncSettingTab native page lifecycle", () => {
|
||||
const tab = createSettingsTab();
|
||||
const pages = collectPages(tab.getSettingDefinitions());
|
||||
|
||||
expect(pages).toHaveLength(14);
|
||||
expect(pages).toHaveLength(15);
|
||||
expect(pages.map(({ name }) => name)).toEqual(
|
||||
expect.arrayContaining(
|
||||
createSettingsPageCatalogue()
|
||||
|
||||
@@ -892,9 +892,9 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
showEditorStatusDetails: () => this.isConfiguredAs("showStatusOnEditor", true),
|
||||
showVerboseLog: () => this.isConfiguredAs("lessInformationInLog", false),
|
||||
});
|
||||
const [appearance, logging] = groups;
|
||||
if (!appearance || !logging) {
|
||||
throw new Error("General settings must define Appearance and Logging groups");
|
||||
const [appearance, logging, fileIdentity] = groups;
|
||||
if (!appearance || !logging || !fileIdentity) {
|
||||
throw new Error("General settings must define Appearance, Logging, and File identity groups");
|
||||
}
|
||||
return this.createRootGroup("general-settings", [
|
||||
{
|
||||
@@ -907,6 +907,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
name: `📝 ${logging.heading}`,
|
||||
items: logging.items,
|
||||
},
|
||||
{
|
||||
type: "page",
|
||||
name: `🗂️ ${fileIdentity.heading}`,
|
||||
items: [this.createRebuildRequiredAction(), ...(fileIdentity.items ?? [])],
|
||||
},
|
||||
this.createExtraMenusPage(),
|
||||
]);
|
||||
}
|
||||
@@ -1128,7 +1133,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
if (method === "remoteOnly" && this.isDirty("pathNormalizationVersion")) {
|
||||
Logger(
|
||||
$msg(
|
||||
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
|
||||
"Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild from a complete authoritative Vault and Fetch on the other devices. With P2P, Rebuild prepares only the source device's local database. Resolve colliding file names first."
|
||||
),
|
||||
LOG_LEVEL_NOTICE
|
||||
);
|
||||
|
||||
@@ -581,11 +581,11 @@ describe("ObsidianLiveSyncSettingTab pending-setting initialisation", () => {
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
expect(vi.mocked($msg)).toHaveBeenCalledWith(
|
||||
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
|
||||
"Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild from a complete authoritative Vault and Fetch on the other devices. With P2P, Rebuild prepares only the source device's local database. Resolve colliding file names first."
|
||||
);
|
||||
expect(loggerMocks.Logger).toHaveBeenCalledWith(
|
||||
expect.stringContaining(
|
||||
"Rebuild both databases from a complete authoritative Vault and Fetch on the other devices."
|
||||
"Rebuild from a complete authoritative Vault and Fetch on the other devices."
|
||||
),
|
||||
LOG_LEVEL_NOTICE
|
||||
);
|
||||
@@ -692,6 +692,30 @@ describe("ObsidianLiveSyncSettingTab declarative settings boundary", () => {
|
||||
return { tab, saveSettingData };
|
||||
}
|
||||
|
||||
it("includes path identity once in the declarative General settings group", () => {
|
||||
const { tab } = createSettingsTab();
|
||||
const group = (
|
||||
tab as unknown as {
|
||||
createGeneralSettingsGroup: () => {
|
||||
items: Array<{
|
||||
name: string;
|
||||
items?: Array<{ control?: { key: string }; action?: () => void; visible?: () => boolean }>;
|
||||
}>;
|
||||
};
|
||||
}
|
||||
).createGeneralSettingsGroup();
|
||||
const fileIdentity = group.items.find(({ name }) => name === "🗂️ File identity");
|
||||
|
||||
expect(fileIdentity).toBeDefined();
|
||||
const apply = fileIdentity?.items?.find(({ action }) => typeof action === "function");
|
||||
expect(apply).toBeDefined();
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
expect(apply?.visible?.()).toBe(true);
|
||||
expect(fileIdentity?.items?.flatMap(({ control }) => (control ? [control.key] : []))).toEqual([
|
||||
"pathNormalizationVersion",
|
||||
]);
|
||||
});
|
||||
|
||||
it("awaits the stored passphrase before initialising the editing buffer", async () => {
|
||||
const { tab } = createSettingsTab();
|
||||
tab.resetEditingSettings();
|
||||
|
||||
@@ -43,7 +43,7 @@ describe("paneAdvanced", () => {
|
||||
};
|
||||
|
||||
paneAdvanced.call(host as never, {} as HTMLElement, { addPanel } as never);
|
||||
await vi.waitFor(() => expect(settingHarness.rendered).toHaveLength(11));
|
||||
await vi.waitFor(() => expect(settingHarness.rendered).toHaveLength(10));
|
||||
|
||||
expect(addPanel.mock.calls.map(([, heading]) => heading)).toEqual([
|
||||
"Memory cache",
|
||||
@@ -56,7 +56,6 @@ describe("paneAdvanced", () => {
|
||||
panelElements.get("Local Database Tweak"),
|
||||
panelElements.get("Local Database Tweak"),
|
||||
panelElements.get("Local Database Tweak"),
|
||||
panelElements.get("Local Database Tweak"),
|
||||
panelElements.get("Transfer Tweak"),
|
||||
panelElements.get("Transfer Tweak"),
|
||||
panelElements.get("Transfer Tweak"),
|
||||
@@ -69,7 +68,6 @@ describe("paneAdvanced", () => {
|
||||
"chunkSplitterVersion",
|
||||
"customChunkSize",
|
||||
"strictContentCheckOnStartup",
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { paneGeneral } from "./PaneGeneral.ts";
|
||||
|
||||
const settingHarness = vi.hoisted(() => ({
|
||||
rendered: [] as unknown[],
|
||||
}));
|
||||
|
||||
vi.mock("@/common/translation", () => ({
|
||||
$msg: (key: string) => key,
|
||||
$t: (key: string) => key,
|
||||
}));
|
||||
|
||||
vi.mock("./LiveSyncSetting.ts", () => ({
|
||||
LiveSyncSetting: class LiveSyncSetting {},
|
||||
}));
|
||||
|
||||
vi.mock("./SettingSpec.ts", async (importOriginal) => {
|
||||
const original = await importOriginal<typeof import("./SettingSpec.ts")>();
|
||||
return {
|
||||
...original,
|
||||
renderLegacySettingSpec: vi.fn((_renderer: unknown, spec: unknown) => {
|
||||
settingHarness.rendered.push(spec);
|
||||
}),
|
||||
};
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
settingHarness.rendered.length = 0;
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
describe("paneGeneral", () => {
|
||||
it("renders path identity once in the File identity panel and preserves its held value", async () => {
|
||||
const addPanel = vi.fn((_parent: HTMLElement, heading: string) =>
|
||||
Promise.resolve({ heading } as unknown as HTMLElement)
|
||||
);
|
||||
const host = {
|
||||
isConfiguredAs: vi.fn(() => false),
|
||||
};
|
||||
|
||||
paneGeneral.call(host as never, {} as HTMLElement, { addPanel } as never);
|
||||
await vi.waitFor(() => expect(settingHarness.rendered).toHaveLength(14));
|
||||
|
||||
expect(addPanel.mock.calls.map(([, heading]) => heading)).toEqual([
|
||||
"obsidianLiveSyncSettingTab.titleAppearance",
|
||||
"obsidianLiveSyncSettingTab.titleLogging",
|
||||
"File identity",
|
||||
"obsidianLiveSyncSettingTab.titleExtraMenus",
|
||||
]);
|
||||
const pathIdentity = settingHarness.rendered.filter(
|
||||
(spec) => (spec as { key: string }).key === "pathNormalizationVersion"
|
||||
);
|
||||
expect(pathIdentity).toHaveLength(1);
|
||||
expect(pathIdentity[0]).toMatchObject({ holdValue: true });
|
||||
expect((pathIdentity[0] as { visible?: () => boolean }).visible).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -17,7 +17,7 @@ const rangeMessages = {
|
||||
};
|
||||
|
||||
describe("Advanced setting specifications", () => {
|
||||
it("lists only the eleven controls currently exposed by the Advanced page", () => {
|
||||
it("lists only the ten controls currently exposed by the Advanced page", () => {
|
||||
const groups = createAdvancedSettingSpecGroups({ isCouchDB: () => true });
|
||||
|
||||
expect(groups.map(({ heading }) => heading)).toEqual([
|
||||
@@ -31,7 +31,6 @@ describe("Advanced setting specifications", () => {
|
||||
"chunkSplitterVersion",
|
||||
"customChunkSize",
|
||||
"strictContentCheckOnStartup",
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
@@ -52,13 +51,12 @@ describe("Advanced setting specifications", () => {
|
||||
expect(spec.disabled).toBeUndefined();
|
||||
});
|
||||
|
||||
it("keeps path identity and the four remote chunk controls CouchDB-only", () => {
|
||||
it("keeps the four remote chunk controls CouchDB-only", () => {
|
||||
let couchDB = false;
|
||||
const groups = createAdvancedSettingSpecGroups({ isCouchDB: () => couchDB });
|
||||
const conditional = groups.flatMap(({ items }) => items).filter(({ visible }) => visible !== undefined);
|
||||
|
||||
expect(conditional.map(({ key }) => key)).toEqual([
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
@@ -70,18 +68,67 @@ describe("Advanced setting specifications", () => {
|
||||
expect(conditional.every(({ visible }) => visible?.() === true)).toBe(true);
|
||||
});
|
||||
|
||||
it("exposes CouchDB path identity as a two-choice numeric dropdown", () => {
|
||||
let couchDB = false;
|
||||
const spec = createAdvancedSettingSpecGroups({ isCouchDB: () => couchDB })
|
||||
it("excludes transient settings-dialogue keys from the persisted key type", () => {
|
||||
type OnDialogOverlap = Extract<keyof OnDialogSettings, PersistedSettingKey>;
|
||||
expectTypeOf<OnDialogOverlap>().toEqualTypeOf<never>();
|
||||
});
|
||||
|
||||
it("resolves translated Commonlib metadata for every explicitly exposed key", () => {
|
||||
const specs = [
|
||||
...createAdvancedSettingSpecGroups({ isCouchDB: () => true }),
|
||||
...createGeneralSettingSpecGroups({
|
||||
showEditorStatusDetails: () => true,
|
||||
showVerboseLog: () => true,
|
||||
}),
|
||||
].flatMap(({ items }) => items);
|
||||
|
||||
for (const spec of specs) {
|
||||
const metadata = getConfig(spec.key);
|
||||
expect(metadata, spec.key).not.toBe(false);
|
||||
if (!metadata) {
|
||||
throw new Error(`Missing setting metadata for ${spec.key}`);
|
||||
}
|
||||
const definition = toObsidianSettingDefinition(spec, metadata, rangeMessages);
|
||||
expect(definition.name).toBe(`${metadata.name}${statusDisplay(metadata.status)}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("General setting specifications", () => {
|
||||
it("shares every General, Logging, and File identity control between the imperative and declarative renderers", () => {
|
||||
const groups = createGeneralSettingSpecGroups({
|
||||
showEditorStatusDetails: () => true,
|
||||
showVerboseLog: () => true,
|
||||
});
|
||||
|
||||
expect(groups.map(({ heading }) => heading)).toEqual(["Appearance", "Logging", "File identity"]);
|
||||
expect(groups.flatMap(({ items }) => items.map(({ key }) => key))).toEqual([
|
||||
"displayLanguage",
|
||||
"showStatusOnEditor",
|
||||
"showOnlyIconsOnEditor",
|
||||
"showStatusOnStatusbar",
|
||||
"hideFileWarningNotice",
|
||||
"syncGuardEnabled",
|
||||
"syncGuardMinimumFiles",
|
||||
"networkWarningStyle",
|
||||
"lessInformationInLog",
|
||||
"showVerboseLog",
|
||||
"pathNormalizationVersion",
|
||||
]);
|
||||
});
|
||||
|
||||
it("exposes path identity for every remote type as a reconstruction-required dropdown", () => {
|
||||
const spec = createGeneralSettingSpecGroups({
|
||||
showEditorStatusDetails: () => false,
|
||||
showVerboseLog: () => false,
|
||||
})
|
||||
.flatMap(({ items }) => items)
|
||||
.find(({ key }) => key === "pathNormalizationVersion");
|
||||
if (!spec) throw new Error("Path identity mode is not exposed on the Advanced page");
|
||||
if (!spec) throw new Error("Path identity mode is not exposed in General settings");
|
||||
if (spec.control.type !== "numeric-dropdown") throw new Error("Expected a numeric dropdown");
|
||||
|
||||
expect(spec.control).toMatchObject({ type: "numeric-dropdown", options: expect.any(Function) });
|
||||
expect(spec.visible?.()).toBe(false);
|
||||
couchDB = true;
|
||||
expect(spec.visible?.()).toBe(true);
|
||||
expect(spec).toMatchObject({ key: "pathNormalizationVersion", holdValue: true });
|
||||
expect(spec.visible).toBeUndefined();
|
||||
expect(spec.control.options()).toEqual({
|
||||
"0": "Legacy path identity",
|
||||
"1": "Unicode NFC path identity",
|
||||
@@ -102,54 +149,11 @@ describe("Advanced setting specifications", () => {
|
||||
key: "pathNormalizationVersion",
|
||||
options: {
|
||||
holdValue: true,
|
||||
onUpdate: expect.any(Function),
|
||||
options: { "0": "Legacy path identity", "1": "Unicode NFC path identity" },
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("excludes transient settings-dialogue keys from the persisted key type", () => {
|
||||
type OnDialogOverlap = Extract<keyof OnDialogSettings, PersistedSettingKey>;
|
||||
expectTypeOf<OnDialogOverlap>().toEqualTypeOf<never>();
|
||||
});
|
||||
|
||||
it("resolves translated Commonlib metadata for every explicitly exposed key", () => {
|
||||
const specs = createAdvancedSettingSpecGroups({ isCouchDB: () => true }).flatMap(({ items }) => items);
|
||||
|
||||
for (const spec of specs) {
|
||||
const metadata = getConfig(spec.key);
|
||||
expect(metadata, spec.key).not.toBe(false);
|
||||
if (!metadata) {
|
||||
throw new Error(`Missing setting metadata for ${spec.key}`);
|
||||
}
|
||||
const definition = toObsidianSettingDefinition(spec, metadata, rangeMessages);
|
||||
expect(definition.name).toBe(`${metadata.name}${statusDisplay(metadata.status)}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("General setting specifications", () => {
|
||||
it("shares every General and Logging control between the imperative and declarative renderers", () => {
|
||||
const groups = createGeneralSettingSpecGroups({
|
||||
showEditorStatusDetails: () => true,
|
||||
showVerboseLog: () => true,
|
||||
});
|
||||
|
||||
expect(groups.map(({ heading }) => heading)).toEqual(["Appearance", "Logging"]);
|
||||
expect(groups.flatMap(({ items }) => items.map(({ key }) => key))).toEqual([
|
||||
"displayLanguage",
|
||||
"showStatusOnEditor",
|
||||
"showOnlyIconsOnEditor",
|
||||
"showStatusOnStatusbar",
|
||||
"hideFileWarningNotice",
|
||||
"syncGuardEnabled",
|
||||
"syncGuardMinimumFiles",
|
||||
"networkWarningStyle",
|
||||
"lessInformationInLog",
|
||||
"showVerboseLog",
|
||||
]);
|
||||
});
|
||||
|
||||
it("keeps the three feature-level controls together under Extra menus", () => {
|
||||
const group = createExtraMenuSettingSpecGroup();
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("@/common/translation", () => ({
|
||||
$msg: (key: string) => key,
|
||||
$t: (key: string) => key,
|
||||
translateLiveSyncMessage: (key: string) => key,
|
||||
}));
|
||||
vi.mock("./PaneChangeLog.ts", () => ({ paneChangeLog: vi.fn() }));
|
||||
@@ -20,6 +21,7 @@ vi.mock("./PaneMaintenance.ts", () => ({ paneMaintenance: vi.fn() }));
|
||||
|
||||
import {
|
||||
createAdvancedSettingDefinitionGroups,
|
||||
createGeneralSettingDefinitionGroups,
|
||||
createSettingsPageCatalogue,
|
||||
getSettingsRootGroupEntry,
|
||||
type SettingsRootGroupId,
|
||||
@@ -87,7 +89,6 @@ describe("settings page catalogue", () => {
|
||||
"chunkSplitterVersion",
|
||||
"customChunkSize",
|
||||
"strictContentCheckOnStartup",
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
@@ -97,4 +98,22 @@ describe("settings page catalogue", () => {
|
||||
]);
|
||||
expect(new Set(keys).size).toBe(keys.length);
|
||||
});
|
||||
|
||||
it("registers path identity once in General settings and leaves it out of Advanced", () => {
|
||||
const generalGroups = createGeneralSettingDefinitionGroups({
|
||||
showEditorStatusDetails: () => true,
|
||||
showVerboseLog: () => true,
|
||||
});
|
||||
const generalKeys = generalGroups.flatMap(({ items = [] }) =>
|
||||
items.flatMap((item) => ("control" in item && item.control ? [item.control.key] : []))
|
||||
);
|
||||
const advancedKeys = createAdvancedSettingDefinitionGroups({ isCouchDB: () => true }).flatMap(
|
||||
({ items = [] }) =>
|
||||
items.flatMap((item) => ("control" in item && item.control ? [item.control.key] : []))
|
||||
);
|
||||
|
||||
expect(generalGroups.map(({ heading }) => heading)).toContain("File identity");
|
||||
expect(generalKeys.filter((key) => key === "pathNormalizationVersion")).toHaveLength(1);
|
||||
expect(advancedKeys).not.toContain("pathNormalizationVersion");
|
||||
});
|
||||
});
|
||||
@@ -7,7 +7,7 @@ import {
|
||||
REMOTE_COUCHDB,
|
||||
REMOTE_MINIO,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { createNewVaultSettings } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { createNewVaultSettings } from "@/common/newVaultSettings";
|
||||
import { upsertRemoteConfigurationInPlace } from "@vrtmrz/livesync-commonlib/remote-configurations";
|
||||
import { isObjectDifferent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import Intro from "./SetupWizard/dialogs/Intro.svelte";
|
||||
@@ -49,6 +49,8 @@ import {
|
||||
type SetupInitialisationMode,
|
||||
} from "@/serviceFeatures/setupObsidian/setupActivationLifecycle.ts";
|
||||
import { isP2PMainRemote } from "@/common/remoteConfiguration.ts";
|
||||
import { REMOTE_RESOURCE_KINDS } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { resolveInitialPathIdentity } from "@/common/initialPathIdentity";
|
||||
import type { P2PConnectionProbeAdmission } from "@vrtmrz/livesync-commonlib/p2p";
|
||||
|
||||
function copySettingsForRemoteProfileUpdate(settings: ObsidianLiveSyncSettings): ObsidianLiveSyncSettings {
|
||||
@@ -58,11 +60,14 @@ function copySettingsForRemoteProfileUpdate(settings: ObsidianLiveSyncSettings):
|
||||
};
|
||||
}
|
||||
|
||||
function normaliseImportedIdDerivationSettings(settings: ObsidianLiveSyncSettings): ObsidianLiveSyncSettings {
|
||||
function normaliseImportedIdentitySettings(settings: ObsidianLiveSyncSettings): ObsidianLiveSyncSettings {
|
||||
// Setup URIs are complete imports even when their encoder omitted default-valued fields.
|
||||
// Fill each missing half so a receiving device cannot supply the unrelated saved key.
|
||||
return {
|
||||
...settings,
|
||||
pathNormalizationVersion: Object.prototype.hasOwnProperty.call(settings, "pathNormalizationVersion")
|
||||
? settings.pathNormalizationVersion
|
||||
: 0,
|
||||
idDerivationVersion: Object.prototype.hasOwnProperty.call(settings, "idDerivationVersion")
|
||||
? settings.idDerivationVersion
|
||||
: 0,
|
||||
@@ -233,7 +238,7 @@ export class SetupManager extends AbstractModule {
|
||||
return false;
|
||||
}
|
||||
this._log("Setup URI dialog closed.", LOG_LEVEL_VERBOSE);
|
||||
return await this.onConfirmApplySettingsFromWizard(normaliseImportedIdDerivationSettings(newSetting), userMode);
|
||||
return await this.onConfirmApplySettingsFromWizard(normaliseImportedIdentitySettings(newSetting), userMode);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -271,6 +276,7 @@ export class SetupManager extends AbstractModule {
|
||||
if (activate) {
|
||||
newSetting.remoteType = REMOTE_COUCHDB;
|
||||
}
|
||||
if (!(await this.selectInitialPathIdentity(newSetting, userMode, activate))) return false;
|
||||
upsertRemoteConfigurationInPlace(newSetting, "couchdb", { activate });
|
||||
return await this.onConfirmApplySettingsFromWizard(newSetting, userMode, activate);
|
||||
}
|
||||
@@ -302,10 +308,37 @@ export class SetupManager extends AbstractModule {
|
||||
if (activate) {
|
||||
newSetting.remoteType = REMOTE_MINIO;
|
||||
}
|
||||
if (!(await this.selectInitialPathIdentity(newSetting, userMode, activate))) return false;
|
||||
upsertRemoteConfigurationInPlace(newSetting, "s3", { activate });
|
||||
return await this.onConfirmApplySettingsFromWizard(newSetting, userMode, activate);
|
||||
}
|
||||
|
||||
private async selectInitialPathIdentity(
|
||||
settings: ObsidianLiveSyncSettings,
|
||||
userMode: UserMode,
|
||||
activate: boolean
|
||||
): Promise<boolean> {
|
||||
if (!activate || (userMode !== UserMode.NewUser && userMode !== UserMode.ExistingUser)) return true;
|
||||
try {
|
||||
const probe = await this.services.replicator.createRemoteResource(
|
||||
REMOTE_RESOURCE_KINDS.CONNECTION,
|
||||
settings
|
||||
);
|
||||
settings.pathNormalizationVersion = await resolveInitialPathIdentity(
|
||||
probe,
|
||||
settings.pathNormalizationVersion
|
||||
);
|
||||
return true;
|
||||
} catch (error) {
|
||||
this._log(
|
||||
"The remote path identity could not be inspected. Settings have not been applied.",
|
||||
LOG_LEVEL_NOTICE
|
||||
);
|
||||
this._log(error, LOG_LEVEL_VERBOSE);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handles manual setup for P2P
|
||||
* @param userMode
|
||||
@@ -345,10 +378,7 @@ export class SetupManager extends AbstractModule {
|
||||
const e2eeConf = await this.dialogManager.openWithExplicitCancel<
|
||||
SetupRemoteE2EEResultType,
|
||||
SetupRemoteE2EEInitialData
|
||||
>(
|
||||
SetupRemoteE2EE,
|
||||
{ settings: currentSetting, newVault: userMode === UserMode.NewUser }
|
||||
);
|
||||
>(SetupRemoteE2EE, { settings: currentSetting, newVault: userMode === UserMode.NewUser });
|
||||
if (e2eeConf === "cancelled") {
|
||||
this._log("E2EE configuration cancelled.", LOG_LEVEL_NOTICE);
|
||||
return false;
|
||||
@@ -397,10 +427,7 @@ export class SetupManager extends AbstractModule {
|
||||
const e2eeConf = await this.dialogManager.openWithExplicitCancel<
|
||||
SetupRemoteE2EEResultType,
|
||||
SetupRemoteE2EEInitialData
|
||||
>(
|
||||
SetupRemoteE2EE,
|
||||
{ settings: originalSetting, newVault: userMode === UserMode.NewUser }
|
||||
);
|
||||
>(SetupRemoteE2EE, { settings: originalSetting, newVault: userMode === UserMode.NewUser });
|
||||
if (e2eeConf === "cancelled") {
|
||||
this._log("Manual configuration cancelled.", LOG_LEVEL_NOTICE);
|
||||
return await this.onOnboard(userMode);
|
||||
@@ -545,7 +572,7 @@ export class SetupManager extends AbstractModule {
|
||||
async decodeQR(qr: string) {
|
||||
let newSettings: ObsidianLiveSyncSettings;
|
||||
try {
|
||||
newSettings = normaliseImportedIdDerivationSettings(decodeSettingsFromQRCodeData(qr));
|
||||
newSettings = normaliseImportedIdentitySettings(decodeSettingsFromQRCodeData(qr));
|
||||
} catch {
|
||||
this._log("The QR configuration could not be decoded or contains unsupported settings.", LOG_LEVEL_NOTICE);
|
||||
return false;
|
||||
|
||||
@@ -3,12 +3,13 @@ import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
DEFAULT_SETTINGS,
|
||||
REMOTE_COUCHDB,
|
||||
REMOTE_MINIO,
|
||||
REMOTE_P2P,
|
||||
type ObsidianLiveSyncSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { SettingService } from "@vrtmrz/livesync-commonlib/compat/services/base/SettingService";
|
||||
import { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { createNewVaultSettings } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { createNewVaultSettings } from "@/common/newVaultSettings";
|
||||
import type {
|
||||
P2PConnectionProbeAdmission,
|
||||
P2PConnectionProbeAdmissionResult,
|
||||
@@ -98,6 +99,12 @@ function createSetupManager() {
|
||||
dialogManager,
|
||||
},
|
||||
setting,
|
||||
replicator: {
|
||||
createRemoteResource: vi.fn(async () => ({
|
||||
inspectPathIdentity: vi.fn(async () => ({ status: "configured", pathNormalizationVersion: 0 })),
|
||||
dispose: vi.fn(async () => undefined),
|
||||
})),
|
||||
},
|
||||
} as any;
|
||||
const core: any = {
|
||||
_services: services,
|
||||
@@ -165,6 +172,99 @@ describe("SetupManager", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.each([REMOTE_COUCHDB, REMOTE_MINIO])(
|
||||
"uses an existing remote declaration during first manual setup for %s",
|
||||
async (remoteType) => {
|
||||
const { manager, core, dialogManager } = createSetupManager();
|
||||
const dispose = vi.fn(async () => undefined);
|
||||
core.services.replicator.createRemoteResource.mockResolvedValue({
|
||||
inspectPathIdentity: vi.fn(async () => ({ status: "configured", pathNormalizationVersion: 0 })),
|
||||
dispose,
|
||||
});
|
||||
const confirm = vi.spyOn(manager, "onConfirmApplySettingsFromWizard").mockResolvedValue(true);
|
||||
dialogManager.openWithExplicitCancel.mockResolvedValue({
|
||||
couchDB_URI: "https://example.test",
|
||||
couchDB_DBNAME: "vault",
|
||||
couchDB_USER: "user",
|
||||
couchDB_PASSWORD: "pass",
|
||||
endpoint: "https://objects.test",
|
||||
bucket: "vault",
|
||||
accessKey: "key",
|
||||
secretKey: "secret",
|
||||
region: "auto",
|
||||
bucketPrefix: "",
|
||||
});
|
||||
const initial = { ...createLegacyRemoteSetting(), pathNormalizationVersion: 1 } as ObsidianLiveSyncSettings;
|
||||
if (remoteType === REMOTE_COUCHDB) await manager.onCouchDBManualSetup(UserMode.NewUser, initial);
|
||||
else await manager.onBucketManualSetup(UserMode.NewUser, initial);
|
||||
expect(confirm).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ pathNormalizationVersion: 0 }),
|
||||
UserMode.NewUser,
|
||||
true
|
||||
);
|
||||
expect(dispose).toHaveBeenCalledOnce();
|
||||
expect(initial.pathNormalizationVersion).toBe(1);
|
||||
}
|
||||
);
|
||||
|
||||
it.each([
|
||||
[{ status: "empty" }, 1],
|
||||
[{ status: "configured", pathNormalizationVersion: 1 }, 1],
|
||||
] as const)("uses inspected setup identity %s", async (inspection, expected) => {
|
||||
const { manager, core, dialogManager } = createSetupManager();
|
||||
core.services.replicator.createRemoteResource.mockResolvedValue({
|
||||
inspectPathIdentity: vi.fn(async () => inspection),
|
||||
dispose: vi.fn(async () => undefined),
|
||||
});
|
||||
const confirm = vi.spyOn(manager, "onConfirmApplySettingsFromWizard").mockResolvedValue(true);
|
||||
dialogManager.openWithExplicitCancel.mockResolvedValue({
|
||||
couchDB_URI: "https://example.test",
|
||||
couchDB_DBNAME: "vault",
|
||||
});
|
||||
await manager.onCouchDBManualSetup(UserMode.NewUser, createNewVaultSettings());
|
||||
expect(confirm).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ pathNormalizationVersion: expected }),
|
||||
UserMode.NewUser,
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
it("does not apply setup settings when identity inspection fails", async () => {
|
||||
const { manager, core, dialogManager, setting } = createSetupManager();
|
||||
const failure = new Error("remote unavailable");
|
||||
const dispose = vi.fn(async () => undefined);
|
||||
core.services.replicator.createRemoteResource.mockResolvedValue({
|
||||
inspectPathIdentity: vi.fn(async () => {
|
||||
throw failure;
|
||||
}),
|
||||
dispose,
|
||||
});
|
||||
const confirm = vi.spyOn(manager, "onConfirmApplySettingsFromWizard").mockResolvedValue(true);
|
||||
dialogManager.openWithExplicitCancel.mockResolvedValue({
|
||||
couchDB_URI: "https://example.test",
|
||||
couchDB_DBNAME: "vault",
|
||||
});
|
||||
await expect(manager.onCouchDBManualSetup(UserMode.NewUser, createNewVaultSettings())).resolves.toBe(false);
|
||||
expect(confirm).not.toHaveBeenCalled();
|
||||
expect(setting.saveSettingData).not.toHaveBeenCalled();
|
||||
expect(dispose).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it.each([
|
||||
[UserMode.Update, true],
|
||||
[UserMode.NewUser, false],
|
||||
] as const)("keeps identity during profile edits in mode %s with activation %s", async (mode, activate) => {
|
||||
const { manager, core, dialogManager } = createSetupManager();
|
||||
const confirm = vi.spyOn(manager, "onConfirmApplySettingsFromWizard").mockResolvedValue(true);
|
||||
dialogManager.openWithExplicitCancel.mockResolvedValue({
|
||||
couchDB_URI: "https://example.test",
|
||||
couchDB_DBNAME: "vault",
|
||||
});
|
||||
await manager.onCouchDBManualSetup(mode, createLegacyRemoteSetting(), activate);
|
||||
expect(core.services.replicator.createRemoteResource).not.toHaveBeenCalled();
|
||||
expect(confirm).toHaveBeenCalledWith(expect.objectContaining({ pathNormalizationVersion: 0 }), mode, activate);
|
||||
});
|
||||
|
||||
it("starts manual new-user setup from the recommended new-Vault settings", async () => {
|
||||
const { manager, dialogManager } = createSetupManager();
|
||||
dialogManager.openWithExplicitCancel.mockResolvedValueOnce("configure-manually");
|
||||
@@ -192,12 +292,13 @@ describe("SetupManager", () => {
|
||||
expect(setting.currentSettings().activeConfigurationId).toBe("legacy-couchdb");
|
||||
});
|
||||
|
||||
it("compatibility: treats omitted ID derivation fields in a Setup URI as legacy defaults", async () => {
|
||||
it("compatibility: treats omitted identity fields in a Setup URI as legacy defaults", async () => {
|
||||
const { manager, setting, dialogManager } = createSetupManager();
|
||||
const savedKey = "12".repeat(32);
|
||||
setting.settings = {
|
||||
...createLegacyRemoteSetting(),
|
||||
isConfigured: true,
|
||||
pathNormalizationVersion: 1,
|
||||
idDerivationVersion: 1,
|
||||
idDerivationKey: savedKey,
|
||||
};
|
||||
@@ -205,6 +306,7 @@ describe("SetupManager", () => {
|
||||
...createLegacyRemoteSetting(),
|
||||
isConfigured: true,
|
||||
} as Partial<ObsidianLiveSyncSettings>;
|
||||
delete imported.pathNormalizationVersion;
|
||||
delete imported.idDerivationVersion;
|
||||
delete imported.idDerivationKey;
|
||||
vi.spyOn(setting, "adjustSettings").mockImplementation((settings) => Promise.resolve(settings));
|
||||
@@ -213,6 +315,7 @@ describe("SetupManager", () => {
|
||||
await manager.onUseSetupURI(UserMode.Unknown, "mock-config://legacy-settings");
|
||||
|
||||
const mergedSettings = vi.mocked(setting.adjustSettings).mock.calls[0][0];
|
||||
expect(mergedSettings.pathNormalizationVersion).toBe(0);
|
||||
expect(mergedSettings.idDerivationVersion).toBe(0);
|
||||
expect(mergedSettings.idDerivationKey).toBe("");
|
||||
expect(setting.currentSettings().idDerivationKey).toBe(savedKey);
|
||||
@@ -489,44 +592,47 @@ describe("SetupManager", () => {
|
||||
it.each([
|
||||
{ importedSize: 0, expectedSize: 0 },
|
||||
{ importedSize: undefined, expectedSize: 60 },
|
||||
])("preserves modern Setup URI profiles and adopts the supplied chunk size $importedSize", async ({ importedSize, expectedSize }) => {
|
||||
const { manager, setting, dialogManager } = createSetupManager();
|
||||
setting.settings = {
|
||||
...setting.currentSettings(),
|
||||
customChunkSize: 60,
|
||||
};
|
||||
const imported = {
|
||||
...DEFAULT_SETTINGS,
|
||||
remoteConfigurations: {
|
||||
couch: {
|
||||
id: "couch",
|
||||
name: "Office CouchDB",
|
||||
uri: "sls+https://alice:secret@couch.example/?db=notes",
|
||||
isEncrypted: false,
|
||||
])(
|
||||
"preserves modern Setup URI profiles and adopts the supplied chunk size $importedSize",
|
||||
async ({ importedSize, expectedSize }) => {
|
||||
const { manager, setting, dialogManager } = createSetupManager();
|
||||
setting.settings = {
|
||||
...setting.currentSettings(),
|
||||
customChunkSize: 60,
|
||||
};
|
||||
const imported = {
|
||||
...DEFAULT_SETTINGS,
|
||||
remoteConfigurations: {
|
||||
couch: {
|
||||
id: "couch",
|
||||
name: "Office CouchDB",
|
||||
uri: "sls+https://alice:secret@couch.example/?db=notes",
|
||||
isEncrypted: false,
|
||||
},
|
||||
archive: {
|
||||
id: "archive",
|
||||
name: "Archive bucket",
|
||||
uri: "sls+s3://key:secret@storage.example/?endpoint=https%3A%2F%2Fstorage.example&bucket=archive®ion=auto",
|
||||
isEncrypted: false,
|
||||
},
|
||||
},
|
||||
archive: {
|
||||
id: "archive",
|
||||
name: "Archive bucket",
|
||||
uri: "sls+s3://key:secret@storage.example/?endpoint=https%3A%2F%2Fstorage.example&bucket=archive®ion=auto",
|
||||
isEncrypted: false,
|
||||
},
|
||||
},
|
||||
activeConfigurationId: "archive",
|
||||
} as ObsidianLiveSyncSettings;
|
||||
if (importedSize === undefined) delete (imported as Partial<ObsidianLiveSyncSettings>).customChunkSize;
|
||||
else imported.customChunkSize = importedSize;
|
||||
dialogManager.openWithExplicitCancel
|
||||
.mockResolvedValueOnce(imported)
|
||||
.mockResolvedValueOnce("compatible-existing-user");
|
||||
activeConfigurationId: "archive",
|
||||
} as ObsidianLiveSyncSettings;
|
||||
if (importedSize === undefined) delete (imported as Partial<ObsidianLiveSyncSettings>).customChunkSize;
|
||||
else imported.customChunkSize = importedSize;
|
||||
dialogManager.openWithExplicitCancel
|
||||
.mockResolvedValueOnce(imported)
|
||||
.mockResolvedValueOnce("compatible-existing-user");
|
||||
|
||||
await manager.onUseSetupURI(UserMode.Unknown, "mock-config://modern-settings");
|
||||
await manager.onUseSetupURI(UserMode.Unknown, "mock-config://modern-settings");
|
||||
|
||||
const current = setting.currentSettings();
|
||||
expect(current.customChunkSize).toBe(expectedSize);
|
||||
expect(current.remoteConfigurations).toEqual(imported.remoteConfigurations);
|
||||
expect(current.activeConfigurationId).toBe("archive");
|
||||
expect(Object.keys(current.remoteConfigurations).some((id) => id.startsWith("legacy-"))).toBe(false);
|
||||
});
|
||||
const current = setting.currentSettings();
|
||||
expect(current.customChunkSize).toBe(expectedSize);
|
||||
expect(current.remoteConfigurations).toEqual(imported.remoteConfigurations);
|
||||
expect(current.activeConfigurationId).toBe("archive");
|
||||
expect(Object.keys(current.remoteConfigurations).some((id) => id.startsWith("legacy-"))).toBe(false);
|
||||
}
|
||||
);
|
||||
|
||||
it("adds and activates a manually configured CouchDB without replacing existing profiles", async () => {
|
||||
const { manager, setting, dialogManager } = createSetupManager();
|
||||
|
||||
@@ -17,6 +17,7 @@ export interface ConflictCheckingDependencies {
|
||||
IConflictService,
|
||||
"getOptionalConflictCheckMethod" | "queueCheckFor" | "resolve" | "resolveByNewest"
|
||||
>;
|
||||
readonly signal: AbortSignal;
|
||||
readonly conflictProcessQueueCount: ReactiveSource<number>;
|
||||
readonly cancelActiveResolution: (filename: FilePathWithPrefix) => void;
|
||||
readonly currentSettings: () => ConflictCheckingSettings;
|
||||
@@ -27,6 +28,7 @@ export interface ConflictCheckingDependencies {
|
||||
export interface ConflictCheckingHandlers {
|
||||
readonly queueCheckForIfOpen: (file: FilePathWithPrefix) => Promise<void>;
|
||||
readonly queueCheckFor: (file: FilePathWithPrefix) => Promise<void>;
|
||||
/** Wait for active checks; false means the queue was cancelled during shutdown. */
|
||||
readonly ensureAllProcessed: () => Promise<boolean>;
|
||||
}
|
||||
|
||||
@@ -45,11 +47,13 @@ export function createConflictCheckingHandlers(dependencies: ConflictCheckingDep
|
||||
// and automatic merges can use the remaining execution slots.
|
||||
concurrency: 10,
|
||||
onProgress(remaining) {
|
||||
// Accepted checks still update their accounting as they settle.
|
||||
dependencies.conflictProcessQueueCount.value = remaining;
|
||||
},
|
||||
});
|
||||
|
||||
const queueCheckForIfOpen = async (file: FilePathWithPrefix): Promise<void> => {
|
||||
if (dependencies.signal.aborted) return;
|
||||
const path = file;
|
||||
if (dependencies.currentSettings().checkConflictOnlyOnOpen) {
|
||||
const activeFile: FilePath | undefined = dependencies.vault.getActiveFilePath();
|
||||
@@ -62,7 +66,9 @@ export function createConflictCheckingHandlers(dependencies: ConflictCheckingDep
|
||||
};
|
||||
|
||||
const queueCheckFor = async (file: FilePathWithPrefix): Promise<void> => {
|
||||
if (dependencies.signal.aborted) return;
|
||||
const optionalConflictResult = await dependencies.conflict.getOptionalConflictCheckMethod(file);
|
||||
if (dependencies.signal.aborted) return;
|
||||
if (optionalConflictResult == true) {
|
||||
// The conflict has been resolved by another process.
|
||||
return;
|
||||
@@ -79,6 +85,16 @@ export function createConflictCheckingHandlers(dependencies: ConflictCheckingDep
|
||||
};
|
||||
|
||||
const ensureAllProcessed = (): Promise<boolean> => conflictQueue.waitForIdle();
|
||||
const abortQueue = (): void => {
|
||||
// Shutdown keeps its existing database-close order. Observe cleanup
|
||||
// without making the lifecycle wait for an active resolver.
|
||||
void conflictQueue.abort().catch((error) => dependencies.log(error, LOG_LEVEL_VERBOSE));
|
||||
};
|
||||
if (dependencies.signal.aborted) {
|
||||
abortQueue();
|
||||
} else {
|
||||
dependencies.signal.addEventListener("abort", abortQueue, { once: true });
|
||||
}
|
||||
|
||||
return {
|
||||
queueCheckForIfOpen,
|
||||
|
||||
@@ -14,7 +14,7 @@ import {
|
||||
type MetaEntry,
|
||||
type ObsidianLiveSyncSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { EVENT_CONFLICT_CANCELLED } from "@vrtmrz/livesync-commonlib/compat/events/coreEvents";
|
||||
import { EVENT_CONFLICT_CANCELLED, EVENT_PLUGIN_UNLOADED } from "@vrtmrz/livesync-commonlib/compat/events/coreEvents";
|
||||
import { NO_INTERACTION } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import type { IAPIService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
|
||||
import {
|
||||
@@ -44,6 +44,7 @@ type HarnessOptions = {
|
||||
function createHarness(options: HarnessOptions = {}) {
|
||||
const context = createServiceContext();
|
||||
const conflict = new InjectableConflictService(context);
|
||||
const unloadHandlers: Array<() => Promise<boolean>> = [];
|
||||
const settings = { ...DEFAULT_SETTINGS, ...options.settings };
|
||||
const tryAutoMerge = vi.fn();
|
||||
const databaseFileAccess = {
|
||||
@@ -62,7 +63,14 @@ function createHarness(options: HarnessOptions = {}) {
|
||||
const activeFile = vi.fn(() => options.activeFile);
|
||||
const services = {
|
||||
API: { addLog },
|
||||
appLifecycle: { isSuspended: vi.fn(() => false) },
|
||||
appLifecycle: {
|
||||
isSuspended: vi.fn(() => false),
|
||||
onUnload: {
|
||||
addHandler: vi.fn((handler: () => Promise<boolean>) => {
|
||||
unloadHandlers.push(handler);
|
||||
}),
|
||||
},
|
||||
},
|
||||
conflict,
|
||||
context,
|
||||
database: { localDatabase: { tryAutoMerge } },
|
||||
@@ -88,9 +96,22 @@ function createHarness(options: HarnessOptions = {}) {
|
||||
serviceModules,
|
||||
storageAccess,
|
||||
tryAutoMerge,
|
||||
unload: async () => {
|
||||
for (const handler of unloadHandlers) await handler();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function deferred<T>() {
|
||||
let resolve!: (value: T) => void;
|
||||
let reject!: (reason?: unknown) => void;
|
||||
const promise = new Promise<T>((resolvePromise, rejectPromise) => {
|
||||
resolve = resolvePromise;
|
||||
reject = rejectPromise;
|
||||
});
|
||||
return { promise, resolve, reject };
|
||||
}
|
||||
|
||||
function leaf(rev: string, data: string, mtime: number): ConflictLeaf {
|
||||
return { rev, data, mtime, ctime: mtime, deleted: false };
|
||||
}
|
||||
@@ -919,6 +940,347 @@ describe("conflict resolution serviceFeature", () => {
|
||||
expect(harness.conflict.conflictProcessQueueCount.value).toBe(0);
|
||||
});
|
||||
|
||||
it("reports queue cancellation and logs a resolver rejection after unload", async () => {
|
||||
const path = "queued-rejection-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness();
|
||||
const started = deferred<void>();
|
||||
const pendingResolve = deferred<void>();
|
||||
const resolve = vi.fn(async () => {
|
||||
started.resolve(undefined);
|
||||
await pendingResolve.promise;
|
||||
});
|
||||
harness.conflict.resolve.setHandler(resolve, true);
|
||||
|
||||
await harness.conflict.queueCheckFor(path);
|
||||
await started.promise;
|
||||
const completion = harness.conflict.ensureAllProcessed().then(
|
||||
(result) => ({ status: "resolved" as const, result }),
|
||||
(error: unknown) => ({ status: "rejected" as const, error })
|
||||
);
|
||||
|
||||
await harness.unload();
|
||||
await harness.unload();
|
||||
const failure = new Error("Resolver rejected after unload");
|
||||
pendingResolve.reject(failure);
|
||||
|
||||
await expect(completion).resolves.toEqual({ status: "resolved", result: false });
|
||||
expect(resolve).toHaveBeenCalledOnce();
|
||||
expect(harness.addLog).toHaveBeenCalledWith(failure, LOG_LEVEL_VERBOSE, "");
|
||||
});
|
||||
|
||||
it("allows an active inspection to complete normally after unload", async () => {
|
||||
const path = "not-conflicted-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness();
|
||||
const queueCheckFor = vi.spyOn(harness.conflict, "queueCheckFor");
|
||||
const started = deferred<void>();
|
||||
const pendingMergeCheck = deferred<{ ok: typeof NOT_CONFLICTED }>();
|
||||
harness.tryAutoMerge.mockImplementation(async () => {
|
||||
started.resolve(undefined);
|
||||
return await pendingMergeCheck.promise;
|
||||
});
|
||||
const emitEvent = vi.spyOn(harness.context.events, "emitEvent");
|
||||
const operation = harness.conflict.resolve(path);
|
||||
const completion = operation.then(
|
||||
() => ({ status: "resolved" as const }),
|
||||
(error: unknown) => ({ status: "rejected" as const, error })
|
||||
);
|
||||
await started.promise;
|
||||
expect(harness.tryAutoMerge).toHaveBeenCalledWith(path, true);
|
||||
|
||||
harness.context.events.emitEvent(EVENT_PLUGIN_UNLOADED);
|
||||
pendingMergeCheck.resolve({ ok: NOT_CONFLICTED });
|
||||
|
||||
await expect(completion).resolves.toEqual({ status: "resolved" });
|
||||
expect(emitEvent).toHaveBeenCalledWith(EVENT_CONFLICT_CANCELLED, path);
|
||||
expect(queueCheckFor).not.toHaveBeenCalled();
|
||||
expect(harness.addLog).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("continues an accepted auto-merge when storage completes after unload", async () => {
|
||||
const path = "stored-merge-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness({ settings: { syncAfterMerge: false } });
|
||||
const queueCheckFor = vi.spyOn(harness.conflict, "queueCheckFor");
|
||||
const started = deferred<void>();
|
||||
const pendingStore = deferred<boolean>();
|
||||
harness.tryAutoMerge.mockResolvedValue({ result: "Merged content\n", conflictedRev: "2-conflict" });
|
||||
harness.databaseFileAccess.storeContent.mockImplementation(async () => {
|
||||
started.resolve(undefined);
|
||||
return await pendingStore.promise;
|
||||
});
|
||||
const emitEvent = vi.spyOn(harness.context.events, "emitEvent");
|
||||
const operation = harness.conflict.resolve(path);
|
||||
const completion = operation.then(
|
||||
() => ({ status: "resolved" as const }),
|
||||
(error: unknown) => ({ status: "rejected" as const, error })
|
||||
);
|
||||
await started.promise;
|
||||
expect(harness.databaseFileAccess.storeContent).toHaveBeenCalledWith(path, "Merged content\n");
|
||||
|
||||
await harness.unload();
|
||||
pendingStore.resolve(true);
|
||||
|
||||
await expect(completion).resolves.toEqual({ status: "resolved" });
|
||||
expect(harness.fileHandler.deleteRevisionFromDB).toHaveBeenCalledWith(path, "2-conflict");
|
||||
expect(harness.fileHandler.dbToStorage).toHaveBeenCalled();
|
||||
expect(queueCheckFor).toHaveBeenCalledWith(path);
|
||||
expect(emitEvent).toHaveBeenCalledWith(EVENT_CONFLICT_CANCELLED, path);
|
||||
expect(harness.addLog).toHaveBeenCalled();
|
||||
await expect(harness.conflict.ensureAllProcessed()).resolves.toBe(false);
|
||||
});
|
||||
|
||||
it("reflects a revision deletion which succeeds after the unload event", async () => {
|
||||
const path = "deleted-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness();
|
||||
const started = deferred<void>();
|
||||
const pendingDelete = deferred<boolean>();
|
||||
harness.fileHandler.deleteRevisionFromDB.mockImplementation(async () => {
|
||||
started.resolve(undefined);
|
||||
return await pendingDelete.promise;
|
||||
});
|
||||
const emitEvent = vi.spyOn(harness.context.events, "emitEvent");
|
||||
const operation = harness.conflict.resolveByDeletingRevision(path, "2-conflict", "UI Selected");
|
||||
const completion = operation.then(
|
||||
(result) => ({ status: "resolved" as const, result }),
|
||||
(error: unknown) => ({ status: "rejected" as const, error })
|
||||
);
|
||||
await started.promise;
|
||||
expect(harness.fileHandler.deleteRevisionFromDB).toHaveBeenCalledWith(path, "2-conflict");
|
||||
|
||||
harness.context.events.emitEvent(EVENT_PLUGIN_UNLOADED);
|
||||
pendingDelete.resolve(true);
|
||||
|
||||
await expect(completion).resolves.toEqual({ status: "resolved", result: AUTO_MERGED });
|
||||
expect(harness.databaseFileAccess.getConflictedRevs).toHaveBeenCalledWith(path);
|
||||
expect(harness.fileHandler.dbToStorage).toHaveBeenCalledWith(path, path, true);
|
||||
expect(emitEvent).toHaveBeenCalledWith(EVENT_CONFLICT_CANCELLED, path);
|
||||
expect(harness.addLog).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("lets active replication finish while rejecting its next conflict check", async () => {
|
||||
const path = "replication-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness({ settings: { syncAfterMerge: true } });
|
||||
const queueCheckFor = vi.spyOn(harness.conflict, "queueCheckFor");
|
||||
const started = deferred<void>();
|
||||
const pendingReplication = deferred<{ status: "completed" }>();
|
||||
harness.tryAutoMerge.mockResolvedValue({ ok: AUTO_MERGED });
|
||||
harness.services.replication.replicateUnattendedByEvent.mockImplementation(async () => {
|
||||
started.resolve(undefined);
|
||||
return await pendingReplication.promise;
|
||||
});
|
||||
const operation = harness.conflict.resolve(path);
|
||||
const completion = operation.then(
|
||||
() => ({ status: "resolved" as const }),
|
||||
(error: unknown) => ({ status: "rejected" as const, error })
|
||||
);
|
||||
await started.promise;
|
||||
expect(harness.services.replication.replicateUnattendedByEvent).toHaveBeenCalledWith({
|
||||
trigger: "merge",
|
||||
interaction: NO_INTERACTION,
|
||||
});
|
||||
|
||||
await harness.unload();
|
||||
pendingReplication.resolve({ status: "completed" });
|
||||
|
||||
await expect(completion).resolves.toEqual({ status: "resolved" });
|
||||
expect(queueCheckFor).toHaveBeenCalledWith(path);
|
||||
expect(harness.tryAutoMerge).toHaveBeenCalledOnce();
|
||||
await expect(harness.conflict.ensureAllProcessed()).resolves.toBe(false);
|
||||
expect(harness.addLog).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not start resolveByNewest when an optional conflict check returns newer after unload", async () => {
|
||||
const path = "optional-newer-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness();
|
||||
const started = deferred<void>();
|
||||
const pendingOptionalCheck = deferred<"newer" | undefined>();
|
||||
const optionalCheck = vi.fn(async () => {
|
||||
started.resolve(undefined);
|
||||
return await pendingOptionalCheck.promise;
|
||||
});
|
||||
const resolveByNewest = vi.fn(async () => true);
|
||||
const unregisterOptionalCheck = harness.conflict.getOptionalConflictCheckMethod.addHandler(optionalCheck);
|
||||
harness.conflict.resolveByNewest.setHandler(resolveByNewest, true);
|
||||
const operation = harness.conflict.queueCheckFor(path);
|
||||
const completion = operation.then(
|
||||
() => ({ status: "resolved" as const }),
|
||||
(error: unknown) => ({ status: "rejected" as const, error })
|
||||
);
|
||||
try {
|
||||
await started.promise;
|
||||
expect(optionalCheck).toHaveBeenCalledWith(path);
|
||||
|
||||
await harness.unload();
|
||||
harness.addLog.mockImplementation(() => {
|
||||
throw new Error("Handler addLog not assigned");
|
||||
});
|
||||
pendingOptionalCheck.resolve("newer");
|
||||
|
||||
await expect(completion).resolves.toEqual({ status: "resolved" });
|
||||
expect(resolveByNewest).not.toHaveBeenCalled();
|
||||
expect(harness.addLog).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
unregisterOptionalCheck();
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["lifecycle", "event"] as const)(
|
||||
"stops admission and preserves ordinary database failures after %s disposal",
|
||||
async (trigger) => {
|
||||
const path = "operation-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness();
|
||||
const optionalCheck = vi.fn(async () => "newer" as const);
|
||||
const unregisterOptionalCheck = harness.conflict.getOptionalConflictCheckMethod.addHandler(optionalCheck);
|
||||
const publicOperations = [
|
||||
vi.spyOn(harness.conflict, "queueCheckForIfOpen"),
|
||||
vi.spyOn(harness.conflict, "queueCheckFor"),
|
||||
vi.spyOn(harness.conflict, "resolve"),
|
||||
vi.spyOn(harness.conflict, "resolveByDeletingRevision"),
|
||||
vi.spyOn(harness.conflict, "resolveByNewest"),
|
||||
vi.spyOn(harness.conflict, "resolveAllConflictedFilesByNewerOnes"),
|
||||
];
|
||||
const emitEvent = vi.spyOn(harness.context.events, "emitEvent");
|
||||
|
||||
if (trigger === "lifecycle") {
|
||||
await harness.unload();
|
||||
} else {
|
||||
harness.context.events.emitEvent(EVENT_PLUGIN_UNLOADED);
|
||||
}
|
||||
const failure = new Error("database closed");
|
||||
harness.tryAutoMerge.mockRejectedValue(failure);
|
||||
harness.databaseFileAccess.fetchEntryMeta.mockResolvedValue(false);
|
||||
harness.fileHandler.deleteRevisionFromDB.mockResolvedValue(false);
|
||||
|
||||
try {
|
||||
const results = await Promise.allSettled([
|
||||
harness.conflict.queueCheckForIfOpen(path),
|
||||
harness.conflict.queueCheckFor(path),
|
||||
harness.conflict.resolve(path),
|
||||
harness.conflict.resolveByDeletingRevision(path, "2-conflict", "UI Selected"),
|
||||
harness.conflict.resolveByNewest(path),
|
||||
harness.conflict.resolveAllConflictedFilesByNewerOnes(),
|
||||
]);
|
||||
|
||||
expect(results).toEqual([
|
||||
{ status: "fulfilled", value: undefined },
|
||||
{ status: "fulfilled", value: undefined },
|
||||
{ status: "rejected", reason: failure },
|
||||
{ status: "fulfilled", value: MISSING_OR_ERROR },
|
||||
{ status: "fulfilled", value: false },
|
||||
{ status: "fulfilled", value: undefined },
|
||||
]);
|
||||
await expect(harness.conflict.ensureAllProcessed()).resolves.toBe(false);
|
||||
for (const operation of publicOperations) expect(operation).toHaveBeenCalledOnce();
|
||||
expect(optionalCheck).not.toHaveBeenCalled();
|
||||
expect(harness.tryAutoMerge).toHaveBeenCalledOnce();
|
||||
expect(harness.databaseFileAccess.fetchEntryMeta).toHaveBeenCalledOnce();
|
||||
expect(harness.databaseFileAccess.getConflictedRevs).not.toHaveBeenCalled();
|
||||
expect(harness.databaseFileAccess.storeContent).not.toHaveBeenCalled();
|
||||
expect(harness.fileHandler.deleteRevisionFromDB).toHaveBeenCalledOnce();
|
||||
expect(harness.fileHandler.dbToStorage).not.toHaveBeenCalled();
|
||||
expect(harness.storageAccess.getFileNames).toHaveBeenCalledOnce();
|
||||
expect(harness.services.replication.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(emitEvent).not.toHaveBeenCalledWith(EVENT_CONFLICT_CANCELLED, path);
|
||||
expect(harness.addLog).toHaveBeenCalled();
|
||||
} finally {
|
||||
unregisterOptionalCheck();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
it("cancels unstarted checks after active checks settle during unload", async () => {
|
||||
const paths = Array.from({ length: 11 }, (_, index) => `unload-concurrent-${index}.md` as FilePathWithPrefix);
|
||||
const harness = createHarness();
|
||||
const tenStarted = deferred<void>();
|
||||
const releaseResolvers = deferred<void>();
|
||||
const resolve = vi.fn(async (_path: FilePathWithPrefix) => {
|
||||
if (resolve.mock.calls.length === 10) tenStarted.resolve(undefined);
|
||||
await releaseResolvers.promise;
|
||||
});
|
||||
harness.conflict.resolve.setHandler(resolve, true);
|
||||
|
||||
await Promise.all(paths.map(async (path) => await harness.conflict.queueCheckFor(path)));
|
||||
await tenStarted.promise;
|
||||
expect(harness.conflict.conflictProcessQueueCount.value).toBe(11);
|
||||
|
||||
const completion = harness.conflict.ensureAllProcessed();
|
||||
harness.context.events.emitEvent(EVENT_PLUGIN_UNLOADED);
|
||||
releaseResolvers.resolve(undefined);
|
||||
|
||||
await expect(completion).resolves.toBe(false);
|
||||
expect(resolve).toHaveBeenCalledTimes(10);
|
||||
expect(harness.conflict.conflictProcessQueueCount.value).toBe(1);
|
||||
});
|
||||
|
||||
it("reports an unavailable database when an active bulk scan completes after unload", async () => {
|
||||
const path = "bulk-resolution-after-unload.md" as FilePathWithPrefix;
|
||||
const harness = createHarness();
|
||||
const namesStarted = deferred<void>();
|
||||
const pendingNames = deferred<FilePathWithPrefix[]>();
|
||||
harness.storageAccess.getFileNames.mockImplementation(async () => {
|
||||
namesStarted.resolve(undefined);
|
||||
return await pendingNames.promise;
|
||||
});
|
||||
|
||||
const operation = harness.conflict.resolveAllConflictedFilesByNewerOnes();
|
||||
await namesStarted.promise;
|
||||
expect(harness.addLog).toHaveBeenCalled();
|
||||
harness.addLog.mockClear();
|
||||
|
||||
await harness.unload();
|
||||
harness.databaseFileAccess.fetchEntryMeta.mockResolvedValue(false);
|
||||
pendingNames.resolve([path]);
|
||||
|
||||
await expect(operation).resolves.toBeUndefined();
|
||||
expect(harness.databaseFileAccess.fetchEntryMeta).toHaveBeenCalledWith(path, undefined, true);
|
||||
expect(harness.databaseFileAccess.getConflictedRevs).not.toHaveBeenCalled();
|
||||
expect(harness.fileHandler.deleteRevisionFromDB).not.toHaveBeenCalled();
|
||||
expect(harness.fileHandler.dbToStorage).not.toHaveBeenCalled();
|
||||
expect(
|
||||
harness.addLog.mock.calls.some(([message]) => String(message).includes("Could not get current revision"))
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps a replacement queue independent of repeated shutdown of the previous queue", async () => {
|
||||
const first = createHarness();
|
||||
const second = createHarness();
|
||||
first.tryAutoMerge.mockResolvedValue({ ok: NOT_CONFLICTED });
|
||||
second.tryAutoMerge.mockResolvedValue({ ok: NOT_CONFLICTED });
|
||||
|
||||
first.context.events.emitEvent(EVENT_PLUGIN_UNLOADED);
|
||||
await first.unload();
|
||||
await first.unload();
|
||||
await first.conflict.queueCheckFor("retired.md" as FilePathWithPrefix);
|
||||
await second.conflict.queueCheckFor("active.md" as FilePathWithPrefix);
|
||||
|
||||
await expect(first.conflict.ensureAllProcessed()).resolves.toBe(false);
|
||||
await expect(second.conflict.ensureAllProcessed()).resolves.toBe(true);
|
||||
expect(first.tryAutoMerge).not.toHaveBeenCalled();
|
||||
expect(second.tryAutoMerge).toHaveBeenCalledExactlyOnceWith("active.md", true);
|
||||
});
|
||||
|
||||
it("reports a storage failure after shutdown without treating the queue as completed", async () => {
|
||||
const harness = createHarness();
|
||||
const started = deferred<void>();
|
||||
const pendingStore = deferred<boolean>();
|
||||
const failure = new Error("database is closed");
|
||||
harness.tryAutoMerge.mockResolvedValue({ result: "Merged content", conflictedRev: "2-conflict" });
|
||||
harness.databaseFileAccess.storeContent.mockImplementation(() => {
|
||||
started.resolve(undefined);
|
||||
return pendingStore.promise;
|
||||
});
|
||||
|
||||
await harness.conflict.queueCheckFor("closed-database.md" as FilePathWithPrefix);
|
||||
await started.promise;
|
||||
const completion = harness.conflict.ensureAllProcessed();
|
||||
await harness.unload();
|
||||
pendingStore.reject(failure);
|
||||
|
||||
await expect(completion).resolves.toBe(false);
|
||||
expect(harness.fileHandler.deleteRevisionFromDB).not.toHaveBeenCalled();
|
||||
expect(harness.fileHandler.dbToStorage).not.toHaveBeenCalled();
|
||||
expect(harness.addLog).toHaveBeenCalledWith(failure, LOG_LEVEL_VERBOSE, "");
|
||||
});
|
||||
|
||||
it("drains requeued automatic checks when every resolver slot is occupied", async () => {
|
||||
const harness = createHarness({ settings: { syncAfterMerge: false } });
|
||||
const paths = Array.from({ length: 10 }, (_, index) => `requeue-${index}.md` as FilePathWithPrefix);
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import PouchDB from "pouchdb-core";
|
||||
import MemoryAdapter from "pouchdb-adapter-memory";
|
||||
import { createServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { InjectableConflictService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableConflictService";
|
||||
import { LOG_LEVEL_VERBOSE, type FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { createConflictCheckingHandlers } from "./checker";
|
||||
|
||||
PouchDB.plugin(MemoryAdapter);
|
||||
|
||||
type Probe = { completed: number };
|
||||
|
||||
function deferred() {
|
||||
let resolve!: () => void;
|
||||
const promise = new Promise<void>((finish) => {
|
||||
resolve = finish;
|
||||
});
|
||||
return { promise, resolve };
|
||||
}
|
||||
|
||||
describe("conflict-check shutdown with a real PouchDB handle", () => {
|
||||
it.each([false, true])(
|
||||
"reports cancellation and preserves the database outcome when closed before completion: %s",
|
||||
async (closeDuringShutdown) => {
|
||||
const name = `conflict-shutdown-${crypto.randomUUID()}`;
|
||||
const db = new PouchDB<Probe>(name, { adapter: "memory" });
|
||||
const started = deferred();
|
||||
const release = deferred();
|
||||
const conflict = new InjectableConflictService(createServiceContext());
|
||||
const lifetime = new AbortController();
|
||||
const log = vi.fn();
|
||||
const resolver = vi.fn(async () => {
|
||||
started.resolve();
|
||||
await release.promise;
|
||||
const probe = await db.get("_local/shutdown-probe");
|
||||
await db.put({ ...probe, completed: probe.completed + 1 });
|
||||
});
|
||||
conflict.resolve.setHandler(resolver);
|
||||
const checking = createConflictCheckingHandlers({
|
||||
conflict,
|
||||
signal: lifetime.signal,
|
||||
conflictProcessQueueCount: conflict.conflictProcessQueueCount,
|
||||
cancelActiveResolution: vi.fn(),
|
||||
currentSettings: () => ({ checkConflictOnlyOnOpen: false }),
|
||||
vault: { getActiveFilePath: () => undefined },
|
||||
log,
|
||||
});
|
||||
conflict.queueCheckFor.setHandler(checking.queueCheckFor);
|
||||
let closed = false;
|
||||
let reopened: PouchDB.Database<Probe> | undefined;
|
||||
let completion: Promise<boolean> | undefined;
|
||||
try {
|
||||
await db.put({ _id: "_local/shutdown-probe", completed: 0 });
|
||||
await checking.queueCheckFor("active.md" as FilePathWithPrefix);
|
||||
await started.promise;
|
||||
completion = checking.ensureAllProcessed();
|
||||
lifetime.abort();
|
||||
lifetime.abort();
|
||||
await checking.queueCheckFor("late.md" as FilePathWithPrefix);
|
||||
if (closeDuringShutdown) {
|
||||
await db.close();
|
||||
closed = true;
|
||||
}
|
||||
release.resolve();
|
||||
|
||||
await expect(completion).resolves.toBe(false);
|
||||
expect(resolver).toHaveBeenCalledOnce();
|
||||
if (closeDuringShutdown) {
|
||||
expect(log).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ message: expect.stringMatching(/closed/i) }),
|
||||
LOG_LEVEL_VERBOSE
|
||||
);
|
||||
} else {
|
||||
expect(log).not.toHaveBeenCalled();
|
||||
await db.close();
|
||||
closed = true;
|
||||
}
|
||||
reopened = new PouchDB<Probe>(name, { adapter: "memory" });
|
||||
const persisted = await reopened.get("_local/shutdown-probe");
|
||||
expect(persisted.completed).toBe(closeDuringShutdown ? 0 : 1);
|
||||
} finally {
|
||||
release.resolve();
|
||||
await completion?.catch(() => undefined);
|
||||
if (!closed) await db.close();
|
||||
await (reopened ?? new PouchDB<Probe>(name, { adapter: "memory" })).destroy();
|
||||
}
|
||||
}
|
||||
);
|
||||
});
|
||||
@@ -6,6 +6,7 @@ import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib
|
||||
import type { InjectableServiceHub } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableServiceHub";
|
||||
import { createConflictCheckingHandlers } from "./checker";
|
||||
import { createConflictResolutionOperations } from "./operations";
|
||||
import { EVENT_PLUGIN_UNLOADED } from "@vrtmrz/livesync-commonlib/compat/events/coreEvents";
|
||||
|
||||
export type ConflictResolutionHost = NecessaryServices<
|
||||
"appLifecycle" | "conflict" | "database" | "replication" | "setting" | "vault",
|
||||
@@ -18,6 +19,7 @@ export type ConflictResolutionHost = NecessaryServices<
|
||||
/** Compose the host-neutral conflict handlers with the host's borrowed logger. */
|
||||
export const useConflictResolutionFeature = createServiceFeature((host: ConflictResolutionHost): void => {
|
||||
const { services, serviceModules, log } = host;
|
||||
const lifetime = new AbortController();
|
||||
const operations = createConflictResolutionOperations({
|
||||
events: services.context.events,
|
||||
databaseFileAccess: serviceModules.databaseFileAccess,
|
||||
@@ -38,6 +40,7 @@ export const useConflictResolutionFeature = createServiceFeature((host: Conflict
|
||||
currentSettings: () => services.setting.currentSettings(),
|
||||
vault: services.vault,
|
||||
log,
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
|
||||
services.conflict.queueCheckForIfOpen.setHandler(checking.queueCheckForIfOpen);
|
||||
@@ -47,6 +50,13 @@ export const useConflictResolutionFeature = createServiceFeature((host: Conflict
|
||||
services.conflict.resolve.setHandler(operations.resolve);
|
||||
services.conflict.resolveByNewest.setHandler(operations.resolveByNewest);
|
||||
services.conflict.resolveAllConflictedFilesByNewerOnes.setHandler(operations.resolveAllConflictedFilesByNewerOnes);
|
||||
|
||||
const stop = (): void => lifetime.abort();
|
||||
services.context.events.onceEvent(EVENT_PLUGIN_UNLOADED, stop, { signal: lifetime.signal });
|
||||
services.appLifecycle.onUnload.addHandler(() => {
|
||||
stop();
|
||||
return Promise.resolve(true);
|
||||
});
|
||||
});
|
||||
|
||||
export { createConflictCheckingHandlers } from "./checker";
|
||||
|
||||
@@ -27,9 +27,10 @@ import type { LiveSyncEventHub } from "@vrtmrz/livesync-commonlib/context";
|
||||
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
|
||||
import { EVENT_CONFLICT_CANCELLED } from "@vrtmrz/livesync-commonlib/compat/events/coreEvents";
|
||||
import { NO_INTERACTION } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { parseDocumentPath, splitDocumentPath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
import { isLockAcquired, withLock } from "octagonal-wheels/concurrency/lock";
|
||||
import diff_match_patch from "diff-match-patch";
|
||||
import { stripAllPrefixes, isPlainText } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
import { isPlainText } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
|
||||
export type ConflictResolutionSettings = Pick<
|
||||
ObsidianLiveSyncSettings,
|
||||
@@ -67,6 +68,7 @@ export interface ConflictResolutionOperations {
|
||||
readonly resolveAllConflictedFilesByNewerOnes: () => Promise<void>;
|
||||
}
|
||||
|
||||
/** Create conflict-resolution operations which retain their normal completion and failure paths. */
|
||||
export function createConflictResolutionOperations(
|
||||
dependencies: ConflictResolutionOperationsDependencies
|
||||
): ConflictResolutionOperations {
|
||||
@@ -108,7 +110,9 @@ export function createConflictResolutionOperations(
|
||||
return AUTO_MERGED;
|
||||
}
|
||||
// If no conflicts were found, write the resolved content to the storage.
|
||||
if (!(await dependencies.fileHandler.dbToStorage(path, stripAllPrefixes(path), true))) {
|
||||
const documentPath = parseDocumentPath(path);
|
||||
const { path: filePath } = splitDocumentPath(documentPath);
|
||||
if (!(await dependencies.fileHandler.dbToStorage(documentPath, filePath, true))) {
|
||||
dependencies.log(`Could not write the resolved content to the storage: ${path}`, LOG_LEVEL_NOTICE);
|
||||
return MISSING_OR_ERROR;
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ import type { FilePath, FilePathWithPrefix, UXFileInfo } from "@vrtmrz/livesync-
|
||||
import { readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import type { DatabaseFileAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/DatabaseFileAccess";
|
||||
import type { IFileHandler } from "@vrtmrz/livesync-commonlib/compat/interfaces/FileHandler";
|
||||
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
import { parseDocumentPath, splitDocumentPath } from "@vrtmrz/livesync-commonlib/storage";
|
||||
|
||||
export type DocumentHistoryRestorationCore = {
|
||||
databaseFileAccess: Pick<
|
||||
@@ -54,8 +54,8 @@ export async function restoreDocumentHistoryRevision(
|
||||
}
|
||||
|
||||
const body = readAsBlob(source);
|
||||
const storagePath = stripAllPrefixes(current.path);
|
||||
if (storagePath !== current.path || !isPathValid(storagePath)) {
|
||||
const { prefix, path: storagePath } = splitDocumentPath(parseDocumentPath(current.path));
|
||||
if (prefix !== "" || !isPathValid(storagePath)) {
|
||||
return { status: "unsupported-path" };
|
||||
}
|
||||
const file: UXFileInfo = {
|
||||
|
||||
@@ -17,8 +17,10 @@ export function useInteractiveConflictResolutionFeature(
|
||||
createDialogue: ConflictResolveDialogueFactory
|
||||
): void {
|
||||
const services = host.services;
|
||||
const lifetime = new AbortController();
|
||||
const operations = createInteractiveConflictResolutionOperations({
|
||||
events: services.context.events,
|
||||
signal: lifetime.signal,
|
||||
databaseFileAccess: host.serviceModules.databaseFileAccess,
|
||||
localDatabase: () => services.database.localDatabase,
|
||||
confirm: services.UI.confirm,
|
||||
@@ -61,24 +63,18 @@ export function useInteractiveConflictResolutionFeature(
|
||||
});
|
||||
services.appLifecycle.getUnresolvedMessages.addHandler(operations.getActiveConflictMessages);
|
||||
services.conflict.resolveByUserInteraction.addHandler(operations.resolveByUserInteraction);
|
||||
const eventSubscriptions = new AbortController();
|
||||
const dispose = () => {
|
||||
// Stop the refresh listener before cancellation so that unloading does
|
||||
// not start a database read which can race with database disposal.
|
||||
eventSubscriptions.abort();
|
||||
operations.dispose();
|
||||
};
|
||||
services.context.events.onEvent(
|
||||
EVENT_CONFLICT_CANCELLED,
|
||||
(filename) => {
|
||||
operations.invalidateWaitingResolution(filename);
|
||||
fireAndForget(() => operations.refreshConflictState(filename));
|
||||
},
|
||||
{ signal: eventSubscriptions.signal }
|
||||
{ signal: lifetime.signal }
|
||||
);
|
||||
services.context.events.onceEvent(EVENT_PLUGIN_UNLOADED, dispose, { signal: eventSubscriptions.signal });
|
||||
const stop = (): void => lifetime.abort();
|
||||
services.context.events.onceEvent(EVENT_PLUGIN_UNLOADED, stop, { signal: lifetime.signal });
|
||||
services.appLifecycle.onUnload.addHandler(() => {
|
||||
dispose();
|
||||
stop();
|
||||
return Promise.resolve(true);
|
||||
});
|
||||
}
|
||||
|
||||
+485
-2
@@ -1,5 +1,7 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { createServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { bindableFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/HandlerUtils";
|
||||
import { createInstanceLogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
|
||||
import {
|
||||
AUTO_MERGED,
|
||||
CANCELLED,
|
||||
@@ -30,8 +32,12 @@ async function* documents(items: unknown[]) {
|
||||
}
|
||||
}
|
||||
|
||||
function createOperations(conflictedRevisions: string[] = ["2-right"]) {
|
||||
function createOperations(
|
||||
conflictedRevisions: string[] = ["2-right"],
|
||||
overrides: Partial<InteractiveConflictResolutionOperationsDependencies> = {}
|
||||
) {
|
||||
const context = createServiceContext();
|
||||
const lifetime = new AbortController();
|
||||
let dialogueResult: unknown = POSTPONED;
|
||||
const constructed = { value: 0 };
|
||||
const getDBEntry = vi.fn(async (): Promise<false | { _rev: string; _conflicts?: string[] }> => false);
|
||||
@@ -50,6 +56,7 @@ function createOperations(conflictedRevisions: string[] = ["2-right"]) {
|
||||
const log = vi.fn();
|
||||
const operations = createInteractiveConflictResolutionOperations({
|
||||
events: context.events,
|
||||
signal: overrides.signal ?? lifetime.signal,
|
||||
databaseFileAccess: {
|
||||
getConflictedRevs,
|
||||
storeContent,
|
||||
@@ -75,8 +82,10 @@ function createOperations(conflictedRevisions: string[] = ["2-right"]) {
|
||||
};
|
||||
}),
|
||||
log,
|
||||
...overrides,
|
||||
} as unknown as InteractiveConflictResolutionOperationsDependencies);
|
||||
return {
|
||||
abort: () => lifetime.abort(),
|
||||
askSelectString,
|
||||
askInPopup,
|
||||
constructed,
|
||||
@@ -148,11 +157,13 @@ function createControlledDialogueFactory(
|
||||
|
||||
function createDialogueConcurrencyHarness() {
|
||||
const context = createServiceContext();
|
||||
const lifetime = new AbortController();
|
||||
const emitEvent = vi.spyOn(context.events, "emitEvent");
|
||||
const dialogues: ControlledDialogue[] = [];
|
||||
const createDialogue = createControlledDialogueFactory(context, dialogues);
|
||||
const operations = createInteractiveConflictResolutionOperations({
|
||||
events: context.events,
|
||||
signal: lifetime.signal,
|
||||
databaseFileAccess: {
|
||||
getConflictedRevs: vi.fn(async () => ["2-right"]),
|
||||
storeContent: vi.fn(async () => true),
|
||||
@@ -177,10 +188,447 @@ function createDialogueConcurrencyHarness() {
|
||||
createDialogue: createDialogue as ConflictResolveDialogueFactory,
|
||||
log: vi.fn(),
|
||||
} as unknown as InteractiveConflictResolutionOperationsDependencies);
|
||||
return { context, createDialogue, dialogues, emitEvent, operations };
|
||||
return { abort: () => lifetime.abort(), context, createDialogue, dialogues, emitEvent, operations };
|
||||
}
|
||||
|
||||
describe("interactive conflict resolution operations", () => {
|
||||
it("returns cancellation to the caller waiting for a requested conflict check", async () => {
|
||||
const harness = createOperations();
|
||||
harness.ensureAllProcessed.mockResolvedValue(false);
|
||||
|
||||
await expect(harness.operations.requestConflictResolution(path)).resolves.toBe(false);
|
||||
expect(harness.queueCheckFor).toHaveBeenCalledWith(path);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ label: "cancelled", result: CANCELLED },
|
||||
{ label: "postponed", result: POSTPONED },
|
||||
{ label: "selected", result: "2-right" },
|
||||
] as const)("ignores a $label dialogue result which settles after disposal", async ({ result }) => {
|
||||
let finish!: (result: MergeDialogResult) => void;
|
||||
const response = new Promise<MergeDialogResult>((resolve) => {
|
||||
finish = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
const unregisterLog = addLog.setHandler(sink);
|
||||
const createDialogue = vi.fn(() => ({ open: vi.fn(), waitForResult: () => response }));
|
||||
const fixture = createOperations(undefined, {
|
||||
createDialogue,
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
const active = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(createDialogue).toHaveBeenCalledOnce());
|
||||
const waiting = fixture.operations.resolveByUserInteraction("waiting.md" as FilePathWithPrefix, conflict);
|
||||
const settled = Promise.allSettled([active, waiting]);
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
unregisterLog();
|
||||
finish(result);
|
||||
|
||||
expect(await settled).toEqual([
|
||||
{ status: "fulfilled", value: false },
|
||||
{ status: "fulfilled", value: false },
|
||||
]);
|
||||
expect(createDialogue).toHaveBeenCalledOnce();
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal);
|
||||
expect(fixture.getDBEntry).not.toHaveBeenCalled();
|
||||
expect(fixture.storeContent).not.toHaveBeenCalled();
|
||||
expect(fixture.resolveByDeletingRevision).not.toHaveBeenCalled();
|
||||
expect(fixture.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(fixture.queueCheckFor).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("logs a failed database read and returns normally after disposal", async () => {
|
||||
let finishRead!: (value: false | { _rev: string; _conflicts?: string[] }) => void;
|
||||
const pendingRead = new Promise<false | { _rev: string; _conflicts?: string[] }>((resolve) => {
|
||||
finishRead = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
createDialogue: vi.fn(() => ({
|
||||
open: vi.fn(),
|
||||
waitForResult: vi.fn(async () => "2-right"),
|
||||
})),
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.getDBEntry.mockReturnValue(pendingRead);
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.getDBEntry).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
finishRead(false);
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal + 1);
|
||||
expect(sink).toHaveBeenCalledWith(
|
||||
expect.stringContaining(`Merge: Could not read ${path} from the local database`),
|
||||
expect.anything(),
|
||||
expect.anything()
|
||||
);
|
||||
expect(fixture.storeContent).not.toHaveBeenCalled();
|
||||
expect(fixture.resolveByDeletingRevision).not.toHaveBeenCalled();
|
||||
expect(fixture.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(fixture.queueCheckFor).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("continues a selected conflict after the database read succeeds following disposal", async () => {
|
||||
let finishRead!: (value: false | { _rev: string; _conflicts?: string[] }) => void;
|
||||
const pendingRead = new Promise<false | { _rev: string; _conflicts?: string[] }>((resolve) => {
|
||||
finishRead = resolve;
|
||||
});
|
||||
const fixture = createOperations(undefined, {
|
||||
createDialogue: vi.fn(() => ({
|
||||
open: vi.fn(),
|
||||
waitForResult: vi.fn(async () => "2-right"),
|
||||
})),
|
||||
});
|
||||
fixture.getDBEntry.mockReturnValue(pendingRead);
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.getDBEntry).toHaveBeenCalledOnce());
|
||||
|
||||
fixture.abort();
|
||||
finishRead({ _rev: "2-left", _conflicts: ["2-right"] });
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(fixture.resolveByDeletingRevision).toHaveBeenCalledWith(path, "2-right", "UI Selected");
|
||||
expect(fixture.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(fixture.queueCheckFor).toHaveBeenCalledWith(path);
|
||||
});
|
||||
|
||||
it("logs a failed content store after disposal", async () => {
|
||||
let finishStore!: (stored: boolean) => void;
|
||||
const pendingStore = new Promise<boolean>((resolve) => {
|
||||
finishStore = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.dialogueResult.value = LEAVE_TO_SUBSEQUENT;
|
||||
fixture.getDBEntry.mockResolvedValue({ _rev: "2-left", _conflicts: ["2-right"] });
|
||||
fixture.storeContent.mockReturnValue(pendingStore);
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.storeContent).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
finishStore(false);
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal + 1);
|
||||
expect(sink).toHaveBeenCalledWith(
|
||||
`Concatenated content cannot be stored:${path}`,
|
||||
expect.anything(),
|
||||
expect.anything()
|
||||
);
|
||||
expect(fixture.resolveByDeletingRevision).not.toHaveBeenCalled();
|
||||
expect(fixture.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(fixture.queueCheckFor).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("continues a selected conflict after content storage succeeds following disposal", async () => {
|
||||
let finishStore!: (stored: boolean) => void;
|
||||
const pendingStore = new Promise<boolean>((resolve) => {
|
||||
finishStore = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.dialogueResult.value = LEAVE_TO_SUBSEQUENT;
|
||||
fixture.getDBEntry.mockResolvedValue({ _rev: "2-left", _conflicts: ["2-right"] });
|
||||
fixture.storeContent.mockReturnValue(pendingStore);
|
||||
fixture.currentSettings.mockReturnValue({ syncAfterMerge: true });
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.storeContent).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
finishStore(true);
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal);
|
||||
expect(fixture.resolveByDeletingRevision).toHaveBeenCalledWith(path, "2-right", "UI Concatenated");
|
||||
expect(fixture.replicateUnattendedByEvent).toHaveBeenCalledWith({
|
||||
trigger: "merge",
|
||||
interaction: expect.anything(),
|
||||
});
|
||||
expect(fixture.queueCheckFor).toHaveBeenCalledWith(path);
|
||||
});
|
||||
|
||||
it("logs a failed revision deletion after disposal", async () => {
|
||||
let finishDeletion!: (result: typeof AUTO_MERGED | typeof MISSING_OR_ERROR) => void;
|
||||
const pendingDeletion = new Promise<typeof AUTO_MERGED | typeof MISSING_OR_ERROR>((resolve) => {
|
||||
finishDeletion = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.dialogueResult.value = "2-right";
|
||||
fixture.getDBEntry.mockResolvedValue({ _rev: "2-left", _conflicts: ["2-right"] });
|
||||
fixture.resolveByDeletingRevision.mockReturnValue(pendingDeletion);
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.resolveByDeletingRevision).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
finishDeletion(MISSING_OR_ERROR);
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal + 1);
|
||||
expect(sink).toHaveBeenCalledWith(
|
||||
expect.stringContaining(`Merge: Something went wrong: ${path}`),
|
||||
expect.anything(),
|
||||
expect.anything()
|
||||
);
|
||||
expect(fixture.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(fixture.queueCheckFor).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("replicates and requeues after revision deletion succeeds following disposal", async () => {
|
||||
let finishDeletion!: (result: typeof AUTO_MERGED | typeof MISSING_OR_ERROR) => void;
|
||||
const pendingDeletion = new Promise<typeof AUTO_MERGED | typeof MISSING_OR_ERROR>((resolve) => {
|
||||
finishDeletion = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.dialogueResult.value = "2-right";
|
||||
fixture.getDBEntry.mockResolvedValue({ _rev: "2-left", _conflicts: ["2-right"] });
|
||||
fixture.resolveByDeletingRevision.mockReturnValue(pendingDeletion);
|
||||
fixture.currentSettings.mockReturnValue({ syncAfterMerge: true });
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.resolveByDeletingRevision).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
finishDeletion(AUTO_MERGED);
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal);
|
||||
expect(fixture.replicateUnattendedByEvent).toHaveBeenCalledWith({
|
||||
trigger: "merge",
|
||||
interaction: expect.anything(),
|
||||
});
|
||||
expect(fixture.queueCheckFor).toHaveBeenCalledWith(path);
|
||||
});
|
||||
|
||||
it("logs a failed conflict inspection but does not refresh UI after disposal", async () => {
|
||||
let rejectInspection!: (reason: unknown) => void;
|
||||
const pendingInspection = new Promise<string[]>((_resolve, reject) => {
|
||||
rejectInspection = reject;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.getConflictedRevs.mockReturnValue(pendingInspection);
|
||||
const emitEvent = vi.spyOn(fixture.context.events, "emitEvent");
|
||||
|
||||
const refreshing = fixture.operations.refreshConflictState(path);
|
||||
await vi.waitFor(() => expect(fixture.getConflictedRevs).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
rejectInspection(new Error("database unavailable"));
|
||||
|
||||
await expect(refreshing).resolves.toBeUndefined();
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal + 2);
|
||||
expect(sink).toHaveBeenCalledWith(
|
||||
expect.stringContaining(`Could not inspect the conflict state of ${path}`),
|
||||
expect.anything(),
|
||||
expect.anything()
|
||||
);
|
||||
expect(sink).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ message: "database unavailable" }),
|
||||
expect.anything(),
|
||||
expect.anything()
|
||||
);
|
||||
expect(emitEvent).not.toHaveBeenCalledWith(EVENT_ON_UNRESOLVED_ERROR);
|
||||
});
|
||||
|
||||
it("does not emit a successful conflict refresh after disposal", async () => {
|
||||
let finishInspection!: (revisions: string[]) => void;
|
||||
const pendingInspection = new Promise<string[]>((resolve) => {
|
||||
finishInspection = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.getConflictedRevs.mockReturnValue(pendingInspection);
|
||||
const emitEvent = vi.spyOn(fixture.context.events, "emitEvent");
|
||||
|
||||
const refreshing = fixture.operations.refreshConflictState(path);
|
||||
await vi.waitFor(() => expect(fixture.getConflictedRevs).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
|
||||
fixture.abort();
|
||||
finishInspection(["2-right"]);
|
||||
|
||||
await expect(refreshing).resolves.toBeUndefined();
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal);
|
||||
expect(emitEvent).not.toHaveBeenCalledWith(EVENT_ON_UNRESOLVED_ERROR);
|
||||
});
|
||||
|
||||
it("returns no active conflict messages when inspection settles after disposal", async () => {
|
||||
let finishInspection!: (revisions: string[]) => void;
|
||||
const pendingInspection = new Promise<string[]>((resolve) => {
|
||||
finishInspection = resolve;
|
||||
});
|
||||
const fixture = createOperations(undefined, {
|
||||
log: vi.fn(),
|
||||
});
|
||||
fixture.getConflictedRevs.mockReturnValue(pendingInspection);
|
||||
|
||||
const messages = fixture.operations.getActiveConflictMessages();
|
||||
await vi.waitFor(() => expect(fixture.getConflictedRevs).toHaveBeenCalledOnce());
|
||||
fixture.abort();
|
||||
finishInspection(["2-right"]);
|
||||
|
||||
await expect(messages).resolves.toEqual([]);
|
||||
});
|
||||
|
||||
it("does not inspect or publish conflict state when called after disposal", async () => {
|
||||
const fixture = createOperations();
|
||||
const emit = vi.spyOn(fixture.context.events, "emitEvent");
|
||||
fixture.abort();
|
||||
emit.mockClear();
|
||||
|
||||
await expect(fixture.operations.getActiveConflictMessages()).resolves.toEqual([]);
|
||||
await fixture.operations.refreshConflictState(path);
|
||||
|
||||
expect(fixture.getConflictedRevs).not.toHaveBeenCalled();
|
||||
expect(fixture.getActiveFilePath).not.toHaveBeenCalled();
|
||||
expect(fixture.log).not.toHaveBeenCalled();
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not requeue a changed comparison when its conflict refresh finishes after disposal", async () => {
|
||||
let finishInspection!: (revisions: string[]) => void;
|
||||
const pendingInspection = new Promise<string[]>((resolve) => {
|
||||
finishInspection = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.dialogueResult.value = "2-right";
|
||||
fixture.getDBEntry.mockResolvedValue({ _rev: "3-new-left", _conflicts: ["2-right"] });
|
||||
fixture.getConflictedRevs.mockReturnValue(pendingInspection);
|
||||
const emit = vi.spyOn(fixture.context.events, "emitEvent");
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.getConflictedRevs).toHaveBeenCalledOnce());
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
fixture.abort();
|
||||
emit.mockClear();
|
||||
finishInspection(["2-right"]);
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal);
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
expect(fixture.storeContent).not.toHaveBeenCalled();
|
||||
expect(fixture.resolveByDeletingRevision).not.toHaveBeenCalled();
|
||||
expect(fixture.queueCheckFor).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([AUTO_MERGED, MISSING_OR_ERROR] as const)(
|
||||
"continues concatenated resolution according to deletion result after disposal (%s)",
|
||||
async (result) => {
|
||||
let finishDeletion!: (value: typeof AUTO_MERGED | typeof MISSING_OR_ERROR) => void;
|
||||
const pendingDeletion = new Promise<typeof AUTO_MERGED | typeof MISSING_OR_ERROR>((resolve) => {
|
||||
finishDeletion = resolve;
|
||||
});
|
||||
const sink = vi.fn();
|
||||
const addLog = bindableFunction<(message: unknown) => void>("addLog");
|
||||
addLog.setHandler(sink);
|
||||
const fixture = createOperations(undefined, {
|
||||
log: createInstanceLogFunction("Conflict shutdown", { addLog } as never),
|
||||
});
|
||||
fixture.dialogueResult.value = LEAVE_TO_SUBSEQUENT;
|
||||
fixture.getDBEntry.mockResolvedValue({ _rev: "2-left", _conflicts: ["2-right"] });
|
||||
fixture.resolveByDeletingRevision.mockReturnValue(pendingDeletion);
|
||||
fixture.currentSettings.mockReturnValue({ syncAfterMerge: true });
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.resolveByDeletingRevision).toHaveBeenCalledOnce());
|
||||
expect(fixture.storeContent).toHaveBeenCalledOnce();
|
||||
const logsBeforeDisposal = sink.mock.calls.length;
|
||||
fixture.abort();
|
||||
finishDeletion(result);
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(fixture.resolveByDeletingRevision).toHaveBeenCalledTimes(1);
|
||||
if (result === MISSING_OR_ERROR) {
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal + 1);
|
||||
expect(sink).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Concatenated saved, but cannot delete conflicted revisions:"),
|
||||
expect.anything(),
|
||||
expect.anything()
|
||||
);
|
||||
expect(fixture.replicateUnattendedByEvent).not.toHaveBeenCalled();
|
||||
expect(fixture.queueCheckFor).not.toHaveBeenCalled();
|
||||
} else {
|
||||
expect(sink).toHaveBeenCalledTimes(logsBeforeDisposal);
|
||||
expect(fixture.replicateUnattendedByEvent).toHaveBeenCalledWith({
|
||||
trigger: "merge",
|
||||
interaction: expect.anything(),
|
||||
});
|
||||
expect(fixture.queueCheckFor).toHaveBeenCalledWith(path);
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
it("requeues after in-flight replication completes following disposal", async () => {
|
||||
let finishReplication!: (result: { status: "completed" }) => void;
|
||||
const pendingReplication = new Promise<{ status: "completed" }>((resolve) => {
|
||||
finishReplication = resolve;
|
||||
});
|
||||
const fixture = createOperations();
|
||||
fixture.dialogueResult.value = "2-right";
|
||||
fixture.getDBEntry.mockResolvedValue({ _rev: "2-left", _conflicts: ["2-right"] });
|
||||
fixture.currentSettings.mockReturnValue({ syncAfterMerge: true });
|
||||
fixture.replicateUnattendedByEvent.mockReturnValue(pendingReplication);
|
||||
|
||||
const resolving = fixture.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(fixture.replicateUnattendedByEvent).toHaveBeenCalledOnce());
|
||||
fixture.abort();
|
||||
finishReplication({ status: "completed" });
|
||||
|
||||
await expect(resolving).resolves.toBe(false);
|
||||
expect(fixture.resolveByDeletingRevision).toHaveBeenCalledOnce();
|
||||
expect(fixture.replicateUnattendedByEvent).toHaveBeenCalledTimes(1);
|
||||
expect(fixture.queueCheckFor).toHaveBeenCalledWith(path);
|
||||
});
|
||||
|
||||
it("replaces an active same-file dialogue with only the newest waiting request", async () => {
|
||||
const fixture = createDialogueConcurrencyHarness();
|
||||
|
||||
@@ -233,6 +681,40 @@ describe("interactive conflict resolution operations", () => {
|
||||
await Promise.all([first, waiting]);
|
||||
});
|
||||
|
||||
it("keeps another instance's active and waiting dialogues when disposal is repeated", async () => {
|
||||
const oldInstance = createDialogueConcurrencyHarness();
|
||||
const currentInstance = createDialogueConcurrencyHarness();
|
||||
|
||||
const oldActive = oldInstance.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(oldInstance.dialogues).toHaveLength(1));
|
||||
const oldWaiting = oldInstance.operations.resolveByUserInteraction(
|
||||
"old-waiting.md" as FilePathWithPrefix,
|
||||
conflict
|
||||
);
|
||||
|
||||
const currentActive = currentInstance.operations.resolveByUserInteraction(path, conflict);
|
||||
await vi.waitFor(() => expect(currentInstance.dialogues).toHaveLength(1));
|
||||
const currentWaiting = currentInstance.operations.resolveByUserInteraction(
|
||||
"current-waiting.md" as FilePathWithPrefix,
|
||||
conflict
|
||||
);
|
||||
await Promise.resolve();
|
||||
|
||||
oldInstance.abort();
|
||||
oldInstance.abort();
|
||||
|
||||
await expect(Promise.all([oldActive, oldWaiting])).resolves.toEqual([false, false]);
|
||||
expect(oldInstance.dialogues).toHaveLength(1);
|
||||
expect(currentInstance.dialogues).toHaveLength(1);
|
||||
expect(currentInstance.emitEvent).not.toHaveBeenCalledWith(EVENT_CONFLICT_CANCELLED, path);
|
||||
|
||||
currentInstance.dialogues[0].finish(CANCELLED);
|
||||
await vi.waitFor(() => expect(currentInstance.dialogues).toHaveLength(2));
|
||||
expect(currentInstance.dialogues[1].filename).toBe("current-waiting.md");
|
||||
currentInstance.dialogues[1].finish(CANCELLED);
|
||||
await Promise.all([currentActive, currentWaiting]);
|
||||
});
|
||||
|
||||
it("does not reopen an unchanged conflict after the user chooses Not now", async () => {
|
||||
const { constructed, operations } = createOperations();
|
||||
|
||||
@@ -770,6 +1252,7 @@ describe("interactive conflict resolution feature composition", () => {
|
||||
|
||||
expect(completedOnUnload).toBe(true);
|
||||
expect(dialogues).toHaveLength(1);
|
||||
expect(fixture.getConflictedRevs).not.toHaveBeenCalled();
|
||||
await fixture.handlers.onUnload?.();
|
||||
});
|
||||
|
||||
|
||||
@@ -23,6 +23,8 @@ import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/
|
||||
import type { LiveSyncEventHub } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { fireAndForget } from "octagonal-wheels/promises";
|
||||
import { withLock } from "octagonal-wheels/concurrency/lock";
|
||||
import { withCancellation } from "octagonal-wheels/concurrency/cancellation";
|
||||
import { withScope } from "octagonal-wheels/resource/scope";
|
||||
import { NO_INTERACTION } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { displayRev } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import { EVENT_CONFLICT_CANCELLED, EVENT_ON_UNRESOLVED_ERROR } from "@/common/events.ts";
|
||||
@@ -38,6 +40,7 @@ import {
|
||||
|
||||
export interface InteractiveConflictResolutionOperationsDependencies {
|
||||
readonly events: Pick<LiveSyncEventHub, "emitEvent">;
|
||||
readonly signal: AbortSignal;
|
||||
readonly databaseFileAccess: Pick<DatabaseFileAccess, "getConflictedRevs" | "storeContent">;
|
||||
readonly localDatabase: () => Pick<LiveSyncLocalDB, "getDBEntry" | "findAllDocs">;
|
||||
readonly confirm: Pick<Confirm, "askSelectString" | "askInPopup">;
|
||||
@@ -52,11 +55,11 @@ export interface InteractiveConflictResolutionOperationsDependencies {
|
||||
}
|
||||
|
||||
export interface InteractiveConflictResolutionOperations {
|
||||
readonly dispose: () => void;
|
||||
readonly invalidateWaitingResolution: (filename: FilePathWithPrefix) => void;
|
||||
readonly getActiveConflictMessages: () => Promise<string[]>;
|
||||
readonly refreshConflictState: (filename: FilePathWithPrefix) => Promise<void>;
|
||||
readonly requestConflictResolution: (filename: FilePathWithPrefix) => Promise<void>;
|
||||
/** Returns false if the requested conflict check was cancelled. */
|
||||
readonly requestConflictResolution: (filename: FilePathWithPrefix) => Promise<boolean>;
|
||||
readonly resolveByUserInteraction: (
|
||||
filename: FilePathWithPrefix,
|
||||
conflictCheckResult: diff_result
|
||||
@@ -76,9 +79,18 @@ export function createInteractiveConflictResolutionOperations(
|
||||
const latestRequestByFilename = new Map<FilePathWithPrefix, number>();
|
||||
let nextRequestId = 0;
|
||||
let activeDialogue: { filename: FilePathWithPrefix; dialogue: ConflictResolveDialogue } | undefined;
|
||||
let disposed = false;
|
||||
const clearPendingState = (): void => {
|
||||
latestRequestByFilename.clear();
|
||||
postponedConflictEpisodes.clear();
|
||||
};
|
||||
if (dependencies.signal.aborted) {
|
||||
clearPendingState();
|
||||
} else {
|
||||
dependencies.signal.addEventListener("abort", clearPendingState, { once: true });
|
||||
}
|
||||
|
||||
const invalidateWaitingResolution = (filename: FilePathWithPrefix): void => {
|
||||
if (dependencies.signal.aborted) return;
|
||||
// An active dialogue consumes this event itself. Removing its request
|
||||
// here would also remove a same-path replacement which emitted the
|
||||
// event to close that active dialogue. A non-active request is stale
|
||||
@@ -87,21 +99,11 @@ export function createInteractiveConflictResolutionOperations(
|
||||
latestRequestByFilename.delete(filename);
|
||||
};
|
||||
|
||||
const dispose = (): void => {
|
||||
if (disposed) return;
|
||||
disposed = true;
|
||||
latestRequestByFilename.clear();
|
||||
postponedConflictEpisodes.clear();
|
||||
const activeFilename = activeDialogue?.filename;
|
||||
if (activeFilename !== undefined) {
|
||||
dependencies.events.emitEvent(EVENT_CONFLICT_CANCELLED, activeFilename);
|
||||
activeDialogue = undefined;
|
||||
}
|
||||
};
|
||||
|
||||
const getConflictVersionCount = async (filename: FilePathWithPrefix): Promise<number | undefined> => {
|
||||
if (dependencies.signal.aborted) return undefined;
|
||||
try {
|
||||
const conflictCount = (await dependencies.databaseFileAccess.getConflictedRevs(filename)).length;
|
||||
if (dependencies.signal.aborted) return undefined;
|
||||
return conflictCount === 0 ? 0 : conflictCount + 1;
|
||||
} catch (error) {
|
||||
dependencies.log(`Could not inspect the conflict state of ${filename}`, LOG_LEVEL_VERBOSE);
|
||||
@@ -111,9 +113,11 @@ export function createInteractiveConflictResolutionOperations(
|
||||
};
|
||||
|
||||
const getActiveConflictMessages = async (): Promise<string[]> => {
|
||||
if (dependencies.signal.aborted) return [];
|
||||
const filename = dependencies.vault.getActiveFilePath();
|
||||
if (!filename) return [];
|
||||
const versionCount = await getConflictVersionCount(filename);
|
||||
if (dependencies.signal.aborted) return [];
|
||||
if (versionCount === 0) {
|
||||
postponedConflictEpisodes.delete(filename);
|
||||
return [];
|
||||
@@ -132,39 +136,50 @@ export function createInteractiveConflictResolutionOperations(
|
||||
};
|
||||
|
||||
const refreshConflictState = async (filename: FilePathWithPrefix): Promise<void> => {
|
||||
if ((await getConflictVersionCount(filename)) === 0) {
|
||||
if (dependencies.signal.aborted) return;
|
||||
const versionCount = await getConflictVersionCount(filename);
|
||||
if (dependencies.signal.aborted) return;
|
||||
if (versionCount === 0) {
|
||||
postponedConflictEpisodes.delete(filename);
|
||||
}
|
||||
dependencies.events.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
|
||||
};
|
||||
|
||||
const requestConflictResolution = async (filename: FilePathWithPrefix): Promise<void> => {
|
||||
const requestConflictResolution = async (filename: FilePathWithPrefix): Promise<boolean> => {
|
||||
if (dependencies.signal.aborted) return false;
|
||||
postponedConflictEpisodes.delete(filename);
|
||||
dependencies.events.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
|
||||
await dependencies.conflict.queueCheckFor(filename);
|
||||
await dependencies.conflict.ensureAllProcessed();
|
||||
if (dependencies.signal.aborted) return false;
|
||||
return await dependencies.conflict.ensureAllProcessed();
|
||||
};
|
||||
|
||||
const resolveByUserInteraction = async (
|
||||
filename: FilePathWithPrefix,
|
||||
conflictCheckResult: diff_result
|
||||
): Promise<boolean> => {
|
||||
if (disposed) return false;
|
||||
const requestId = ++nextRequestId;
|
||||
if (activeDialogue?.filename === filename) {
|
||||
dependencies.events.emitEvent(EVENT_CONFLICT_CANCELLED, filename);
|
||||
activeDialogue = undefined;
|
||||
}
|
||||
latestRequestByFilename.set(filename, requestId);
|
||||
|
||||
// UI for resolving different files should proceed one-by-one. A newer
|
||||
// request for the active file replaces its dialogue instead of waiting
|
||||
// behind a comparison which is already stale.
|
||||
return await withLock(dialogueSerialisationKey, async () => {
|
||||
if (disposed || latestRequestByFilename.get(filename) !== requestId) {
|
||||
return false;
|
||||
if (dependencies.signal.aborted) return false;
|
||||
return await withScope(undefined, async (request) => {
|
||||
if (dependencies.signal.aborted) return false;
|
||||
const requestId = ++nextRequestId;
|
||||
if (activeDialogue?.filename === filename) {
|
||||
dependencies.events.emitEvent(EVENT_CONFLICT_CANCELLED, filename);
|
||||
activeDialogue = undefined;
|
||||
}
|
||||
try {
|
||||
latestRequestByFilename.set(filename, requestId);
|
||||
request.defer(() => {
|
||||
if (latestRequestByFilename.get(filename) === requestId) {
|
||||
latestRequestByFilename.delete(filename);
|
||||
}
|
||||
});
|
||||
|
||||
// UI for resolving different files should proceed one-by-one. A newer
|
||||
// request for the active file replaces its dialogue instead of waiting
|
||||
// behind a comparison which is already stale.
|
||||
return await withLock(dialogueSerialisationKey, async () => {
|
||||
if (dependencies.signal.aborted || latestRequestByFilename.get(filename) !== requestId) {
|
||||
return false;
|
||||
}
|
||||
if (postponedConflictEpisodes.has(filename)) {
|
||||
dependencies.log(`Merge: Postponed ${filename}`, LOG_LEVEL_VERBOSE);
|
||||
dependencies.events.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
|
||||
@@ -175,13 +190,24 @@ export function createInteractiveConflictResolutionOperations(
|
||||
activeDialogue = { filename, dialogue };
|
||||
let selected: MergeDialogResult;
|
||||
try {
|
||||
dialogue.open();
|
||||
selected = await dialogue.waitForResult();
|
||||
selected = await withCancellation(dialogue, { signal: dependencies.signal }, async (waiting) => {
|
||||
const cancelDialogue = (): void => {
|
||||
if (activeDialogue?.dialogue === dialogue) {
|
||||
dependencies.events.emitEvent(EVENT_CONFLICT_CANCELLED, filename);
|
||||
}
|
||||
};
|
||||
waiting.signal.addEventListener("abort", cancelDialogue, { once: true });
|
||||
waiting.defer(() => waiting.signal.removeEventListener("abort", cancelDialogue));
|
||||
dialogue.open();
|
||||
if (waiting.isCancelled()) cancelDialogue();
|
||||
return await dialogue.waitForResult();
|
||||
});
|
||||
} finally {
|
||||
if (activeDialogue?.dialogue === dialogue) {
|
||||
activeDialogue = undefined;
|
||||
}
|
||||
}
|
||||
if (dependencies.signal.aborted) return false;
|
||||
if (selected === POSTPONED) {
|
||||
postponedConflictEpisodes.add(filename);
|
||||
dependencies.events.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
|
||||
@@ -203,6 +229,7 @@ export function createInteractiveConflictResolutionOperations(
|
||||
if (!testDoc._conflicts || testDoc._conflicts.length === 0) {
|
||||
dependencies.log(`Merge: Nothing to do ${filename}`, LOG_LEVEL_VERBOSE);
|
||||
await refreshConflictState(filename);
|
||||
if (dependencies.signal.aborted) return false;
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
@@ -214,6 +241,7 @@ export function createInteractiveConflictResolutionOperations(
|
||||
LOG_LEVEL_INFO
|
||||
);
|
||||
await refreshConflictState(filename);
|
||||
if (dependencies.signal.aborted) return false;
|
||||
await dependencies.conflict.queueCheckFor(filename);
|
||||
return false;
|
||||
}
|
||||
@@ -224,15 +252,18 @@ export function createInteractiveConflictResolutionOperations(
|
||||
// Create a new file by concatenating both conflicted revisions.
|
||||
const p = conflictCheckResult.diff.map((e) => e[1]).join("");
|
||||
const delRev = conflictCheckResult.right.rev;
|
||||
if (!(await dependencies.databaseFileAccess.storeContent(filename, p))) {
|
||||
const contentStored = await dependencies.databaseFileAccess.storeContent(filename, p);
|
||||
if (!contentStored) {
|
||||
dependencies.log(`Concatenated content cannot be stored:${filename}`, LOG_LEVEL_NOTICE);
|
||||
return false;
|
||||
}
|
||||
// 2. As usual, delete the conflicted revision and if there are no conflicts, write the resolved content to the storage.
|
||||
if (
|
||||
(await dependencies.conflict.resolveByDeletingRevision(filename, delRev, "UI Concatenated")) ==
|
||||
MISSING_OR_ERROR
|
||||
) {
|
||||
const deletionResult = await dependencies.conflict.resolveByDeletingRevision(
|
||||
filename,
|
||||
delRev,
|
||||
"UI Concatenated"
|
||||
);
|
||||
if (deletionResult == MISSING_OR_ERROR) {
|
||||
dependencies.log(
|
||||
`Concatenated saved, but cannot delete conflicted revisions: ${filename}, (${displayRev(delRev)})`,
|
||||
LOG_LEVEL_NOTICE
|
||||
@@ -244,10 +275,12 @@ export function createInteractiveConflictResolutionOperations(
|
||||
(toDelete === conflictCheckResult.left.rev || toDelete === conflictCheckResult.right.rev)
|
||||
) {
|
||||
// Select one of the conflicted revision to delete.
|
||||
if (
|
||||
(await dependencies.conflict.resolveByDeletingRevision(filename, toDelete, "UI Selected")) ==
|
||||
MISSING_OR_ERROR
|
||||
) {
|
||||
const deletionResult = await dependencies.conflict.resolveByDeletingRevision(
|
||||
filename,
|
||||
toDelete,
|
||||
"UI Selected"
|
||||
);
|
||||
if (deletionResult == MISSING_OR_ERROR) {
|
||||
dependencies.log(`Merge: Something went wrong: ${filename}, (${toDelete})`, LOG_LEVEL_NOTICE);
|
||||
return false;
|
||||
}
|
||||
@@ -270,17 +303,15 @@ export function createInteractiveConflictResolutionOperations(
|
||||
// And, check it again.
|
||||
await dependencies.conflict.queueCheckFor(filename);
|
||||
return false;
|
||||
} finally {
|
||||
if (latestRequestByFilename.get(filename) === requestId) {
|
||||
latestRequestByFilename.delete(filename);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
};
|
||||
|
||||
const pickFileForResolve = async (notifyIfEmpty = true): Promise<boolean> => {
|
||||
if (dependencies.signal.aborted) return false;
|
||||
const notes: { id: DocumentID; path: FilePathWithPrefix; dispPath: string; mtime: number }[] = [];
|
||||
for await (const doc of dependencies.localDatabase().findAllDocs({ conflicts: true })) {
|
||||
if (dependencies.signal.aborted) return false;
|
||||
if (!("_conflicts" in doc)) continue;
|
||||
notes.push({
|
||||
id: doc._id,
|
||||
@@ -289,6 +320,7 @@ export function createInteractiveConflictResolutionOperations(
|
||||
mtime: doc.mtime,
|
||||
});
|
||||
}
|
||||
if (dependencies.signal.aborted) return false;
|
||||
notes.sort((a, b) => b.mtime - a.mtime);
|
||||
const notesList = notes.map((e) => e.dispPath);
|
||||
if (notesList.length == 0) {
|
||||
@@ -298,6 +330,7 @@ export function createInteractiveConflictResolutionOperations(
|
||||
return false;
|
||||
}
|
||||
const target = await dependencies.confirm.askSelectString("File to resolve conflict", notesList);
|
||||
if (dependencies.signal.aborted) return false;
|
||||
if (target) {
|
||||
const targetItem = notes.find((e) => e.dispPath == target)!;
|
||||
await requestConflictResolution(targetItem.path);
|
||||
@@ -308,19 +341,22 @@ export function createInteractiveConflictResolutionOperations(
|
||||
|
||||
const allConflictCheck = async (): Promise<void> => {
|
||||
let notifyIfEmpty = true;
|
||||
while (await pickFileForResolve(notifyIfEmpty)) {
|
||||
while (!dependencies.signal.aborted && (await pickFileForResolve(notifyIfEmpty))) {
|
||||
notifyIfEmpty = false;
|
||||
}
|
||||
};
|
||||
|
||||
const scanStartupIssues = async (): Promise<boolean> => {
|
||||
if (dependencies.signal.aborted) return false;
|
||||
const notes: { path: string; mtime: number }[] = [];
|
||||
dependencies.log(`Checking conflicted files`, LOG_LEVEL_VERBOSE);
|
||||
try {
|
||||
for await (const doc of dependencies.localDatabase().findAllDocs({ conflicts: true })) {
|
||||
if (dependencies.signal.aborted) return false;
|
||||
if (!("_conflicts" in doc)) continue;
|
||||
notes.push({ path: dependencies.path.getPath(doc), mtime: doc.mtime });
|
||||
}
|
||||
if (dependencies.signal.aborted) return false;
|
||||
if (notes.length > 0) {
|
||||
dependencies.confirm.askInPopup(
|
||||
`conflicting-detected-on-safety`,
|
||||
@@ -351,7 +387,6 @@ export function createInteractiveConflictResolutionOperations(
|
||||
};
|
||||
|
||||
return {
|
||||
dispose,
|
||||
invalidateWaitingResolution,
|
||||
getActiveConflictMessages,
|
||||
refreshConflictState,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getStoragePathComparisonKey, type FilePath, type PathComparisonKey } from "@vrtmrz/livesync-commonlib/storage";
|
||||
|
||||
/**
|
||||
* Keeps storage from being changed through a name which the file system resolves but no directory lists.
|
||||
*
|
||||
@@ -20,7 +22,13 @@ export interface PathAliasGuardHost {
|
||||
/** How long the names which a directory listed are taken as still listed. A name which is missed is listed afresh. */
|
||||
const LISTING_LIFETIME_MS = 1_000;
|
||||
|
||||
const foldName = (name: string) => name.normalize("NFC").toLowerCase();
|
||||
// Host alias checks retain their fixed NFC and case-insensitive policy in both identity modes.
|
||||
// Original host names keep the host vocabulary; comparison does not apply rooted-storage input validation.
|
||||
const foldName = (name: string) =>
|
||||
getStoragePathComparisonKey(name as FilePath, {
|
||||
pathNormalizationVersion: 1,
|
||||
handleFilenameCaseSensitive: false,
|
||||
});
|
||||
|
||||
const nameOf = (entry: string) => {
|
||||
const segments = entry.split("/").filter((segment) => segment !== "");
|
||||
@@ -28,14 +36,14 @@ const nameOf = (entry: string) => {
|
||||
};
|
||||
|
||||
export class PathAliasGuard {
|
||||
private readonly listings = new Map<string, { names: Set<string>; listedAt: number }>();
|
||||
private readonly listings = new Map<string, { names: Set<PathComparisonKey<"storage">>; listedAt: number }>();
|
||||
|
||||
constructor(
|
||||
private readonly host: PathAliasGuardHost,
|
||||
private readonly now: () => number = Date.now
|
||||
) {}
|
||||
|
||||
private recentListing(directory: string): Set<string> | undefined {
|
||||
private recentListing(directory: string): Set<PathComparisonKey<"storage">> | undefined {
|
||||
const listing = this.listings.get(directory);
|
||||
if (!listing) return undefined;
|
||||
if (this.now() - listing.listedAt <= LISTING_LIFETIME_MS) return listing.names;
|
||||
@@ -43,7 +51,7 @@ export class PathAliasGuard {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
private async listAfresh(directory: string): Promise<Set<string>> {
|
||||
private async listAfresh(directory: string): Promise<Set<PathComparisonKey<"storage">>> {
|
||||
const names = new Set((await this.host.list(directory)).map((entry) => foldName(nameOf(entry))));
|
||||
this.listings.set(directory, { names, listedAt: this.now() });
|
||||
return names;
|
||||
|
||||
@@ -15,7 +15,7 @@ import type { App, TFile } from "obsidian";
|
||||
import { afterAll, describe, expect, it, vi } from "vitest";
|
||||
import { ObsidianStorageAdapter } from "./ObsidianStorageAdapter";
|
||||
import { ObsidianVaultAdapter } from "./ObsidianVaultAdapter";
|
||||
import { createPathAliasGuard } from "./pathAliasGuard";
|
||||
import { createPathAliasGuard, PathAliasGuard } from "./pathAliasGuard";
|
||||
|
||||
const REFUSAL = "Refusing a path which reaches an existing entry by a name its folder does not list.";
|
||||
const CONFIG_DIRECTORY = ".obsidian";
|
||||
@@ -141,6 +141,30 @@ describe("paths which resolve to an existing entry under an unlisted name", () =
|
||||
expect(calls.trashFile).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("allows NFC and case equivalents of names exposed by the host listing", async () => {
|
||||
const folder = "Café".normalize("NFD");
|
||||
const guard = new PathAliasGuard({
|
||||
isIndexed: () => false,
|
||||
exists: async () => true,
|
||||
list: async (path) => (path === "" ? [folder] : [`${folder}/Note.md`]),
|
||||
});
|
||||
|
||||
await expect(guard.assertNoAlias("CAFÉ/note.md")).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it.each(["x:cache", "host\\name"])(
|
||||
"allows a listed target beside the unrelated host-listed name %s",
|
||||
async (unrelatedName) => {
|
||||
const guard = new PathAliasGuard({
|
||||
isIndexed: () => false,
|
||||
exists: async () => true,
|
||||
list: async () => [unrelatedName, "note.md"],
|
||||
});
|
||||
|
||||
await expect(guard.assertNoAlias("note.md")).resolves.toBeUndefined();
|
||||
}
|
||||
);
|
||||
|
||||
it("allows listed names, case-only differences, indexed paths, and new files", async () => {
|
||||
const { app, calls } = fakeApp();
|
||||
const adapter = new ObsidianStorageAdapter(app);
|
||||
|
||||
@@ -167,6 +167,140 @@ describe("WebAppRuntime lifecycle", () => {
|
||||
runtimeMocks.updateToDatabase.mockResolvedValue(undefined);
|
||||
});
|
||||
|
||||
function settingsRoot(settings: Partial<ObsidianLiveSyncSettings>): FileSystemDirectoryHandle {
|
||||
return {
|
||||
name: "identity-test-vault",
|
||||
getDirectoryHandle: async () => ({
|
||||
getFileHandle: async () => ({
|
||||
getFile: async () => new File([JSON.stringify(settings)], "settings.json"),
|
||||
}),
|
||||
}),
|
||||
} as unknown as FileSystemDirectoryHandle;
|
||||
}
|
||||
|
||||
it.each([undefined, 0] as const)("starts with supported legacy identity %s", async (version) => {
|
||||
const runtime = new WebAppRuntime(settingsRoot({ pathNormalizationVersion: version }), {
|
||||
deviceValues: new MemoryValueStore(),
|
||||
});
|
||||
await runtime.start();
|
||||
expect(runtimeMocks.onLoad).toHaveBeenCalledOnce();
|
||||
expect(runtimeMocks.scanDirectory).toHaveBeenCalledOnce();
|
||||
await runtime.shutdown();
|
||||
});
|
||||
|
||||
it.each([1, 2] as const)(
|
||||
"rejects unsupported identity %s before constructing services or starting file work",
|
||||
async (version) => {
|
||||
const reportStatus = vi.fn();
|
||||
const runtime = new WebAppRuntime(settingsRoot({ pathNormalizationVersion: version as 1 }), {
|
||||
reportStatus,
|
||||
deviceValues: new MemoryValueStore(),
|
||||
});
|
||||
await expect(runtime.start()).rejects.toThrow(/path identity|path normalisation/i);
|
||||
expect(runtimeMocks.options).toBeUndefined();
|
||||
expect(runtimeMocks.onLoad).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.onReady).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.scanDirectory).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.collectFilesOnStorage).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.updateToDatabase).not.toHaveBeenCalled();
|
||||
expect(reportStatus).toHaveBeenCalledWith(
|
||||
"error",
|
||||
expect.stringMatching(/path identity|path normalisation/i)
|
||||
);
|
||||
}
|
||||
);
|
||||
|
||||
it("rejects NFC during settings reload instead of substituting defaults", async () => {
|
||||
const runtime = new WebAppRuntime(settingsRoot({ pathNormalizationVersion: 1 }), {
|
||||
deviceValues: new MemoryValueStore(),
|
||||
});
|
||||
const options = (
|
||||
runtime as unknown as { createServiceOptions(): LiveSyncBrowserServiceHubOptions<never> }
|
||||
).createServiceOptions();
|
||||
if (!options.settings) throw new Error("The runtime must provide settings callbacks");
|
||||
await expect(options.settings.load()).rejects.toThrow(/WebApp supports Legacy path identity/);
|
||||
});
|
||||
|
||||
it("rejects saving unsupported NFC settings before opening the settings directory", async () => {
|
||||
const getDirectoryHandle = vi.fn();
|
||||
const runtime = new WebAppRuntime(
|
||||
{ name: "identity-test-vault", getDirectoryHandle } as unknown as FileSystemDirectoryHandle,
|
||||
{ deviceValues: new MemoryValueStore() }
|
||||
);
|
||||
const options = (
|
||||
runtime as unknown as { createServiceOptions(): LiveSyncBrowserServiceHubOptions<never> }
|
||||
).createServiceOptions();
|
||||
if (!options.settings) throw new Error("The runtime must provide settings callbacks");
|
||||
await expect(
|
||||
options.settings.save({ pathNormalizationVersion: 1 } as ObsidianLiveSyncSettings)
|
||||
).rejects.toThrow(/WebApp supports Legacy path identity/);
|
||||
expect(getDirectoryHandle).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([undefined, 0] as const)("saves supported Legacy settings without adopting NFC (%s)", async (version) => {
|
||||
const write = vi.fn(async (_data: string) => undefined);
|
||||
const close = vi.fn(async () => undefined);
|
||||
const getFileHandle = vi.fn(async () => ({ createWritable: async () => ({ write, close }) }));
|
||||
const getDirectoryHandle = vi.fn(async () => ({ getFileHandle }));
|
||||
const runtime = new WebAppRuntime(
|
||||
{ name: "legacy-settings", getDirectoryHandle } as unknown as FileSystemDirectoryHandle,
|
||||
{ deviceValues: new MemoryValueStore() }
|
||||
);
|
||||
const options = (
|
||||
runtime as unknown as { createServiceOptions(): LiveSyncBrowserServiceHubOptions<never> }
|
||||
).createServiceOptions();
|
||||
if (!options.settings) throw new Error("The runtime must provide settings callbacks");
|
||||
const settings = {
|
||||
...unconfiguredSettings,
|
||||
...(version === undefined ? {} : { pathNormalizationVersion: version }),
|
||||
} as ObsidianLiveSyncSettings;
|
||||
|
||||
await options.settings.save(settings);
|
||||
|
||||
expect(getDirectoryHandle).toHaveBeenCalledWith(".livesync", { create: true });
|
||||
expect(getFileHandle).toHaveBeenCalledWith("settings.json", { create: true });
|
||||
expect(write).toHaveBeenCalledOnce();
|
||||
expect(JSON.parse(write.mock.calls[0][0])).toEqual(settings);
|
||||
expect(JSON.parse(write.mock.calls[0][0]).pathNormalizationVersion).toBe(version);
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("reports a Legacy settings write failure without logging a successful save", async () => {
|
||||
const denied = new DOMException("Settings write denied", "NotAllowedError");
|
||||
const write = vi.fn(async () => {
|
||||
throw denied;
|
||||
});
|
||||
const close = vi.fn(async () => undefined);
|
||||
const runtime = new WebAppRuntime(
|
||||
{
|
||||
name: "denied-settings",
|
||||
getDirectoryHandle: async () => ({
|
||||
getFileHandle: async () => ({ createWritable: async () => ({ write, close }) }),
|
||||
}),
|
||||
} as unknown as FileSystemDirectoryHandle,
|
||||
{ deviceValues: new MemoryValueStore() }
|
||||
);
|
||||
const options = (
|
||||
runtime as unknown as { createServiceOptions(): LiveSyncBrowserServiceHubOptions<never> }
|
||||
).createServiceOptions();
|
||||
if (!options.settings) throw new Error("The runtime must provide settings callbacks");
|
||||
const settings = { ...unconfiguredSettings, pathNormalizationVersion: 0 } as ObsidianLiveSyncSettings;
|
||||
await runtime.start();
|
||||
runtimeMocks.addLog.mockClear();
|
||||
|
||||
await expect(options.settings.save(settings)).resolves.toBeUndefined();
|
||||
|
||||
expect(write).toHaveBeenCalledOnce();
|
||||
expect(close).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.addLog).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Failed to save settings: NotAllowedError: Settings write denied"),
|
||||
expect.any(Number),
|
||||
"settings"
|
||||
);
|
||||
expect(runtimeMocks.addLog.mock.calls.some(([message]) => String(message).startsWith("Saved to "))).toBe(false);
|
||||
expect(settings.pathNormalizationVersion).toBe(0);
|
||||
});
|
||||
|
||||
it("owns core start, directory scan, status reporting, and shutdown", async () => {
|
||||
const reportStatus = vi.fn();
|
||||
const runtime = new WebAppRuntime(createRootHandle(), { reportStatus, deviceValues: new MemoryValueStore() });
|
||||
|
||||
@@ -6,7 +6,7 @@ Application unit tests are stored in `test/apps/webapp/` and `test/apps/webpeer/
|
||||
|
||||
Each application has its own production-bundle smoke-test directory outside `src`:
|
||||
|
||||
- `test/browser-apps/webapp/` covers Vault selection, OPFS start-up, and isolation between optional P2P settings and the main remote.
|
||||
- `test/browser-apps/webapp/` covers Vault selection, Legacy OPFS start-up, isolation between optional P2P settings and the main remote, and explicit NFC refusal before database, scan, file-write, or remote activity. The refusal case retains the settings and native filename content.
|
||||
- `test/browser-apps/webpeer/` covers start-up, settings persistence, and reload.
|
||||
- `test/browser-apps/pages/` covers the final GitHub Pages layout, application subpaths, and relative assets.
|
||||
|
||||
@@ -18,6 +18,8 @@ npm run test:browser-apps
|
||||
|
||||
`test/browser-apps/interop.test.ts` is the only cross-application scenario. It configures the real WebApp and WebPeer interfaces, transfers a file from WebApp to WebPeer, then starts the built CLI as the final peer and verifies the file through the production CLI. The CLI is a test fixture in this scenario; the PoC test and its support code are not owned by the CLI package.
|
||||
|
||||
The interoperability scenario uses legacy path identity on all three peers. WebApp and WebPeer remain outside the NFC support matrix, so the CLI fixture explicitly selects legacy identity instead of inheriting the NFC default for new CLI settings. The scenario still verifies the transferred file through the production CLI.
|
||||
|
||||
Run the isolated relay and TURN scenario in Compose with:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -243,6 +243,8 @@ async function configureCliSettings(
|
||||
encrypt: false,
|
||||
isConfigured: true,
|
||||
passphrase: "",
|
||||
// WebApp and WebPeer retain legacy identity; new CLI settings default to NFC.
|
||||
pathNormalizationVersion: 0,
|
||||
remoteType: "ONLY_P2P",
|
||||
usePathObfuscation: false,
|
||||
});
|
||||
|
||||
@@ -112,9 +112,7 @@ Deno.test({
|
||||
"WebApp must not render a peer-menu action without a menu capability"
|
||||
);
|
||||
await runtimePage.locator(".p2p-control input[type='checkbox']").first().check();
|
||||
await runtimePage
|
||||
.getByPlaceholder("wss://exp-relay.vrtmrz.net, wss://xxxxx")
|
||||
.fill("ws://127.0.0.1:4010/");
|
||||
await runtimePage.getByPlaceholder("wss://exp-relay.vrtmrz.net, wss://xxxxx").fill("ws://127.0.0.1:4010/");
|
||||
await runtimePage.getByPlaceholder("anything-you-like").fill("browser-e2e-room");
|
||||
await runtimePage.getByPlaceholder("password").fill("browser-e2e-passphrase");
|
||||
await runtimePage.getByPlaceholder("iphone-16").fill("browser-e2e-webapp");
|
||||
@@ -159,3 +157,118 @@ Deno.test({
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
Deno.test({
|
||||
name: "WebApp: rejects NFC settings before opening a database, scanning, writing, or synchronising",
|
||||
sanitizeOps: false,
|
||||
sanitizeResources: false,
|
||||
async fn() {
|
||||
const server = await startStaticServer(webAppDist);
|
||||
const browser = await chromium.launch({ headless: true });
|
||||
try {
|
||||
const page = await browser.newPage();
|
||||
const assertNoFailures = observePageFailures(page);
|
||||
await page.addInitScript(`
|
||||
Object.defineProperty(globalThis, "showDirectoryPicker", {
|
||||
configurable: true,
|
||||
value: async () => {
|
||||
const originRoot = await navigator.storage.getDirectory();
|
||||
const root = await originRoot.getDirectoryHandle("livesync-webapp-nfc-refusal", { create: true });
|
||||
const settingsDirectory = await root.getDirectoryHandle(".livesync", { create: true });
|
||||
const settingsText = JSON.stringify({
|
||||
pathNormalizationVersion: 1, isConfigured: true, liveSync: true,
|
||||
couchDB_URI: "https://couchdb.example.invalid", couchDB_DBNAME: "nfc-refusal-test"
|
||||
});
|
||||
const settingsWriter = await (await settingsDirectory.getFileHandle("settings.json", { create: true })).createWritable();
|
||||
await settingsWriter.write(settingsText);
|
||||
await settingsWriter.close();
|
||||
const sentinelWriter = await (await root.getFileHandle("Résumé.md".normalize("NFD"), { create: true })).createWritable();
|
||||
await sentinelWriter.write("retained native content");
|
||||
await sentinelWriter.close();
|
||||
const audit = globalThis.webAppIdentityAudit = {
|
||||
settingsText, writes: 0, scans: 0, opens: [], remoteRequests: [],
|
||||
databaseNamesBefore: (await indexedDB.databases()).map(item => item.name).sort()
|
||||
};
|
||||
const open = IDBFactory.prototype.open;
|
||||
IDBFactory.prototype.open = function(...args) {
|
||||
audit.opens.push(String(args[0]));
|
||||
return Reflect.apply(open, this, args);
|
||||
};
|
||||
const createWritable = FileSystemFileHandle.prototype.createWritable;
|
||||
FileSystemFileHandle.prototype.createWritable = function(...args) {
|
||||
audit.writes++;
|
||||
return Reflect.apply(createWritable, this, args);
|
||||
};
|
||||
for (const key of ["entries", "values", Symbol.asyncIterator]) {
|
||||
const original = FileSystemDirectoryHandle.prototype[key];
|
||||
if (typeof original !== "function") continue;
|
||||
FileSystemDirectoryHandle.prototype[key] = function(...args) {
|
||||
audit.scans++;
|
||||
return Reflect.apply(original, this, args);
|
||||
};
|
||||
}
|
||||
const fetch = globalThis.fetch;
|
||||
globalThis.fetch = function(input, ...args) {
|
||||
const url = String(input?.url ?? input);
|
||||
if (url.startsWith("https://couchdb.example.invalid")) {
|
||||
audit.remoteRequests.push(url);
|
||||
return Promise.resolve(new Response("{}", { status: 503 }));
|
||||
}
|
||||
return Reflect.apply(fetch, this, [input, ...args]);
|
||||
};
|
||||
return root;
|
||||
}
|
||||
});
|
||||
`);
|
||||
await page.goto(new URL("webapp.html", server.baseUrl).href);
|
||||
await page.locator("#status").filter({ hasText: "Select a vault folder" }).waitFor();
|
||||
await page.getByRole("button", { name: "Choose new vault folder" }).click();
|
||||
await page.locator("#status.error").filter({ hasText: "WebApp supports Legacy path identity" }).waitFor();
|
||||
const state = await page.evaluate(async () => {
|
||||
const globals = globalThis as typeof globalThis & {
|
||||
webAppIdentityAudit: {
|
||||
settingsText: string;
|
||||
writes: number;
|
||||
scans: number;
|
||||
opens: string[];
|
||||
remoteRequests: string[];
|
||||
databaseNamesBefore: Array<string | undefined>;
|
||||
};
|
||||
livesyncApp: { getRuntime(): unknown };
|
||||
};
|
||||
const audit = globals.webAppIdentityAudit;
|
||||
const root = await (
|
||||
await navigator.storage.getDirectory()
|
||||
).getDirectoryHandle("livesync-webapp-nfc-refusal");
|
||||
const settingsDirectory = await root.getDirectoryHandle(".livesync");
|
||||
return {
|
||||
runtime: globals.livesyncApp.getRuntime(),
|
||||
writes: audit.writes,
|
||||
scans: audit.scans,
|
||||
appDatabaseOpens: audit.opens.filter((name) => name !== "livesync-webapp-handles"),
|
||||
remoteRequests: audit.remoteRequests,
|
||||
databaseNamesBefore: audit.databaseNamesBefore,
|
||||
databaseNamesAfter: (await indexedDB.databases()).map((item) => item.name).sort(),
|
||||
settingsUnchanged:
|
||||
(await (await (await settingsDirectory.getFileHandle("settings.json")).getFile()).text()) ===
|
||||
audit.settingsText,
|
||||
sentinel: await (await (await root.getFileHandle("Résumé.md".normalize("NFD"))).getFile()).text(),
|
||||
};
|
||||
});
|
||||
assertEquals(state.runtime, null);
|
||||
assertEquals(state.writes, 0);
|
||||
assertEquals(state.scans, 0);
|
||||
assertEquals(state.appDatabaseOpens, []);
|
||||
assertEquals(state.remoteRequests, []);
|
||||
assertEquals(state.databaseNamesAfter, state.databaseNamesBefore);
|
||||
assertEquals(state.settingsUnchanged, true);
|
||||
assertEquals(state.sentinel, "retained native content");
|
||||
assertEquals(await page.locator("#vault-selector").isVisible(), true);
|
||||
assertEquals(await page.getByRole("button", { name: "Scan local files", exact: true }).count(), 0);
|
||||
assertNoFailures();
|
||||
} finally {
|
||||
await browser.close();
|
||||
await server.close();
|
||||
}
|
||||
},
|
||||
});
|
||||
@@ -21,10 +21,10 @@
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { copyFile, mkdir, readFile, stat, utimes, writeFile } from "node:fs/promises";
|
||||
import { copyFile, mkdir, readFile, readdir, rm, stat, utimes, writeFile } from "node:fs/promises";
|
||||
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { runCliOrFail, runCliWithInputOrFail } from "./helpers/cli.ts";
|
||||
import { runCli, runCliOrFail, runCliWithInputOrFail } from "./helpers/cli.ts";
|
||||
import { initSettingsFile, markSettingsConfigured } from "./helpers/settings.ts";
|
||||
|
||||
test("mirror: storage <-> DB synchronisation", async (t) => {
|
||||
@@ -136,7 +136,10 @@ test("mirror: storage <-> DB synchronisation", async (t) => {
|
||||
|
||||
// Reflect the shared base into the actual Vault before editing it.
|
||||
await runMirror();
|
||||
assert.deepStrictEqual(await readFile(workDir.join("vault", "test", "sync-storage-newer.md"), "utf8"), "old content\n");
|
||||
assert.deepStrictEqual(
|
||||
await readFile(workDir.join("vault", "test", "sync-storage-newer.md"), "utf8"),
|
||||
"old content\n"
|
||||
);
|
||||
|
||||
// Write new content to storage with a timestamp 1 hour in the future
|
||||
const storageFile = workDir.join("vault", "test", "sync-storage-newer.md");
|
||||
@@ -249,3 +252,96 @@ async function exists(path: string): Promise<boolean> {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
test("mirror: NFC identity retains physical Unicode spelling", async () => {
|
||||
await using workDir = await TempDir.create("livesync-cli-nfc-paths");
|
||||
const vault = workDir.join("vault");
|
||||
const database = workDir.join("db");
|
||||
const settings = workDir.join("settings.json");
|
||||
const parent = "Café".normalize("NFD");
|
||||
const leaf = "Résumé.md".normalize("NFD");
|
||||
const logical = "Café/Résumé.md";
|
||||
const physical = workDir.join("vault", parent, leaf);
|
||||
await mkdir(workDir.join("vault", parent), { recursive: true });
|
||||
await mkdir(database);
|
||||
await initSettingsFile(settings);
|
||||
await markSettingsConfigured(settings);
|
||||
const data = JSON.parse(await readFile(settings, "utf8"));
|
||||
data.pathNormalizationVersion = 1;
|
||||
await writeFile(settings, JSON.stringify(data));
|
||||
await writeFile(physical, "original Unicode content\n");
|
||||
await utimes(physical, new Date(), new Date(Date.now() - 3600_000));
|
||||
|
||||
const run = (...args: string[]) => runCliOrFail(database, "--settings", settings, ...args);
|
||||
await run("mirror", vault);
|
||||
const received = workDir.join("received.md");
|
||||
await run("pull", logical, received);
|
||||
assert.equal(await readFile(received, "utf8"), "original Unicode content\n");
|
||||
|
||||
const replacement = workDir.join("replacement.md");
|
||||
await writeFile(replacement, "updated Unicode content\n");
|
||||
await run("push", replacement, logical);
|
||||
await run("mirror", vault);
|
||||
assert.equal(await readFile(physical, "utf8"), "updated Unicode content\n");
|
||||
assert.deepEqual(await readdir(vault), [parent]);
|
||||
assert.deepEqual(await readdir(workDir.join("vault", parent)), [leaf]);
|
||||
});
|
||||
|
||||
test("mirror: rejects an ambiguous NFC inventory without changing prior data and recovers", async (t) => {
|
||||
await using workDir = await TempDir.create("livesync-cli-nfc-scan-failure");
|
||||
const vault = workDir.join("vault");
|
||||
const database = workDir.join("db");
|
||||
const settings = workDir.join("settings.json");
|
||||
const existingContent = "retained before failed scan\n";
|
||||
const existingFile = workDir.join("vault", "known.md");
|
||||
const nfcParent = "Café";
|
||||
const nfdParent = nfcParent.normalize("NFD");
|
||||
const nfcFile = workDir.join("vault", nfcParent, "from-nfc.md");
|
||||
const nfdFile = workDir.join("vault", nfdParent, "from-nfd.md");
|
||||
|
||||
await mkdir(vault, { recursive: true });
|
||||
await mkdir(database);
|
||||
await initSettingsFile(settings);
|
||||
await markSettingsConfigured(settings);
|
||||
const data = JSON.parse(await readFile(settings, "utf8"));
|
||||
data.pathNormalizationVersion = 1;
|
||||
await writeFile(settings, JSON.stringify(data));
|
||||
await writeFile(existingFile, existingContent);
|
||||
|
||||
const runMirror = () => runCliOrFail(database, "--settings", settings, "mirror", vault);
|
||||
const runDatabase = (...args: string[]) => runCliOrFail(database, "--settings", settings, ...args);
|
||||
|
||||
await runMirror();
|
||||
const before = JSON.parse(await runDatabase("info", "known.md"));
|
||||
assert.equal(await runDatabase("cat-rev", "known.md", before.revision), existingContent);
|
||||
|
||||
await mkdir(workDir.join("vault", nfcParent), { recursive: true });
|
||||
await mkdir(workDir.join("vault", nfdParent), { recursive: true });
|
||||
await writeFile(nfcFile, "NFC spelling content\n");
|
||||
await writeFile(nfdFile, "NFD spelling content\n");
|
||||
const inventoryNames = await readdir(vault);
|
||||
if (!inventoryNames.includes(nfcParent) || !inventoryNames.includes(nfdParent)) {
|
||||
t.skip("This filesystem cannot represent both NFC and NFD directory entries separately");
|
||||
return;
|
||||
}
|
||||
|
||||
const failedMirror = await runCli(database, "--settings", settings, "mirror", vault);
|
||||
assert.notEqual(failedMirror.code, 0, "An ambiguous inventory must fail the mirror command");
|
||||
assert(
|
||||
!failedMirror.combined.includes("[Done] Command 'mirror' completed"),
|
||||
"A failed inventory must not claim successful mirror completion"
|
||||
);
|
||||
assert.equal(await readFile(existingFile, "utf8"), existingContent);
|
||||
assert.equal(await readFile(nfcFile, "utf8"), "NFC spelling content\n");
|
||||
assert.equal(await readFile(nfdFile, "utf8"), "NFD spelling content\n");
|
||||
assert.deepEqual(JSON.parse(await runDatabase("info", "known.md")), before);
|
||||
assert.equal(await runDatabase("cat-rev", "known.md", before.revision), existingContent);
|
||||
|
||||
await rm(workDir.join("vault", nfdParent), { recursive: true });
|
||||
const recoveredMirror = await runMirror();
|
||||
assert(recoveredMirror.includes("[Done] Command 'mirror' completed"));
|
||||
const recovered = JSON.parse(await runDatabase("info", `${nfcParent}/from-nfc.md`));
|
||||
assert.equal(await runDatabase("cat-rev", `${nfcParent}/from-nfc.md`, recovered.revision), "NFC spelling content\n");
|
||||
assert.equal(await readFile(existingFile, "utf8"), existingContent);
|
||||
assert.deepEqual(JSON.parse(await runDatabase("info", "known.md")), before);
|
||||
});
|
||||
+256
-137
@@ -19,152 +19,271 @@ import {
|
||||
import { runCli, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts";
|
||||
import { getOptimalLoopbackIp } from "./helpers/net.ts";
|
||||
|
||||
test("p2p-sync: transfers with the same ID key and rejects a different document ID key", async () => {
|
||||
const loopbackIp = await getOptimalLoopbackIp();
|
||||
const loopbackHost = loopbackIp === "::1" ? "[::1]" : loopbackIp;
|
||||
for (const [pathNormalizationVersion, encrypted] of [
|
||||
[0, true],
|
||||
[1, true],
|
||||
[1, false],
|
||||
] as const) {
|
||||
test(`p2p-sync: identity mode ${pathNormalizationVersion}, encryption ${encrypted} transfers matching IDs and rejects incompatible peers`, async () => {
|
||||
const loopbackIp = await getOptimalLoopbackIp();
|
||||
const loopbackHost = loopbackIp === "::1" ? "[::1]" : loopbackIp;
|
||||
|
||||
const relay = process.env["RELAY"] ?? `ws://${loopbackHost}:4000/`;
|
||||
const roomId = process.env["ROOM_ID"] ?? `room-${Date.now()}`;
|
||||
const passphrase = process.env["PASSPHRASE"] ?? "test";
|
||||
const peersTimeout = Number(process.env["PEERS_TIMEOUT"] ?? "12");
|
||||
const syncTimeout = Number(process.env["SYNC_TIMEOUT"] ?? "15");
|
||||
const nonce = `${Date.now()}-${crypto.randomUUID().slice(0, 8)}`;
|
||||
const hostPeerName = process.env["HOST_PEER_NAME"] ?? `p2p-host-${nonce}`;
|
||||
const clientPeerName = process.env["CLIENT_PEER_NAME"] ?? `p2p-client-${nonce}`;
|
||||
const useCoturn = process.env["LIVESYNC_USE_COTURN"] !== "0";
|
||||
const turnServers = process.env["TURN_SERVERS"] ?? (useCoturn ? `turn:${loopbackHost}:3478` : "none");
|
||||
const relay = process.env["RELAY"] ?? `ws://${loopbackHost}:4000/`;
|
||||
const roomId = process.env["ROOM_ID"] ?? `room-${Date.now()}`;
|
||||
const passphrase = process.env["PASSPHRASE"] ?? "test";
|
||||
const peersTimeout = Number(process.env["PEERS_TIMEOUT"] ?? "12");
|
||||
const syncTimeout = Number(process.env["SYNC_TIMEOUT"] ?? "15");
|
||||
const nonce = `${Date.now()}-${crypto.randomUUID().slice(0, 8)}`;
|
||||
const hostPeerName = process.env["HOST_PEER_NAME"] ?? `p2p-host-${nonce}`;
|
||||
const clientPeerName = process.env["CLIENT_PEER_NAME"] ?? `p2p-client-${nonce}`;
|
||||
const useCoturn = process.env["LIVESYNC_USE_COTURN"] !== "0";
|
||||
const turnServers = process.env["TURN_SERVERS"] ?? (useCoturn ? `turn:${loopbackHost}:3478` : "none");
|
||||
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-sync");
|
||||
const hostVault = workDir.join("vault-host");
|
||||
const hostSettings = workDir.join("settings-host.json");
|
||||
const clientVault = workDir.join("vault-sync");
|
||||
const clientSettings = workDir.join("settings-sync.json");
|
||||
const rejectedVault = workDir.join("vault-rejected");
|
||||
const rejectedSettings = workDir.join("settings-rejected.json");
|
||||
await mkdir(hostVault, { recursive: true });
|
||||
await mkdir(clientVault, { recursive: true });
|
||||
await mkdir(rejectedVault, { recursive: true });
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-sync");
|
||||
const hostVault = workDir.join("vault-host");
|
||||
const hostSettings = workDir.join("settings-host.json");
|
||||
const clientVault = workDir.join("vault-sync");
|
||||
const clientSettings = workDir.join("settings-sync.json");
|
||||
const rejectedVault = workDir.join("vault-rejected");
|
||||
const rejectedSettings = workDir.join("settings-rejected.json");
|
||||
await mkdir(hostVault, { recursive: true });
|
||||
await mkdir(clientVault, { recursive: true });
|
||||
await mkdir(rejectedVault, { recursive: true });
|
||||
|
||||
const relayStarted = await maybeStartLocalRelay(relay);
|
||||
const coturnStarted = await maybeStartCoturn(turnServers);
|
||||
try {
|
||||
await initSettingsFile(hostSettings);
|
||||
await initSettingsFile(clientSettings);
|
||||
await initSettingsFile(rejectedSettings);
|
||||
await applyP2pSettings(
|
||||
hostSettings,
|
||||
roomId,
|
||||
passphrase,
|
||||
"self-hosted-livesync-cli-tests",
|
||||
relay,
|
||||
"~.*",
|
||||
turnServers
|
||||
);
|
||||
await applyP2pSettings(
|
||||
clientSettings,
|
||||
roomId,
|
||||
passphrase,
|
||||
"self-hosted-livesync-cli-tests",
|
||||
relay,
|
||||
"~.*",
|
||||
turnServers
|
||||
);
|
||||
await applyP2pSettings(
|
||||
rejectedSettings,
|
||||
roomId,
|
||||
passphrase,
|
||||
"self-hosted-livesync-cli-tests",
|
||||
relay,
|
||||
"~.*",
|
||||
turnServers
|
||||
);
|
||||
await applyP2pTestTweaks(hostSettings, hostPeerName, passphrase);
|
||||
await applyP2pTestTweaks(clientSettings, clientPeerName, passphrase);
|
||||
await applyP2pTestTweaks(rejectedSettings, "p2p-rejected-" + nonce, passphrase);
|
||||
for (const [vault, path, key, label] of [
|
||||
[hostVault, hostSettings, "ab".repeat(32), "host"],
|
||||
[clientVault, clientSettings, "ab".repeat(32), "client"],
|
||||
[rejectedVault, rejectedSettings, "cd".repeat(32), "rejected"],
|
||||
]) {
|
||||
const settings = JSON.parse(await readFile(path, "utf8"));
|
||||
settings.idDerivationVersion = 1;
|
||||
settings.idDerivationKey = key;
|
||||
const sourcePath = workDir.join("setup-source-" + label + ".json");
|
||||
await writeFile(sourcePath, JSON.stringify(settings));
|
||||
const setupPassphrase = "independent-id-setup-passphrase";
|
||||
const setupUri = await generateSetupUriFromSettings(sourcePath, setupPassphrase, true);
|
||||
await runCliWithInputOrFail(setupPassphrase + "\n", vault, "--settings", path, "setup", setupUri);
|
||||
const persisted = JSON.parse(await readFile(path, "utf8"));
|
||||
assert(persisted.idDerivationVersion === 1, "The Setup URI lost the ID derivation version.");
|
||||
assert(persisted.idDerivationKey === "", "The CLI stored the ID key in plain text.");
|
||||
assert(
|
||||
typeof persisted.encryptedIdDerivationKey === "string" && persisted.encryptedIdDerivationKey.length > 0,
|
||||
"The CLI did not encrypt the saved ID key."
|
||||
);
|
||||
assert(persisted.P2P_Enabled === true, "The Setup URI disabled P2P.");
|
||||
assert(persisted.P2P_roomID === roomId, "The Setup URI changed the P2P room.");
|
||||
assert(persisted.P2P_relays === relay, "The Setup URI changed the P2P relay.");
|
||||
assert(persisted.remoteType === "ONLY_P2P", "The Setup URI changed the remote type.");
|
||||
}
|
||||
const notePath = "p2p/independent-id-note.md";
|
||||
await runCliWithInputOrFail(
|
||||
"A note transferred with the saved ID key.\n",
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"put",
|
||||
notePath
|
||||
);
|
||||
|
||||
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
|
||||
const relayStarted = await maybeStartLocalRelay(relay);
|
||||
const coturnStarted = await maybeStartCoturn(turnServers);
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const peer = await discoverPeer(
|
||||
clientVault,
|
||||
await initSettingsFile(hostSettings);
|
||||
await initSettingsFile(clientSettings);
|
||||
await initSettingsFile(rejectedSettings);
|
||||
await applyP2pSettings(
|
||||
hostSettings,
|
||||
roomId,
|
||||
passphrase,
|
||||
"self-hosted-livesync-cli-tests",
|
||||
relay,
|
||||
"~.*",
|
||||
turnServers
|
||||
);
|
||||
await applyP2pSettings(
|
||||
clientSettings,
|
||||
peersTimeout,
|
||||
process.env["TARGET_PEER"] ?? hostPeerName
|
||||
roomId,
|
||||
passphrase,
|
||||
"self-hosted-livesync-cli-tests",
|
||||
relay,
|
||||
"~.*",
|
||||
turnServers
|
||||
);
|
||||
const syncResult = await runCli(
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"p2p-sync",
|
||||
peer.id,
|
||||
String(syncTimeout)
|
||||
);
|
||||
assert(
|
||||
syncResult.code === 0,
|
||||
`p2p-sync failed\nstdout: ${syncResult.stdout}\nstderr: ${syncResult.stderr}`
|
||||
);
|
||||
const rejectedPeer = await discoverPeer(rejectedVault, rejectedSettings, peersTimeout, hostPeerName);
|
||||
const rejectedSync = await runCli(
|
||||
rejectedVault,
|
||||
"--settings",
|
||||
await applyP2pSettings(
|
||||
rejectedSettings,
|
||||
"p2p-sync",
|
||||
rejectedPeer.id,
|
||||
String(syncTimeout)
|
||||
roomId,
|
||||
passphrase,
|
||||
"self-hosted-livesync-cli-tests",
|
||||
relay,
|
||||
"~.*",
|
||||
turnServers
|
||||
);
|
||||
await applyP2pTestTweaks(hostSettings, hostPeerName, passphrase);
|
||||
await applyP2pTestTweaks(clientSettings, clientPeerName, passphrase);
|
||||
await applyP2pTestTweaks(rejectedSettings, "p2p-rejected-" + nonce, passphrase);
|
||||
for (const [vault, path, key, label] of [
|
||||
[hostVault, hostSettings, "ab".repeat(32), "host"],
|
||||
[clientVault, clientSettings, "ab".repeat(32), "client"],
|
||||
[
|
||||
rejectedVault,
|
||||
rejectedSettings,
|
||||
(pathNormalizationVersion === 1 ? "ab" : "cd").repeat(32),
|
||||
"rejected",
|
||||
],
|
||||
]) {
|
||||
const settings = JSON.parse(await readFile(path, "utf8"));
|
||||
settings.pathNormalizationVersion =
|
||||
pathNormalizationVersion === 1 && label === "rejected" ? 0 : pathNormalizationVersion;
|
||||
settings.encrypt = encrypted;
|
||||
settings.usePathObfuscation = encrypted;
|
||||
settings.idDerivationVersion = 1;
|
||||
settings.idDerivationKey = key;
|
||||
const sourcePath = workDir.join("setup-source-" + label + ".json");
|
||||
await writeFile(sourcePath, JSON.stringify(settings));
|
||||
const setupPassphrase = "independent-id-setup-passphrase";
|
||||
const setupUri = await generateSetupUriFromSettings(sourcePath, setupPassphrase, true);
|
||||
await runCliWithInputOrFail(setupPassphrase + "\n", vault, "--settings", path, "setup", setupUri);
|
||||
const persisted = JSON.parse(await readFile(path, "utf8"));
|
||||
assert(persisted.idDerivationVersion === 1, "The Setup URI lost the ID derivation version.");
|
||||
assert(
|
||||
persisted.pathNormalizationVersion === settings.pathNormalizationVersion,
|
||||
"The Setup URI lost the path identity version."
|
||||
);
|
||||
assert(persisted.idDerivationKey === "", "The CLI stored the ID key in plain text.");
|
||||
assert(
|
||||
typeof persisted.encryptedIdDerivationKey === "string" &&
|
||||
persisted.encryptedIdDerivationKey.length > 0,
|
||||
"The CLI did not encrypt the saved ID key."
|
||||
);
|
||||
assert(persisted.P2P_Enabled === true, "The Setup URI disabled P2P.");
|
||||
assert(persisted.P2P_roomID === roomId, "The Setup URI changed the P2P room.");
|
||||
assert(persisted.P2P_relays === relay, "The Setup URI changed the P2P relay.");
|
||||
assert(persisted.remoteType === "ONLY_P2P", "The Setup URI changed the remote type.");
|
||||
}
|
||||
const notePath =
|
||||
pathNormalizationVersion === 1 ? "Cafe\u0301/re\u0301sume\u0301.md" : "p2p/independent-id-note.md";
|
||||
const receivedPath = notePath.normalize("NFC");
|
||||
await runCliWithInputOrFail(
|
||||
"A note transferred with the saved ID key.\n",
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"put",
|
||||
notePath
|
||||
);
|
||||
|
||||
const originalInfo = JSON.parse(
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "info", notePath)
|
||||
) as { id: string; revision: string };
|
||||
if (pathNormalizationVersion === 1) {
|
||||
assert.equal(
|
||||
sanitiseCatStdout(
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "cat", receivedPath)
|
||||
),
|
||||
"A note transferred with the saved ID key.\n",
|
||||
"The NFC spelling did not address the NFD Metadata."
|
||||
);
|
||||
}
|
||||
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const peer = await discoverPeer(
|
||||
clientVault,
|
||||
clientSettings,
|
||||
peersTimeout,
|
||||
process.env["TARGET_PEER"] ?? hostPeerName
|
||||
);
|
||||
const syncResult = await runCli(
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"p2p-sync",
|
||||
peer.id,
|
||||
String(syncTimeout)
|
||||
);
|
||||
assert(
|
||||
syncResult.code === 0,
|
||||
`p2p-sync failed\nstdout: ${syncResult.stdout}\nstderr: ${syncResult.stderr}`
|
||||
);
|
||||
const rejectedPeer = await discoverPeer(rejectedVault, rejectedSettings, peersTimeout, hostPeerName);
|
||||
const rejectedSync = await runCli(
|
||||
rejectedVault,
|
||||
"--settings",
|
||||
rejectedSettings,
|
||||
"p2p-sync",
|
||||
rejectedPeer.id,
|
||||
String(syncTimeout)
|
||||
);
|
||||
assert(
|
||||
rejectedSync.code !== 0,
|
||||
`P2P accepted incompatible peer settings.\nstdout: ${rejectedSync.stdout}\nstderr: ${rejectedSync.stderr}`
|
||||
);
|
||||
|
||||
assert(
|
||||
rejectedSync.combined.includes("Tweak values are not matched"),
|
||||
`P2P failed before checking peer settings.\nstdout: ${rejectedSync.stdout}\nstderr: ${rejectedSync.stderr}`
|
||||
);
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
const received = sanitiseCatStdout(
|
||||
await runCliOrFail(hostVault, "--settings", hostSettings, "cat", receivedPath)
|
||||
).trimEnd();
|
||||
assert(
|
||||
rejectedSync.code !== 0,
|
||||
`P2P accepted a different key for obfuscated document IDs.\nstdout: ${rejectedSync.stdout}\nstderr: ${rejectedSync.stderr}`
|
||||
);
|
||||
assert(
|
||||
rejectedSync.combined.includes("Tweak values are not matched"),
|
||||
`P2P failed before checking peer settings.\nstdout: ${rejectedSync.stdout}\nstderr: ${rejectedSync.stderr}`
|
||||
received === "A note transferred with the saved ID key.",
|
||||
"The host did not receive the keyed note."
|
||||
);
|
||||
const rejectedRead = await runCli(rejectedVault, "--settings", rejectedSettings, "cat", notePath);
|
||||
assert(rejectedRead.code !== 0, "The rejected device received the keyed note.");
|
||||
if (pathNormalizationVersion === 1) {
|
||||
const canonicalRead = await runCli(rejectedVault, "--settings", rejectedSettings, "cat", receivedPath);
|
||||
assert(
|
||||
canonicalRead.code !== 0,
|
||||
"The rejected device received canonical Metadata despite its different identity mode."
|
||||
);
|
||||
}
|
||||
if (pathNormalizationVersion === 1) {
|
||||
const exchange = async (
|
||||
senderVault: string,
|
||||
senderSettings: string,
|
||||
receiverVault: string,
|
||||
receiverSettings: string,
|
||||
receiverName: string
|
||||
) => {
|
||||
const restartedHost = startCliInBackground(
|
||||
receiverVault,
|
||||
"--settings",
|
||||
receiverSettings,
|
||||
"p2p-host"
|
||||
);
|
||||
try {
|
||||
await restartedHost.waitUntilContains("P2P host is running", 20000);
|
||||
const peer = await discoverPeer(senderVault, senderSettings, peersTimeout, receiverName);
|
||||
await runCliOrFail(
|
||||
senderVault,
|
||||
"--settings",
|
||||
senderSettings,
|
||||
"p2p-sync",
|
||||
peer.id,
|
||||
String(syncTimeout)
|
||||
);
|
||||
} finally {
|
||||
await restartedHost.stop();
|
||||
}
|
||||
};
|
||||
const editedBody = "Edited through the NFC spelling after process restart.\n";
|
||||
await runCliWithInputOrFail(editedBody, hostVault, "--settings", hostSettings, "put", receivedPath);
|
||||
await exchange(hostVault, hostSettings, clientVault, clientSettings, clientPeerName);
|
||||
assert.equal(
|
||||
sanitiseCatStdout(await runCliOrFail(clientVault, "--settings", clientSettings, "cat", notePath)),
|
||||
editedBody
|
||||
);
|
||||
const storedPath = (await runCliOrFail(clientVault, "--settings", clientSettings, "ls"))
|
||||
.trim()
|
||||
.split("\t")[0];
|
||||
assert.equal(storedPath.normalize("NFC"), receivedPath);
|
||||
const editedInfo = JSON.parse(
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "info", storedPath)
|
||||
) as { id: string };
|
||||
assert.equal(editedInfo.id, originalInfo.id, "The reverse edit changed the canonical Metadata ID.");
|
||||
assert.equal(
|
||||
sanitiseCatStdout(
|
||||
await runCliOrFail(
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"cat-rev",
|
||||
notePath,
|
||||
originalInfo.revision
|
||||
)
|
||||
),
|
||||
"A note transferred with the saved ID key.\n",
|
||||
"P2P discarded the original revision body."
|
||||
);
|
||||
|
||||
const renamedPath = "Cafe\u0301/renamed.md";
|
||||
await runCliWithInputOrFail(editedBody, clientVault, "--settings", clientSettings, "put", renamedPath);
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "rm", receivedPath);
|
||||
await exchange(clientVault, clientSettings, hostVault, hostSettings, hostPeerName);
|
||||
assert.notEqual((await runCli(hostVault, "--settings", hostSettings, "cat", notePath)).code, 0);
|
||||
assert.equal(
|
||||
sanitiseCatStdout(
|
||||
await runCliOrFail(hostVault, "--settings", hostSettings, "cat", renamedPath.normalize("NFC"))
|
||||
),
|
||||
editedBody
|
||||
);
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "rm", renamedPath.normalize("NFC"));
|
||||
await exchange(clientVault, clientSettings, hostVault, hostSettings, hostPeerName);
|
||||
assert.notEqual((await runCli(hostVault, "--settings", hostSettings, "cat", renamedPath)).code, 0);
|
||||
}
|
||||
} finally {
|
||||
await host.stop();
|
||||
await stopLocalRelayIfStarted(relayStarted);
|
||||
await stopCoturnIfStarted(coturnStarted);
|
||||
}
|
||||
const received = sanitiseCatStdout(
|
||||
await runCliOrFail(hostVault, "--settings", hostSettings, "cat", notePath)
|
||||
).trimEnd();
|
||||
assert(received === "A note transferred with the saved ID key.", "The host did not receive the keyed note.");
|
||||
const rejectedRead = await runCli(rejectedVault, "--settings", rejectedSettings, "cat", notePath);
|
||||
assert(rejectedRead.code !== 0, "The rejected device received the keyed note.");
|
||||
} finally {
|
||||
await stopLocalRelayIfStarted(relayStarted);
|
||||
await stopCoturnIfStarted(coturnStarted);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
|
||||
FROM node:26-slim AS node26
|
||||
|
||||
FROM mcr.microsoft.com/playwright:v1.62.0-noble
|
||||
|
||||
COPY --from=node26 /usr/local/bin/node /usr/local/bin/node
|
||||
|
||||
ENV NODE_OPTIONS=--max-old-space-size=3072
|
||||
|
||||
WORKDIR /workspace
|
||||
|
||||
ENTRYPOINT ["node", "--import", "tsx", "test/e2e-obsidian/scripts/p2p-nfc-sync.ts"]
|
||||
@@ -230,6 +230,8 @@ Set `E2E_LIVESYNC_TARGET_ARTIFACT_ROOT` to the directory containing the exact re
|
||||
Set `E2E_OBSIDIAN_INDEPENDENT_IDS=true` to run the same upload with E2EE, Path Obfuscation, and a separately derived ID key. The scenario verifies the local document and Chunk ID shapes before the Journal transfer.
|
||||
Set `E2E_OBSIDIAN_CUSTOM_HTTP_HANDLER=true` when the local Object Storage fixture does not allow browser requests from Obsidian's renderer.
|
||||
|
||||
`test:e2e:obsidian:journal-nfc-resend` uses the same Object Storage variables with NFC path identity. Run `npm run test:e2e:obsidian:focused -- journal-nfc-resend --case all` for the plain and E2EE/Path Obfuscation cases with an independent ID key, or select `plain` or `encrypted`. It creates two conflict leaves with real Chunks with an exact shared ancestor, completes the initial upload, and deletes only its unique remote prefix while retaining the source database and populated transfer checkpoint. Rebuilding the remote must resend enough data for a fresh Obsidian Standard Fetch to recover both exact leaf bodies, Chunk references, revision IDs, and each leaf's `_revisions`. The source must retain its readable ancestor body. The receiver records whether that older body is available; ancestry IDs alone do not make a historical body readable. A full receiver restart must retain the same leaves, history, ancestor availability, and one NFC-equivalent physical control file. The restart consumes the saved settings and device state.
|
||||
|
||||
`test:e2e:obsidian:object-storage-setup-uri-workflow` uses the public Commonlib-backed tool to generate the initial Setup URI for a unique Object Storage prefix, completes visible initialisation on the first device, and then asks that working real Obsidian device to create a new Setup URI through the registered command. A second real Obsidian device imports only the device-generated URI. The workflow verifies the A-to-B note through explicit replication, then verifies that the B-to-A note arrives through `syncOnStart` after restarting the first device, without requesting manual replication. It captures the documented onboarding choices, and removes the Object Storage prefix only after both sessions have stopped. The test requires the current version marker and absence of a compatibility pause after Fetch and after restarting the same Vault, without accepting a review automatically.
|
||||
|
||||
`test:e2e:obsidian:object-storage-qr-workflow` runs the same Object Storage round trip with QR settings on the second device. It takes the first device's settings, assigns a distinct database suffix in the QR fixture, encodes them with Commonlib's QR encoder, passes the payload to the real QR decoding entry point, and selects **Join this device** in the visible dialogue. Unlike the Setup URI, the QR payload includes a database suffix; explicitly choosing one makes the namespace change independent of Obsidian's initial defaults. Before Fetch begins, the scenario verifies that the imported namespace has its current compatibility marker without a pause. Fetch then selects the receiving device's own suffix when resetting the local database. The test verifies the marker and absence of a pause again after Fetch and after a natural restart. It covers the QR settings and setup flow, without requiring a camera or exercising operating-system URI dispatch.
|
||||
@@ -244,6 +246,12 @@ The P2P Setup URI workflow also schedules another Fetch and injects an exception
|
||||
|
||||
`test:e2e:obsidian:p2p-connection-check` owns the browser-to-Obsidian preflight path. It serves the WebPeer production build from loopback, asks the page to generate a disposable Setup URI using the local relay, starts its browser reference peer, and applies that exact URI through visible onboarding in an isolated empty real Obsidian Vault. After the first successful WebRTC diagnostic appears, it selects the action for another device in the same room, proves that the Setup URI was not regenerated, applies it to a second isolated empty real Obsidian Vault, and requires both the successful total and the baseline number of simultaneous active connections to advance. It captures the result card without Setup URI credentials and does not claim to verify note synchronisation. Run `test:e2e:obsidian:p2p-connection-check:services` to build both production artefacts and let the scenario start and stop the Compose relay.
|
||||
|
||||
`test:e2e:obsidian:p2p-nfc` runs the plain case and the E2EE case using Path Obfuscation and an independent ID key, serially, in an isolated Compose project with a local signalling relay and two real Obsidian processes at a time. Each case seeds an NFD-named source file, fetches it by its NFC spelling, and compares the real local Metadata ID, exact body, and referenced Chunks on both peers. It edits the note from the receiving peer, verifies that the Metadata ID remains stable and that the original revision body remains readable, then fully stops and relaunches the same Vault to verify the ID, updated body, original revision, and a further transfer after restart. A fresh path version 0 peer must show the visible P2P compatibility refusal; the test verifies that it receives no source Metadata and retains its local sentinel. Run `npm run test:e2e:obsidian:p2p-nfc -- --case plain` or `-- --case encrypted` to select one case for debugging. The host wrapper builds the plug-in, uses the existing extracted Obsidian directory and its `obsidian-cli`, mounts the repository read-only, and tears down its unique Compose project after the run. It does not download Obsidian or npm dependencies.
|
||||
|
||||
Each P2P case also transfers another Unicode note after restart, renames it on the receiving peer, compares the new Metadata IDs and exact content, and verifies deletion on both peers.
|
||||
|
||||
Add `-- --migration` to start with legacy databases and exercise source Rebuild followed by receiving-device Standard Fetch. Because Obsidian's file cache normalises names, the scenario seeds valid legacy NFD Metadata IDs using the real stored Chunks before reconstruction. It verifies replacement of those IDs, physical NFD-name retention, ordinary legacy-peer refusal, and successful migration after a source restart. It also attempts Fetch with mismatched settings, verifies the local database is reset before peer refusal while the physical note remains intact, then retries with NFC and verifies convergence. The migration phase calls the running plug-in's production Rebuild and Fetch methods and uses real peer selection; reservation through the settings dialogue has separate coverage.
|
||||
|
||||
`test:e2e:obsidian:startup-scan` starts from a CouchDB fixture using current settings with its device-local compatibility marker already acknowledged, stops Obsidian, writes a note directly into the Vault, restarts the same isolated Vault and profile without rewriting its plug-in data, and verifies from CouchDB that the start-up scan picked up the offline file. Onboarding remains covered by `onboarding-invitation`; this scenario owns the ordinary configured restart and start-up scan.
|
||||
|
||||
`test:e2e:obsidian:partial-startup-file-failure` is a focused Linux release-acceptance scenario for an ordinary configured restart. It stores one valid database-only note and one database-only note whose path component is 258 UTF-8 bytes, then restarts the same isolated Vault and profile. On a Linux test Vault which enforces the conventional 255-byte component limit, the scenario requires the valid file to be reflected, the application to become ready, the partial-failure Notice to appear, and the failed path to remain readable and eligible for a later scan with its exact path in the verbose log. It remains outside `local-suite` because the failure fixture is deliberately platform-specific.
|
||||
@@ -264,7 +272,9 @@ The workflow creates a random dedicated database, records only SHA-256 Seed fing
|
||||
|
||||
This proves in real Obsidian the plug-in behaviour shared by supported platforms, including the encrypted bidirectional round-trip and protection against a stale client restoring the old remote Seed. It does not verify iPadOS-specific background or reconnect lifecycle behaviour, and it does not count as Android device evidence. The workflow remains outside `test:e2e:obsidian:local-suite` because it is a focused release-acceptance check.
|
||||
|
||||
`test:e2e:obsidian:conflict-dialog-policy` creates three real local revision leaves without a remote service and opens the pairwise merge dialogue in Obsidian. It verifies the three-version count, requires the four decision buttons to be stacked vertically, concatenates the displayed pair as a child of the displayed winner, confirms that the untouched leaf remains as one conflict, postpones that remaining pair, restarts the same isolated Vault and profile, and confirms that only the two current versions are reconstructed. It also verifies that an ordinary repeated conflict check does not reopen a postponed dialogue, that **Resolve if conflicted.** explicitly reopens it, and that the active editor retains the appropriate unresolved-conflict warning. The scenario then invokes the same Commonlib consumer boundary used for an incoming replicated document and checks that a postponed warning disappears, an open stale dialogue closes, and the conflict-processing queue completes even when the dialogue closes immediately. This isolates the Obsidian UI contract from transport and second-device setup. The fixture owns one temporary Vault and profile, and the session runner stops Obsidian before removing them.
|
||||
`test:e2e:obsidian:conflict-dialog-policy` creates three real local revision leaves without a remote service and opens the pairwise merge dialogue in Obsidian. It verifies the three-version count, requires the four decision buttons to be stacked vertically, concatenates the displayed pair as a child of the displayed winner, confirms that the untouched leaf remains as one conflict, postpones that remaining pair, restarts the same isolated Vault and profile, and confirms that only the two current versions are reconstructed. It also verifies that an ordinary repeated conflict check does not reopen a postponed dialogue, that **Resolve if conflicted.** explicitly reopens it, and that the active editor retains the appropriate unresolved-conflict warning. The scenario then invokes the same Commonlib consumer boundary used for an incoming replicated document and checks that a postponed warning disappears, an open stale dialogue closes, and the conflict-processing queue completes even when the dialogue closes immediately. This isolates the Obsidian UI contract from transport and second-device setup. The fixture owns one temporary Vault and profile, and the session runner stops Obsidian before removing them. Its unload check requires the queue to report cancellation, closes active and waiting dialogues, and compares Vault contents, conflict leaves, and revision ancestry after reload.
|
||||
|
||||
`test:e2e:obsidian:log-lifecycle` delays the local database close and tests both successful and failed shutdown. It verifies that logs remain available during shutdown, that later logs go to the console after display subscriptions are released, and that logs from the previous instance do not reach the reloaded instance. It uses an isolated Vault and profile without a remote service.
|
||||
|
||||
`test:e2e:obsidian:revision-repair` creates an ordinary healthy logical deletion and two current leaf revisions in a temporary real Obsidian Vault, then removes a chunk used only by the non-winning revision. It proves that automatic conflict checking does not discard the unreadable branch, and that a healthy logical deletion with no Vault file is neither reported nor retained as Vault provenance. **Inspect conflicts and file/database differences** must show the winner and conflict separately, identify the exact unreadable revision and missing chunk, show the compact `Δsize` and `Δtime` diagnostics, and expose a wrench menu with the appropriate actions for each branch. The scenario opens the existing comparison dialogue in read-only mode, applies the readable winner to the Vault, shows the compact matching-winner and remaining-conflict status, records the exact winner as Vault provenance without creating a child, and confirms that retrying the unreadable branch leaves the revision tree unchanged. It then verifies both the cancellation path and the explicit confirmation path for discarding only that selected current branch, requires the winner and its Vault provenance to remain unchanged, and captures the repair card, a 360-pixel-wide reflow check, the matching-winner status, both revision menus, and the read-only comparison. The narrow capture checks responsive layout, not a mobile operating-system lifecycle. The scenario uses no remote service; a retry is therefore expected to remain unreadable unless the chunk is already available locally.
|
||||
|
||||
@@ -404,6 +414,11 @@ When these checks pass for the release artefacts, BRAT acceptance can be limited
|
||||
- `E2E_OBSIDIAN_REVIEW_HARNESS_TIMEOUT_MS`: timeout for Review Harness view and action boundaries; default is 15 seconds.
|
||||
- `E2E_OBSIDIAN_P2P_PANE_TIMEOUT_MS`: timeout for the P2P status pane and its principal connection control; default is 10 seconds.
|
||||
- `E2E_OBSIDIAN_P2P_WORKFLOW_TIMEOUT_MS`: timeout for each visible P2P Setup URI, peer-discovery, approval, and replication control; default is 60 seconds.
|
||||
- `E2E_OBSIDIAN_P2P_NFC_TIMEOUT_MS`: timeout for P2P NFC transfers and the incompatible-peer refusal; default is 120 seconds.
|
||||
- `E2E_OBSIDIAN_P2P_NFC_UI_TIMEOUT_MS`: timeout for P2P NFC status and setup controls; default is 60 seconds.
|
||||
- `E2E_P2P_NFC_KEEP_ARTIFACTS=1`: retain successful Compose screenshots and diagnostics; failures are always retained.
|
||||
- `E2E_P2P_NFC_CASE`: case selected by the Compose wrapper (`all`, `plain`, or `encrypted`); default is `all`. The `--case` option overrides this value.
|
||||
- `E2E_P2P_NFC_OBSIDIAN_DIR`: extracted Linux Obsidian directory mounted into the Compose runner. The wrapper normally infers it from the discovered `OBSIDIAN_BINARY` and requires the corresponding `obsidian-cli` in the same directory.
|
||||
- `E2E_P2P_CHECK_CONNECTION_TIMEOUT_MS`: timeout for the browser-to-Obsidian successful WebRTC diagnostic; default is 60 seconds.
|
||||
- `E2E_P2P_CHECK_SCREENSHOT`: explicit path for the successful browser result screenshot; default is `p2p-connection-check-browser-success.png` under `E2E_OBSIDIAN_DIAGNOSTICS_DIR`.
|
||||
- `E2E_P2P_RELAY_URL`: signalling relay used by the real-Obsidian P2P workflow; default is the local relay at `ws://127.0.0.1:4010/`.
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
name: livesync-p2p-nfc
|
||||
|
||||
services:
|
||||
p2p-relay:
|
||||
image: ghcr.io/hoytech/strfry:latest
|
||||
entrypoint: ["/app/strfry"]
|
||||
command: ["--config", "/etc/strfry/strfry.conf", "relay"]
|
||||
volumes:
|
||||
- ../../test/fixtures/p2p-relay/strfry.conf:/etc/strfry/strfry.conf:ro
|
||||
tmpfs:
|
||||
- /app/strfry-db:rw,size=256m,mode=1777
|
||||
|
||||
p2p-nfc:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/e2e-obsidian/Dockerfile.p2p
|
||||
depends_on:
|
||||
p2p-relay:
|
||||
condition: service_started
|
||||
environment:
|
||||
OBSIDIAN_BINARY: ${E2E_P2P_NFC_OBSIDIAN_BINARY:?Run this scenario through run-compose-p2p-nfc.ts}
|
||||
OBSIDIAN_CLI: ${E2E_P2P_NFC_OBSIDIAN_CLI:?Run this scenario through run-compose-p2p-nfc.ts}
|
||||
E2E_P2P_RELAY_URL: ws://p2p-relay:7777/
|
||||
E2E_P2P_NFC_CASE: ${E2E_P2P_NFC_CASE:-all}
|
||||
E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT: "9222"
|
||||
E2E_OBSIDIAN_SECONDARY_REMOTE_DEBUGGING_PORT: "9223"
|
||||
E2E_OBSIDIAN_DIAGNOSTICS_DIR: /artifacts
|
||||
volumes:
|
||||
- ../..:/workspace:ro
|
||||
- ${E2E_P2P_NFC_OBSIDIAN_DIR:?Run this scenario through run-compose-p2p-nfc.ts}:/opt/obsidian:ro
|
||||
- ${E2E_P2P_NFC_ARTIFACT_DIR:?Run this scenario through run-compose-p2p-nfc.ts}:/artifacts
|
||||
working_dir: /workspace
|
||||
mem_limit: 6g
|
||||
cpus: 2
|
||||
init: true
|
||||
ipc: host
|
||||
@@ -0,0 +1,243 @@
|
||||
import { deepStrictEqual } from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { assertEqual } from "./liveSyncWorkflow.ts";
|
||||
import { deleteObjectStoragePrefix, readObjectStorageJson, type ObjectStorageConfig } from "./objectStorage.ts";
|
||||
|
||||
const notePath = "Cafe\u0301/re\u0301sume\u0301.md";
|
||||
const noteContent = "# CLI journal migration\n\nThe NFD physical path must keep its identity during migration.\n";
|
||||
|
||||
type CliFileInfo = {
|
||||
id: string;
|
||||
revision: string;
|
||||
path: string;
|
||||
};
|
||||
|
||||
type JournalMilestone = {
|
||||
node_chunk_info?: Record<string, { current?: number }>;
|
||||
};
|
||||
|
||||
type CliResult = {
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
};
|
||||
|
||||
type CliRunner = (...args: string[]) => Promise<CliResult>;
|
||||
|
||||
function cliErrorOutput(error: unknown): string {
|
||||
if (error === null || typeof error !== "object") {
|
||||
return String(error);
|
||||
}
|
||||
const details = error as { message?: unknown; stdout?: unknown; stderr?: unknown };
|
||||
return [details.message, details.stdout, details.stderr]
|
||||
.filter((value): value is string => typeof value === "string")
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
async function readSingleMetadata(run: CliRunner, label: string): Promise<CliFileInfo> {
|
||||
const output = (await run("ls")).stdout.trimEnd();
|
||||
const paths = output === "" ? [] : output.split(/\r?\n/u).map((line) => line.split("\t", 1)[0] ?? "");
|
||||
assertEqual(paths.length, 1, `${label} did not contain exactly one Metadata document.`);
|
||||
const storedPath = paths[0];
|
||||
if (!storedPath) {
|
||||
throw new Error(`${label} did not report a stored path.`);
|
||||
}
|
||||
return JSON.parse((await run("info", storedPath)).stdout) as CliFileInfo;
|
||||
}
|
||||
|
||||
export async function runCliJournalMigration(
|
||||
objectStorage: ObjectStorageConfig,
|
||||
bucketPrefix: string,
|
||||
sourceSettings: Record<string, unknown>
|
||||
): Promise<void> {
|
||||
const migrationPrefix = `${bucketPrefix}migration/`;
|
||||
const cliRoot = await mkdtemp(join(tmpdir(), "livesync-cli-journal-migration-"));
|
||||
const sourceRoot = join(cliRoot, "source");
|
||||
const targetRoot = join(cliRoot, "target");
|
||||
const sourceDatabasePath = join(sourceRoot, "database");
|
||||
const targetDatabasePath = join(targetRoot, "database");
|
||||
const sourceVaultPath = join(sourceRoot, "vault");
|
||||
const targetVaultPath = join(targetRoot, "vault");
|
||||
const sourceSettingsPath = join(sourceRoot, "settings.json");
|
||||
const targetSettingsPath = join(targetRoot, "settings.json");
|
||||
const physicalParent = notePath.split("/")[0] ?? "";
|
||||
const physicalFilename = notePath.split("/").slice(-1)[0] ?? "";
|
||||
|
||||
const writeCliSettings = async (settingsPath: string, pathNormalizationVersion: number) => {
|
||||
const previous = await readFile(settingsPath, "utf8").then(
|
||||
(text) => JSON.parse(text) as Record<string, unknown>,
|
||||
(error: NodeJS.ErrnoException) => {
|
||||
if (error.code !== "ENOENT") throw error;
|
||||
return undefined;
|
||||
}
|
||||
);
|
||||
await writeFile(
|
||||
settingsPath,
|
||||
JSON.stringify({
|
||||
...(previous ?? {
|
||||
...DEFAULT_SETTINGS,
|
||||
...objectStorage,
|
||||
...sourceSettings,
|
||||
remoteType: "MINIO",
|
||||
bucketPrefix: migrationPrefix,
|
||||
isConfigured: true,
|
||||
}),
|
||||
pathNormalizationVersion,
|
||||
liveSync: false,
|
||||
syncOnStart: false,
|
||||
syncOnSave: false,
|
||||
syncOnEditorSave: false,
|
||||
syncOnFileOpen: false,
|
||||
syncAfterMerge: false,
|
||||
periodicReplication: false,
|
||||
})
|
||||
);
|
||||
};
|
||||
|
||||
try {
|
||||
await mkdir(sourceDatabasePath, { recursive: true });
|
||||
await mkdir(targetDatabasePath, { recursive: true });
|
||||
await mkdir(dirname(join(sourceVaultPath, notePath)), { recursive: true });
|
||||
await mkdir(dirname(join(targetVaultPath, notePath)), { recursive: true });
|
||||
await writeFile(join(sourceVaultPath, notePath), noteContent);
|
||||
await writeFile(join(targetVaultPath, notePath), noteContent);
|
||||
await writeCliSettings(sourceSettingsPath, 0);
|
||||
await writeCliSettings(targetSettingsPath, 0);
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const createRunner =
|
||||
(databasePath: string, vaultPath: string, settingsPath: string): CliRunner =>
|
||||
async (...args: string[]) =>
|
||||
await execute(
|
||||
process.execPath,
|
||||
[
|
||||
resolve("src/apps/cli/dist/index.cjs"),
|
||||
databasePath,
|
||||
"--vault",
|
||||
vaultPath,
|
||||
"--settings",
|
||||
settingsPath,
|
||||
...args,
|
||||
],
|
||||
{ timeout: 60_000, maxBuffer: 4 * 1024 * 1024 }
|
||||
);
|
||||
|
||||
const source = createRunner(sourceDatabasePath, sourceVaultPath, sourceSettingsPath);
|
||||
const target = createRunner(targetDatabasePath, targetVaultPath, targetSettingsPath);
|
||||
|
||||
await source("--force", "--write-settings", "rebuild");
|
||||
await target("mirror");
|
||||
|
||||
const legacySource = await readSingleMetadata(source, "Legacy CLI source");
|
||||
const legacyTarget = await readSingleMetadata(target, "Legacy CLI target");
|
||||
const targetVaultEntriesBeforeFetch = await readdir(targetVaultPath);
|
||||
const targetParentEntriesBeforeFetch = await readdir(dirname(join(targetVaultPath, notePath)));
|
||||
assertEqual(
|
||||
targetVaultEntriesBeforeFetch.includes(physicalParent),
|
||||
true,
|
||||
"The legacy CLI target did not retain the NFD physical parent."
|
||||
);
|
||||
assertEqual(
|
||||
targetParentEntriesBeforeFetch.includes(physicalFilename),
|
||||
true,
|
||||
"The legacy CLI target did not retain the NFD physical filename."
|
||||
);
|
||||
|
||||
await writeCliSettings(sourceSettingsPath, 1);
|
||||
await source("--force", "--write-settings", "rebuild");
|
||||
|
||||
const migratedSource = await readSingleMetadata(source, "Migrated CLI source");
|
||||
if (migratedSource.id === legacySource.id) {
|
||||
throw new Error("The NFC migration did not change the source Metadata ID.");
|
||||
}
|
||||
assertEqual(
|
||||
(await source("cat", notePath.normalize("NFC"))).stdout,
|
||||
noteContent,
|
||||
"The migrated CLI source did not retain the canonical Chunk content."
|
||||
);
|
||||
|
||||
const milestoneKey = `${migrationPrefix}_00000000-milestone.json`;
|
||||
const milestoneBeforeFailedFetch = await readObjectStorageJson<JournalMilestone>(objectStorage, milestoneKey);
|
||||
const migrationVersions = Object.values(milestoneBeforeFailedFetch.node_chunk_info ?? {}).map(
|
||||
(node) => node.current
|
||||
);
|
||||
assertEqual(migrationVersions.includes(4), true, "The migrated Journal milestone did not declare version 4.");
|
||||
|
||||
let legacyFetchError: unknown;
|
||||
try {
|
||||
await target("--force", "--write-settings", "fetch");
|
||||
} catch (error) {
|
||||
legacyFetchError = error;
|
||||
}
|
||||
if (legacyFetchError === undefined) {
|
||||
throw new Error("The legacy CLI target unexpectedly fetched a Journal with NFC path identity.");
|
||||
}
|
||||
const legacyFetchOutput = cliErrorOutput(legacyFetchError);
|
||||
if (!/path identity/iu.test(legacyFetchOutput)) {
|
||||
throw new Error(
|
||||
`The legacy CLI Fetch failed without the expected path identity error. Output: ${legacyFetchOutput}`
|
||||
);
|
||||
}
|
||||
|
||||
const targetAfterFailedFetch = await readSingleMetadata(target, "Legacy CLI target after failed Fetch");
|
||||
assertEqual(
|
||||
targetAfterFailedFetch.id,
|
||||
legacyTarget.id,
|
||||
"The failed legacy Fetch changed the target Metadata ID."
|
||||
);
|
||||
assertEqual(
|
||||
targetAfterFailedFetch.revision,
|
||||
legacyTarget.revision,
|
||||
"The failed legacy Fetch changed the target Metadata revision."
|
||||
);
|
||||
const milestoneAfterFailedFetch = await readObjectStorageJson<JournalMilestone>(objectStorage, milestoneKey);
|
||||
deepStrictEqual(
|
||||
milestoneAfterFailedFetch,
|
||||
milestoneBeforeFailedFetch,
|
||||
"The failed legacy Fetch changed the remote Journal milestone."
|
||||
);
|
||||
|
||||
await writeCliSettings(targetSettingsPath, 1);
|
||||
await target("--force", "--write-settings", "fetch");
|
||||
|
||||
const migratedTarget = await readSingleMetadata(target, "Migrated CLI target");
|
||||
assertEqual(migratedTarget.id, migratedSource.id, "The CLI source and target Metadata IDs did not converge.");
|
||||
assertEqual(
|
||||
(await target("cat", notePath.normalize("NFC"))).stdout,
|
||||
noteContent,
|
||||
"The migrated CLI target did not retain the canonical Chunk content."
|
||||
);
|
||||
assertEqual(
|
||||
await readFile(join(targetVaultPath, notePath), "utf8"),
|
||||
noteContent,
|
||||
"The migrated CLI Fetch changed the NFD physical file contents."
|
||||
);
|
||||
|
||||
const targetVaultEntriesAfterFetch = await readdir(targetVaultPath);
|
||||
const targetParentEntriesAfterFetch = await readdir(dirname(join(targetVaultPath, notePath)));
|
||||
deepStrictEqual(
|
||||
targetVaultEntriesAfterFetch,
|
||||
targetVaultEntriesBeforeFetch,
|
||||
"The migrated CLI Fetch changed the physical Vault parent names."
|
||||
);
|
||||
deepStrictEqual(
|
||||
targetParentEntriesAfterFetch,
|
||||
targetParentEntriesBeforeFetch,
|
||||
"The migrated CLI Fetch changed the physical NFD filename."
|
||||
);
|
||||
|
||||
console.log(
|
||||
"CLI_JOURNAL_MIGRATION: legacy Fetch rejection, safe migration, and NFD physical path preservation verified."
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
await rm(cliRoot, { recursive: true, force: true });
|
||||
} finally {
|
||||
await deleteObjectStoragePrefix(objectStorage, migrationPrefix);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -34,6 +34,7 @@ export type RemoteActivitySnapshot = {
|
||||
responseCount: number;
|
||||
statusBarFound: boolean;
|
||||
statusBarText: string;
|
||||
indicatorElements?: Array<{ text: string; inactive: boolean; ariaHidden: string | null; display: string }>;
|
||||
};
|
||||
|
||||
export type ExpectedRemoteActivityState =
|
||||
@@ -88,14 +89,26 @@ async function readRemoteActivitySnapshotFromPage(page: Page): Promise<RemoteAct
|
||||
gateKind: gate?.kind,
|
||||
requestCount: Number(core.services?.API?.requestCount?.value ?? -1),
|
||||
remoteOperationIndicatorVisible: statusBars.some((element) =>
|
||||
(element.textContent ?? "").includes(operationIcon)
|
||||
),
|
||||
remoteRequestIndicatorVisible: statusBars.some((element) =>
|
||||
(element.textContent ?? "").includes(requestIcon)
|
||||
element.innerText.includes(operationIcon)
|
||||
),
|
||||
remoteRequestIndicatorVisible: statusBars.some((element) => element.innerText.includes(requestIcon)),
|
||||
responseCount: Number(core.services?.API?.responseCount?.value ?? -1),
|
||||
statusBarFound: statusBars.length > 0,
|
||||
statusBarText: statusBars.map((element) => element.textContent ?? "").join("\n"),
|
||||
statusBarText: statusBars.map((element) => element.innerText).join("\n"),
|
||||
indicatorElements: Array.from(
|
||||
document.querySelectorAll<HTMLElement>(".syncstatusbar .livesync-stat-item")
|
||||
)
|
||||
.filter(
|
||||
(element) =>
|
||||
(element.textContent ?? "").includes(operationIcon) ||
|
||||
(element.textContent ?? "").includes(requestIcon)
|
||||
)
|
||||
.map((element) => ({
|
||||
text: element.textContent ?? "",
|
||||
inactive: element.classList.contains("livesync-stat-inactive"),
|
||||
ariaHidden: element.getAttribute("aria-hidden"),
|
||||
display: getComputedStyle(element).display,
|
||||
})),
|
||||
} satisfies RemoteActivitySnapshot;
|
||||
},
|
||||
{
|
||||
@@ -141,7 +154,7 @@ export async function waitForRemoteActivityState(
|
||||
if (!core) return false;
|
||||
const gate = (globalThis as unknown as Record<string, RuntimeGate | undefined>)[stateKey];
|
||||
const statusBarText = Array.from(document.querySelectorAll<HTMLElement>(".syncstatusbar"))
|
||||
.map((element) => element.textContent ?? "")
|
||||
.map((element) => element.innerText)
|
||||
.join("\n");
|
||||
const bounded = Number(core.services?.replicator?.boundedRemoteActivityCount?.value ?? -1);
|
||||
const finite = Number(core.services?.replicator?.finiteReplicationActivityCount?.value ?? -1);
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { DoctorRegulation } from "@vrtmrz/livesync-commonlib/compat/common/configForDoc";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import {
|
||||
@@ -28,6 +31,11 @@ type ConflictFixture = {
|
||||
conflicts: string[];
|
||||
};
|
||||
|
||||
type ObservedConflictService = {
|
||||
ensureAllProcessed(): Promise<boolean>;
|
||||
conflictProcessQueueCount: { value: number };
|
||||
};
|
||||
|
||||
type ObsidianTestApp = {
|
||||
commands?: { executeCommandById(commandId: string): boolean };
|
||||
plugins?: {
|
||||
@@ -38,10 +46,7 @@ type ObsidianTestApp = {
|
||||
| {
|
||||
core?: {
|
||||
services?: {
|
||||
conflict?: {
|
||||
ensureAllProcessed(): Promise<boolean>;
|
||||
conflictProcessQueueCount: { value: number };
|
||||
};
|
||||
conflict?: ObservedConflictService;
|
||||
};
|
||||
};
|
||||
}
|
||||
@@ -53,9 +58,46 @@ type ObsidianTestApp = {
|
||||
type ObsidianTestGlobal = typeof globalThis & {
|
||||
app?: ObsidianTestApp;
|
||||
__livesyncConflictChecksCompleted?: boolean;
|
||||
__livesyncConflictChecksResult?: boolean;
|
||||
__livesyncWaitingConflictCompleted?: boolean;
|
||||
__livesyncConflictChecksError?: string;
|
||||
__livesyncWaitingConflictError?: string;
|
||||
__livesyncObservedConflict?: ObservedConflictService;
|
||||
};
|
||||
|
||||
async function recordConflictDiagnostics(port: number, label: string) {
|
||||
const state = await withObsidianPage(
|
||||
port,
|
||||
async (page) =>
|
||||
await page.evaluate(() => {
|
||||
const host = globalThis as ObsidianTestGlobal;
|
||||
const plugin = host.app?.plugins?.plugins?.["obsidian-livesync"];
|
||||
const conflict = host.__livesyncObservedConflict ?? plugin?.core?.services?.conflict;
|
||||
return {
|
||||
pluginLoaded: plugin !== undefined,
|
||||
queueCount: conflict?.conflictProcessQueueCount.value,
|
||||
checksCompleted: host.__livesyncConflictChecksCompleted,
|
||||
checksResult: host.__livesyncConflictChecksResult,
|
||||
waitingCompleted: host.__livesyncWaitingConflictCompleted,
|
||||
checksError: host.__livesyncConflictChecksError,
|
||||
waitingError: host.__livesyncWaitingConflictError,
|
||||
visibility: document.visibilityState,
|
||||
dialogues: Array.from(document.querySelectorAll(".modal-container")).map((element) => ({
|
||||
visible: element.getClientRects().length > 0,
|
||||
title: element.querySelector(".modal-title")?.textContent,
|
||||
text: element.textContent?.slice(0, 300),
|
||||
actions: element.querySelectorAll(".conflict-action-button").length,
|
||||
})),
|
||||
};
|
||||
})
|
||||
);
|
||||
const directory = process.env.E2E_OBSIDIAN_DIAGNOSTICS_DIR ?? "/tmp/obsidian-livesync-e2e";
|
||||
await mkdir(directory, { recursive: true });
|
||||
await writeFile(join(directory, `conflict-${label}.json`), JSON.stringify(state, null, 2) + "\n");
|
||||
console.log(`Conflict ${label}: ${JSON.stringify(state)}`);
|
||||
return state;
|
||||
}
|
||||
|
||||
async function createAndOpenBaseFile(cliBinary: string, env: NodeJS.ProcessEnv, targetPath = path): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
@@ -249,7 +291,7 @@ async function createTwoVersionConflict(
|
||||
if (fixture.conflicts.length !== 1) {
|
||||
throw new Error(`Expected exactly two live leaves for ${targetPath}: ${JSON.stringify(fixture)}`);
|
||||
}
|
||||
return fixture;
|
||||
return { ...fixture, currentParentRev: base.rev };
|
||||
}
|
||||
|
||||
async function setShowMergeDialogOnlyOnActive(
|
||||
@@ -308,13 +350,19 @@ async function requestInteractiveConflictResolution(
|
||||
`const conflictRev=${JSON.stringify(fixture.conflicts[0])};`,
|
||||
`const trackCompletion=${JSON.stringify(trackCompletion)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"if(trackCompletion) globalThis.__livesyncWaitingConflictCompleted=false;",
|
||||
"if(trackCompletion){",
|
||||
" globalThis.__livesyncWaitingConflictCompleted=false;",
|
||||
" globalThis.__livesyncWaitingConflictError=undefined;",
|
||||
"}",
|
||||
"const pending=core.services.conflict.resolveByUserInteraction(path,{",
|
||||
"left:{rev:currentRev,data:'Current branch',ctime:1,mtime:2},",
|
||||
"right:{rev:conflictRev,data:'Conflict branch',ctime:1,mtime:3},",
|
||||
"diff:[[0,'Current and conflict branches']],",
|
||||
"});",
|
||||
"if(trackCompletion) void pending.then(()=>{globalThis.__livesyncWaitingConflictCompleted=true;});",
|
||||
"if(trackCompletion) void pending.then(",
|
||||
" ()=>{globalThis.__livesyncWaitingConflictCompleted=true;},",
|
||||
" (error)=>{globalThis.__livesyncWaitingConflictError=error?.stack??String(error);}",
|
||||
");",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
@@ -328,10 +376,20 @@ async function disableLiveSyncAndWaitForConflictChecks(port: number): Promise<vo
|
||||
const host = globalThis as ObsidianTestGlobal;
|
||||
const conflict = host.app?.plugins?.plugins?.["obsidian-livesync"]?.core?.services?.conflict;
|
||||
if (conflict === undefined) throw new Error("LiveSync conflict service is unavailable before unload");
|
||||
host.__livesyncObservedConflict = conflict;
|
||||
host.__livesyncConflictChecksCompleted = false;
|
||||
void conflict.ensureAllProcessed().then(() => {
|
||||
host.__livesyncConflictChecksCompleted = true;
|
||||
});
|
||||
host.__livesyncConflictChecksResult = undefined;
|
||||
host.__livesyncConflictChecksError = undefined;
|
||||
void conflict.ensureAllProcessed().then(
|
||||
(result) => {
|
||||
host.__livesyncConflictChecksResult = result;
|
||||
host.__livesyncConflictChecksCompleted = true;
|
||||
},
|
||||
(error) => {
|
||||
host.__livesyncConflictChecksError =
|
||||
error instanceof Error ? (error.stack ?? String(error)) : String(error);
|
||||
}
|
||||
);
|
||||
});
|
||||
await page.evaluate(async () => {
|
||||
const plugins = (globalThis as ObsidianTestGlobal).app?.plugins;
|
||||
@@ -349,6 +407,8 @@ async function disableLiveSyncAndWaitForConflictChecks(port: number): Promise<vo
|
||||
undefined,
|
||||
{ timeout: uiTimeoutMs }
|
||||
);
|
||||
const result = await page.evaluate(() => (globalThis as ObsidianTestGlobal).__livesyncConflictChecksResult);
|
||||
if (result !== false) throw new Error("Unloading LiveSync did not report conflict-check cancellation");
|
||||
});
|
||||
}
|
||||
|
||||
@@ -362,6 +422,76 @@ async function enableLiveSync(port: number): Promise<void> {
|
||||
});
|
||||
}
|
||||
|
||||
type UnloadConflictFixture = { path: string; fixture: ConflictFixture };
|
||||
type PreservedConflictState = {
|
||||
path: string;
|
||||
currentRev: string;
|
||||
baseContent: string | false;
|
||||
vaultContent: string;
|
||||
leaves: { rev: string; parentRev: string; content: string | false; deleted: boolean }[];
|
||||
};
|
||||
|
||||
async function recordUnloadPreservation(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
label: string,
|
||||
fixtures: UnloadConflictFixture[],
|
||||
before?: PreservedConflictState[]
|
||||
): Promise<PreservedConflictState[]> {
|
||||
const states = await evalObsidianJson<PreservedConflictState[]>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const fixtures=${JSON.stringify(fixtures)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const contentOf=(entry)=>entry===false?false:Array.isArray(entry.data)?entry.data.join(''):entry.data;",
|
||||
"const states=[];",
|
||||
"for(const {path,fixture} of fixtures){",
|
||||
" const id=await core.services.path.path2id(path);",
|
||||
" const meta=await core.localDatabase.getRaw(id,{conflicts:true});",
|
||||
" const leaves=[];",
|
||||
" for(const rev of [meta._rev,...(meta._conflicts??[])].sort()){",
|
||||
" const raw=await core.localDatabase.getRaw(id,{rev,revs:true});",
|
||||
" const history=raw._revisions;",
|
||||
" const parentRev=history?.ids?.length>1?`${history.start-1}-${history.ids[1]}`:undefined;",
|
||||
" const entry=await core.localDatabase.getDBEntry(path,{rev},false,true,true);",
|
||||
" leaves.push({rev,parentRev,content:contentOf(entry),deleted:!!(raw.deleted||raw._deleted)});",
|
||||
" }",
|
||||
" const base=await core.localDatabase.getDBEntry(path,{rev:fixture.currentParentRev},false,true,true);",
|
||||
" const vaultContent=await app.vault.adapter.read(path);",
|
||||
" states.push({path,currentRev:meta._rev,baseContent:contentOf(base),vaultContent,leaves});",
|
||||
"}",
|
||||
"return JSON.stringify(states);",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
const directory = process.env.E2E_OBSIDIAN_DIAGNOSTICS_DIR ?? "/tmp/obsidian-livesync-e2e";
|
||||
await mkdir(directory, { recursive: true });
|
||||
await writeFile(join(directory, `conflict-${label}-preservation.json`), JSON.stringify(states, null, 2) + "\n");
|
||||
for (const { path: targetPath, fixture } of fixtures) {
|
||||
const state = states.find((entry) => entry.path === targetPath);
|
||||
const expectedRevs = [fixture.currentRev, ...fixture.conflicts].sort();
|
||||
if (
|
||||
!state ||
|
||||
state.currentRev !== fixture.currentRev ||
|
||||
state.baseContent !== baseContent ||
|
||||
JSON.stringify(state.leaves.map((leaf) => leaf.rev)) !== JSON.stringify(expectedRevs) ||
|
||||
JSON.stringify(state.leaves.map((leaf) => leaf.content).sort()) !==
|
||||
JSON.stringify([leftContent, rightContent].sort()) ||
|
||||
state.leaves.some((leaf) => leaf.deleted || leaf.parentRev !== fixture.currentParentRev)
|
||||
) {
|
||||
throw new Error(`Conflict contents or ancestry changed across ${label}: ${JSON.stringify(state)}`);
|
||||
}
|
||||
const previous = before?.find((entry) => entry.path === targetPath);
|
||||
if (previous && state.vaultContent !== previous.vaultContent) {
|
||||
throw new Error(`Vault content changed across ${label}: ${targetPath}`);
|
||||
}
|
||||
}
|
||||
console.log(`Verified conflict contents, leaf revisions, and ancestry: ${label}`);
|
||||
return states;
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
@@ -372,14 +502,17 @@ async function main(): Promise<void> {
|
||||
|
||||
const vault = await createTemporaryVault("obsidian-livesync-conflict-dialog-");
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
let unloadFixtures: UnloadConflictFixture[] = [];
|
||||
let unloadPreservationBefore: PreservedConflictState[] | undefined;
|
||||
try {
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary,
|
||||
vault,
|
||||
artifactRoot: process.env.E2E_OBSIDIAN_ARTIFACT_ROOT,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
pluginData: {
|
||||
doctorProcessedVersion: "1.0.0",
|
||||
doctorProcessedVersion: DoctorRegulation.version,
|
||||
isConfigured: true,
|
||||
liveSync: false,
|
||||
remoteType: "",
|
||||
@@ -433,34 +566,31 @@ async function main(): Promise<void> {
|
||||
(page) => conflictDialogue(page).locator(".modal").first()
|
||||
);
|
||||
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||
const modal = conflictDialogue(page);
|
||||
const actionButtons = modal.locator(".conflict-action-button");
|
||||
await actionButtons.nth(3).waitFor({ state: "visible", timeout: uiTimeoutMs });
|
||||
const actionButtonBounds = await actionButtons.evaluateAll((buttons) =>
|
||||
buttons.map((button) => {
|
||||
const bounds = button.getBoundingClientRect();
|
||||
return { top: bounds.top, bottom: bounds.bottom };
|
||||
})
|
||||
await page.waitForFunction(
|
||||
(targetPath) => {
|
||||
const dialogues = Array.from(document.querySelectorAll(".modal-container")).filter(
|
||||
(element) =>
|
||||
element.getClientRects().length > 0 &&
|
||||
element.querySelector(".modal-title")?.textContent?.includes("Conflicting changes") &&
|
||||
element.textContent?.includes(targetPath)
|
||||
);
|
||||
if (dialogues.length !== 1) return false;
|
||||
const bounds = Array.from(dialogues[0].querySelectorAll(".conflict-action-button")).map((button) =>
|
||||
button.getBoundingClientRect()
|
||||
);
|
||||
return (
|
||||
bounds.length === 4 &&
|
||||
bounds.every(
|
||||
(button, index) =>
|
||||
button.width > 0 &&
|
||||
button.height > 0 &&
|
||||
(index === 0 || button.top >= bounds[index - 1].bottom)
|
||||
)
|
||||
);
|
||||
},
|
||||
path,
|
||||
{ timeout: uiTimeoutMs }
|
||||
);
|
||||
if (
|
||||
actionButtonBounds.length !== 4 ||
|
||||
actionButtonBounds.some(
|
||||
(bounds, index) => index > 0 && bounds.top < actionButtonBounds[index - 1].bottom
|
||||
)
|
||||
) {
|
||||
const buttonDetails = await modal.locator("button").evaluateAll((buttons) =>
|
||||
buttons.map((button) => ({
|
||||
text: button.textContent,
|
||||
className: button.className,
|
||||
}))
|
||||
);
|
||||
throw new Error(
|
||||
`Conflict action buttons are not stacked vertically: ${JSON.stringify({
|
||||
actionButtonBounds,
|
||||
buttonDetails,
|
||||
})}`
|
||||
);
|
||||
}
|
||||
});
|
||||
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||
const modal = conflictDialogue(page);
|
||||
@@ -506,6 +636,7 @@ async function main(): Promise<void> {
|
||||
binary,
|
||||
cliBinary,
|
||||
vault,
|
||||
artifactRoot: process.env.E2E_OBSIDIAN_ARTIFACT_ROOT,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||
@@ -542,6 +673,7 @@ async function main(): Promise<void> {
|
||||
binary,
|
||||
cliBinary,
|
||||
vault,
|
||||
artifactRoot: process.env.E2E_OBSIDIAN_ARTIFACT_ROOT,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||
@@ -734,8 +866,12 @@ async function main(): Promise<void> {
|
||||
});
|
||||
await waitForConflictChecks(cliBinary, session.cliEnv);
|
||||
|
||||
await createTwoVersionConflict(cliBinary, session.cliEnv, unloadActivePath);
|
||||
const unloadActiveFixture = await createTwoVersionConflict(cliBinary, session.cliEnv, unloadActivePath);
|
||||
const unloadWaitingFixture = await createTwoVersionConflict(cliBinary, session.cliEnv, unloadWaitingPath);
|
||||
unloadFixtures = [
|
||||
{ path: unloadActivePath, fixture: unloadActiveFixture },
|
||||
{ path: unloadWaitingPath, fixture: unloadWaitingFixture },
|
||||
];
|
||||
await requestConflictCheck(cliBinary, session.cliEnv, false, unloadActivePath);
|
||||
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||
await conflictDialogue(page, unloadActivePath).waitFor({ state: "visible", timeout: uiTimeoutMs });
|
||||
@@ -752,7 +888,14 @@ async function main(): Promise<void> {
|
||||
"conflict-dialog-before-unload.png",
|
||||
(page) => conflictDialogue(page, unloadActivePath).locator(".modal").first()
|
||||
);
|
||||
unloadPreservationBefore = await recordUnloadPreservation(
|
||||
cliBinary,
|
||||
session.cliEnv,
|
||||
"before-unload",
|
||||
unloadFixtures
|
||||
);
|
||||
await disableLiveSyncAndWaitForConflictChecks(session.remoteDebuggingPort);
|
||||
await recordConflictDiagnostics(session.remoteDebuggingPort, "unloaded");
|
||||
const afterUnloadScreenshot = await captureObsidianPage(
|
||||
session.remoteDebuggingPort,
|
||||
"conflict-dialog-after-unload.png",
|
||||
@@ -770,6 +913,13 @@ async function main(): Promise<void> {
|
||||
|
||||
await enableLiveSync(session.remoteDebuggingPort);
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||
await recordUnloadPreservation(
|
||||
cliBinary,
|
||||
session.cliEnv,
|
||||
"after-reload",
|
||||
unloadFixtures,
|
||||
unloadPreservationBefore
|
||||
);
|
||||
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||
await page.waitForTimeout(500);
|
||||
if (await conflictDialogue(page).isVisible()) {
|
||||
@@ -794,7 +944,7 @@ async function main(): Promise<void> {
|
||||
await waitForConflictChecks(cliBinary, session.cliEnv);
|
||||
|
||||
console.log(
|
||||
"Real Obsidian preserved pairwise conflict resolution and dialogue presentation; replaced only stale same-file dialogues; serialised different files; discarded externally resolved waiting requests; and drained active and waiting requests across unload and reload."
|
||||
"Real Obsidian preserved pairwise conflict resolution and dialogue presentation; replaced only stale same-file dialogues; serialised different files; discarded externally resolved waiting requests; and reported cancellation while closing active and waiting requests across unload and reload."
|
||||
);
|
||||
console.log(`Dialogue screenshot: ${firstDialogueScreenshot}`);
|
||||
console.log(`Postponed warning screenshot: ${warningScreenshot}`);
|
||||
@@ -803,6 +953,27 @@ async function main(): Promise<void> {
|
||||
console.log(`Before unload screenshot: ${beforeUnloadScreenshot}`);
|
||||
console.log(`After unload screenshot: ${afterUnloadScreenshot}`);
|
||||
console.log(`After reload screenshot: ${afterReloadScreenshot}`);
|
||||
} catch (error) {
|
||||
if (session) {
|
||||
try {
|
||||
const state = await recordConflictDiagnostics(session.remoteDebuggingPort, "failure");
|
||||
await captureObsidianPage(session.remoteDebuggingPort, "conflict-failure.png", async () => {});
|
||||
if (!state.pluginLoaded && unloadPreservationBefore) {
|
||||
await enableLiveSync(session.remoteDebuggingPort);
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||
await recordUnloadPreservation(
|
||||
cliBinary,
|
||||
session.cliEnv,
|
||||
"after-failed-unload-reload",
|
||||
unloadFixtures,
|
||||
unloadPreservationBefore
|
||||
);
|
||||
}
|
||||
} catch (diagnosticError) {
|
||||
console.error("Could not capture conflict diagnostics:", diagnosticError);
|
||||
}
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
|
||||
@@ -0,0 +1,843 @@
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { createServer } from "node:net";
|
||||
import { readFile, readdir } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { deriveIdKey } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
configureObjectStorage,
|
||||
createE2eObjectStoragePluginData,
|
||||
createE2eObsidianDeviceLocalState,
|
||||
prepareRemote,
|
||||
pushLocalChanges,
|
||||
waitForLiveSyncCoreReady,
|
||||
waitForLocalDatabaseEntry,
|
||||
type LocalDatabaseEntry,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import {
|
||||
deleteObjectStoragePrefix,
|
||||
ensureObjectStorageBucket,
|
||||
listObjectStorageObjects,
|
||||
loadObjectStorageConfig,
|
||||
makeUniqueBucketPrefix,
|
||||
readObjectStorageJson,
|
||||
type ObjectStorageConfig,
|
||||
} from "../runner/objectStorage.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
||||
import { withObsidianPage } from "../runner/ui.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
|
||||
|
||||
const timeoutMs = Number(process.env.E2E_OBSIDIAN_JOURNAL_NFC_TIMEOUT_MS ?? 60000);
|
||||
const objectTimeoutMs = Number(process.env.E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS ?? 30000);
|
||||
const uiTimeoutMs = Number(process.env.E2E_OBSIDIAN_JOURNAL_NFC_UI_TIMEOUT_MS ?? 10000);
|
||||
const relativeFolder = "E2E/journal-nfc";
|
||||
const folderNfd = "Cafe\u0301";
|
||||
const conflictNameNfd = "conflict-e\u0301.md";
|
||||
const conflictPathNfd = `${relativeFolder}/${folderNfd}/${conflictNameNfd}`;
|
||||
const conflictPathNfc = conflictPathNfd.normalize("NFC");
|
||||
const controlNameNfd = "ordinary-e\u0301.md";
|
||||
const controlPathNfd = `${relativeFolder}/${folderNfd}/${controlNameNfd}`;
|
||||
const controlContent = "# NFC control\n\nThis ordinary, non-conflicted note keeps one NFC-equivalent path.\n";
|
||||
const baseContent = [
|
||||
"# Journal resend conflict fixture",
|
||||
"",
|
||||
"The common ancestor contains this exact sentence and enough bytes for real Chunks.",
|
||||
"The source file begins from this baseline before either device makes a change.",
|
||||
"Shared value: the original value remains here.",
|
||||
"",
|
||||
].join("\n");
|
||||
const sourceLeafContent = [
|
||||
"# Journal resend conflict fixture",
|
||||
"",
|
||||
"The common ancestor contains this exact sentence and enough bytes for real Chunks.",
|
||||
"The source file begins from this baseline before either device makes a change.",
|
||||
"Shared value: the source device replaced this line with its own edit.",
|
||||
"",
|
||||
].join("\n");
|
||||
const competingLeafContent = [
|
||||
"# Journal resend conflict fixture",
|
||||
"",
|
||||
"The common ancestor contains this exact sentence and enough bytes for real Chunks.",
|
||||
"The source file begins from this baseline before either device makes a change.",
|
||||
"Shared value: another device replaced this line differently.",
|
||||
"",
|
||||
].join("\n");
|
||||
|
||||
type ScenarioCase = { name: "plain" | "encrypted"; independentIds: boolean };
|
||||
type RevisionSnapshot = {
|
||||
rev: string;
|
||||
revisionTree: { start: number; ids: string[] };
|
||||
revisions: string[];
|
||||
body: string;
|
||||
children: string[];
|
||||
chunksPresent: boolean;
|
||||
};
|
||||
type ConflictTreeSnapshot = {
|
||||
id: string;
|
||||
path: string;
|
||||
leafRevs: string[];
|
||||
leaves: RevisionSnapshot[];
|
||||
ancestor?: RevisionSnapshot;
|
||||
ancestorRev?: string;
|
||||
ancestorBodyAvailable: boolean;
|
||||
};
|
||||
type TransferCheckpoint = {
|
||||
lastLocalSeq: number | string | undefined;
|
||||
journalEpoch: string;
|
||||
knownIDs: string[];
|
||||
sentIDs: string[];
|
||||
receivedFiles: string[];
|
||||
sentFiles: string[];
|
||||
};
|
||||
type ScenarioSettings = Record<string, unknown>;
|
||||
|
||||
function selectedCases(args: readonly string[]): ScenarioCase[] {
|
||||
let requested = process.env.E2E_OBSIDIAN_INDEPENDENT_IDS === "true" ? "encrypted" : "plain";
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const argument = args[index];
|
||||
if (argument === "--help" || argument === "-h") {
|
||||
console.log("Usage: journal-nfc-resend.ts [--case all|plain|encrypted|independent-ids]");
|
||||
process.exit(0);
|
||||
}
|
||||
if (argument === "--case") {
|
||||
const value = args[index + 1];
|
||||
if (!value) throw new Error("--case requires all, plain, encrypted, or independent-ids.");
|
||||
requested = value;
|
||||
index += 1;
|
||||
continue;
|
||||
}
|
||||
if (argument.startsWith("--case=")) {
|
||||
requested = argument.slice("--case=".length);
|
||||
continue;
|
||||
}
|
||||
throw new Error(`Unknown Journal NFC resend argument: ${argument}`);
|
||||
}
|
||||
const cases: ScenarioCase[] = [
|
||||
{ name: "plain", independentIds: false },
|
||||
{ name: "encrypted", independentIds: true },
|
||||
];
|
||||
if (requested === "all") return cases;
|
||||
if (requested === "independent-ids") requested = "encrypted";
|
||||
const selected = cases.filter((candidate) => candidate.name === requested);
|
||||
if (selected.length === 0) throw new Error(`Unsupported Journal NFC resend case: ${requested}`);
|
||||
return selected;
|
||||
}
|
||||
|
||||
async function allocatePort(excluded: ReadonlySet<number>): Promise<number> {
|
||||
for (let attempt = 0; attempt < 10; attempt += 1) {
|
||||
const server = createServer();
|
||||
const port = await new Promise<number>((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
const address = server.address();
|
||||
if (!address || typeof address === "string") {
|
||||
reject(new Error("Could not allocate an Obsidian debugging port."));
|
||||
return;
|
||||
}
|
||||
resolve(address.port);
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve, reject) => server.close((error) => (error ? reject(error) : resolve())));
|
||||
if (!excluded.has(port)) return port;
|
||||
}
|
||||
throw new Error("Could not allocate distinct Obsidian debugging ports.");
|
||||
}
|
||||
|
||||
async function createVaultNote(
|
||||
cliBinary: string,
|
||||
environment: NodeJS.ProcessEnv,
|
||||
relativePath: string,
|
||||
content: string
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<boolean>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const path=" + JSON.stringify(relativePath.normalize("NFC")) + ";",
|
||||
"const content=" + JSON.stringify(content) + ";",
|
||||
"const parent=path.slice(0,path.lastIndexOf('/'));",
|
||||
"if(!(await app.vault.adapter.exists(parent)))await app.vault.createFolder(parent);",
|
||||
"await app.vault.create(path,content);",
|
||||
"await app.plugins.plugins['obsidian-livesync'].core.services.fileProcessing.commitPendingFileEvents();",
|
||||
"return JSON.stringify(true);",
|
||||
"})()",
|
||||
].join(""),
|
||||
environment,
|
||||
timeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
async function assertSinglePhysicalUnicodeFile(
|
||||
vaultPath: string,
|
||||
relativePath: string,
|
||||
expectedContent: string
|
||||
): Promise<void> {
|
||||
const segments = relativePath.split("/");
|
||||
const stableParent = join(vaultPath, ...segments.slice(0, -2));
|
||||
const expectedFolder = segments[segments.length - 2].normalize("NFC");
|
||||
const expectedFilename = segments[segments.length - 1].normalize("NFC");
|
||||
const physicalFolders = (await readdir(stableParent, { withFileTypes: true })).filter(
|
||||
(entry) => entry.isDirectory() && entry.name.normalize("NFC") === expectedFolder
|
||||
);
|
||||
assertEqual(physicalFolders.length, 1, "NFC-equivalent folder spellings created duplicate physical folders.");
|
||||
const actualFolder = join(stableParent, physicalFolders[0].name);
|
||||
const matchingFiles = (await readdir(actualFolder)).filter((name) => name.normalize("NFC") === expectedFilename);
|
||||
assertEqual(matchingFiles.length, 1, "NFC-equivalent file spellings created duplicate physical files.");
|
||||
assertEqual(
|
||||
await readFile(join(actualFolder, matchingFiles[0]), "utf8"),
|
||||
expectedContent,
|
||||
"The ordinary NFC control file has unexpected bytes."
|
||||
);
|
||||
}
|
||||
|
||||
async function waitForSinglePhysicalUnicodeFile(
|
||||
vaultPath: string,
|
||||
relativePath: string,
|
||||
expectedContent: string
|
||||
): Promise<void> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastError: unknown;
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
await assertSinglePhysicalUnicodeFile(vaultPath, relativePath, expectedContent);
|
||||
return;
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
}
|
||||
throw new Error(
|
||||
`Timed out waiting for one NFC-equivalent physical file: ${lastError instanceof Error ? lastError.message : String(lastError)}`
|
||||
);
|
||||
}
|
||||
|
||||
async function makeScenarioSettings(scenario: ScenarioCase): Promise<ScenarioSettings> {
|
||||
return {
|
||||
pathNormalizationVersion: 1,
|
||||
...(scenario.independentIds
|
||||
? {
|
||||
encrypt: true,
|
||||
usePathObfuscation: true,
|
||||
passphrase: randomBytes(32).toString("base64url"),
|
||||
idDerivationVersion: 1,
|
||||
idDerivationKey: await deriveIdKey(randomBytes(32).toString("base64url")),
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
async function startSession(
|
||||
binary: string,
|
||||
cliBinary: string,
|
||||
vault: TemporaryVault,
|
||||
port: number,
|
||||
storage: ObjectStorageConfig & { bucketPrefix: string },
|
||||
overrides: ScenarioSettings,
|
||||
configure: boolean
|
||||
): Promise<ObsidianLiveSyncSession> {
|
||||
const session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
env: { E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT: String(port) },
|
||||
...(configure
|
||||
? {
|
||||
pluginData: createE2eObjectStoragePluginData(storage, { pathNormalizationVersion: 1 }),
|
||||
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
|
||||
}
|
||||
: {}),
|
||||
});
|
||||
try {
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv, timeoutMs);
|
||||
if (configure) {
|
||||
const configured = await configureObjectStorage(cliBinary, session.cliEnv, storage, overrides);
|
||||
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
|
||||
assertEqual(configured.remoteType, "MINIO", "The configured remote was not Object Storage.");
|
||||
assertEqual(configured.bucket, storage.bucket, "The Object Storage bucket changed during configuration.");
|
||||
assertEqual(
|
||||
configured.bucketPrefix,
|
||||
storage.bucketPrefix,
|
||||
"The Object Storage prefix changed during configuration."
|
||||
);
|
||||
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during the one-shot workflow.");
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv, timeoutMs);
|
||||
}
|
||||
return session;
|
||||
} catch (error) {
|
||||
await session.app.stop().catch((stopError: unknown) => {
|
||||
console.warn(stopError instanceof Error ? stopError.message : stopError);
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function readTreeSource(path: string, expectedBaseRev: string, requireAncestorBody: boolean): string {
|
||||
return [
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
`const expectedBaseRev=${JSON.stringify(expectedBaseRev)};`,
|
||||
`const requireAncestorBody=${JSON.stringify(requireAncestorBody)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const db=core.localDatabase;",
|
||||
"const readRevision=async(rev)=>{",
|
||||
"const metadata=await db.getDBEntryMeta(path,{rev,revs:true},true);",
|
||||
"const entry=await db.getDBEntry(path,{rev},false,true,true);",
|
||||
"if(!metadata||!entry) throw new Error('Could not read exact Metadata revision '+rev);",
|
||||
"const tree=metadata._revisions;",
|
||||
"if(!tree||!Number.isInteger(tree.start)||!Array.isArray(tree.ids)||tree.ids.length===0) throw new Error('Revision ancestry is unavailable for '+rev);",
|
||||
"const revisionTree={start:tree.start,ids:[...tree.ids]};",
|
||||
"const revisions=revisionTree.ids.map((id,index)=>`${revisionTree.start-index}-${id}`);",
|
||||
"const data=entry.data;",
|
||||
"const body=Array.isArray(data)?data.join(''):typeof data==='string'?data:data instanceof Blob?await data.text():null;",
|
||||
"if(typeof body!=='string') throw new Error('Could not decode exact revision body '+rev);",
|
||||
"const children=Array.isArray(metadata.children)?[...metadata.children]:Array.isArray(entry.children)?[...entry.children]:[];",
|
||||
"const rows=children.length?await db.allDocsRaw({keys:children,include_docs:true}):{rows:[]};",
|
||||
"const available=rows.rows.filter(row=>Boolean(row.doc)&&!Boolean(row.value?.deleted)&&!Boolean(row.doc?.deleted)&&!Boolean(row.doc?._deleted)).length;",
|
||||
"const chunksPresent=children.length>0&&available===children.length;",
|
||||
"if(!chunksPresent) throw new Error('Exact Metadata revision has missing referenced Chunks '+rev);",
|
||||
"return {rev:entry._rev,revisionTree,revisions,body,children,chunksPresent};",
|
||||
"};",
|
||||
"const current=await db.getDBEntryMeta(path,{conflicts:true},true);",
|
||||
"if(!current?._rev) throw new Error('Conflict Metadata is missing: '+path);",
|
||||
"const leafRevs=[current._rev,...(current._conflicts??[])].sort();",
|
||||
"const leaves=[];",
|
||||
"for(const rev of leafRevs) leaves.push(await readRevision(rev));",
|
||||
"let ancestor;let ancestorRev;let ancestorBodyAvailable=false;",
|
||||
"if(leaves.length>=2){",
|
||||
"const shared=leaves[0].revisions.filter(rev=>leaves.slice(1).every(leaf=>leaf.revisions.includes(rev)));",
|
||||
"if(shared.length===0) throw new Error('Conflict leaves have no exact shared revision ID.');",
|
||||
"ancestorRev=shared[0];",
|
||||
"if(ancestorRev!==expectedBaseRev) throw new Error('Nearest common revision did not match the seeded baseline.');",
|
||||
"const metadata=await db.getDBEntryMeta(path,{rev:ancestorRev,revs:true},true).catch(error=>{if(error?.status===404||error?.name==='not_found')return false;throw error;});",
|
||||
"if(metadata){ancestor=await readRevision(ancestorRev);ancestorBodyAvailable=true;}",
|
||||
"if(requireAncestorBody&&!ancestorBodyAvailable)throw new Error('The source lost its available common ancestor body.');",
|
||||
"}",
|
||||
"return JSON.stringify({id:current._id,path:current.path,leafRevs,leaves,ancestor,ancestorRev,ancestorBodyAvailable});",
|
||||
"})()",
|
||||
].join("");
|
||||
}
|
||||
|
||||
async function readConflictTree(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
path: string,
|
||||
expectedBaseRev: string,
|
||||
requireAncestorBody = true
|
||||
): Promise<ConflictTreeSnapshot> {
|
||||
return await evalObsidianJson<ConflictTreeSnapshot>(
|
||||
cliBinary,
|
||||
readTreeSource(path, expectedBaseRev, requireAncestorBody),
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
async function waitForConflictTree(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
path: string,
|
||||
expectedBaseRev: string,
|
||||
expectedBodies: readonly string[],
|
||||
requireAncestorBody = true
|
||||
): Promise<ConflictTreeSnapshot> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let last: ConflictTreeSnapshot | undefined;
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
last = await readConflictTree(cliBinary, env, path, expectedBaseRev, requireAncestorBody);
|
||||
const bodies = last.leaves.map((leaf) => leaf.body).sort();
|
||||
if (last.leafRevs.length === 2 && JSON.stringify(bodies) === JSON.stringify([...expectedBodies].sort())) {
|
||||
return last;
|
||||
}
|
||||
} catch (error) {
|
||||
if (Date.now() + 250 >= deadline) throw error;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for both exact conflict leaves: ${JSON.stringify(last)}`);
|
||||
}
|
||||
|
||||
function assertSameConflictTree(actual: ConflictTreeSnapshot, expected: ConflictTreeSnapshot, label: string): void {
|
||||
assertEqual(actual.id, expected.id, `${label}: Metadata ID changed.`);
|
||||
assertEqual(
|
||||
JSON.stringify(actual.leafRevs),
|
||||
JSON.stringify(expected.leafRevs),
|
||||
`${label}: leaf revision IDs changed.`
|
||||
);
|
||||
assertEqual(
|
||||
JSON.stringify(
|
||||
actual.leaves.map(({ rev, revisionTree, body, children, chunksPresent }) => ({
|
||||
rev,
|
||||
revisionTree,
|
||||
body,
|
||||
children,
|
||||
chunksPresent,
|
||||
}))
|
||||
),
|
||||
JSON.stringify(
|
||||
expected.leaves.map(({ rev, revisionTree, body, children, chunksPresent }) => ({
|
||||
rev,
|
||||
revisionTree,
|
||||
body,
|
||||
children,
|
||||
chunksPresent,
|
||||
}))
|
||||
),
|
||||
`${label}: a leaf body, exact _revisions, or referenced Chunk changed.`
|
||||
);
|
||||
assertEqual(actual.ancestorRev, expected.ancestorRev, `${label}: the exact common ancestor changed.`);
|
||||
if (!actual.ancestorBodyAvailable) return;
|
||||
assertEqual(
|
||||
JSON.stringify(
|
||||
actual.ancestor && {
|
||||
revisionTree: actual.ancestor.revisionTree,
|
||||
revisions: actual.ancestor.revisions,
|
||||
body: actual.ancestor.body,
|
||||
children: actual.ancestor.children,
|
||||
chunksPresent: actual.ancestor.chunksPresent,
|
||||
}
|
||||
),
|
||||
JSON.stringify(
|
||||
expected.ancestor && {
|
||||
revisionTree: expected.ancestor.revisionTree,
|
||||
revisions: expected.ancestor.revisions,
|
||||
body: expected.ancestor.body,
|
||||
children: expected.ancestor.children,
|
||||
chunksPresent: expected.ancestor.chunksPresent,
|
||||
}
|
||||
),
|
||||
`${label}: the common ancestor body or its Chunks changed.`
|
||||
);
|
||||
}
|
||||
|
||||
async function storeSourceLeaf(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
path: string,
|
||||
content: string,
|
||||
baseRev: string
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
`const content=${JSON.stringify(content)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const file=app.vault.getFiles().find(candidate=>candidate.path.normalize('NFC')===path.normalize('NFC'));",
|
||||
"if(!file) throw new Error('Could not find the NFC-equivalent source file '+path);",
|
||||
"await app.vault.modify(file,content);",
|
||||
"await core.services.fileProcessing.commitPendingFileEvents();",
|
||||
"const deadline=Date.now()+20000;",
|
||||
"const sleep=(ms)=>new Promise(resolve=>setTimeout(resolve,ms));",
|
||||
"let entry=false;",
|
||||
"let stored=false;",
|
||||
"while(Date.now()<deadline){",
|
||||
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true,true).catch(()=>false);",
|
||||
"const data=entry?.data;",
|
||||
"const body=Array.isArray(data)?data.join(''):typeof data==='string'?data:data instanceof Blob?await data.text():null;",
|
||||
`if(entry&&entry._rev!==${JSON.stringify(baseRev)}&&body===content){stored=true;break;}`,
|
||||
"await sleep(250);",
|
||||
"}",
|
||||
"if(!stored) throw new Error('The real Obsidian source edit did not create a child revision with the expected body.');",
|
||||
"const children=Array.isArray(entry.children)?entry.children:[];",
|
||||
"if(children.length===0) throw new Error('The real Obsidian source revision has no Chunks.');",
|
||||
"return JSON.stringify(true);",
|
||||
"})()",
|
||||
].join(""),
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
async function storeCompetingLeaf(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
path: string,
|
||||
content: string,
|
||||
baseRev: string
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
`const content=${JSON.stringify(content)};`,
|
||||
`const baseRev=${JSON.stringify(baseRev)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const id=await core.services.path.path2id(path);",
|
||||
"const blob=new Blob([content],{type:'text/plain'});",
|
||||
"const now=Date.now();",
|
||||
"const result=await core.localDatabase.putDBEntry({",
|
||||
"_id:id,path,data:blob,ctime:now,mtime:now,size:blob.size,children:[],datatype:'plain',type:'plain',eden:{},",
|
||||
"},false,baseRev);",
|
||||
"if(!result?.ok) throw new Error('Could not create the sibling conflict leaf from its exact baseline.');",
|
||||
"const entry=await core.localDatabase.getDBEntry(path,{rev:result.rev},false,true,true);",
|
||||
"if(!entry||!Array.isArray(entry.children)||entry.children.length===0) throw new Error('The sibling conflict leaf has no real Chunks.');",
|
||||
"return JSON.stringify(true);",
|
||||
"})()",
|
||||
].join(""),
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
async function readTransferCheckpoint(cliBinary: string, env: NodeJS.ProcessEnv): Promise<TransferCheckpoint> {
|
||||
return await evalObsidianJson<TransferCheckpoint>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const replicator=core.services.replicator.getActiveReplicator();",
|
||||
"const client=replicator?.client;",
|
||||
"if(!client||typeof client.getCheckpointInfo!=='function') throw new Error('The active Object Storage replicator did not expose its Journal checkpoint.');",
|
||||
"const info=await client.getCheckpointInfo();",
|
||||
"const values=value=>[...(value??[])].sort();",
|
||||
"return JSON.stringify({lastLocalSeq:info.lastLocalSeq,journalEpoch:info.journalEpoch,knownIDs:values(info.knownIDs),sentIDs:values(info.sentIDs),receivedFiles:values(info.receivedFiles),sentFiles:values(info.sentFiles)});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
function assertCheckpointPopulated(checkpoint: TransferCheckpoint, tree: ConflictTreeSnapshot): void {
|
||||
if (checkpoint.sentFiles.length === 0 || checkpoint.sentIDs.length === 0) {
|
||||
throw new Error("The initial Journal transfer did not populate its persistent send checkpoint.");
|
||||
}
|
||||
const localSequence = checkpoint.lastLocalSeq;
|
||||
const sequenceIsPopulated =
|
||||
typeof localSequence === "number"
|
||||
? Number.isFinite(localSequence) && localSequence > 0
|
||||
: typeof localSequence === "string" && localSequence.trim() !== "" && localSequence.trim() !== "0";
|
||||
assertEqual(
|
||||
sequenceIsPopulated,
|
||||
true,
|
||||
"The initial Journal transfer did not advance its local sequence checkpoint."
|
||||
);
|
||||
const sentIds = new Set(checkpoint.sentIDs);
|
||||
const expectedSentIds = tree.leaves.map((leaf) => `${tree.id}-${leaf.rev}`);
|
||||
for (const expectedId of expectedSentIds) {
|
||||
if (!sentIds.has(expectedId))
|
||||
throw new Error(`The Journal checkpoint does not record sent revision ${expectedId}.`);
|
||||
}
|
||||
}
|
||||
|
||||
function assertIndependentIdShape(tree: ConflictTreeSnapshot): void {
|
||||
if (!/^f:[0-9a-f]{64}$/u.test(tree.id)) {
|
||||
throw new Error(`The encrypted case did not use an independent Metadata ID: ${tree.id}`);
|
||||
}
|
||||
for (const revision of [...tree.leaves, ...(tree.ancestor ? [tree.ancestor] : [])]) {
|
||||
if (revision.children.some((id) => !/^h:\+[0-9a-f]{64}$/u.test(id))) {
|
||||
throw new Error("The encrypted case did not use independent Chunk IDs.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForJournalObjects(
|
||||
storage: ObjectStorageConfig,
|
||||
prefix: string,
|
||||
operationLabel: string
|
||||
): Promise<string[]> {
|
||||
const milestoneKey = `${prefix}_00000000-milestone.json`;
|
||||
const deadline = Date.now() + objectTimeoutMs;
|
||||
let keys: string[] = [];
|
||||
while (Date.now() < deadline) {
|
||||
keys = (await listObjectStorageObjects(storage, prefix)).flatMap((object) => (object.Key ? [object.Key] : []));
|
||||
if (keys.includes(milestoneKey) && keys.some((key) => key.endsWith("-docs.jsonl.gz"))) return keys;
|
||||
await new Promise((resolve) => setTimeout(resolve, 300));
|
||||
}
|
||||
throw new Error(
|
||||
`${operationLabel} did not create both the Journal milestone and document objects: ${keys.join(", ")}`
|
||||
);
|
||||
}
|
||||
|
||||
async function postponeConflictIfOpen(session: ObsidianLiveSyncSession): Promise<void> {
|
||||
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||
const dialogue = page.locator(".modal-container:visible").filter({ hasText: "Conflicting changes" });
|
||||
if ((await dialogue.count()) === 0) return;
|
||||
const notNow = dialogue.getByRole("button", { name: "Not now", exact: true });
|
||||
if ((await notNow.count()) > 0) await notNow.click({ timeout: uiTimeoutMs });
|
||||
});
|
||||
}
|
||||
|
||||
async function rebuildRemote(cliBinary: string, env: NodeJS.ProcessEnv, port: number): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(()=>{",
|
||||
"const rebuilder=app.plugins.plugins['obsidian-livesync'].core.rebuilder;",
|
||||
"const state={done:false,error:null};",
|
||||
"globalThis.__journalNfcRemoteRebuild=state;",
|
||||
"void Promise.resolve().then(()=>rebuilder.$rebuildRemote()).then(()=>{state.done=true;},error=>{state.error=String(error?.stack??error);});",
|
||||
"return JSON.stringify(true);",
|
||||
"})()",
|
||||
].join(""),
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
await withObsidianPage(port, async (page) => {
|
||||
const title = "All optional features are disabled";
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let acknowledged = false;
|
||||
while (Date.now() < deadline) {
|
||||
const state = await evalObsidianJson<{ done: boolean; error: string | null }>(
|
||||
cliBinary,
|
||||
"JSON.stringify(globalThis.__journalNfcRemoteRebuild)",
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
if (state.error) throw new Error(state.error);
|
||||
if (state.done) {
|
||||
if (!acknowledged) {
|
||||
throw new Error(`Remote-only Rebuild finished without showing the '${title}' dialogue.`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (!acknowledged) {
|
||||
for (const candidate of page.context().pages()) {
|
||||
const dialogue = candidate
|
||||
.locator(".modal-container")
|
||||
.filter({ has: candidate.locator(".modal-title").filter({ hasText: title }) })
|
||||
.last();
|
||||
if (!(await dialogue.isVisible().catch(() => false))) continue;
|
||||
await dialogue.getByRole("button", { name: "OK", exact: true }).click({ timeout: uiTimeoutMs });
|
||||
await dialogue.waitFor({ state: "hidden", timeout: uiTimeoutMs });
|
||||
acknowledged = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
const state = await evalObsidianJson<{ done: boolean; error: string | null }>(
|
||||
cliBinary,
|
||||
"JSON.stringify(globalThis.__journalNfcRemoteRebuild)",
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
if (state.error) throw new Error(state.error);
|
||||
throw new Error("Timed out waiting for remote-only Rebuild and its optional-features dialogue.");
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchIntoFreshVault(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const existing=await core.localDatabase.getDBEntry(path,undefined,false,true,true).catch(()=>false);",
|
||||
"if(existing) throw new Error('The fresh receiving database already contains the conflict Metadata.');",
|
||||
"await core.rebuilder.$fetchLocal(false,true);",
|
||||
"return JSON.stringify(true);",
|
||||
"})()",
|
||||
].join(""),
|
||||
env,
|
||||
timeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
async function runCase(
|
||||
binary: string,
|
||||
cliBinary: string,
|
||||
objectStorage: ObjectStorageConfig,
|
||||
scenario: ScenarioCase
|
||||
): Promise<void> {
|
||||
const bucketPrefix = makeUniqueBucketPrefix(`journal-nfc-resend-${scenario.name}`);
|
||||
const storage = { ...objectStorage, bucketPrefix };
|
||||
const overrides = await makeScenarioSettings(scenario);
|
||||
const sourceVault = await createTemporaryVault(`obsidian-livesync-journal-nfc-source-${scenario.name}-`);
|
||||
let receiverVault: TemporaryVault | undefined;
|
||||
let sourceSession: ObsidianLiveSyncSession | undefined;
|
||||
let receiverSession: ObsidianLiveSyncSession | undefined;
|
||||
try {
|
||||
receiverVault = await createTemporaryVault(`obsidian-livesync-journal-nfc-receiver-${scenario.name}-`);
|
||||
const sourcePort = await allocatePort(new Set());
|
||||
const receiverPort = await allocatePort(new Set([sourcePort]));
|
||||
assertEqual(sourcePort === receiverPort, false, "Source and receiver Obsidian debugging ports must differ.");
|
||||
|
||||
console.log(`Journal NFC resend case: ${scenario.name}`);
|
||||
console.log(`Temporary source and receiver Vaults: ${sourceVault.name}, ${receiverVault.name}`);
|
||||
console.log(`Temporary Object Storage prefix: ${bucketPrefix}`);
|
||||
sourceSession = await startSession(binary, cliBinary, sourceVault, sourcePort, storage, overrides, true);
|
||||
await prepareRemote(cliBinary, sourceSession.cliEnv);
|
||||
await createVaultNote(cliBinary, sourceSession.cliEnv, conflictPathNfd, baseContent);
|
||||
await createVaultNote(cliBinary, sourceSession.cliEnv, controlPathNfd, controlContent);
|
||||
await assertSinglePhysicalUnicodeFile(sourceVault.path, controlPathNfd, controlContent);
|
||||
|
||||
const initialEntry: LocalDatabaseEntry = await waitForLocalDatabaseEntry(
|
||||
cliBinary,
|
||||
sourceSession.cliEnv,
|
||||
conflictPathNfc,
|
||||
{
|
||||
timeoutMs,
|
||||
}
|
||||
);
|
||||
if (!initialEntry.rev || initialEntry.children.length === 0) {
|
||||
throw new Error("The source baseline was not saved as a Chunk-backed Metadata revision.");
|
||||
}
|
||||
await storeSourceLeaf(cliBinary, sourceSession.cliEnv, conflictPathNfc, sourceLeafContent, initialEntry.rev);
|
||||
await storeCompetingLeaf(
|
||||
cliBinary,
|
||||
sourceSession.cliEnv,
|
||||
conflictPathNfc,
|
||||
competingLeafContent,
|
||||
initialEntry.rev
|
||||
);
|
||||
const sourceTree = await waitForConflictTree(
|
||||
cliBinary,
|
||||
sourceSession.cliEnv,
|
||||
conflictPathNfc,
|
||||
initialEntry.rev,
|
||||
[sourceLeafContent, competingLeafContent]
|
||||
);
|
||||
if (!sourceTree.ancestor || sourceTree.ancestor.rev !== initialEntry.rev) {
|
||||
throw new Error(
|
||||
"The two source leaves did not retain the exact available baseline as their nearest common ancestor."
|
||||
);
|
||||
}
|
||||
assertEqual(
|
||||
sourceTree.ancestor.body,
|
||||
baseContent,
|
||||
"The source conflict ancestor did not preserve the exact seeded baseline body."
|
||||
);
|
||||
await postponeConflictIfOpen(sourceSession);
|
||||
if (scenario.independentIds) {
|
||||
assertIndependentIdShape(sourceTree);
|
||||
} else {
|
||||
assertEqual(
|
||||
sourceTree.id,
|
||||
conflictPathNfc.toLowerCase(),
|
||||
"The plain case did not derive its Metadata ID from the NFC path."
|
||||
);
|
||||
}
|
||||
|
||||
await pushLocalChanges(cliBinary, sourceSession.cliEnv);
|
||||
const initialKeys = await waitForJournalObjects(objectStorage, bucketPrefix, "Initial Journal upload");
|
||||
const initialCheckpoint = await readTransferCheckpoint(cliBinary, sourceSession.cliEnv);
|
||||
assertCheckpointPopulated(initialCheckpoint, sourceTree);
|
||||
await readObjectStorageJson<Record<string, unknown>>(objectStorage, `${bucketPrefix}_00000000-milestone.json`);
|
||||
await postponeConflictIfOpen(sourceSession);
|
||||
|
||||
// Removing this fixture prefix leaves the source PouchDB tree and its device-local transfer checkpoint intact.
|
||||
await deleteObjectStoragePrefix(objectStorage, bucketPrefix);
|
||||
assertEqual(
|
||||
(await listObjectStorageObjects(objectStorage, bucketPrefix)).length,
|
||||
0,
|
||||
"The test's isolated Object Storage prefix was not empty after deletion."
|
||||
);
|
||||
assertEqual(
|
||||
JSON.stringify(await readTransferCheckpoint(cliBinary, sourceSession.cliEnv)),
|
||||
JSON.stringify(initialCheckpoint),
|
||||
"Deleting the remote fixture prefix changed the source transfer checkpoint."
|
||||
);
|
||||
assertSameConflictTree(
|
||||
await readConflictTree(cliBinary, sourceSession.cliEnv, conflictPathNfc, initialEntry.rev),
|
||||
sourceTree,
|
||||
"Source before remote-only Rebuild"
|
||||
);
|
||||
|
||||
await rebuildRemote(cliBinary, sourceSession.cliEnv, sourceSession.remoteDebuggingPort);
|
||||
const rebuiltKeys = await waitForJournalObjects(objectStorage, bucketPrefix, "Remote-only Rebuild");
|
||||
await postponeConflictIfOpen(sourceSession);
|
||||
assertSameConflictTree(
|
||||
await readConflictTree(cliBinary, sourceSession.cliEnv, conflictPathNfc, initialEntry.rev),
|
||||
sourceTree,
|
||||
"Source after remote-only Rebuild"
|
||||
);
|
||||
if (scenario.independentIds) assertIndependentIdShape(sourceTree);
|
||||
await sourceSession.app.stop();
|
||||
sourceSession = undefined;
|
||||
|
||||
receiverSession = await startSession(binary, cliBinary, receiverVault, receiverPort, storage, overrides, true);
|
||||
await fetchIntoFreshVault(cliBinary, receiverSession.cliEnv, conflictPathNfc);
|
||||
await waitForLiveSyncCoreReady(cliBinary, receiverSession.cliEnv, timeoutMs);
|
||||
await postponeConflictIfOpen(receiverSession);
|
||||
const receivedTree = await waitForConflictTree(
|
||||
cliBinary,
|
||||
receiverSession.cliEnv,
|
||||
conflictPathNfc,
|
||||
initialEntry.rev,
|
||||
[sourceLeafContent, competingLeafContent],
|
||||
false
|
||||
);
|
||||
await postponeConflictIfOpen(receiverSession);
|
||||
assertSameConflictTree(receivedTree, sourceTree, "Fresh receiver after Standard Fetch");
|
||||
await waitForSinglePhysicalUnicodeFile(receiverVault.path, controlPathNfd, controlContent);
|
||||
|
||||
await receiverSession.app.stop();
|
||||
receiverSession = undefined;
|
||||
const restartPort = await allocatePort(new Set([sourcePort, receiverPort]));
|
||||
receiverSession = await startSession(binary, cliBinary, receiverVault, restartPort, storage, overrides, false);
|
||||
await waitForLiveSyncCoreReady(cliBinary, receiverSession.cliEnv, timeoutMs);
|
||||
await postponeConflictIfOpen(receiverSession);
|
||||
const restartedTree = await waitForConflictTree(
|
||||
cliBinary,
|
||||
receiverSession.cliEnv,
|
||||
conflictPathNfc,
|
||||
initialEntry.rev,
|
||||
[sourceLeafContent, competingLeafContent],
|
||||
false
|
||||
);
|
||||
await postponeConflictIfOpen(receiverSession);
|
||||
assertSameConflictTree(restartedTree, sourceTree, "Receiver after full Obsidian restart");
|
||||
assertEqual(
|
||||
restartedTree.ancestorBodyAvailable,
|
||||
receivedTree.ancestorBodyAvailable,
|
||||
"Restart changed the common ancestor body's availability."
|
||||
);
|
||||
await waitForSinglePhysicalUnicodeFile(receiverVault.path, controlPathNfd, controlContent);
|
||||
|
||||
const initialJournalObjects = initialKeys.filter((key) => key.endsWith("-docs.jsonl.gz")).length;
|
||||
const rebuiltJournalObjects = rebuiltKeys.filter((key) => key.endsWith("-docs.jsonl.gz")).length;
|
||||
console.log(
|
||||
`Journal NFC resend ${scenario.name}: source checkpoint survived external prefix deletion, ${rebuiltJournalObjects} rebuilt Journal object(s), exact conflict leaves and ancestry fetched, and restart verified (initial objects: ${initialJournalObjects}; receiver ancestor body available: ${receivedTree.ancestorBodyAvailable}).`
|
||||
);
|
||||
console.log(
|
||||
"The history assertion covers this deliberately uncompacted fixture and does not claim arbitrary historical body availability."
|
||||
);
|
||||
} finally {
|
||||
if (receiverSession) {
|
||||
await receiverSession.app.stop().catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
if (sourceSession) {
|
||||
await sourceSession.app.stop().catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
await sourceVault.dispose().catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
if (receiverVault) {
|
||||
await receiverVault.dispose().catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
await deleteObjectStoragePrefix(objectStorage, bucketPrefix).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const scenarios = selectedCases(process.argv.slice(2));
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
const objectStorage = await loadObjectStorageConfig();
|
||||
await ensureObjectStorageBucket(objectStorage);
|
||||
for (const scenario of scenarios) await runCase(binary, cli.binary, objectStorage, scenario);
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -2,7 +2,7 @@ import { createHash, randomUUID } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { LOG_LEVEL_URGENT } from "octagonal-wheels/common/logger";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { withObsidianPage } from "../runner/ui.ts";
|
||||
import { requireObsidianBinary, requireObsidianCli } from "../runner/environment.ts";
|
||||
import {
|
||||
createE2eCouchDbPluginData,
|
||||
@@ -25,25 +25,35 @@ type ShutdownObservation = {
|
||||
disposeCalls: number;
|
||||
disposeCompleted: boolean;
|
||||
fixtureLogs: FixtureLog[];
|
||||
consoleLogs: {
|
||||
marker: string;
|
||||
messageType: string;
|
||||
messageText: string;
|
||||
level?: number;
|
||||
key?: string;
|
||||
}[];
|
||||
events: string[];
|
||||
};
|
||||
|
||||
async function evaluate<T>(cliBinary: string, session: ObsidianLiveSyncSession, body: string): Promise<T> {
|
||||
return await evalObsidianJson<T>(
|
||||
cliBinary,
|
||||
`(async()=>{
|
||||
const require=(condition,message)=>{if(!condition)throw new Error(message);};
|
||||
const waitFor=async(predicate,message)=>{
|
||||
const until=Date.now()+15000;
|
||||
while(!predicate()) {
|
||||
if(Date.now()>until)throw new Error(message);
|
||||
await new Promise(resolve=>setTimeout(resolve,20));
|
||||
}
|
||||
};
|
||||
${body}
|
||||
})()`,
|
||||
session.cliEnv
|
||||
);
|
||||
async function evaluate<T>(session: ObsidianLiveSyncSession, body: string): Promise<T> {
|
||||
// Observe the renderer console directly; CLI eval captures console output
|
||||
// as part of its response and changes the logging surface under test.
|
||||
return await withObsidianPage(session.remoteDebuggingPort, async (page) => {
|
||||
const result = await page.evaluate<string>(
|
||||
`(async()=>{
|
||||
const require=(condition,message)=>{if(!condition)throw new Error(message);};
|
||||
const waitFor=async(predicate,message)=>{
|
||||
const until=Date.now()+15000;
|
||||
while(!predicate()) {
|
||||
if(Date.now()>until)throw new Error(message);
|
||||
await new Promise(resolve=>setTimeout(resolve,20));
|
||||
}
|
||||
};
|
||||
${body}
|
||||
})()`
|
||||
);
|
||||
return JSON.parse(result) as T;
|
||||
});
|
||||
}
|
||||
|
||||
function createPluginData(dbName: string): Record<string, unknown> {
|
||||
@@ -95,7 +105,9 @@ async function startSession(
|
||||
function createShutdownScript(options: {
|
||||
duringMarker: string;
|
||||
failureMarker: string;
|
||||
afterMarker: string;
|
||||
reloadMarker: string;
|
||||
oldReloadMarker: string;
|
||||
injectFailure: boolean;
|
||||
urgentLevel: number;
|
||||
}): string {
|
||||
@@ -118,6 +130,8 @@ function createShutdownScript(options: {
|
||||
api,
|
||||
module,
|
||||
fixtureLogs:[],
|
||||
consoleLogs:[],
|
||||
uiAddLogCalls:0,
|
||||
events:[],
|
||||
closeStarted:false,
|
||||
closeCalls:0,
|
||||
@@ -129,7 +143,39 @@ function createShutdownScript(options: {
|
||||
app.__livesyncLogLifecycleState=state;
|
||||
|
||||
const originalAddLog=module.__addLog;
|
||||
const originalConsoleLog=console.log;
|
||||
const consoleMarkers=[
|
||||
fixtures.duringMarker,
|
||||
fixtures.failureMarker,
|
||||
fixtures.afterMarker,
|
||||
fixtures.reloadMarker,
|
||||
fixtures.oldReloadMarker
|
||||
].filter(Boolean);
|
||||
console.log=function(...args) {
|
||||
try {
|
||||
const entry=args[0];
|
||||
if(entry && typeof entry==="object" && "message" in entry) {
|
||||
const message=entry.message;
|
||||
const messageText=typeof message==="string"
|
||||
? message
|
||||
: message instanceof Error ? message.message
|
||||
: message && typeof message==="object" &&
|
||||
typeof message.__e2eLogLifecycleMarker==="string"
|
||||
? message.__e2eLogLifecycleMarker
|
||||
: "";
|
||||
const marker=consoleMarkers.find(candidate=>messageText.includes(candidate));
|
||||
if(marker) state.consoleLogs.push({
|
||||
marker,
|
||||
message,
|
||||
level:entry.level,
|
||||
key:entry.key
|
||||
});
|
||||
}
|
||||
} catch {}
|
||||
return originalConsoleLog.apply(this,args);
|
||||
};
|
||||
module.__addLog=function(message,level,key) {
|
||||
state.uiAddLogCalls++;
|
||||
if(typeof message==="string" && [
|
||||
fixtures.duringMarker,
|
||||
fixtures.failureMarker,
|
||||
@@ -179,6 +225,7 @@ function createShutdownScript(options: {
|
||||
module.__addLog=originalAddLog;
|
||||
resource.dispose=originalDispose;
|
||||
database.close=originalClose;
|
||||
console.log=originalConsoleLog;
|
||||
};
|
||||
|
||||
const run=async()=>{
|
||||
@@ -203,6 +250,10 @@ function createShutdownScript(options: {
|
||||
state.events.indexOf("close-finished")<state.events.indexOf("dispose-start"),
|
||||
"The log resource was disposed before the local database close completed");
|
||||
|
||||
await originalDispose.call(resource);
|
||||
require(state.disposeCompleted && state.disposeCalls===1,
|
||||
"Repeated log resource disposal was not idempotent");
|
||||
|
||||
const failedLog=fixtures.failureMarker
|
||||
? state.fixtureLogs.find(entry=>entry.message.includes(fixtures.failureMarker))
|
||||
: undefined;
|
||||
@@ -219,13 +270,41 @@ function createShutdownScript(options: {
|
||||
require(!failedLog,"The normal shutdown unexpectedly reported the injected failure");
|
||||
}
|
||||
|
||||
let detached=false;
|
||||
try {
|
||||
api.addLog("e2e-log-lifecycle-after-disposal");
|
||||
} catch(error) {
|
||||
detached=String(error).includes("not assigned");
|
||||
}
|
||||
require(detached,"The old API log handler remained assigned after disposal");
|
||||
const afterMessage=new Error(fixtures.afterMarker);
|
||||
const afterKey="e2e-log-lifecycle-after-disposal";
|
||||
const afterUnknownMessage={__e2eLogLifecycleMarker:fixtures.afterMarker};
|
||||
const afterUnknownKey="e2e-log-lifecycle-after-disposal-unknown";
|
||||
const uiCallsBeforeAfterLog=state.uiAddLogCalls;
|
||||
api.addLog(afterMessage,fixtures.urgentLevel,afterKey);
|
||||
api.addLog(afterUnknownMessage,fixtures.urgentLevel,afterUnknownKey);
|
||||
await waitFor(
|
||||
()=>state.consoleLogs.some(entry=>entry.marker===fixtures.afterMarker &&
|
||||
entry.message===afterMessage) &&
|
||||
state.consoleLogs.some(entry=>entry.marker===fixtures.afterMarker &&
|
||||
entry.message===afterUnknownMessage),
|
||||
"The old API did not forward its post-disposal messages to console.log"
|
||||
);
|
||||
const afterErrorLogs=state.consoleLogs.filter(entry=>entry.marker===fixtures.afterMarker &&
|
||||
entry.message===afterMessage);
|
||||
const afterUnknownLogs=state.consoleLogs.filter(entry=>entry.marker===fixtures.afterMarker &&
|
||||
entry.message===afterUnknownMessage);
|
||||
require(afterErrorLogs.length===1,"The post-disposal Error was not logged exactly once");
|
||||
require(afterUnknownLogs.length===1,"The post-disposal unknown message was not logged exactly once");
|
||||
require(
|
||||
afterErrorLogs[0].message===afterMessage && afterErrorLogs[0].message instanceof Error &&
|
||||
afterErrorLogs[0].message.message===fixtures.afterMarker,
|
||||
"The post-disposal Error was not preserved in the console log"
|
||||
);
|
||||
require(afterUnknownLogs[0].message===afterUnknownMessage,
|
||||
"The post-disposal unknown message was not preserved in the console log");
|
||||
require(
|
||||
afterErrorLogs[0].level===fixtures.urgentLevel && afterErrorLogs[0].key===afterKey &&
|
||||
afterUnknownLogs[0].level===fixtures.urgentLevel &&
|
||||
afterUnknownLogs[0].key===afterUnknownKey,
|
||||
"The post-disposal console logs changed their level or key"
|
||||
);
|
||||
require(state.uiAddLogCalls===uiCallsBeforeAfterLog,
|
||||
"The disposed ModuleLog UI handler received a post-disposal log");
|
||||
await state.disablePromise;
|
||||
require(!state.disableError,"Disabling the plug-in failed: "+state.disableError);
|
||||
return JSON.stringify({
|
||||
@@ -233,6 +312,19 @@ function createShutdownScript(options: {
|
||||
disposeCalls:state.disposeCalls,
|
||||
disposeCompleted:state.disposeCompleted,
|
||||
fixtureLogs:state.fixtureLogs,
|
||||
consoleLogs:state.consoleLogs.map(entry=>({
|
||||
marker:entry.marker,
|
||||
messageType:entry.message instanceof Error ? "Error" : typeof entry.message,
|
||||
messageText:entry.message instanceof Error
|
||||
? entry.message.message
|
||||
: typeof entry.message==="string" ? entry.message
|
||||
: entry.message && typeof entry.message==="object" &&
|
||||
typeof entry.message.__e2eLogLifecycleMarker==="string"
|
||||
? entry.message.__e2eLogLifecycleMarker
|
||||
: "",
|
||||
level:entry.level,
|
||||
key:entry.key
|
||||
})),
|
||||
events:state.events
|
||||
});
|
||||
};
|
||||
@@ -260,10 +352,14 @@ async function runCase(options: {
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
const markers = {
|
||||
duringMarker: `log-lifecycle-${options.caseName}-during-${randomUUID()}`,
|
||||
afterMarker: "log-lifecycle-" + options.caseName + "-after-disposal-" + randomUUID(),
|
||||
failureMarker: options.injectFailure
|
||||
? `log-lifecycle-${options.caseName}-injected-close-failure-${randomUUID()}`
|
||||
: "",
|
||||
reloadMarker: options.injectFailure ? "" : `log-lifecycle-${options.caseName}-reload-${randomUUID()}`,
|
||||
oldReloadMarker: options.injectFailure
|
||||
? ""
|
||||
: "log-lifecycle-" + options.caseName + "-old-api-after-reload-" + randomUUID(),
|
||||
injectFailure: options.injectFailure,
|
||||
urgentLevel: LOG_LEVEL_URGENT,
|
||||
};
|
||||
@@ -271,19 +367,17 @@ async function runCase(options: {
|
||||
try {
|
||||
session = await startSession(options.binary, options.cliBinary, vault);
|
||||
await waitForLiveSyncCoreReady(options.cliBinary, session.cliEnv);
|
||||
const shutdown = await evaluate<ShutdownObservation>(options.cliBinary, session, createShutdownScript(markers));
|
||||
const shutdown = await evaluate<ShutdownObservation>(session, createShutdownScript(markers));
|
||||
if (options.injectFailure) {
|
||||
return { caseName: options.caseName, shutdown };
|
||||
}
|
||||
|
||||
await evaluate(
|
||||
options.cliBinary,
|
||||
session,
|
||||
`await app.plugins.enablePlugin("obsidian-livesync"); return JSON.stringify({enabled:true});`
|
||||
);
|
||||
await waitForLiveSyncCoreReady(options.cliBinary, session.cliEnv);
|
||||
const reload = await evaluate<Record<string, unknown>>(
|
||||
options.cliBinary,
|
||||
session,
|
||||
`
|
||||
const state=app.__livesyncLogLifecycleState;
|
||||
@@ -295,8 +389,12 @@ async function runCase(options: {
|
||||
require(module,"The reloaded core did not contain ModuleLog");
|
||||
const originalAddLog=module.__addLog;
|
||||
const observed=[];
|
||||
const oldApiKey="e2e-log-lifecycle-reloaded-old-api";
|
||||
module.__addLog=function(message,level,key) {
|
||||
if(typeof message==="string" && message.includes(${JSON.stringify(markers.reloadMarker)})) {
|
||||
if(key===oldApiKey || (typeof message==="string" && [
|
||||
${JSON.stringify(markers.reloadMarker)},
|
||||
${JSON.stringify(markers.oldReloadMarker)}
|
||||
].some(marker=>marker && message.includes(marker)))) {
|
||||
observed.push({message,level,key});
|
||||
}
|
||||
return originalAddLog.apply(this,arguments);
|
||||
@@ -309,16 +407,36 @@ async function runCase(options: {
|
||||
!state.fixtureLogs.some(entry=>entry.message.includes(${JSON.stringify(markers.reloadMarker)})),
|
||||
"The old ModuleLog observer received the reloaded core's fixture log"
|
||||
);
|
||||
let oldHandlerDetached=false;
|
||||
try {
|
||||
state.api.addLog("e2e-log-lifecycle-reloaded-old-api");
|
||||
} catch(error) {
|
||||
oldHandlerDetached=String(error).includes("not assigned");
|
||||
}
|
||||
require(oldHandlerDetached,"The old API log handler was reassigned after reload");
|
||||
const oldApiMarker=${JSON.stringify(markers.oldReloadMarker)};
|
||||
const oldApiMessage=new Error(oldApiMarker);
|
||||
const uiCallsBeforeOldApi=state.uiAddLogCalls;
|
||||
const newModuleFixtureCallsBeforeOldApi=observed.length;
|
||||
state.api.addLog(oldApiMessage,${JSON.stringify(markers.urgentLevel)},oldApiKey);
|
||||
await waitFor(
|
||||
()=>state.consoleLogs.some(entry=>entry.marker===oldApiMarker &&
|
||||
entry.message===oldApiMessage),
|
||||
"The old API did not forward its post-reload Error to console.log"
|
||||
);
|
||||
const oldApiLogs=state.consoleLogs.filter(entry=>entry.marker===oldApiMarker);
|
||||
require(oldApiLogs.length===1,"The old API post-reload Error was not logged exactly once");
|
||||
require(
|
||||
oldApiLogs[0].message===oldApiMessage && oldApiLogs[0].message instanceof Error &&
|
||||
oldApiLogs[0].message.message===oldApiMarker,
|
||||
"The old API post-reload Error was not preserved in the console log"
|
||||
);
|
||||
require(
|
||||
oldApiLogs[0].level===${JSON.stringify(markers.urgentLevel)} &&
|
||||
oldApiLogs[0].key===oldApiKey,
|
||||
"The old API post-reload console log changed its level or key"
|
||||
);
|
||||
require(state.uiAddLogCalls===uiCallsBeforeOldApi,
|
||||
"The old ModuleLog UI handler received the old API's post-reload log");
|
||||
require(observed.length===newModuleFixtureCallsBeforeOldApi,
|
||||
"The new ModuleLog received the old API's post-reload log");
|
||||
return JSON.stringify({
|
||||
coreChanged:core!==state.core,
|
||||
observedByNewModule:observed.length,
|
||||
oldApiConsoleLogs:oldApiLogs.length,
|
||||
observedByOldModule:state.fixtureLogs.filter(
|
||||
entry=>entry.message.includes(${JSON.stringify(markers.reloadMarker)})
|
||||
).length
|
||||
@@ -335,13 +453,15 @@ async function runCase(options: {
|
||||
if (session) {
|
||||
try {
|
||||
await evaluate(
|
||||
options.cliBinary,
|
||||
session,
|
||||
`
|
||||
const state=app.__livesyncLogLifecycleState;
|
||||
state?.restore?.();
|
||||
delete app.__livesyncLogLifecycleState;
|
||||
return JSON.stringify({cleaned:true});
|
||||
try {
|
||||
return JSON.stringify({cleaned:true});
|
||||
} finally {
|
||||
state?.restore?.();
|
||||
delete app.__livesyncLogLifecycleState;
|
||||
}
|
||||
`
|
||||
);
|
||||
} catch {}
|
||||
|
||||
@@ -16,8 +16,15 @@
|
||||
* same upload.
|
||||
*/
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, mkdtemp, readFile, readdir, rm, utimes, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { DEFAULT_SETTINGS, TweakValuesTemplate } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { deriveIdKey } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { runCliJournalMigration } from "../runner/cliJournalMigration.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
@@ -43,7 +50,8 @@ import { REMOTE_ACTIVITY_EXPECTED_STATE, waitForRemoteActivityState } from "../r
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
|
||||
const notePath = "E2E/minio-upload.md";
|
||||
const unicodePaths = process.env.E2E_OBSIDIAN_UNICODE_PATHS === "true";
|
||||
const notePath = unicodePaths ? "Cafe\u0301/re\u0301sume\u0301.md" : "E2E/minio-upload.md";
|
||||
const useIndependentIds = process.env.E2E_OBSIDIAN_INDEPENDENT_IDS === "true";
|
||||
const useCustomRequestHandler = process.env.E2E_OBSIDIAN_CUSTOM_HTTP_HANDLER === "true";
|
||||
const noteContent = [
|
||||
@@ -63,10 +71,11 @@ async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.Proces
|
||||
`const path=${JSON.stringify(notePath)};`,
|
||||
`const content=${JSON.stringify(noteContent)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
|
||||
"const parent=path.slice(0,path.lastIndexOf('/'));",
|
||||
"if(!(await app.vault.adapter.exists(parent))) await app.vault.createFolder(parent);",
|
||||
"const existing=app.vault.getAbstractFileByPath(path);",
|
||||
"if(existing) await app.vault.delete(existing);",
|
||||
"await app.vault.create(path,content);",
|
||||
"const createdFile=await app.vault.create(path,content);",
|
||||
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
|
||||
"let entry=false;",
|
||||
"for(let i=0;i<40;i++){",
|
||||
@@ -76,7 +85,7 @@ async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.Proces
|
||||
"await sleep(250);",
|
||||
"}",
|
||||
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
|
||||
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
|
||||
"return JSON.stringify({id:entry._id,path:createdFile.path,type:entry.type,children:entry.children||[]});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
@@ -124,10 +133,13 @@ async function main(): Promise<void> {
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
pluginData: createE2eObjectStoragePluginData({
|
||||
...objectStorage,
|
||||
bucketPrefix,
|
||||
}),
|
||||
pluginData: createE2eObjectStoragePluginData(
|
||||
{
|
||||
...objectStorage,
|
||||
bucketPrefix,
|
||||
},
|
||||
{ pathNormalizationVersion: unicodePaths ? 1 : 0 }
|
||||
),
|
||||
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
@@ -137,6 +149,7 @@ async function main(): Promise<void> {
|
||||
session.cliEnv,
|
||||
{ ...objectStorage, bucketPrefix },
|
||||
{
|
||||
pathNormalizationVersion: unicodePaths ? 1 : 0,
|
||||
...(useIndependentIds
|
||||
? {
|
||||
encrypt: true,
|
||||
@@ -173,6 +186,13 @@ async function main(): Promise<void> {
|
||||
);
|
||||
}
|
||||
}
|
||||
if (unicodePaths && !useIndependentIds) {
|
||||
assertEqual(
|
||||
localEntry.id,
|
||||
notePath.normalize("NFC").toLowerCase(),
|
||||
"Obsidian did not derive the canonical Metadata ID."
|
||||
);
|
||||
}
|
||||
await pushLocalChanges(cli.binary, session.cliEnv);
|
||||
const activityAfterUpload = await waitForRemoteActivityState(
|
||||
session.remoteDebuggingPort,
|
||||
@@ -198,6 +218,131 @@ async function main(): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
if (unicodePaths) {
|
||||
const cliRoot = await mkdtemp(join(tmpdir(), "livesync-nfc-journal-"));
|
||||
try {
|
||||
const databasePath = join(cliRoot, "db");
|
||||
const vaultPath = join(cliRoot, "vault");
|
||||
const settingsPath = join(cliRoot, "settings.json");
|
||||
await mkdir(databasePath, { recursive: true });
|
||||
await mkdir(dirname(join(vaultPath, notePath)), { recursive: true });
|
||||
await writeFile(join(vaultPath, notePath), noteContent);
|
||||
const sourceSettings = await evalObsidianJson<Record<string, unknown>>(
|
||||
cli.binary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const service=app.plugins.plugins['obsidian-livesync'].core.services.setting;",
|
||||
"const s=service.currentSettings();",
|
||||
`const keys=${JSON.stringify([...Object.keys(TweakValuesTemplate), "encrypt", "passphrase", "usePathObfuscation", "E2EEAlgorithm", "idDerivationKey"])};`,
|
||||
"const values=Object.fromEntries(keys.map(key=>[key,s[key]]));",
|
||||
"if(s.idDerivationVersion===1){",
|
||||
"values.encryptedIdDerivationKey=await service.encryptConfigurationItem(s.idDerivationKey,s,'*');",
|
||||
"values.idDerivationKey='';values.configPassphraseStore='';",
|
||||
"}",
|
||||
"return JSON.stringify(values);",
|
||||
"})()",
|
||||
].join(""),
|
||||
session.cliEnv
|
||||
);
|
||||
await runCliJournalMigration(objectStorage, bucketPrefix, sourceSettings);
|
||||
await writeFile(
|
||||
settingsPath,
|
||||
JSON.stringify({
|
||||
...DEFAULT_SETTINGS,
|
||||
...objectStorage,
|
||||
...sourceSettings,
|
||||
remoteType: "MINIO",
|
||||
bucketPrefix,
|
||||
pathNormalizationVersion: 1,
|
||||
isConfigured: true,
|
||||
})
|
||||
);
|
||||
const execute = promisify(execFile);
|
||||
const run = async (...args: string[]) =>
|
||||
await execute(
|
||||
process.execPath,
|
||||
[
|
||||
resolve("src/apps/cli/dist/index.cjs"),
|
||||
databasePath,
|
||||
"--vault",
|
||||
vaultPath,
|
||||
"--settings",
|
||||
settingsPath,
|
||||
...args,
|
||||
],
|
||||
{ timeout: 60_000, maxBuffer: 4 * 1024 * 1024 }
|
||||
);
|
||||
await run("--force", "--write-settings", "fetch");
|
||||
assertEqual(
|
||||
await readFile(join(vaultPath, notePath), "utf8"),
|
||||
noteContent,
|
||||
"CLI Fetch changed the file contents."
|
||||
);
|
||||
assertEqual(
|
||||
(await run("cat", notePath.normalize("NFC"))).stdout,
|
||||
noteContent,
|
||||
"CLI Fetch did not retain the remote Chunk content."
|
||||
);
|
||||
const listedPaths = (await run("ls")).stdout
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => line.split("\t")[0]);
|
||||
assertEqual(listedPaths.length, 1, "CLI Fetch retained duplicate Metadata.");
|
||||
assertEqual(
|
||||
listedPaths[0].normalize("NFC"),
|
||||
notePath.normalize("NFC"),
|
||||
"CLI Fetch changed the logical path."
|
||||
);
|
||||
const fileInfo = JSON.parse((await run("info", listedPaths[0])).stdout) as { id: string };
|
||||
assertEqual(fileInfo.id, localEntry.id, "CLI Fetch used a different Metadata ID.");
|
||||
const expectedNames = await readdir(dirname(join(vaultPath, notePath)));
|
||||
assertEqual(expectedNames.length, 1, "CLI Fetch created a duplicate physical file.");
|
||||
assertEqual(
|
||||
expectedNames[0],
|
||||
notePath.split("/").slice(-1)[0],
|
||||
"CLI Fetch renamed the NFD physical file."
|
||||
);
|
||||
const editedContent = noteContent + "\nEdited after CLI restart.\n";
|
||||
await writeFile(join(vaultPath, notePath), editedContent);
|
||||
const nextTime = new Date(Date.now() + 5000);
|
||||
await utimes(join(vaultPath, notePath), nextTime, nextTime);
|
||||
await run("mirror");
|
||||
await run("sync");
|
||||
await pushLocalChanges(cli.binary, session.cliEnv);
|
||||
const sourceFile = join(vault.path, localEntry.path);
|
||||
const deadline = Date.now() + 20_000;
|
||||
while ((await readFile(sourceFile, "utf8")) !== editedContent && Date.now() < deadline) {
|
||||
await new Promise((resolveWait) => setTimeout(resolveWait, 200));
|
||||
}
|
||||
assertEqual(
|
||||
await readFile(sourceFile, "utf8"),
|
||||
editedContent,
|
||||
"Obsidian did not receive the CLI edit."
|
||||
);
|
||||
await run("rm", notePath.normalize("NFC"));
|
||||
await run("sync");
|
||||
await pushLocalChanges(cli.binary, session.cliEnv);
|
||||
const deleteDeadline = Date.now() + 20_000;
|
||||
let exists = true;
|
||||
while (exists && Date.now() < deleteDeadline) {
|
||||
exists = await readFile(sourceFile).then(
|
||||
() => true,
|
||||
(error: NodeJS.ErrnoException) => {
|
||||
if (error.code === "ENOENT") return false;
|
||||
throw error;
|
||||
}
|
||||
);
|
||||
if (exists) await new Promise((resolveWait) => setTimeout(resolveWait, 200));
|
||||
}
|
||||
assertEqual(exists, false, "Obsidian did not reflect the CLI deletion.");
|
||||
console.log(
|
||||
"NFC_JOURNAL_ROUND_TRIP: canonical ID, NFD physical name, CLI Fetch, edit after process restart, and deletion verified."
|
||||
);
|
||||
} finally {
|
||||
await rm(cliRoot, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Uploaded ${localEntry.path} through Journal Sync to ${objectStorage.bucket}/${bucketPrefix} (${keys.length} object(s)); tracked requests: ${activityAfterUpload.requestCount - activityBeforeUpload.requestCount}`
|
||||
);
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,102 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { mkdtemp, realpath, rm, stat } from "node:fs/promises";
|
||||
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { tmpdir } from "node:os";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
|
||||
type CommandResult = { code: number; signal: NodeJS.Signals | null };
|
||||
|
||||
function run(command: string, args: readonly string[], env: NodeJS.ProcessEnv = process.env): Promise<CommandResult> {
|
||||
return new Promise((resolveResult, reject) => {
|
||||
const child = spawn(command, [...args], { cwd: process.cwd(), env, stdio: "inherit" });
|
||||
child.once("error", reject);
|
||||
child.once("close", (code, signal) => resolveResult({ code: code ?? 1, signal }));
|
||||
});
|
||||
}
|
||||
|
||||
function pathInside(root: string, candidate: string): string {
|
||||
const localPath = relative(root, candidate);
|
||||
if (localPath === ".." || localPath.startsWith(".." + sep) || isAbsolute(localPath)) {
|
||||
throw new Error(
|
||||
"The discovered Obsidian executable paths must share one extracted application directory. " +
|
||||
"Set E2E_P2P_NFC_OBSIDIAN_DIR to their common root."
|
||||
);
|
||||
}
|
||||
return localPath.split(sep).join("/");
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const repoRoot = resolve(process.cwd());
|
||||
const binary = await realpath(requireObsidianBinary());
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error("Could not find obsidian-cli. Checked paths: " + cli.checked.join(", "));
|
||||
}
|
||||
const cliBinary = await realpath(cli.binary);
|
||||
const appDirectory = await realpath(resolve(process.env.E2E_P2P_NFC_OBSIDIAN_DIR?.trim() || dirname(binary)));
|
||||
if (!(await stat(appDirectory)).isDirectory()) {
|
||||
throw new Error("The selected Obsidian application root is not a directory: " + appDirectory);
|
||||
}
|
||||
const containerBinary = "/opt/obsidian/" + pathInside(appDirectory, binary);
|
||||
const containerCli = "/opt/obsidian/" + pathInside(appDirectory, cliBinary);
|
||||
|
||||
const build = await run(process.platform === "win32" ? "npm.cmd" : "npm", ["run", "build"]);
|
||||
if (build.code !== 0) {
|
||||
throw new Error("The host plug-in build failed with exit code " + build.code + ".");
|
||||
}
|
||||
|
||||
const artifactsDirectory = await mkdtemp(join(tmpdir(), "livesync-e2e-p2p-nfc-"));
|
||||
const projectName = "livesync-p2p-nfc-" + process.pid + "-" + Date.now() + "-" + randomBytes(4).toString("hex");
|
||||
const composeFile = join(repoRoot, "test/e2e-obsidian/compose.p2p.yml");
|
||||
const composeArgs = ["compose", "--project-name", projectName, "--file", composeFile];
|
||||
const composeEnvironment: NodeJS.ProcessEnv = {
|
||||
...process.env,
|
||||
E2E_P2P_NFC_OBSIDIAN_DIR: appDirectory,
|
||||
E2E_P2P_NFC_OBSIDIAN_BINARY: containerBinary,
|
||||
E2E_P2P_NFC_OBSIDIAN_CLI: containerCli,
|
||||
E2E_P2P_NFC_ARTIFACT_DIR: artifactsDirectory,
|
||||
};
|
||||
let exitCode = 0;
|
||||
let failure: unknown;
|
||||
|
||||
try {
|
||||
const result = await run(
|
||||
"docker",
|
||||
[...composeArgs, "run", "--rm", "--build", "p2p-nfc", ...process.argv.slice(2)],
|
||||
composeEnvironment
|
||||
);
|
||||
exitCode = result.code;
|
||||
if (result.signal) exitCode = 1;
|
||||
} catch (error) {
|
||||
failure = error;
|
||||
exitCode = 1;
|
||||
} finally {
|
||||
try {
|
||||
const teardown = await run(
|
||||
"docker",
|
||||
[...composeArgs, "down", "--volumes", "--remove-orphans"],
|
||||
composeEnvironment
|
||||
);
|
||||
if (teardown.code !== 0 && exitCode === 0) exitCode = teardown.code;
|
||||
} catch (error) {
|
||||
if (exitCode === 0) {
|
||||
failure = error;
|
||||
exitCode = 1;
|
||||
}
|
||||
}
|
||||
if (exitCode === 0 && process.env.E2E_P2P_NFC_KEEP_ARTIFACTS !== "1") {
|
||||
await rm(artifactsDirectory, { recursive: true, force: true });
|
||||
} else {
|
||||
console.log("P2P NFC diagnostics were retained at " + artifactsDirectory);
|
||||
}
|
||||
}
|
||||
|
||||
if (failure) throw failure;
|
||||
if (exitCode !== 0) process.exitCode = exitCode;
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -30,6 +30,7 @@ const focusedScenarios = new Set([
|
||||
"couchdb-manual-setup-workflow",
|
||||
"cli-to-obsidian-sync",
|
||||
"minio-upload",
|
||||
"journal-nfc-resend",
|
||||
"object-storage-setup-uri-workflow",
|
||||
"object-storage-compatible-setup-uri-workflow",
|
||||
"object-storage-qr-workflow",
|
||||
|
||||
+19
@@ -12,6 +12,10 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Unicode NFC path identity can be used with Object Storage and P2P on CLI and Obsidian clients. The setting is available under General Settings → File identity. Existing databases require a coordinated Rebuild and Fetch; physical file names and Chunk IDs stay unchanged. All P2P participants must be updated and migrated.
|
||||
- New CLI settings and Obsidian manual onboarding use NFC. Existing settings and old Setup URIs retain legacy identity; initial central-remote setup follows the remote declaration. Existing databases still require the migration above.
|
||||
- Object Storage checks the selected identity before Fetch replaces the local database and before ordinary remote administration changes its milestone. Rebuild preserves the old identity until the remote journals have been deleted.
|
||||
|
||||
### Fixed
|
||||
|
||||
#### Synchronisation
|
||||
@@ -20,11 +24,26 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
||||
- Metadata and Chunk reads use a cache URL derived from the startup time and Security Seed together. The URL remains stable within the same startup and Seed.
|
||||
- The advanced CouchDB setting **Use headers only for HTTP cache control (compatibility)** is disabled by default. Enable it only if your server or proxy rejects the additional query parameter; affected native caches may then reuse stale encrypted responses after Rebuild.
|
||||
|
||||
#### Conflict resolution
|
||||
|
||||
- Disabling or reloading the plug-in no longer causes a secondary error when pending work writes to the log. Logs arriving after the log display closes go to Obsidian's console.
|
||||
- Open conflict dialogues for ordinary files close immediately, and queued checks report cancellation. Operations already running retain their normal completion or failure path.
|
||||
|
||||
#### WebApp
|
||||
|
||||
- The experimental WebApp stops before opening its database or processing files when its settings select unsupported NFC path identity. It reports the Legacy-only requirement and preserves the selected Vault files and settings. This does not migrate an NFC database to Legacy identity.
|
||||
|
||||
#### Translations
|
||||
|
||||
- The notice shown when a translation is available now displays the **Display Language** setting label correctly in English, French, Hebrew, Russian, and Simplified Chinese. (#1298) Thank you to @cwatanab for the contribution!
|
||||
- Hebrew interface messages also display their referenced labels correctly.
|
||||
|
||||
### CLI
|
||||
|
||||
#### Fixed
|
||||
|
||||
- NFC file lookups and directory scans report permission and I/O failures instead of treating unreadable files as absent or returning a successful partial scan. A scan retains the comparison settings with which it began.
|
||||
|
||||
## 1.0.44
|
||||
|
||||
1st November, 2026
|
||||
|
||||
@@ -31,6 +31,7 @@ export default mergeConfig(
|
||||
"**/*.test.ts",
|
||||
"**/*unit.test.ts",
|
||||
"**/*.unit.spec.ts",
|
||||
"**/*.integration.spec.ts",
|
||||
"test/**",
|
||||
"**/_*",
|
||||
"**/*_obsolete.ts",
|
||||
|
||||
Reference in new issue
Block a user