Merge main and verify ID keys through Setup URI and QR

This commit is contained in:
vorotamoroz
2026-09-29 14:28:51 +00:00
51 changed files with 2823 additions and 181 deletions
+12
View File
@@ -235,6 +235,18 @@ Commonlib owns the typed English fallback for messages requested by its services
### Logging & Debugging ### Logging & Debugging
#### ID generation measurements on a device
Enable **Enable Developers' Debug Tools.**, restart Obsidian, and run **Open review harness** from the command palette. Choose **Run** beside **ID generation performance**, keep Obsidian in the foreground, and use **Copy Markdown report** to retain the results. The **Automatic** action does not run this measurement; **Full review** includes it.
The measurement uses fixed in-memory inputs and keys, with no Vault, database, settings, or remote writes. It compares legacy `xxhash64` and independent Chunk IDs for 256-byte, 4-KiB, and 32-KiB inputs, and compares obfuscated document IDs. Each result reports the median and range of three 1,000-ID samples and the median time per ID. Key derivation at save time is measured separately. Warm-up and pauses between batches are excluded from the timings. These measurements do not represent a full Rebuild.
Where `performance.memory` is available, the report includes approximate JavaScript heap samples before, during, and after measurement. These may include other Obsidian activity and garbage collection; they are neither total process RAM nor an exact peak. Unsupported devices explicitly report that heap measurements are unavailable.
The developer-only adapter in `src/features/ReviewHarness/reviewHarnessIdBenchmarkRuntime.ts` imports `HashManager` from Commonlib's public `/hashing` entry. Compilation, packed-package checks, and runtime tests cover this boundary. The algorithms remain owned by Commonlib.
#### Logs
- Use `this._log(msg, LOG_LEVEL_INFO)` in modules (automatically prefixes with module name) - Use `this._log(msg, LOG_LEVEL_INFO)` in modules (automatically prefixes with module name)
- Log levels: `LOG_LEVEL_DEBUG`, `LOG_LEVEL_VERBOSE`, `LOG_LEVEL_INFO`, `LOG_LEVEL_NOTICE`, `LOG_LEVEL_URGENT` - Log levels: `LOG_LEVEL_DEBUG`, `LOG_LEVEL_VERBOSE`, `LOG_LEVEL_INFO`, `LOG_LEVEL_NOTICE`, `LOG_LEVEL_URGENT`
- LOG_LEVEL_NOTICE and above are reported to the user via Obsidian notices - LOG_LEVEL_NOTICE and above are reported to the user via Obsidian notices
@@ -0,0 +1,515 @@
---
date: 2026-09-29
commonlib-version: "0.1.33"
self-hosted-livesync-version: "1.0.32"
status: unreleased
---
# Configurable ID derivation
## Purpose and baseline
Introduce an optional, saved secret for deterministic Chunk IDs and obfuscated
Metadata document IDs. This allows an E2EE passphrase to change without also
changing those IDs, and allows their derivation to use an independent secret.
Identical inputs must produce identical IDs on participating devices so that
Chunks can be reused and edits to the same path share one document identity.
The baseline is [PR #1222](https://github.com/vrtmrz/obsidian-livesync/pull/1222),
including its passphrase-persistence correction at commit
`126d6eadb858a79a08ad7f600061e54fc8d31196`. Commonlib `0.1.33`, published with
the `next` tag, provides the construction described here. It replaces the
independent Chunk algorithm from the `0.1.32` prerelease without a
compatibility branch or a new settings version. Legacy ID generation remains
unchanged. LiveSync pins the published `0.1.33` package and its registry
integrity in the lockfile.
Its [Internal Metadata encryption design](https://github.com/vrtmrz/obsidian-livesync/blob/126d6eadb858a79a08ad7f600061e54fc8d31196/docs/design_docs/internal_metadata_encryption.md)
remains the basis for Properties encryption and CouchDB feature admission.
This document records an unreleased LiveSync feature. It does not select a
plug-in release version.
## Feasibility
The change is feasible within the existing architecture. Commonlib already
centralises Chunk hashing, path-to-ID conversion, settings persistence, and
Setup URI encoding. Chunk reads follow stored IDs, so changing the generator
does not require a new Chunk reader or content representation.
The work spans Commonlib and its consumers. The principal constraints are
agreement on document IDs, complete propagation of the saved secret, and cache
behaviour after a setting change. The construction and transport-specific
agreement checks are described below. No database migration framework is
required.
## Scope
| Value or operation | Proposed behaviour |
| -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Encrypted Chunk IDs | Use the saved ID secret in the new mode. Keep legacy generation when the option is absent. |
| Obfuscated Metadata document IDs | Use the same saved secret with a separate derivation purpose. Preserve existing path normalisation and namespace handling, including ordinary files, `i:`, `ix:`, and supported legacy `ps:` entries. |
| Unobfuscated document IDs | Retain the current path-based identity. |
| Content and Properties encryption | Continue using the E2EE passphrase and existing encryption format. |
| CouchDB/PouchDB `_rev` | Retain existing revision generation and replication behaviour. |
| Internal content digests and transport bookkeeping | Retain existing behaviour unless they directly construct one of the IDs above. |
Document IDs are already assigned in the local database. Properties encryption
protects the path and other fields during transfer, while preserving `_id`.
Consequently, the saved ID secret must reach local path conversion as well as
remote-facing code. Path Obfuscation continues to control whether this
conversion is used; this proposal does not enable it automatically.
Journal keys which incorporate document IDs inherit the resulting IDs. They do
not need another secret. Content digests inside encrypted Customisation Sync
content also do not need a separate setting.
Automatic migration or enablement of existing or already migrated users,
Setup URI expiry or revocation, QR format security changes, revision redesign,
remote-only E2EE passphrase rotation, and a new remote configuration management
protocol are outside this change.
## Input and saved state
New Vault setup selects independent ID derivation and a random source by
default when E2EE is enabled. An existing Vault with no saved key retains
legacy mode until the user selects a new key explicitly. The setup dialogue
shows three radio choices:
1. Keep current configuration, selected by default for an existing Vault. It
retains the saved key when present and otherwise retains legacy ID
generation. A small description below this choice shows which configuration
is currently saved. In legacy mode, changing the E2EE passphrase still
changes IDs.
2. Generate a random ID key, selected by default for a new Vault.
3. Set an ID key. Three nested radio choices derive it once from the current
E2EE passphrase, accept a source string, or import a tagged recovery code.
Only the latter two show the text input.
The action is not persisted. An ordinary source is converted to a key when the
settings are applied, and then discarded. A tagged `sls-id-v1:` recovery code
imports its exact 256-bit key without deriving it again, including when pasted
into the source-string input. The recovery-code choice accepts only tagged
codes. A malformed tagged code is rejected. If either input is empty, a saved
key is kept; without a saved key, the dialogue requests input. Changing the
E2EE passphrase later preserves the saved ID key. Cancelling or failing to
save preserves the previous settings.
The source itself cannot be recovered from its key. A user can explicitly
display and copy the saved key as a tagged recovery code on the local device;
it is hidden when the dialogue opens. The dialogue warns that anyone who needs
recovery after losing every device should save that code or choose a source
they can reproduce.
Turning E2EE off retains the saved value but suspends its use for ID generation.
The setup dialogue disables new ID-key configuration while E2EE is off. Turning
E2EE on again reactivates the same value. Existing E2EE re-encryption and
Rebuild requirements still apply when the passphrase changes.
The source-input warning concerns only that input. The E2EE passphrase retains
its separate, existing storage behaviour. The recovery code contains the actual
saved ID key and must be handled as a secret.
Settings need to represent legacy mode or a supported version plus a derived
secret. Final field names belong in Commonlib. A declared new version with a
missing, malformed, or unavailable secret is an error; it must not silently
fall back to legacy generation. Loading or exporting an already derived value
must not derive it again.
## Deterministic derivation
The required contract is:
```text
source string --versioned derivation at save--> saved ID secret
saved ID secret + Chunk content ------------> Chunk ID
saved ID secret + canonical path -----------> obfuscated document ID
E2EE passphrase ----------------------------> content and Properties encryption
```
Derivation is offline and deterministic across supported runtimes. Its version
fixes the text encoding, treatment of Unicode and whitespace, salt, parameters,
and saved representation. It must not depend on server state, the E2EE Security
Seed, a device identifier, time, or device-specific iteration calibration.
Repeated saving of the same source under the same version produces the same
value. Reusing that source in another Vault consequently also reuses the value.
Version 1 uses PBKDF2-HMAC-SHA-256 with 310,000 iterations, the UTF-8 bytes of
the source after NFC normalisation, the fixed salt
`self-hosted-livesync:id-source:v1`, and a 256-bit output encoded as 64 lowercase
hexadecimal characters. Whitespace is preserved. The existing
`idDerivationVersion: 1` setting, saved-settings fields, and recovery-code format
remain unchanged; this implementation change does not add an ID format version
or migration path.
Obfuscated document IDs continue to use the saved 256-bit value directly as the
key for full HMAC-SHA-256. Their message remains UTF-8 encoding of
`self-hosted-livesync:id-v1:document`, a NUL byte, and the canonical path.
Agreement proofs likewise retain their existing full-HMAC messages. Neither
path uses the new Chunk-specific cache.
For encrypted Chunk IDs, first compute xxHash64 over the UTF-8 bytes of the
exact Chunk text with seed 0. Encode its result as a fixed 16-character
lowercase hexadecimal prehash. Derive a Chunk-specific subkey from the saved
32-byte value, then HMAC the domain-separated prehash:
```text
Kchunk = HMAC-SHA-256(
saved 32-byte key,
UTF8('self-hosted-livesync:id-v1:chunk-key:xxhash64')
)
prehash = fixed16lowerhex(xxHash64(UTF8(exact Chunk text), seed 0))
Chunk ID = full64lowerhex(HMAC-SHA-256(
Kchunk,
UTF8('self-hosted-livesync:id-v1:chunk:xxhash64' + NUL + prehash)
))
```
The resulting Chunk ID is the full 64-character lowercase hexadecimal HMAC
output. Existing namespace prefixes remain outside the digest. Keyed Chunk IDs
use this fixed prehash regardless of `hashAlg`; legacy mode and its existing
hash selection remain unchanged.
The Chunk-specific HMAC subkey and imported key, along with the WASM xxHash64
generator, are cached per `HashManager` and active saved key. Concurrent
preparation is shared. Replacing the manager or key, turning E2EE off, or
returning to legacy mode clears the cache; failed preparation can be retried.
This cache adds no persistent state. The current E2EE setting also selects the
legacy encrypted or plain Chunk route when a manager remains alive while E2EE is
turned off.
### Security properties and limits
A derived value remains a secret capable of generating IDs. Hashing does not
increase the entropy of its source. A password KDF adds guessing cost; it does
not make a weak source strong. HKDF alone does not provide that password
stretching. See [RFC 8018](https://www.rfc-editor.org/rfc/rfc8018.html#section-8)
and [RFC 5869](https://www.rfc-editor.org/rfc/rfc5869.html#section-4).
The random default separates ID generation from the E2EE passphrase. Deriving
both secrets from the same source retains a relationship with the original
passphrase, even after that passphrase changes. An independent source with
sufficient entropy provides the intended separation. HMAC with purpose
separation is the construction for using that secret; see
[RFC 2104](https://www.rfc-editor.org/rfc/rfc2104.html).
The fixed derivation salt means that reusing a source across Vaults reuses the
ID key; use separate sources when independent Vault identities are required.
CouchDB authentication and database access control remain the first access
boundary. This design also considers exposure through database credentials,
server administration, or backups. It does not promise to conceal equality,
document counts, revision history, or ciphertext lengths from database readers.
For Chunk IDs, xxHash64 is a public, non-cryptographic prehash. Distinct Chunk
texts which produce the same 64-bit prehash produce the same ID under the same
saved key. The final 256-bit HMAC does not restore distinctions lost at that
stage, so collision resistance for Chunk IDs is bounded by xxHash64 rather than
by the HMAC output width. This limit is an accepted trade-off for bounding the
content processed by HMAC.
The independent ID key preserves existing file contents, Chunk representation,
and `_rev` behaviour. It does not change the privacy properties of those
formats. Payload and virtual file padding remain outside this change.
## Sharing, import, and storage
Include the saved derived value and its version in Setup URIs, protected by the
existing, separate Setup URI passphrase. Import the saved value directly.
Additional devices therefore need neither the original source nor another
derivation step. Manual setup can reproduce it by entering the same source and
version, or by importing the tagged recovery code. After an E2EE passphrase
change, the current passphrase cannot be assumed to reproduce the old ID secret.
The standalone Setup URI generator uses a fresh random 256-bit ID key by
default, prints its tagged recovery code, and accepts that code for repeatable
generation for the same Vault. `id_mode=legacy` selects the old ID behaviour.
Running it again without the code produces a different key, so the generated
URI must not be treated as an update for an existing remote.
QR sharing includes the same fields through the existing QR representation and
warnings. Its current payload is not encrypted like a Setup URI. The agreed
scope accepts that existing sharing model and user responsibility for keeping
QR material private; it adds no QR storage or expiry mechanism.
| Boundary | Required handling |
| ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Existing settings and old complete Setup URI/QR/P2P imports | Missing fields select legacy behaviour. Do not inherit an unrelated value already present on the receiving device. |
| Ordinary partial setting updates | Preserve the current secret and version when neither is supplied. |
| New-format imports | Validate version and value together before applying or starting database work. |
| Local persistence | Integrate the secret explicitly with sensitive-configuration encryption and loading. Adding an arbitrary field does not currently provide this protection. |
| Reports, logs, and Markdown settings | Redact the secret in reports and logs; treat it as a credential in the existing Markdown export/import policy. An export which omits credentials must omit this secret. |
| Remote profiles, CLI, WebApp, WebPeer, and direct writers | Carry the effective value and version through every supported configuration path. A selected new mode must never degrade silently to legacy mode. |
Setup URI JSON encoding can carry ordinary new settings, but QR encoding uses
an explicit key-index table. Append stable QR entries without reordering old
ones. Complete imports and partial edits must have distinct missing-value
semantics even where the current implementation merges settings objects.
In particular, `SetupManager` currently merges decoded URI settings over the
receiving device's settings. Complete imports must normalise the new fields
before that merge to prevent accidental inheritance.
## Compatibility and changes to existing data
| Difference or change | Consequence |
| ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| Different Chunk derivation only | Existing content remains readable through Metadata `children`; new writes can duplicate Chunks and reduce reuse. |
| Different obfuscated document ID derivation | The same path can become separate documents. Treat this as an incompatible configuration requiring resolution. |
| New E2EE passphrase, unchanged saved ID secret | IDs remain stable for unchanged content, paths, and other ID settings. Re-encryption still requires the existing E2EE workflow. |
| Enabling, replacing, or disabling the option with Path Obfuscation active | Document identity changes. Use the established authoritative Rebuild and secondary-device Fetch workflow. |
| Existing installation with no new option | Preserve its exact legacy behaviour; do not derive or copy a value during upgrade. |
Copying the legacy passphrase into the new derivation does not preserve legacy
IDs, because the derivation itself changes. This proposal therefore makes no
automatic or seamless migration promise. Before an explicit transition, update
and stop the participating devices, select the authoritative data, and use the
existing [Rebuild and Fetch procedures](../recovery.md). Share the resulting
configuration before other devices rejoin. One-entry
[Metadata ID repair](metadata_document_id_validation_and_repair.md) does not
perform this transition.
Commonlib's Chunk cache includes content-to-ID lookup before hashing. Changing
the hash function alone can keep producing old IDs, and reading old Chunks can
populate that lookup again. The implementation must distinguish read reuse
from the ID selected for a new write, including after manager replacement and
restart. Readers continue accepting referenced legacy Chunks.
## Agreement checks and older clients
Keep checks focused on preventing incompatible document identities. Reuse the
existing configuration review and replication admission paths. A mismatch
must not be treated as an automatically alignable Chunk setting when document
IDs depend on it. Show a mismatch or unsupported version without exposing the
saved secret.
The advertised ID version is used for comparison only. Ordinary Tweak alignment
preserves each device's ID version and key together, including when only Chunk
IDs differ. A document ID mode mismatch requires explicit configuration through
a Setup URI or the matching key, rather than adopting a version without its key.
For CouchDB, extend the supported feature set in the remote feature contract
introduced by PR #1222, then declare the new requirement before writing data
under it. That mechanism rejects unsupported features at admission and provides
a best-effort stop when an unsupported requirement arrives later. It checks
format support, not equality of saved secrets, and does not make a live
migration atomic. Journal can extend its existing milestone compatibility path;
P2P needs its separate admission handling. The CouchDB feature contract alone
cannot protect those transports.
The implementation checks up to two remote documents in each ordinary and
internal obfuscated-ID namespace before CouchDB replication or direct writes.
This includes a legacy-mode caller connecting to a remote which uses keyed IDs.
For each available sample it recomputes the ID from the decrypted path; any
mismatch rejects the connection. An empty database, or one with no usable
sample, is reported as unverified and may proceed because there is no observed
document identity to conflict with. A sampled match is evidence, not a proof
that every document has the same identity; an unsampled mixture remains a
limitation of this bounded check.
When E2EE and Path Obfuscation are both active, Journal stores an
E2EE-encrypted, domain-separated proof in its existing milestone. It is
encrypted before the milestone is uploaded and compared on later connections.
An established milestone without this proof requires a Rebuild before the new
document IDs can be used. Journal advertises a new compatibility range for
keyed document IDs so older clients reject it. P2P compares a
purpose-separated HMAC over a fresh challenge during peer admission; the proof
is not stored. When Path Obfuscation is off, different keys affect only Chunk
IDs, so Journal keeps its legacy compatibility range and P2P does not require
key agreement. Neither transport publishes the key or a plaintext verifier in
Tweak values. CouchDB and direct writers use the document sample check above
rather than a stored verifier. The sample check uses the host's path service
with the attempted settings snapshot so stored non-canonical paths are treated
the same way as ID generation.
The existing `_rev` behaviour for ordinary content remains unchanged.
## Implementation responsibilities
The following are the confirmed integration points in the reviewed baseline.
Commonlib paths refer to its package implementation, not a source mirror in
this repository.
| Owner and entry points | Work |
| -------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Commonlib `HashManagerCore`, concrete hash managers, `PathService`, and `path2id_base` | Select legacy or new derivation consistently; preserve path and namespace semantics. |
| Commonlib `EntryManagerImpls`, `LayeredChunkManager`, and `LiveSyncManagers` | Keep referenced Chunks readable and invalidate or partition generation-dependent caches. |
| Commonlib settings definitions/lifecycle, `SettingService`, `pickEncryptionSettings`, and `API/processSetting` | Own version validation, persistence, copying, imports, Setup URI encoding, and QR slots. |
| Commonlib compatibility assessment and replication implementations | Classify identity differences, protect any comparison data, and enforce supported formats at each transport boundary. |
| Commonlib `API/DirectFileManipulatorV2` | Carry the option through its explicit settings and path-obfuscation configuration. |
| LiveSync `SetupRemoteE2EE.svelte`, `PaneRemoteConfig.ts`, and `SetupManager.ts` | Implement the three configuration actions and three nested ID-key inputs, configured state, local recovery-code reveal, and existing Apply/Rebuild/Fetch choices. |
| LiveSync `replicatorConfigurationIdentity.ts`, `reportTool.ts`, and `ModuleObsidianSettingAsMarkdown.ts` | Replace connections when effective settings change, redact the secret, and apply credential-sharing rules. |
| CLI, browser applications, and setup tools | Use the same Commonlib contract in manual setup and imports; generate new Setup URIs with a reusable random ID key by default. |
Implement Commonlib changes in its own repository, validate its packed artefact,
and validate LiveSync against that exact dependency before adopting a released
version. Translations remain outside this implementation scope.
## Validation
The fixed-vector and cache tests first failed against unchanged Commonlib
`0.1.32`, then passed after the implementation change. Commonlib's 2,036 Unit
tests, type check, package boundary, and isolated packed-package checks pass.
Three Integration tests against real CouchDB and Object Storage verify direct
access, Journal agreement, and rejection before control-document changes.
The same-manager E2EE-off regression was reproduced and fixed.
LiveSync's 1,049 Unit tests, type and lint checks, production build, and iOS 15
bundle syntax check pass after installing the exact published `0.1.33`
package. Its tarball matches the validated publication candidate, and all 559
installed package files match the registry artefact. The resulting bundle is
identical to the one checked before publication.
Real Obsidian two-Vault checks with the accepted Chunk construction cover
matching-key synchronisation, incompatible document key rejection, and
differing Chunk keys with visible paths. The Review Harness also passes with
the published package, verifying actual ID calculations and report copying
without changing live settings. Its adapter imports `HashManager` through
Commonlib's focused `/hashing` entry, whose package checks cover public types,
Node execution, and browser bundling.
### Current Chunk calculation performance
The actual Commonlib `HashManager` implementations were compared in Obsidian
1.12.7 on ARM64 Linux. Six samples rotate all three variants through each
execution position twice. The table reports median total ID calculation time;
1,000 means the total for 1,000 IDs, not the time per ID. Inputs are synthetic.
| Input | IDs per sample | Legacy xxHash64 | Previous independent HMAC | Updated independent ID |
| --------------------------------- | -------------: | --------------: | ------------------------: | ---------------------: |
| 256-byte text | 1,000 | 4.50 ms | 41.40 ms | 24.45 ms |
| 4 KiB text | 1,000 | 10.00 ms | 96.25 ms | 29.70 ms |
| 32 KiB text | 1,000 | 48.40 ms | 494.40 ms | 68.95 ms |
| 10 MiB binary, default splitting | 103 | 19.60 ms | 192.50 ms | 21.45 ms |
| 50 MiB binary, default splitting | 512 | 99.85 ms | 972.75 ms | 106.65 ms |
| 10 MiB binary, Self-hosted preset | 5 | 24.10 ms | 216.90 ms | 18.50 ms |
| 50 MiB binary, Self-hosted preset | 35 | 118.40 ms | 1,067.00 ms | 91.75 ms |
The first updated ID, including Chunk-key preparation, took 1.3 ms in this
run. Repeated measurements exclude preparation, warm-up, and pauses. Binary
cases use the actual splitter and Base64 representation; all decoded bytes
and repeated IDs were checked. Splitting, Base64 conversion, database work,
payload encryption, and transfer are outside the measured interval. These
results establish lower ID calculation cost on this host, not a complete
Rebuild speedup or native mobile performance.
### Native-device ID measurements
User-supplied Review Harness reports compare the previous and updated builds
on Android 13 and iOS 18.7. Each value is the median total time for 1,000
independent Chunk IDs, using three samples in each run.
| Input | Android, previous | Android, updated | iOS, previous | iOS, updated |
| ------------- | ----------------: | ---------------: | ------------: | -----------: |
| 256-byte text | 53.6 ms | 37.8 ms | 21 ms | 20 ms |
| 4 KiB text | 70.9 ms | 43.4 ms | 22 ms | 22 ms |
| 32 KiB text | 151.1 ms | 66.5 ms | 36 ms | 39 ms |
Android's 32-KiB result takes about 56% less time. The corresponding iOS
result increases by 3 ms per 1,000 IDs; separate runs with three samples do
not establish the cause of that difference. Across these sizes, the updated
independent calculation adds approximately 18–25 ms per 1,000 IDs over each
device's legacy xxHash64 calculation. Save-time key derivation has medians of
46.6 ms on Android and 51 ms on iOS.
These reports measure synthetic ID calculations, excluding database work,
payload encryption, and transfer. Android's heap samples remain constant,
and iOS does not expose them, so the reports do not establish memory usage or
improvement. The updated build has not been measured on Windows.
The checks below are historical reference evidence for the predecessor
independent-ID implementation, which used full-content HMAC-SHA-256 for Chunk
IDs. They do not validate the current xxHash64-prehash construction.
### Historical predecessor checks
Earlier consumer validation used a local Commonlib `0.1.32` candidate. Clean
installations with npm 10 and npm 11, type checking, lint, Svelte checks, the
production build, and the iOS 15 bundle compatibility check passed. LiveSync
had 1,039 passing Unit tests, six passing Integration tests against real
CouchDB, and seven passing Setup URI utility tests with the frozen Deno
lockfile.
Predecessor Commonlib candidate checks covered deterministic vectors, Unicode
normalisation, legacy behaviour, encrypted settings persistence, imports,
cache transitions, and transport admission. Its Integration tests against real
Object Storage accept a matching Journal key, reject a different document ID
key before changing the milestone, and allow different Chunk keys when paths
remain visible. A direct-access Integration test against real CouchDB reads
with the same key and rejects a different key before changing the version
document. These library tests complemented the consumer checks for that
predecessor; they were not additional LiveSync Unit tests.
Real Obsidian 1.12.7 on ARM64 Linux verifies the following consumer boundaries:
| Boundary | Verified behaviour |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CouchDB synchronisation | Two Vaults exchange notes in both directions with matching document and Chunk IDs. Different Chunk keys also work with Path Obfuscation off. |
| CouchDB rejection | Ordinary replication rejects different or legacy document ID keys before downloading files or changing remote documents and checkpoints. |
| Visible onboarding | Both a separate source and the random default persist an encrypted key, transfer it through a Setup URI, complete Fast Fetch, and synchronise in both directions. The `%`-prefixed E2EE passphrase survives restart. |
| Input and recovery | The radio controls and disabled styling are exercised. An empty first source keeps the dialogue open with an error; empty input keeps an existing key. A recovery code restores the same key. |
| Journal upload | The uploaded documents and Chunks have keyed IDs, and the first Object Storage milestone contains the encrypted agreement proof. |
| Credential-free Markdown | Neither the saved ID key nor its encrypted representation appears in exported settings Markdown. |
A CLI P2P E2E run with a local relay imports encrypted Setup URIs, transfers a
note with matching keys, and rejects a peer with a different document ID key.
A separate check loads the published `0.1.31` DirectFileManipulator in another
process: it reads a legacy remote, and rejects an independent-ID remote with
or without Path Obfuscation, leaving remote documents and checkpoints
unchanged. This checks the previous library API, rather than an older
Obsidian installation. Bounded sampling tests cover empty and mixed document
collections; they do not establish that every document in a remote is
compatible.
### Performance reference for the predecessor
The measurements below are historical results for a local predecessor
Commonlib `0.1.32` candidate which used full-content HMAC-SHA-256 for
independent Chunk IDs. They are reference evidence only, not performance
results for the accepted xxHash64-prehash construction. Both modes enable E2EE
and Path Obfuscation; the legacy baseline uses `xxhash64`. Three trials per
mode alternate their order. These are synthetic corpora and serial local
writes, excluding Vault enumeration, remote payload encryption, and transfer;
they are not timings of the complete Rebuild action.
Saving an ordinary ID source takes 52–57 ms in Node.js 24, with a median of
56 ms. This PBKDF2 operation happens once when saving the source. Per-Chunk
and per-document IDs use the saved key and do not repeat PBKDF2.
The actual Obsidian renderer gives these median times for 1,000 serial calls
to the Commonlib hash manager or Path Service, after warm-up:
| Input | Legacy | Independent ID |
| ------------------------ | ------: | -------------: |
| Distinct 256-byte Chunks | 5.4 ms | 43.3 ms |
| Distinct 4 KiB Chunks | 15.0 ms | 104.8 ms |
| Distinct 32 KiB Chunks | 52.8 ms | 581.7 ms |
| Distinct document paths | 36.5 ms | 50.0 ms |
These direct calls include no Chunk-content cache hits. The predecessor hash
had a measurable cost, especially when there were many small Chunks. The
local-write experiments exercise splitting, ID generation, Chunk reuse, and
PouchDB writes:
| Workload and adapter | Legacy median (range) | Independent median (range) |
| ------------------------------------------------------------- | --------------------: | -------------------------: |
| 5,000 text/binary files, 100 MiB, Node PouchDB memory adapter | 102.6 s (87.4–109.3) | 86.6 s (75.4–93.1) |
| 1,000 text files, 19.5 MiB, actual Obsidian local database | 78.7 s (52.0–82.8) | 63.2 s (62.8–63.8) |
The measurements do not show a large overall slowdown for these workloads,
but the variation does not support a general speedup claim. Both experiments
checked document counts, Chunk-reference counts, and sample content readback.
The Node experiment also checked that every referenced Chunk was present. The
100 MiB corpus includes 250 duplicate files and produces 213,788 Chunk
references to 199,468 distinct Chunks in both modes, preserving reuse.
For that corpus, stored document JSON grows from 133,157,581 to 154,342,743
UTF-8 bytes, an increase of 15.9%. The text-only Obsidian corpus produces many
small Chunks and grows from 27,893,656 to 33,594,994 bytes, or 20.4%, including
32 warm-up documents. Longer Chunk IDs occur in both Chunk documents and
Metadata references. Ordinary obfuscated document IDs remain 66 characters.
These totals measure serialised document JSON; physical database and index
growth depend on the adapter and have not been measured.
### Remaining validation
Larger binary workloads on mobile, a representative user's Vault, physical
storage growth, and the complete Rebuild wall time remain unmeasured. The
native-device reports above verify the synthetic ID calculation scenario;
desktop E2E and mobile viewport checks do not establish other mobile
operating-system behaviour. URI revocation, QR redesign, and automatic
migration remain outside this change.
+20 -10
View File
@@ -402,14 +402,24 @@ passphrase and cancellation. The WebPeer browser test confirms that its monitor
remains available after the URI window ends for a device which has already remains available after the URI window ends for a device which has already
imported the URI. imported the URI.
The subsequent update to published Commonlib 0.1.33 passes a clean `npm ci`, The current Commonlib 0.1.33 integration includes the merged host changes from
matching npm and frozen Deno lock integrity, source checks, the production build, PR #1222 and PR #1225. npm 10 and normal clean installations preserve the lockfile;
1,042 unit tests, and eight Deno setup-tool tests. A consumer check of the public source checks, the production build, 1,064 unit tests, and ten Deno setup-tool
hashing entry covers key changes, E2EE suspension, and cache retirement. The real tests pass. Both URI modes preserve ID recovery and explicit legacy ID selection.
Obsidian two-Vault workflow also passes ordinary and encrypted note transfers.
The downstream change depends on PR #1222's host compatibility integration; Real Obsidian tests cover independent ID keys through Time-bound, Compatible,
relaxing only the version check would bypass its compatibility contract. These and QR setup, encrypted local persistence, natural restarts of both devices,
results do not establish a mobile performance bound. The first implementation and bidirectional note transfers with stable document and Chunk IDs. Incorrect
changes only primary-language resources; translation changes require separate passphrases and past-window URIs leave runtime and persisted settings unchanged.
scope. The past clock belongs to an isolated fixture worker, not Obsidian or the host.
CouchDB tests also cover a custom ID source and recovery code, rejection of
incompatible document keys before remote writes, and the Doctor's decline,
reminder, dismissal, and later acceptance paths. The detailed commands and the
remaining published-artefact and physical-device checks are recorded in the
[real Obsidian test guide](../../test/e2e-obsidian/README.md#combined-setup-and-security-regression-checks).
Compatible preserves the URI encryption format; each receiving client must
still support the shared settings and the remote's declared features. The host
compatibility checks from PR #1222 remain necessary. These results do not
establish a mobile performance bound. The first implementation changes only
primary-language resources; translation changes require separate scope.
+6
View File
@@ -92,6 +92,12 @@ recovery guidance, or diagnostics intended for users.
reports, and advanced edge-case settings. reports, and advanced edge-case settings.
- **Hidden File Sync:** The feature which synchronises files in hidden - **Hidden File Sync:** The feature which synchronises files in hidden
directories, such as `.obsidian`. directories, such as `.obsidian`.
- **ID key:** A saved secret used to generate encrypted Chunk IDs and
obfuscated Metadata document IDs when independent ID derivation is enabled.
It is separate from the current E2EE passphrase.
- **ID recovery code:** A versioned text form of the saved ID key which can be
shown on the current device and imported without deriving a different key.
Treat it as a secret.
- **JWT Authentication:** An experimental CouchDB authentication option which - **JWT Authentication:** An experimental CouchDB authentication option which
uses a JSON Web Token instead of standard credentials. It requires a private uses a JSON Web Token instead of standard credentials. It requires a private
key or secret, algorithm, expiry duration, subject, and key ID. key or secret, algorithm, expiry duration, subject, and key ID.
+16
View File
@@ -239,6 +239,20 @@ Setting key: passphrase
Encrypting passphrase. If you change the passphrase, you need to rebuild databases (You will be informed). Encrypting passphrase. If you change the passphrase, you need to rebuild databases (You will be informed).
#### Independent ID derivation
Setting keys: `idDerivationVersion`, `idDerivationKey`
This setting saves a separate key for encrypted Chunk IDs and obfuscated Metadata document IDs. New Vault setup selects **Generate a random ID key** by default when E2EE is enabled. Existing Vaults select **Keep current configuration** by default. The radio choices show the available configurations together. A small description under **Keep current configuration** identifies the saved configuration: an existing ID key, or legacy IDs linked to the E2EE passphrase. That choice retains either one; on a new Vault, choosing it explicitly uses legacy IDs. If the configuration is legacy, changing the E2EE passphrase also changes IDs.
To set a key yourself, choose **Set an ID key**. Three further radio choices then appear: **Derive from current E2EE passphrase**, **Enter an ID source**, and **Import an ID recovery code**. The last two choices show a text input. The source input also recognises a tagged recovery code. An empty input keeps an existing key; a first key requires input. An ordinary source is converted to a key when you apply the settings and cannot be shown again. A recovery code imports the saved key directly.
Use **Show current recovery code** to display and copy the saved key on this device. The code starts with `sls-id-v1:` and can be pasted into the manual input on another device without deriving a different key. A Setup URI carries the same saved key under its separate passphrase. If you need to restore the configuration after losing every device, save the recovery code or choose a source you can reproduce before relying on the random default. Keep the code private.
Using the E2EE passphrase as the source keeps IDs stable after later passphrase changes, but it does not separate the original passphrase from guesses based on known IDs. Use a long, unpredictable, separate source when that separation matters. Hashing a weak source does not make it strong.
Changing the E2EE passphrase later does not change the saved ID key, although the existing re-encryption and Rebuild procedure still applies to the encrypted data. While E2EE is off, the saved ID key is retained but is not used; existing legacy ID generation applies until E2EE is enabled again. Devices with different ID keys can synchronise when Path Obfuscation is off, although identical content may produce duplicate Chunks. Enabling, replacing, or disabling the ID key can change document IDs when Path Obfuscation is active. Update participating devices, Rebuild from the authoritative Vault, and Fetch on other devices before resuming ordinary synchronisation. A QR code includes the saved key under the existing QR sharing rules, so keep the QR code private.
#### Path Obfuscation #### Path Obfuscation
Setting key: usePathObfuscation Setting key: usePathObfuscation
@@ -1033,6 +1047,8 @@ Setting key: hashAlg
`xxhash64` is the supported current value. Older algorithms remain selectable only as an edge-case compatibility path for existing databases. Changing the algorithm can reduce chunk reuse between devices and requires the normal tweak review. `xxhash64` is the supported current value. Older algorithms remain selectable only as an edge-case compatibility path for existing databases. Changing the algorithm can reduce chunk reuse between devices and requires the normal tweak review.
When independent ID derivation is enabled, encrypted Chunk IDs use its versioned HMAC construction instead of `hashAlg`. The selected `hashAlg` continues to apply to legacy IDs.
### 6. Edge case addressing (Behaviour) ### 6. Edge case addressing (Behaviour)
#### Fetch database with previous behaviour #### Fetch database with previous behaviour
+1 -1
View File
@@ -31,7 +31,7 @@ deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.co
For providers which require them, set `force_path_style`, `use_custom_request_handler`, or `bucket_custom_headers` as described in the [setup utility reference](../utils/readme.md#object-storage). For providers which require them, set `force_path_style`, `use_custom_request_handler`, or `bucket_custom_headers` as described in the [setup utility reference](../utils/readme.md#object-storage).
Store the generated Setup URI and Setup URI passphrase separately. The URI is encrypted, but it contains the Object Storage credentials. Store the generated Setup URI and Setup URI passphrase separately. The URI is encrypted, but it contains the Object Storage credentials. The generator also prints an ID recovery code; reuse it through `id_recovery_code` if you regenerate a URI for the same Vault. A new run without it creates a different ID key. The [setup utility reference](../utils/readme.md#setup-uri-generation) describes the `id_mode=legacy` option for existing Vaults.
The generator prints the exact end time of its default Ephemeral URI. Set `uri_mode=persistent` before running it if the URI must remain usable without a time condition. The generator prints the exact end time of its default Ephemeral URI. Set `uri_mode=persistent` before running it if the URI must remain usable without a time condition.
## Set up the first device ## Set up the first device
+4
View File
@@ -190,6 +190,8 @@ deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.co
> >
> If `uri_passphrase` is omitted, the generator creates a cryptographically random value and prints it once. > If `uri_passphrase` is omitted, the generator creates a cryptographically random value and prints it once.
The generator also prints an ID recovery code for its random ID key. Save that code if you may need to regenerate a Setup URI for the same Vault. Pass it back as `id_recovery_code`; otherwise a later run creates a different key. Set `id_mode=legacy` only when connecting to a Vault which uses the previous ID behaviour. See the [setup utility reference](../utils/readme.md#setup-uri-generation).
The generator consumes the exact registry-pinned Commonlib release used by the provisioning utility. It creates a configured CouchDB remote profile, applies the current defaults for a new Vault, and encodes them with Commonlib's Setup URI contract. The generator consumes the exact registry-pinned Commonlib release used by the provisioning utility. It creates a configured CouchDB remote profile, applies the current defaults for a new Vault, and encodes them with Commonlib's Setup URI contract.
By default, the URI is Ephemeral and can be opened only until the exact UTC time printed by the generator. This is the end of a fixed seven-day window, not seven days after creation. Set `uri_mode=persistent` before running the command if the URI needs no time condition or must be readable by an older client that supports the existing encrypted format. By default, the URI is Ephemeral and can be opened only until the exact UTC time printed by the generator. This is the end of a fixed seven-day window, not seven days after creation. Set `uri_mode=persistent` before running the command if the URI needs no time condition or must be readable by an older client that supports the existing encrypted format.
@@ -201,6 +203,8 @@ Generated couchdb Setup URI.
Ephemeral: usable until 2026-10-01T00:00:00.000Z (UTC). Ephemeral: usable until 2026-10-01T00:00:00.000Z (UTC).
Your passphrase for the Setup URI is: H7vX...a-random-32-character-value Your passphrase for the Setup URI is: H7vX...a-random-32-character-value
This passphrase is never shown again, so store it safely. This passphrase is never shown again, so store it safely.
ID recovery code: sls-id-v1:<64 lowercase hexadecimal characters>
Use id_recovery_code with this value and reuse the same remote settings when generating another Setup URI for the same Vault.
obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D
``` ```
+1 -1
View File
@@ -135,5 +135,5 @@ export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
``` ```
The generated Setup URI contains the encrypted room, relay, and Vault settings. It deliberately omits the device-specific name. Store the URI and its passphrase separately. After importing it on the first device, continue from the initialisation step above, then generate a fresh Setup URI for an additional device from that working device. The generated Setup URI contains the encrypted room, relay, and Vault settings. It deliberately omits the device-specific name. Store the URI and its passphrase separately. The generator prints an ID recovery code; pass it as `id_recovery_code` if you regenerate a URI for the same Vault. A run without it creates a different ID key. Also reuse the original `p2p_room_id` and `p2p_passphrase`, since omitted values are generated afresh. The [setup utility reference](../utils/readme.md#setup-uri-generation) describes the `id_mode=legacy` option for existing Vaults. After importing the URI on the first device, continue from the initialisation step above, then generate a fresh Setup URI for an additional device from that working device.
The generator prints the exact end time of its default Ephemeral URI. Set `uri_mode=persistent` before running it if the URI must remain usable without a time condition. The generator prints the exact end time of its default Ephemeral URI. Set `uri_mode=persistent` before running it if the URI must remain usable without a time condition.
+2 -1
View File
@@ -71,7 +71,8 @@
"test:e2e:obsidian:cli-to-obsidian-sync": "tsx test/e2e-obsidian/scripts/cli-to-obsidian-sync.ts", "test:e2e:obsidian:cli-to-obsidian-sync": "tsx test/e2e-obsidian/scripts/cli-to-obsidian-sync.ts",
"test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts", "test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts",
"test:e2e:obsidian:object-storage-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts", "test:e2e:obsidian:object-storage-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts",
"test:e2e:obsidian:object-storage-qr-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --qr", "test:e2e:obsidian:object-storage-compatible-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --compatible",
"test:e2e:obsidian:object-storage-qr-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --qr",
"test:e2e:obsidian:object-storage-custom-http-handler-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --custom-http-handler", "test:e2e:obsidian:object-storage-custom-http-handler-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --custom-http-handler",
"test:e2e:obsidian:p2p-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/p2p-setup-uri-workflow.ts", "test:e2e:obsidian:p2p-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/p2p-setup-uri-workflow.ts",
"pretest:e2e:obsidian:p2p-connection-check": "npm run build && npm run build --workspace webpeer", "pretest:e2e:obsidian:p2p-connection-check": "npm run build && npm run build --workspace webpeer",
+1 -1
View File
@@ -628,7 +628,7 @@ export async function startP2pRelay(): Promise<void> {
//TODO: port mapping should be configurable. //TODO: port mapping should be configurable.
"4000:7777", "4000:7777",
"--tmpfs", "--tmpfs",
"/app/strfry-db:rw,size=256m", "/app/strfry-db:rw,size=256m,mode=1777",
"--entrypoint", "--entrypoint",
"sh", "sh",
P2P_RELAY_IMAGE, P2P_RELAY_IMAGE,
+16 -8
View File
@@ -13,7 +13,11 @@ export async function initSettingsFile(settingsFile: string): Promise<void> {
* Generate a full setup URI from a settings file via the Commonlib package API. * Generate a full setup URI from a settings file via the Commonlib package API.
* Mirrors the bash flow in test-setup-put-cat-linux.sh. * Mirrors the bash flow in test-setup-put-cat-linux.sh.
*/ */
export async function generateSetupUriFromSettings(settingsFile: string, setupPassphrase: string): Promise<string> { export async function generateSetupUriFromSettings(
settingsFile: string,
setupPassphrase: string,
preserveRemoteSettings = false
): Promise<string> {
const script = [ const script = [
"import { fs } from '@vrtmrz/livesync-commonlib/node';", "import { fs } from '@vrtmrz/livesync-commonlib/node';",
"import { encodeSettingsToSetupURI } from '@vrtmrz/livesync-commonlib/compat/API/processSetting';", "import { encodeSettingsToSetupURI } from '@vrtmrz/livesync-commonlib/compat/API/processSetting';",
@@ -21,13 +25,17 @@ export async function generateSetupUriFromSettings(settingsFile: string, setupPa
" const settingsPath = process.env.SETTINGS_FILE;", " const settingsPath = process.env.SETTINGS_FILE;",
" const passphrase = process.env.SETUP_PASSPHRASE;", " const passphrase = process.env.SETUP_PASSPHRASE;",
" const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf-8'));", " const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf-8'));",
" settings.couchDB_DBNAME = 'setup-put-cat-db';", ...(preserveRemoteSettings
" settings.couchDB_URI = 'http://127.0.0.1:5999';", ? []
" settings.couchDB_USER = 'dummy';", : [
" settings.couchDB_PASSWORD = 'dummy';", " settings.couchDB_DBNAME = 'setup-put-cat-db';",
" settings.liveSync = false;", " settings.couchDB_URI = 'http://127.0.0.1:5999';",
" settings.syncOnStart = false;", " settings.couchDB_USER = 'dummy';",
" settings.syncOnSave = false;", " settings.couchDB_PASSWORD = 'dummy';",
" settings.liveSync = false;",
" settings.syncOnStart = false;",
" settings.syncOnSave = false;",
]),
" const uri = await encodeSettingsToSetupURI(settings, passphrase);", " const uri = await encodeSettingsToSetupURI(settings, passphrase);",
" process.stdout.write(uri.trim());", " process.stdout.write(uri.trim());",
"})();", "})();",
+79 -3
View File
@@ -1,6 +1,11 @@
import { assert } from "@std/assert"; import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts"; import { TempDir } from "./helpers/temp.ts";
import { initSettingsFile, applyP2pSettings, applyP2pTestTweaks } from "./helpers/settings.ts"; import {
initSettingsFile,
applyP2pSettings,
applyP2pTestTweaks,
generateSetupUriFromSettings,
} from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts"; import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { import {
discoverPeer, discoverPeer,
@@ -9,10 +14,10 @@ import {
maybeStartCoturn, maybeStartCoturn,
stopCoturnIfStarted, stopCoturnIfStarted,
} from "./helpers/p2p.ts"; } from "./helpers/p2p.ts";
import { runCli } from "./helpers/cli.ts"; import { runCli, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts";
import { getOptimalLoopbackIp } from "./helpers/net.ts"; import { getOptimalLoopbackIp } from "./helpers/net.ts";
Deno.test("p2p-sync: discovers peer and completes sync", async () => { Deno.test("p2p-sync: transfers with the same ID key and rejects a different document ID key", async () => {
const loopbackIp = await getOptimalLoopbackIp(); const loopbackIp = await getOptimalLoopbackIp();
const loopbackHost = loopbackIp === "::1" ? "[::1]" : loopbackIp; const loopbackHost = loopbackIp === "::1" ? "[::1]" : loopbackIp;
@@ -32,14 +37,18 @@ Deno.test("p2p-sync: discovers peer and completes sync", async () => {
const hostSettings = workDir.join("settings-host.json"); const hostSettings = workDir.join("settings-host.json");
const clientVault = workDir.join("vault-sync"); const clientVault = workDir.join("vault-sync");
const clientSettings = workDir.join("settings-sync.json"); const clientSettings = workDir.join("settings-sync.json");
const rejectedVault = workDir.join("vault-rejected");
const rejectedSettings = workDir.join("settings-rejected.json");
await Deno.mkdir(hostVault, { recursive: true }); await Deno.mkdir(hostVault, { recursive: true });
await Deno.mkdir(clientVault, { recursive: true }); await Deno.mkdir(clientVault, { recursive: true });
await Deno.mkdir(rejectedVault, { recursive: true });
const relayStarted = await maybeStartLocalRelay(relay); const relayStarted = await maybeStartLocalRelay(relay);
const coturnStarted = await maybeStartCoturn(turnServers); const coturnStarted = await maybeStartCoturn(turnServers);
try { try {
await initSettingsFile(hostSettings); await initSettingsFile(hostSettings);
await initSettingsFile(clientSettings); await initSettingsFile(clientSettings);
await initSettingsFile(rejectedSettings);
await applyP2pSettings( await applyP2pSettings(
hostSettings, hostSettings,
roomId, roomId,
@@ -58,8 +67,52 @@ Deno.test("p2p-sync: discovers peer and completes sync", async () => {
"~.*", "~.*",
turnServers turnServers
); );
await applyP2pSettings(
rejectedSettings,
roomId,
passphrase,
"self-hosted-livesync-cli-tests",
relay,
"~.*",
turnServers
);
await applyP2pTestTweaks(hostSettings, hostPeerName, passphrase); await applyP2pTestTweaks(hostSettings, hostPeerName, passphrase);
await applyP2pTestTweaks(clientSettings, clientPeerName, passphrase); await applyP2pTestTweaks(clientSettings, clientPeerName, passphrase);
await applyP2pTestTweaks(rejectedSettings, "p2p-rejected-" + nonce, passphrase);
for (const [vault, path, key, label] of [
[hostVault, hostSettings, "ab".repeat(32), "host"],
[clientVault, clientSettings, "ab".repeat(32), "client"],
[rejectedVault, rejectedSettings, "cd".repeat(32), "rejected"],
]) {
const settings = JSON.parse(await Deno.readTextFile(path));
settings.idDerivationVersion = 1;
settings.idDerivationKey = key;
const sourcePath = workDir.join("setup-source-" + label + ".json");
await Deno.writeTextFile(sourcePath, JSON.stringify(settings));
const setupPassphrase = "independent-id-setup-passphrase";
const setupUri = await generateSetupUriFromSettings(sourcePath, setupPassphrase, true);
await runCliWithInputOrFail(setupPassphrase + "\n", vault, "--settings", path, "setup", setupUri);
const persisted = JSON.parse(await Deno.readTextFile(path));
assert(persisted.idDerivationVersion === 1, "The Setup URI lost the ID derivation version.");
assert(persisted.idDerivationKey === "", "The CLI stored the ID key in plain text.");
assert(
typeof persisted.encryptedIdDerivationKey === "string" && persisted.encryptedIdDerivationKey.length > 0,
"The CLI did not encrypt the saved ID key."
);
assert(persisted.P2P_Enabled === true, "The Setup URI disabled P2P.");
assert(persisted.P2P_roomID === roomId, "The Setup URI changed the P2P room.");
assert(persisted.P2P_relays === relay, "The Setup URI changed the P2P relay.");
assert(persisted.remoteType === "ONLY_P2P", "The Setup URI changed the remote type.");
}
const notePath = "p2p/independent-id-note.md";
await runCliWithInputOrFail(
"A note transferred with the saved ID key.\n",
clientVault,
"--settings",
clientSettings,
"put",
notePath
);
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host"); const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try { try {
@@ -82,9 +135,32 @@ Deno.test("p2p-sync: discovers peer and completes sync", async () => {
syncResult.code === 0, syncResult.code === 0,
`p2p-sync failed\nstdout: ${syncResult.stdout}\nstderr: ${syncResult.stderr}` `p2p-sync failed\nstdout: ${syncResult.stdout}\nstderr: ${syncResult.stderr}`
); );
const rejectedPeer = await discoverPeer(rejectedVault, rejectedSettings, peersTimeout, hostPeerName);
const rejectedSync = await runCli(
rejectedVault,
"--settings",
rejectedSettings,
"p2p-sync",
rejectedPeer.id,
String(syncTimeout)
);
assert(
rejectedSync.code !== 0,
`P2P accepted a different key for obfuscated document IDs.\nstdout: ${rejectedSync.stdout}\nstderr: ${rejectedSync.stderr}`
);
assert(
rejectedSync.combined.includes("Tweak values are not matched"),
`P2P failed before checking peer settings.\nstdout: ${rejectedSync.stdout}\nstderr: ${rejectedSync.stderr}`
);
} finally { } finally {
await host.stop(); await host.stop();
} }
const received = sanitiseCatStdout(
await runCliOrFail(hostVault, "--settings", hostSettings, "cat", notePath)
).trimEnd();
assert(received === "A note transferred with the saved ID key.", "The host did not receive the keyed note.");
const rejectedRead = await runCli(rejectedVault, "--settings", rejectedSettings, "cat", notePath);
assert(rejectedRead.code !== 0, "The rejected device received the keyed note.");
} finally { } finally {
await stopLocalRelayIfStarted(relayStarted); await stopLocalRelayIfStarted(relayStarted);
await stopCoturnIfStarted(coturnStarted); await stopCoturnIfStarted(coturnStarted);
@@ -207,6 +207,50 @@ export const liveSyncProvisionalEnglishMessages = {
"Repair failed before the source was removed. Run inspection again before retrying.", "Repair failed before the source was removed. Run inspection again before retrying.",
"Connection settings": "Connection settings", "Connection settings": "Connection settings",
"Saved connections": "Saved connections", "Saved connections": "Saved connections",
"ID generation": "ID generation",
"Keep current configuration": "Keep current configuration",
"Set an ID key": "Set an ID key",
"Current configuration: a saved ID key is used.": "Current configuration: a saved ID key is used.",
"Current configuration: the saved ID key is retained while E2EE is off.":
"Current configuration: the saved ID key is retained while E2EE is off.",
"Current configuration: no ID key is saved. With E2EE enabled, keeping it uses legacy IDs tied to the E2EE passphrase.":
"Current configuration: no ID key is saved. With E2EE enabled, keeping it uses legacy IDs tied to the E2EE passphrase.",
"Changing the E2EE passphrase changes IDs generated by the legacy configuration.":
"Changing the E2EE passphrase changes IDs generated by the legacy configuration.",
"This uses a saved key for new Chunk IDs and obfuscated Metadata document IDs, so changing the E2EE passphrase does not derive a new key automatically.":
"This uses a saved key for new Chunk IDs and obfuscated Metadata document IDs, so changing the E2EE passphrase does not derive a new key automatically.",
Configured: "Configured",
"The saved ID key is configured. Its source cannot be shown again.":
"The saved ID key is configured. Its source cannot be shown again.",
"Leave this input empty to keep the saved ID key.": "Leave this input empty to keep the saved ID key.",
"Generate a random ID key": "Generate a random ID key",
"How to set the ID key": "How to set the ID key",
"Derive from current E2EE passphrase": "Derive from current E2EE passphrase",
"Enter an ID source": "Enter an ID source",
"Import an ID recovery code": "Import an ID recovery code",
"ID source": "ID source",
"ID recovery code": "ID recovery code",
"Enter an ID recovery code": "Enter an ID recovery code",
"Choose a long, unpredictable source. It is used once and cannot be shown again after saving. A recovery code can be displayed on this device later. This input also accepts a tagged recovery code.":
"Choose a long, unpredictable source. It is used once and cannot be shown again after saving. A recovery code can be displayed on this device later. This input also accepts a tagged recovery code.",
"Paste a tagged recovery code from an existing device to restore the same ID key.":
"Paste a tagged recovery code from an existing device to restore the same ID key.",
"For recovery after losing every device, save the recovery code after setup or choose an ID source you can reproduce.":
"For recovery after losing every device, save the recovery code after setup or choose an ID source you can reproduce.",
"Show current recovery code": "Show current recovery code",
"Hide current recovery code": "Hide current recovery code",
"Current ID recovery code": "Current ID recovery code",
"Copy recovery code": "Copy recovery code",
"Recovery code copied.": "Recovery code copied.",
"The displayed recovery code belongs to the current key. Reopen this dialogue after saving to copy the replacement key.":
"The displayed recovery code belongs to the current key. Reopen this dialogue after saving to copy the replacement key.",
"The recovery code could not be copied. Select and copy the visible code instead.":
"The recovery code could not be copied. Select and copy the visible code instead.",
"The ID key is derived from the current E2EE passphrase and saved separately. Changing that passphrase later does not change the saved ID key. To reduce the risk of guessing that passphrase from known IDs, use a separate, unpredictable ID source instead.":
"The ID key is derived from the current E2EE passphrase and saved separately. Changing that passphrase later does not change the saved ID key. To reduce the risk of guessing that passphrase from known IDs, use a separate, unpredictable ID source instead.",
"An ID source is required to enable this option.": "An ID source is required to enable this option.",
"The ID source or recovery code is invalid. Check it and try again.":
"The ID source or recovery code is invalid. Check it and try again.",
} as const; } as const;
export type LiveSyncProvisionalMessageKey = keyof typeof liveSyncProvisionalEnglishMessages; export type LiveSyncProvisionalMessageKey = keyof typeof liveSyncProvisionalEnglishMessages;
@@ -43,7 +43,10 @@ function projectHeaders(value: string): readonly (readonly [name: string, value:
} }
function projectRemoteSecurity(settings: RemoteDBSettings) { function projectRemoteSecurity(settings: RemoteDBSettings) {
return settings.encrypt return [
settings.idDerivationVersion,
settings.idDerivationKey,
settings.encrypt
? ([ ? ([
"encrypted", "encrypted",
settings.passphrase, settings.passphrase,
@@ -51,7 +54,8 @@ function projectRemoteSecurity(settings: RemoteDBSettings) {
settings.E2EEAlgorithm, settings.E2EEAlgorithm,
settings.permitEmptyPassphrase, settings.permitEmptyPassphrase,
] as const) ] as const)
: (["plain"] as const); : (["plain"] as const),
] as const;
} }
/** /**
@@ -31,6 +31,17 @@ describe("active Replicator configuration identity", () => {
}); });
} }
it("replaces a connection when the independent ID key changes", () => {
const first = configuredSettings({ idDerivationVersion: 1, idDerivationKey: "a".repeat(64) });
const second = { ...first, idDerivationKey: "b".repeat(64) };
expect(getCouchDBReplicatorConfigurationIdentity(second)).not.toBe(
getCouchDBReplicatorConfigurationIdentity(first)
);
expect(getObjectStorageReplicatorConfigurationIdentity(second)).not.toBe(
getObjectStorageReplicatorConfigurationIdentity(first)
);
});
it.each([ it.each([
["couchDB_URI", "https://other.example.test/base"], ["couchDB_URI", "https://other.example.test/base"],
["couchDB_DBNAME", "other-vault"], ["couchDB_DBNAME", "other-vault"],
+2
View File
@@ -80,6 +80,8 @@ export async function generateReport(settings: ObsidianLiveSyncSettings, core: L
pluginConfig.couchDB_USER = REDACTED; pluginConfig.couchDB_USER = REDACTED;
pluginConfig.passphrase = REDACTED; pluginConfig.passphrase = REDACTED;
pluginConfig.encryptedPassphrase = REDACTED; pluginConfig.encryptedPassphrase = REDACTED;
pluginConfig.idDerivationKey = REDACTED;
pluginConfig.encryptedIdDerivationKey = REDACTED;
pluginConfig.encryptedCouchDBConnection = REDACTED; pluginConfig.encryptedCouchDBConnection = REDACTED;
pluginConfig.accessKey = REDACTED; pluginConfig.accessKey = REDACTED;
pluginConfig.secretKey = REDACTED; pluginConfig.secretKey = REDACTED;
+16
View File
@@ -10,6 +10,22 @@ vi.mock("@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions", () => ({
})); }));
describe("TURN credentials in diagnostic reports", () => { describe("TURN credentials in diagnostic reports", () => {
it("redacts the derived ID key and its encrypted local wrapper", async () => {
const key = "f3205cc41d24116d8c2484993c9d9a2e667373af338ba02f2ee71199adb82f2e";
const wrapper = "encrypted-id-key-test-wrapper";
const settings = {
...DEFAULT_SETTINGS,
idDerivationVersion: 1 as const,
idDerivationKey: key,
encryptedIdDerivationKey: wrapper,
};
const core = { services: { vault: { isStorageInsensitive: () => false } } } as unknown as LiveSyncBaseCore;
const report = await generateReport(settings, core);
const text = JSON.stringify(report);
expect(text).not.toContain(key);
expect(text).not.toContain(wrapper);
});
it("redacts provider tokens in all profiles and runtime credentials", async () => { it("redacts provider tokens in all profiles and runtime credentials", async () => {
const token = "private+token/with=symbols"; const token = "private+token/with=symbols";
const provider = { P2P_managedType: "CF", P2P_managedId: "private-key", P2P_managedToken: token }; const provider = { P2P_managedType: "CF", P2P_managedId: "private-key", P2P_managedToken: token };
@@ -1,9 +1,6 @@
import type { ObsidianLiveSyncSettings, SettingsMigrationState } from "@vrtmrz/livesync-commonlib/settings"; import type { ObsidianLiveSyncSettings, SettingsMigrationState } from "@vrtmrz/livesync-commonlib/settings";
import type { CompatibilityPause } from "@/common/databaseCompatibility.ts"; import type { CompatibilityPause } from "@/common/databaseCompatibility.ts";
import type { import type { ReviewHarnessScenarioResult, ReviewHarnessScenarioStatus } from "./reviewHarnessTypes";
ReviewHarnessScenarioResult,
ReviewHarnessScenarioStatus,
} from "./reviewHarnessTypes";
export type { ReviewHarnessScenarioResult, ReviewHarnessScenarioStatus } from "./reviewHarnessTypes"; export type { ReviewHarnessScenarioResult, ReviewHarnessScenarioStatus } from "./reviewHarnessTypes";
@@ -32,6 +29,14 @@ export const REVIEW_HARNESS_SCENARIOS = [
mode: "automatic", mode: "automatic",
access: "dedicated-vault-fixtures", access: "dedicated-vault-fixtures",
}, },
{
id: "id-generation-performance",
title: "ID generation performance",
description:
"Measures legacy and independent IDs with fixed in-memory inputs. Reports time per 1,000 IDs and per ID, key derivation time, and JavaScript heap samples where available. Keep Obsidian in the foreground.",
mode: "automatic",
access: "read-only",
},
] as const; ] as const;
export const REVIEW_HARNESS_SCENARIO_IDS = REVIEW_HARNESS_SCENARIOS.map(({ id }) => id); export const REVIEW_HARNESS_SCENARIO_IDS = REVIEW_HARNESS_SCENARIOS.map(({ id }) => id);
@@ -114,7 +119,9 @@ const NEW_VAULT_RECOMMENDATION_KEYS = [
"E2EEAlgorithm", "E2EEAlgorithm",
] as const; ] as const;
type LifecycleSettingKey = (typeof PRESERVED_SYNC_SETTING_KEYS)[number] | (typeof NEW_VAULT_RECOMMENDATION_KEYS)[number]; type LifecycleSettingKey =
| (typeof PRESERVED_SYNC_SETTING_KEYS)[number]
| (typeof NEW_VAULT_RECOMMENDATION_KEYS)[number];
type SettingsForLifecycleInspection = Partial<Pick<ObsidianLiveSyncSettings, LifecycleSettingKey>>; type SettingsForLifecycleInspection = Partial<Pick<ObsidianLiveSyncSettings, LifecycleSettingKey>>;
export function inspectSettingsLifecycle(input: { export function inspectSettingsLifecycle(input: {
@@ -130,9 +137,7 @@ export function inspectSettingsLifecycle(input: {
}; };
} }
const invalidSyncSettings = PRESERVED_SYNC_SETTING_KEYS.filter( const invalidSyncSettings = PRESERVED_SYNC_SETTING_KEYS.filter((key) => typeof input.settings[key] !== "boolean");
(key) => typeof input.settings[key] !== "boolean"
);
if (invalidSyncSettings.length > 0) { if (invalidSyncSettings.length > 0) {
return { return {
status: "failed", status: "failed",
@@ -205,6 +210,7 @@ export interface ReviewHarnessReportScenario {
readonly mode: ReviewHarnessScenarioMode; readonly mode: ReviewHarnessScenarioMode;
readonly status: ReviewHarnessScenarioStatus; readonly status: ReviewHarnessScenarioStatus;
readonly detail: string; readonly detail: string;
readonly observations?: readonly string[];
} }
export interface ReviewHarnessReportInput { export interface ReviewHarnessReportInput {
@@ -248,13 +254,15 @@ export function formatReviewHarnessReport(input: ReviewHarnessReportInput): stri
); );
const scenarios = table( const scenarios = table(
["Scenario", "Mode", "Status", "Detail"], ["Scenario", "Mode", "Status", "Detail"],
input.scenarios.map(({ id, title, mode, status, detail }) => [ input.scenarios.map(({ id, title, mode, status, detail }) => [`${title} (${id})`, mode, status, detail])
`${title} (${id})`,
mode,
status,
detail,
])
); );
const observations = input.scenarios
.filter((scenario) => scenario.observations?.length)
.map(
({ title, observations }) =>
`### ${title}\n\n${observations!.map((value) => `- ${tableCell(value)}`).join("\n")}`
)
.join("\n\n");
return `## Self-hosted LiveSync Review Harness report return `## Self-hosted LiveSync Review Harness report
Generated at \`${tableCell(input.generatedAt)}\`. Generated at \`${tableCell(input.generatedAt)}\`.
@@ -267,6 +275,8 @@ ${environment}
${scenarios} ${scenarios}
${observations}
<details> <details>
<summary>Event transcript</summary> <summary>Event transcript</summary>
@@ -75,6 +75,7 @@ describe("Review Harness contract", () => {
"settings-lifecycle", "settings-lifecycle",
"compatibility-review", "compatibility-review",
"vault-round-trip", "vault-round-trip",
"id-generation-performance",
]); ]);
}); });
@@ -21,6 +21,7 @@ export interface ReviewHarnessRuntime {
getCompatibilityPause(): CompatibilityPause | undefined; getCompatibilityPause(): CompatibilityPause | undefined;
openCompatibilityReview(): Promise<void>; openCompatibilityReview(): Promise<void>;
runVaultRoundTrip(): Promise<ReviewHarnessScenarioResult>; runVaultRoundTrip(): Promise<ReviewHarnessScenarioResult>;
runIdBenchmark(): Promise<ReviewHarnessScenarioResult>;
readContinuation(): string | null; readContinuation(): string | null;
writeContinuation(value: string): void; writeContinuation(value: string): void;
deleteContinuation(): void; deleteContinuation(): void;
@@ -159,6 +160,8 @@ export class ReviewHarnessController {
}); });
} else if (id === "vault-round-trip") { } else if (id === "vault-round-trip") {
result = await this.runtime.runVaultRoundTrip(); result = await this.runtime.runVaultRoundTrip();
} else if (id === "id-generation-performance") {
result = await this.runtime.runIdBenchmark();
} else { } else {
const inspection = this.inspectCompatibilityReview(); const inspection = this.inspectCompatibilityReview();
result = result =
@@ -206,10 +209,7 @@ export class ReviewHarnessController {
detail: "The device-local compatibility review remains pending.", detail: "The device-local compatibility review remains pending.",
observations: inspection.observations, observations: inspection.observations,
}; };
this.record( this.record("compatibility-review-updated", this.results["compatibility-review"].status);
"compatibility-review-updated",
this.results["compatibility-review"].status
);
} catch (error) { } catch (error) {
this.setUnexpectedFailure("compatibility-review", error); this.setUnexpectedFailure("compatibility-review", error);
} finally { } finally {
@@ -259,6 +259,7 @@ export class ReviewHarnessController {
mode, mode,
status: this.results[id].status, status: this.results[id].status,
detail: this.results[id].detail, detail: this.results[id].detail,
observations: this.results[id].observations,
})), })),
transcript: this.transcript, transcript: this.transcript,
}); });
@@ -80,6 +80,11 @@ function createRuntime(): ReviewHarnessRuntime & {
detail: "The owned fixture tree was exercised and removed.", detail: "The owned fixture tree was exercised and removed.",
observations: [], observations: [],
})), })),
runIdBenchmark: vi.fn(async () => ({
status: "passed" as const,
detail: "ID generation measurements completed.",
observations: ["Chunk 256 B: 1000 IDs total=43.00 ms; per ID=0.0430 ms"],
})),
readContinuation() { readContinuation() {
return this.continuation; return this.continuation;
}, },
@@ -150,6 +155,60 @@ describe("ReviewHarnessController", () => {
expect(runtime.reportError).toHaveBeenCalledOnce(); expect(runtime.reportError).toHaveBeenCalledOnce();
}); });
it("runs ID measurements on request and includes their units in the copied report", async () => {
const runtime = createRuntime();
const controller = new ReviewHarnessController(runtime);
await controller.runAutomaticScenarios();
expect(runtime.runIdBenchmark).not.toHaveBeenCalled();
await controller.runScenario("id-generation-performance");
await controller.copyReport();
expect(runtime.runIdBenchmark).toHaveBeenCalledOnce();
expect(controller.snapshot().results["id-generation-performance"].status).toBe("passed");
expect(vi.mocked(runtime.copyText).mock.calls[0][0]).toContain("1000 IDs total=43.00 ms; per ID=0.0430 ms");
expect(runtime.runVaultRoundTrip).not.toHaveBeenCalled();
expect(runtime.events).toEqual([]);
expect(runtime.continuation).toBeNull();
});
it("excludes an unexpected measurement error from the copied report", async () => {
const runtime = createRuntime();
runtime.runIdBenchmark = vi.fn().mockRejectedValue(new Error("private measurement error"));
const controller = new ReviewHarnessController(runtime);
await controller.runScenario("id-generation-performance");
expect(controller.snapshot().results["id-generation-performance"].status).toBe("failed");
expect(controller.createReport()).not.toContain("private measurement error");
expect(runtime.reportError).toHaveBeenCalledOnce();
});
it("does not overlap an ID measurement with another scenario", async () => {
const runtime = createRuntime();
let finish!: () => void;
const pending = new Promise<void>((resolve) => {
finish = resolve;
});
runtime.runIdBenchmark = vi.fn(async () => {
await pending;
return { status: "passed" as const, detail: "Measured", observations: [] };
});
const controller = new ReviewHarnessController(runtime);
const running = controller.runScenario("id-generation-performance");
await controller.runScenario("id-generation-performance");
await controller.runScenario("vault-round-trip");
expect(runtime.runIdBenchmark).toHaveBeenCalledOnce();
expect(runtime.runVaultRoundTrip).not.toHaveBeenCalled();
expect(controller.snapshot().running).toBe(true);
finish();
await running;
expect(controller.snapshot().running).toBe(false);
});
it("deletes a one-shot continuation before exposing the resumed guided step", () => { it("deletes a one-shot continuation before exposing the resumed guided step", () => {
const runtime = createRuntime(); const runtime = createRuntime();
runtime.continuation = JSON.stringify({ runtime.continuation = JSON.stringify({
@@ -167,9 +226,7 @@ describe("ReviewHarnessController", () => {
expect(controller.snapshot().results["compatibility-review"]).toMatchObject({ expect(controller.snapshot().results["compatibility-review"]).toMatchObject({
status: "waiting-for-user", status: "waiting-for-user",
}); });
expect(controller.snapshot().resumedRequestId).toBe( expect(controller.snapshot().resumedRequestId).toBe("compatibility-review-2026-07-18T11:59:00.000Z");
"compatibility-review-2026-07-18T11:59:00.000Z"
);
}); });
it("does not copy rejected continuation values into the report", () => { it("does not copy rejected continuation values into the report", () => {
@@ -0,0 +1,109 @@
import type { ReviewHarnessScenarioResult } from "./reviewHarnessTypes";
export interface IdBenchmarkOperations {
deriveKey(): Promise<unknown>;
chunkId(piece: string, independent: boolean): Promise<string>;
documentId(path: string, independent: boolean): Promise<string>;
}
type BenchmarkPerformance = Pick<Performance, "now"> & {
readonly memory?: { readonly usedJSHeapSize: number };
};
const ID_COUNT = 1000;
const SAMPLES = 3;
const BATCH_SIZE = 100;
const WARMUP_COUNT = 32;
function readHeap(clock: BenchmarkPerformance): number | undefined {
try {
const bytes = clock.memory?.usedJSHeapSize;
return typeof bytes === "number" && Number.isFinite(bytes) && bytes >= 0 ? bytes : undefined;
} catch {
return undefined;
}
}
function summary(samples: readonly number[]): string {
const sorted = [...samples].sort((a, b) => a - b);
return `median=${sorted[1].toFixed(2)} ms; range=${sorted[0].toFixed(2)}–${sorted[2].toFixed(2)} ms`;
}
export async function runReviewHarnessIdBenchmark(
operations: IdBenchmarkOperations,
clock: BenchmarkPerformance = performance,
yieldControl: () => Promise<void> = () => new Promise((resolve) => window.setTimeout(resolve, 0))
): Promise<ReviewHarnessScenarioResult> {
const before = readHeap(clock);
let highest = before;
const sampleHeap = () => {
const value = readHeap(clock);
if (value !== undefined) highest = Math.max(highest ?? value, value);
return value;
};
const observations = [
"Fixed synthetic inputs; 3 samples, alternating legacy/independent order; 32 warm-up IDs per sample. Legacy Chunk algorithm: xxhash64.",
"Compute timings include input construction and awaited ID generation. Initialisation, warm-up, and pauses between batches are excluded. This does not measure a Rebuild or remote transfer.",
];
const derivationSamples: number[] = [];
for (let sample = 0; sample < SAMPLES; sample++) {
await yieldControl();
const started = clock.now();
await operations.deriveKey();
derivationSamples.push(clock.now() - started);
sampleHeap();
}
observations.push(`ID key derivation at save time: ${summary(derivationSamples)} per derivation.`);
const cases = [
...[256, 4096, 32768].map((bytes) => {
const prefix = "r".repeat(bytes - 8);
return {
label: `Chunk IDs, ${bytes} B`,
run: (i: number, independent: boolean) =>
operations.chunkId(prefix + i.toString(36).padStart(8, "0"), independent),
};
}),
{
label: "Obfuscated document IDs",
run: (i: number, independent: boolean) => operations.documentId(`benchmark/path-${i}.md`, independent),
},
];
for (const scenario of cases) {
const samples: [number[], number[]] = [[], []];
for (let sample = 0; sample < SAMPLES; sample++) {
for (const independent of sample % 2 === 0 ? [false, true] : [true, false]) {
for (let i = 0; i < WARMUP_COUNT; i++) await scenario.run(i, independent);
let elapsed = 0;
for (let batch = 0; batch < ID_COUNT; batch += BATCH_SIZE) {
await yieldControl();
const started = clock.now();
for (let i = batch; i < batch + BATCH_SIZE; i++) await scenario.run(i, independent);
elapsed += clock.now() - started;
sampleHeap();
}
samples[independent ? 1 : 0].push(elapsed);
}
}
for (const [index, values] of samples.entries()) {
const median = [...values].sort((a, b) => a - b)[1];
observations.push(
`${scenario.label}, ${index === 0 ? "legacy" : "independent"}: ${ID_COUNT} IDs total ${summary(values)}; per ID=${(median / ID_COUNT).toFixed(4)} ms.`
);
}
}
const after = sampleHeap();
if (highest === undefined) {
observations.push("JavaScript heap: unavailable on this device.");
} else {
const mib = (bytes: number | undefined) =>
bytes === undefined ? "unavailable" : `${(bytes / 1048576).toFixed(2)} MiB`;
observations.push(
`JavaScript heap: before=${mib(before)}; highest sampled=${mib(highest)}; after=${mib(after)}.`
);
}
observations.push(
"Heap samples are approximate, may include other Obsidian work, and are affected by garbage collection. They are neither total app RAM nor a true peak."
);
return { status: "passed", detail: "ID generation measurements completed.", observations };
}
@@ -0,0 +1,99 @@
import { describe, expect, it } from "vitest";
import { runReviewHarnessIdBenchmark, type IdBenchmarkOperations } from "./reviewHarnessIdBenchmark";
function fixture() {
let elapsed = 0;
let derivations = 0;
const chunkCounts = [0, 0];
const documentCounts = [0, 0];
const chunkSizes = new Set<number>();
const operations: IdBenchmarkOperations = {
deriveKey: () => {
derivations++;
elapsed += 42;
return Promise.resolve("private-derived-key");
},
chunkId: (piece, independent) => {
chunkCounts[independent ? 1 : 0]++;
chunkSizes.add(piece.length);
elapsed += independent ? 2 : 1;
return Promise.resolve("private-chunk-id");
},
documentId: (_path, independent) => {
documentCounts[independent ? 1 : 0]++;
elapsed += independent ? 4 : 3;
return Promise.resolve("private-document-id");
},
};
return {
operations,
now: () => elapsed,
yieldControl: () => {
elapsed += 100;
return Promise.resolve();
},
counts: () => ({ derivations, chunkCounts, documentCounts, chunkSizes: [...chunkSizes] }),
};
}
describe("Review Harness ID measurements", () => {
it("reports totals and per-ID timings separately, excluding warm-up and cooperative pauses", async () => {
const f = fixture();
const result = await runReviewHarnessIdBenchmark(f.operations, { now: f.now }, f.yieldControl);
const report = result.observations.join("\n");
expect(result.status).toBe("passed");
expect(report).toContain("1000 IDs total median=1000.00 ms; range=1000.00–1000.00 ms; per ID=1.0000 ms");
expect(report).toContain("1000 IDs total median=2000.00 ms; range=2000.00–2000.00 ms; per ID=2.0000 ms");
expect(report).toContain("Obfuscated document IDs, legacy: 1000 IDs total median=3000.00 ms");
expect(report).toContain("Obfuscated document IDs, independent: 1000 IDs total median=4000.00 ms");
expect(report).toContain("ID key derivation at save time: median=42.00 ms");
expect(report).toContain("JavaScript heap: unavailable on this device.");
expect(report).not.toContain("private-");
expect(f.counts()).toEqual({
derivations: 3,
chunkCounts: [9288, 9288],
documentCounts: [3096, 3096],
chunkSizes: [256, 4096, 32768],
});
});
it("labels the highest sampled heap separately from total app RAM and allows a lower final sample", async () => {
const f = fixture();
let reads = 0;
const clock = {
now: f.now,
get memory() {
return { usedJSHeapSize: (reads++ === 0 ? 2 : reads === 2 ? 5 : 1) * 1048576 };
},
};
const result = await runReviewHarnessIdBenchmark(f.operations, clock, f.yieldControl);
expect(result.observations).toContain(
"JavaScript heap: before=2.00 MiB; highest sampled=5.00 MiB; after=1.00 MiB."
);
expect(result.observations.join("\n")).toContain("neither total app RAM nor a true peak");
});
it.each([Number.NaN, Number.POSITIVE_INFINITY, -1, "throws"])(
"keeps timings usable when the heap API returns %s",
async (value) => {
const f = fixture();
const result = await runReviewHarnessIdBenchmark(
f.operations,
{
now: f.now,
get memory() {
if (value === "throws") throw new Error("Heap API unavailable");
return { usedJSHeapSize: value as number };
},
},
f.yieldControl
);
expect(result.status).toBe("passed");
expect(result.observations).toContain("JavaScript heap: unavailable on this device.");
expect(result.observations.join("\n")).not.toMatch(/NaN|Infinity|private-/u);
}
);
});
@@ -0,0 +1,35 @@
import { DEFAULT_SETTINGS, deriveIdKey } from "@vrtmrz/livesync-commonlib/settings";
import { path2id_base } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { HashManager } from "@vrtmrz/livesync-commonlib/hashing";
import type { IdBenchmarkOperations } from "./reviewHarnessIdBenchmark";
const FIXTURE_PASSPHRASE = "Self-hosted LiveSync ID benchmark passphrase";
const FIXTURE_SOURCE = "Self-hosted LiveSync ID benchmark source";
const FIXTURE_KEY = "ab".repeat(32);
export async function createIdBenchmarkOperations(): Promise<IdBenchmarkOperations> {
const managers: HashManager[] = [];
for (const independent of [false, true]) {
const settings = Object.freeze({
...DEFAULT_SETTINGS,
encrypt: true,
passphrase: FIXTURE_PASSPHRASE,
hashAlg: "xxhash64" as const,
idDerivationVersion: independent ? (1 as const) : (0 as const),
idDerivationKey: independent ? FIXTURE_KEY : "",
});
// HashManager only reads currentSettings; this fixture has no storage or live service access.
const settingService = { currentSettings: () => settings } as HashManager["options"]["settingService"];
const manager = new HashManager({ settingService });
if (!(await manager.initialise())) throw new Error("The benchmark hash manager could not initialise.");
managers.push(manager);
}
return {
deriveKey: () => deriveIdKey(FIXTURE_SOURCE),
chunkId: (piece, independent) => managers[independent ? 1 : 0].computeHash(piece),
// Fixture paths are already normalised; use the same ID calculation as PathService.
documentId: (path, independent) =>
path2id_base(path as FilePath, FIXTURE_PASSPHRASE, false, independent ? FIXTURE_KEY : undefined),
};
}
@@ -0,0 +1,38 @@
import { describe, expect, it, vi } from "vitest";
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/settings";
import { createIdBenchmarkOperations } from "./reviewHarnessIdBenchmarkRuntime";
describe("Review Harness benchmark implementation", () => {
it("uses the packaged legacy and independent algorithms with isolated fixed settings", async () => {
const originalDefaults = structuredClone(DEFAULT_SETTINGS);
const fetch = vi.spyOn(globalThis, "fetch").mockRejectedValue(new Error("Network access is forbidden"));
try {
const operations = await createIdBenchmarkOperations();
const chunk = "r".repeat(256);
const legacy = await operations.chunkId(chunk, false);
const independent = await operations.chunkId(chunk, true);
expect(legacy).toMatch(/^\+[0-9a-z]{1,13}$/u);
expect(independent).toMatch(/^\+[0-9a-f]{64}$/u);
expect(independent).toBe("+9223e53d99e80c29effee9e95e38ed168d13c14f717054f9e996a1cd0a597000");
expect(await operations.chunkId(chunk, false)).toBe(legacy);
expect(await operations.chunkId(chunk, true)).toBe(independent);
expect(await operations.chunkId("s".repeat(256), true)).not.toBe(independent);
const legacyPath = await operations.documentId("benchmark/path-1.md", false);
const independentPath = await operations.documentId("benchmark/path-1.md", true);
expect(legacyPath).toMatch(/^f:[0-9a-f]{64}$/u);
expect(independentPath).toMatch(/^f:[0-9a-f]{64}$/u);
expect(legacyPath).not.toBe(independentPath);
expect(await operations.documentId("benchmark/path-1.md", true)).toBe(independentPath);
const second = await createIdBenchmarkOperations();
expect(await second.chunkId(chunk, true)).toBe(independent);
expect(await operations.deriveKey()).toMatch(/^[0-9a-f]{64}$/u);
expect(fetch).not.toHaveBeenCalled();
expect(DEFAULT_SETTINGS).toEqual(originalDefaults);
} finally {
fetch.mockRestore();
}
});
});
@@ -99,6 +99,17 @@ function resolutionSettingsSignature(settings: ObsidianLiveSyncSettings): string
} }
export class ModuleResolvingMismatchedTweaks extends AbstractModule { export class ModuleResolvingMismatchedTweaks extends AbstractModule {
private requiresIdConfigurationReview(assessment: TweakAssessment): boolean {
if (!assessment.entries.some(({ key, relation }) => key === "idDerivationVersion" && relation === "different")) {
return false;
}
Logger(
"The document ID configurations differ. Import the correct Setup URI, or configure the matching ID key, before synchronising.",
LOG_LEVEL_NOTICE
);
return true;
}
private _selectNewerTweakSide(current: TweakValues, preferred: Partial<TweakValues>): "REMOTE" | "CURRENT" { private _selectNewerTweakSide(current: TweakValues, preferred: Partial<TweakValues>): "REMOTE" | "CURRENT" {
Logger(`Modified: ${current.tweakModified} (current) vs ${preferred.tweakModified} (preferred)`); Logger(`Modified: ${current.tweakModified} (current) vs ${preferred.tweakModified} (preferred)`);
const currentModified = current.tweakModified; const currentModified = current.tweakModified;
@@ -196,6 +207,7 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
assessment = assessTweakCompatibility(this.settings, preferred) assessment = assessTweakCompatibility(this.settings, preferred)
): Promise<[TweakValues | boolean, boolean]> { ): Promise<[TweakValues | boolean, boolean]> {
if (assessment.alignment === "matched") return [false, false]; if (assessment.alignment === "matched") return [false, false];
if (this.requiresIdConfigurationReview(assessment)) return [false, false];
const acceptedSettings = settingsAfterAdoption(assessment, "adoptPreferred"); const acceptedSettings = settingsAfterAdoption(assessment, "adoptPreferred");
const autoAcceptSide = await this._shouldAutoAcceptCompatibleLossy(assessment); const autoAcceptSide = await this._shouldAutoAcceptCompatibleLossy(assessment);
if (autoAcceptSide === "REMOTE") return [acceptedSettings, false]; if (autoAcceptSide === "REMOTE") return [acceptedSettings, false];
@@ -363,6 +375,7 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
const trialSignature = JSON.stringify(trialSetting); const trialSignature = JSON.stringify(trialSetting);
const currentSignature = resolutionSettingsSignature(this.settings); const currentSignature = resolutionSettingsSignature(this.settings);
const assessment = assessTweakCompatibility(trialSetting, preferred); const assessment = assessTweakCompatibility(trialSetting, preferred);
if (this.requiresIdConfigurationReview(assessment)) return { result: false, requireFetch: false };
if (assessment.alignment === "matched") { if (assessment.alignment === "matched") {
this._log("The settings in the remote database are the same as the local database.", LOG_LEVEL_NOTICE); this._log("The settings in the remote database are the same as the local database.", LOG_LEVEL_NOTICE);
return { result: false, requireFetch: false }; return { result: false, requireFetch: false };
@@ -7,7 +7,7 @@ import {
type TweakValues, type TweakValues,
} from "@vrtmrz/livesync-commonlib/compat/common/types"; } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { extractObject } from "octagonal-wheels/object"; import { extractObject } from "octagonal-wheels/object";
import { assessTweakCompatibility } from "@vrtmrz/livesync-commonlib/settings"; import { assessTweakCompatibility, configuredIdKey } from "@vrtmrz/livesync-commonlib/settings";
import { ModuleResolvingMismatchedTweaks } from "./ModuleResolveMismatchedTweaks"; import { ModuleResolvingMismatchedTweaks } from "./ModuleResolveMismatchedTweaks";
import { setLang } from "@/common/translation"; import { setLang } from "@/common/translation";
import { import {
@@ -74,6 +74,68 @@ function createModule(settingsOverride: Partial<typeof DEFAULT_SETTINGS> = {}) {
} }
describe("ModuleResolvingMismatchedTweaks", () => { describe("ModuleResolvingMismatchedTweaks", () => {
it.each([0, 1] as const)(
"keeps ID configuration %s when automatically aligning Chunk settings",
async (idDerivationVersion) => {
const idDerivationKey = idDerivationVersion === 1 ? "ab".repeat(32) : "";
const { module, core, askSelectStringDialogue } = createModule({
encrypt: true,
usePathObfuscation: false,
idDerivationVersion,
idDerivationKey,
autoAcceptCompatibleTweak: true,
hashAlg: "xxhash64",
tweakModified: 1,
});
const preferred: TweakValues = {
...extractObject(TweakValuesTemplate, core.settings),
idDerivationVersion: idDerivationVersion === 1 ? 0 : 1,
hashAlg: "xxhash32",
tweakModified: 2,
};
core._services.tweakValue = {
checkAndAskResolvingMismatched: module._checkAndAskResolvingMismatchedTweaks.bind(module),
};
core._services.setting.saveSettingData.mockImplementation(async () => {
configuredIdKey(core.settings);
});
await expect(module._askResolvingMismatchedTweaks(preferred, async () => true)).resolves.toBe("CHECKAGAIN");
expect(core.settings).toMatchObject({ idDerivationVersion, idDerivationKey, hashAlg: "xxhash32" });
expect(askSelectStringDialogue).not.toHaveBeenCalled();
}
);
it.each(["active", "trial"] as const)(
"withholds ordinary tweak adoption for different document ID modes (%s)",
async (route) => {
const { module, core, askSelectStringDialogue } = createModule({
encrypt: true,
usePathObfuscation: true,
idDerivationVersion: 0,
idDerivationKey: "",
});
const preferred: TweakValues = {
...extractObject(TweakValuesTemplate, core.settings),
idDerivationVersion: 1,
};
if (route === "active") {
await expect(module._checkAndAskResolvingMismatchedTweaks(preferred)).resolves.toEqual([false, false]);
} else {
await expect(module._askUseRemoteConfiguration(core.settings, preferred)).resolves.toEqual({
result: false,
requireFetch: false,
});
}
expect(askSelectStringDialogue).not.toHaveBeenCalled();
expect(core._services.setting.saveSettingData).not.toHaveBeenCalled();
expect(core.settings).toMatchObject({ idDerivationVersion: 0, idDerivationKey: "" });
}
);
it("compatibility: offers ordinary application for a missing legacy filename-case setting", async () => { it("compatibility: offers ordinary application for a missing legacy filename-case setting", async () => {
const { module, askSelectStringDialogue } = createModule({ const { module, askSelectStringDialogue } = createModule({
autoAcceptCompatibleTweak: false, autoAcceptCompatibleTweak: false,
@@ -140,6 +140,8 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractModule {
settingToApply.couchDB_USER = this.settings.couchDB_USER; settingToApply.couchDB_USER = this.settings.couchDB_USER;
settingToApply.couchDB_PASSWORD = this.settings.couchDB_PASSWORD; settingToApply.couchDB_PASSWORD = this.settings.couchDB_PASSWORD;
settingToApply.passphrase = this.settings.passphrase; settingToApply.passphrase = this.settings.passphrase;
settingToApply.idDerivationVersion = this.settings.idDerivationVersion;
settingToApply.idDerivationKey = this.settings.idDerivationKey;
} }
const oldSetting = this.generateSettingForMarkdown( const oldSetting = this.generateSettingForMarkdown(
this.settings, this.settings,
@@ -203,11 +205,13 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractModule {
const saveData = { ...(settings ? settings : this.settings) } as Partial<ObsidianLiveSyncSettings>; const saveData = { ...(settings ? settings : this.settings) } as Partial<ObsidianLiveSyncSettings>;
delete saveData.encryptedCouchDBConnection; delete saveData.encryptedCouchDBConnection;
delete saveData.encryptedPassphrase; delete saveData.encryptedPassphrase;
delete saveData.encryptedIdDerivationKey;
delete saveData.additionalSuffixOfDatabaseName; delete saveData.additionalSuffixOfDatabaseName;
if (!saveData.writeCredentialsForSettingSync && !keepCredential) { if (!saveData.writeCredentialsForSettingSync && !keepCredential) {
delete saveData.couchDB_USER; delete saveData.couchDB_USER;
delete saveData.couchDB_PASSWORD; delete saveData.couchDB_PASSWORD;
delete saveData.passphrase; delete saveData.passphrase;
delete saveData.idDerivationKey;
delete saveData.jwtKey; delete saveData.jwtKey;
delete saveData.jwtKid; delete saveData.jwtKid;
delete saveData.jwtSub; delete saveData.jwtSub;
@@ -47,6 +47,12 @@ function getSettingsFromEditingSettings(editingSettings: AllSettings): ObsidianL
} }
return workObj; return workObj;
} }
function syncIdDerivationSettings(target: Partial<ObsidianLiveSyncSettings>, source: ObsidianLiveSyncSettings): void {
target.idDerivationVersion = source.idDerivationVersion;
target.idDerivationKey = source.idDerivationKey;
}
function createRemoteConfigurationId(): string { function createRemoteConfigurationId(): string {
return `remote-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}`; return `remote-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}`;
} }
@@ -116,6 +122,8 @@ export function paneRemoteConfig(
.onClick(async () => { .onClick(async () => {
const setupManager = this.core.getModule(SetupManager); const setupManager = this.core.getModule(SetupManager);
const originalSettings = getSettingsFromEditingSettings(this.editingSettings); const originalSettings = getSettingsFromEditingSettings(this.editingSettings);
const originalIdDerivationVersion = this.core.settings.idDerivationVersion;
const originalIdDerivationKey = this.core.settings.idDerivationKey;
const applied = await setupManager.onlyE2EEConfiguration(UserMode.Update, originalSettings); const applied = await setupManager.onlyE2EEConfiguration(UserMode.Update, originalSettings);
if (applied) { if (applied) {
this.editingSettings.encryptInternalMetadata = this.editingSettings.encryptInternalMetadata =
@@ -126,6 +134,16 @@ export function paneRemoteConfig(
} }
this.requestUpdate(); this.requestUpdate();
} }
if (
this.core.settings.idDerivationVersion !== originalIdDerivationVersion ||
this.core.settings.idDerivationKey !== originalIdDerivationKey
) {
syncIdDerivationSettings(this.editingSettings, this.core.settings);
if (this.initialSettings) {
syncIdDerivationSettings(this.initialSettings, this.core.settings);
}
this.requestUpdate();
}
updateE2EESummary(); updateE2EESummary();
}) })
.setButtonText("Configure") .setButtonText("Configure")
@@ -164,9 +182,11 @@ export function paneRemoteConfig(
const currentConfigs = cloneRemoteConfigurations(this.core.settings.remoteConfigurations); const currentConfigs = cloneRemoteConfigurations(this.core.settings.remoteConfigurations);
this.editingSettings.remoteConfigurations = currentConfigs; this.editingSettings.remoteConfigurations = currentConfigs;
this.editingSettings.activeConfigurationId = this.core.settings.activeConfigurationId; this.editingSettings.activeConfigurationId = this.core.settings.activeConfigurationId;
syncIdDerivationSettings(this.editingSettings, this.core.settings);
if (this.initialSettings) { if (this.initialSettings) {
this.initialSettings.remoteConfigurations = cloneRemoteConfigurations(currentConfigs); this.initialSettings.remoteConfigurations = cloneRemoteConfigurations(currentConfigs);
this.initialSettings.activeConfigurationId = this.core.settings.activeConfigurationId; this.initialSettings.activeConfigurationId = this.core.settings.activeConfigurationId;
syncIdDerivationSettings(this.initialSettings, this.core.settings);
} }
}; };
const persistRemoteConfigurations = async (synchroniseActiveRemote: boolean = false) => { const persistRemoteConfigurations = async (synchroniseActiveRemote: boolean = false) => {
@@ -254,6 +274,8 @@ export function paneRemoteConfig(
usePathObfuscation: this.editingSettings.usePathObfuscation, usePathObfuscation: this.editingSettings.usePathObfuscation,
encryptInternalMetadata: this.editingSettings.encryptInternalMetadata, encryptInternalMetadata: this.editingSettings.encryptInternalMetadata,
passphrase: this.editingSettings.passphrase, passphrase: this.editingSettings.passphrase,
idDerivationVersion: this.editingSettings.idDerivationVersion,
idDerivationKey: this.editingSettings.idDerivationKey,
configPassphraseStore: this.editingSettings.configPassphraseStore, configPassphraseStore: this.editingSettings.configPassphraseStore,
}); });
const addRemoteConfiguration = async () => { const addRemoteConfiguration = async () => {
@@ -194,4 +194,51 @@ describe("paneRemoteConfig", () => {
expect(host.initialSettings.encryptInternalMetadata).toBe(true); expect(host.initialSettings.encryptInternalMetadata).toBe(true);
expect(host.requestUpdate).toHaveBeenCalledOnce(); expect(host.requestUpdate).toHaveBeenCalledOnce();
}); });
it("copies applied ID derivation settings into both dialogue buffers", async () => {
const nextIdKey = "ab".repeat(32);
const originalSettings = {
encrypt: true,
passphrase: "passphrase",
E2EEAlgorithm: "v2",
usePathObfuscation: true,
encryptInternalMetadata: false,
idDerivationVersion: 0,
idDerivationKey: "",
remoteConfigurations: {},
};
const setupManager = {
onlyE2EEConfiguration: vi.fn(() => {
host.core.settings.idDerivationVersion = 1;
host.core.settings.idDerivationKey = nextIdKey;
return Promise.resolve(false);
}),
};
const host = {
editingSettings: { ...originalSettings },
initialSettings: { ...originalSettings },
core: {
settings: { ...originalSettings },
getModule: vi.fn(() => setupManager),
},
lifetimeComponent: { register: vi.fn() },
requestUpdate: vi.fn(),
};
const addPanel = vi.fn((_parent: HTMLElement, heading: string) => ({
then(callback: (paneEl: HTMLElement) => void) {
if (heading === "E2EE Configuration") {
callback(createPanelElement());
}
},
}));
paneRemoteConfig.call(host as never, {} as HTMLElement, { addPanel } as never);
await runtime.clickHandlers[0]();
expect(host.editingSettings.idDerivationVersion).toBe(1);
expect(host.editingSettings.idDerivationKey).toBe(nextIdKey);
expect(host.initialSettings.idDerivationVersion).toBe(1);
expect(host.initialSettings.idDerivationKey).toBe(nextIdKey);
expect(host.requestUpdate).toHaveBeenCalledOnce();
});
}); });
@@ -68,6 +68,7 @@ export function getE2EEConfigSummary(setting: ObsidianLiveSyncSettings, showAdva
export function getSummaryFromPartialSettings(setting: Partial<ObsidianLiveSyncSettings>, showAdvanced = false) { export function getSummaryFromPartialSettings(setting: Partial<ObsidianLiveSyncSettings>, showAdvanced = false) {
const outputSummary: Record<string, string> = {}; const outputSummary: Record<string, string> = {};
for (const key of Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[]) { for (const key of Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[]) {
if (key === "idDerivationKey" || key === "encryptedIdDerivationKey") continue;
const config = getConfig(key as AllSettingItemKey); const config = getConfig(key as AllSettingItemKey);
if (!config) continue; if (!config) continue;
if (config.isAdvanced && !showAdvanced) continue; if (config.isAdvanced && !showAdvanced) continue;
+36 -8
View File
@@ -1,6 +1,5 @@
import { import {
type BucketSyncSetting, type BucketSyncSetting,
type EncryptionSettings,
type ObsidianLiveSyncSettings, type ObsidianLiveSyncSettings,
type P2PSyncSetting, type P2PSyncSetting,
LOG_LEVEL_NOTICE, LOG_LEVEL_NOTICE,
@@ -36,6 +35,7 @@ import type {
SetupRemoteCouchDBResultType, SetupRemoteCouchDBResultType,
SetupRemoteCouchDBInitialData, SetupRemoteCouchDBInitialData,
SetupRemoteE2EEResultType, SetupRemoteE2EEResultType,
SetupRemoteE2EEInitialData,
SetupRemoteP2PInitialData, SetupRemoteP2PInitialData,
SetupRemoteP2PResultType, SetupRemoteP2PResultType,
SetupRemoteResultType, SetupRemoteResultType,
@@ -58,6 +58,20 @@ function copySettingsForRemoteProfileUpdate(settings: ObsidianLiveSyncSettings):
}; };
} }
function normaliseImportedIdDerivationSettings(settings: ObsidianLiveSyncSettings): ObsidianLiveSyncSettings {
// Setup URIs are complete imports even when their encoder omitted default-valued fields.
// Fill each missing half so a receiving device cannot supply the unrelated saved key.
return {
...settings,
idDerivationVersion: Object.prototype.hasOwnProperty.call(settings, "idDerivationVersion")
? settings.idDerivationVersion
: 0,
idDerivationKey: Object.prototype.hasOwnProperty.call(settings, "idDerivationKey")
? settings.idDerivationKey
: "",
};
}
/** /**
* User modes for onboarding and setup * User modes for onboarding and setup
*/ */
@@ -219,7 +233,7 @@ export class SetupManager extends AbstractModule {
return false; return false;
} }
this._log("Setup URI dialog closed.", LOG_LEVEL_VERBOSE); this._log("Setup URI dialog closed.", LOG_LEVEL_VERBOSE);
return await this.onConfirmApplySettingsFromWizard(newSetting, userMode); return await this.onConfirmApplySettingsFromWizard(normaliseImportedIdDerivationSettings(newSetting), userMode);
} }
/** /**
@@ -328,9 +342,12 @@ export class SetupManager extends AbstractModule {
* @returns * @returns
*/ */
async onlyE2EEConfiguration(userMode: UserMode, currentSetting: ObsidianLiveSyncSettings): Promise<boolean> { async onlyE2EEConfiguration(userMode: UserMode, currentSetting: ObsidianLiveSyncSettings): Promise<boolean> {
const e2eeConf = await this.dialogManager.openWithExplicitCancel<SetupRemoteE2EEResultType, EncryptionSettings>( const e2eeConf = await this.dialogManager.openWithExplicitCancel<
SetupRemoteE2EEResultType,
SetupRemoteE2EEInitialData
>(
SetupRemoteE2EE, SetupRemoteE2EE,
currentSetting { settings: currentSetting, newVault: userMode === UserMode.NewUser }
); );
if (e2eeConf === "cancelled") { if (e2eeConf === "cancelled") {
this._log("E2EE configuration cancelled.", LOG_LEVEL_NOTICE); this._log("E2EE configuration cancelled.", LOG_LEVEL_NOTICE);
@@ -341,7 +358,9 @@ export class SetupManager extends AbstractModule {
currentSetting.encrypt === e2eeConf.encrypt && currentSetting.encrypt === e2eeConf.encrypt &&
currentSetting.passphrase === e2eeConf.passphrase && currentSetting.passphrase === e2eeConf.passphrase &&
currentSetting.E2EEAlgorithm === e2eeConf.E2EEAlgorithm && currentSetting.E2EEAlgorithm === e2eeConf.E2EEAlgorithm &&
currentSetting.usePathObfuscation === e2eeConf.usePathObfuscation; currentSetting.usePathObfuscation === e2eeConf.usePathObfuscation &&
currentSetting.idDerivationVersion === e2eeConf.idDerivationVersion &&
currentSetting.idDerivationKey === e2eeConf.idDerivationKey;
if (userMode === UserMode.Update && onlyInternalMetadataPreferenceChanged) { if (userMode === UserMode.Update && onlyInternalMetadataPreferenceChanged) {
if (e2eeConf.encryptInternalMetadata && currentSetting.remoteType === REMOTE_COUCHDB) { if (e2eeConf.encryptInternalMetadata && currentSetting.remoteType === REMOTE_COUCHDB) {
const proceed = "Enable without rebuilding — update every other device first"; const proceed = "Enable without rebuilding — update every other device first";
@@ -375,9 +394,12 @@ export class SetupManager extends AbstractModule {
* @returns * @returns
*/ */
async onConfigureManually(originalSetting: ObsidianLiveSyncSettings, userMode: UserMode): Promise<boolean> { async onConfigureManually(originalSetting: ObsidianLiveSyncSettings, userMode: UserMode): Promise<boolean> {
const e2eeConf = await this.dialogManager.openWithExplicitCancel<SetupRemoteE2EEResultType, EncryptionSettings>( const e2eeConf = await this.dialogManager.openWithExplicitCancel<
SetupRemoteE2EEResultType,
SetupRemoteE2EEInitialData
>(
SetupRemoteE2EE, SetupRemoteE2EE,
originalSetting { settings: originalSetting, newVault: userMode === UserMode.NewUser }
); );
if (e2eeConf === "cancelled") { if (e2eeConf === "cancelled") {
this._log("Manual configuration cancelled.", LOG_LEVEL_NOTICE); this._log("Manual configuration cancelled.", LOG_LEVEL_NOTICE);
@@ -521,7 +543,13 @@ export class SetupManager extends AbstractModule {
* @returns Promise that resolves to true if settings applied successfully, false otherwise * @returns Promise that resolves to true if settings applied successfully, false otherwise
*/ */
async decodeQR(qr: string) { async decodeQR(qr: string) {
const newSettings = decodeSettingsFromQRCodeData(qr); let newSettings: ObsidianLiveSyncSettings;
try {
newSettings = normaliseImportedIdDerivationSettings(decodeSettingsFromQRCodeData(qr));
} catch {
this._log("The QR configuration could not be decoded or contains unsupported settings.", LOG_LEVEL_NOTICE);
return false;
}
return await this.onConfirmApplySettingsFromWizard(newSettings, UserMode.Unknown); return await this.onConfirmApplySettingsFromWizard(newSettings, UserMode.Unknown);
} }
@@ -193,6 +193,58 @@ describe("SetupManager", () => {
expect(setting.currentSettings().activeConfigurationId).toBe("legacy-couchdb"); expect(setting.currentSettings().activeConfigurationId).toBe("legacy-couchdb");
}); });
it("compatibility: treats omitted ID derivation fields in a Setup URI as legacy defaults", async () => {
const { manager, setting, dialogManager } = createSetupManager();
const savedKey = "12".repeat(32);
setting.settings = {
...createLegacyRemoteSetting(),
isConfigured: true,
idDerivationVersion: 1,
idDerivationKey: savedKey,
};
const imported = {
...createLegacyRemoteSetting(),
isConfigured: true,
} as Partial<ObsidianLiveSyncSettings>;
delete imported.idDerivationVersion;
delete imported.idDerivationKey;
vi.spyOn(setting, "adjustSettings").mockImplementation((settings) => Promise.resolve(settings));
dialogManager.openWithExplicitCancel.mockResolvedValueOnce(imported).mockResolvedValueOnce("cancelled");
await manager.onUseSetupURI(UserMode.Unknown, "mock-config://legacy-settings");
const mergedSettings = vi.mocked(setting.adjustSettings).mock.calls[0][0];
expect(mergedSettings.idDerivationVersion).toBe(0);
expect(mergedSettings.idDerivationKey).toBe("");
expect(setting.currentSettings().idDerivationKey).toBe(savedKey);
});
it("does not inherit the missing half of a partially present Setup URI ID configuration", async () => {
const { manager, setting, dialogManager } = createSetupManager();
const savedKey = "34".repeat(32);
setting.settings = {
...createLegacyRemoteSetting(),
isConfigured: true,
idDerivationVersion: 1,
idDerivationKey: savedKey,
};
const imported = {
...createLegacyRemoteSetting(),
isConfigured: true,
idDerivationVersion: 1,
} as Partial<ObsidianLiveSyncSettings>;
delete imported.idDerivationKey;
vi.spyOn(setting, "adjustSettings").mockImplementation((settings) => Promise.resolve(settings));
dialogManager.openWithExplicitCancel.mockResolvedValueOnce(imported).mockResolvedValueOnce("cancelled");
await manager.onUseSetupURI(UserMode.Unknown, "mock-config://partial-settings");
const mergedSettings = vi.mocked(setting.adjustSettings).mock.calls[0][0];
expect(mergedSettings.idDerivationVersion).toBe(1);
expect(mergedSettings.idDerivationKey).toBe("");
expect(setting.currentSettings().idDerivationKey).toBe(savedKey);
});
it("compatibility: normalises imported flat remote settings from QR data before applying", async () => { it("compatibility: normalises imported flat remote settings from QR data before applying", async () => {
const { manager, setting, dialogManager } = createSetupManager(); const { manager, setting, dialogManager } = createSetupManager();
vi.mocked(decodeSettingsFromQRCodeData).mockReturnValue(createLegacyRemoteSetting()); vi.mocked(decodeSettingsFromQRCodeData).mockReturnValue(createLegacyRemoteSetting());
@@ -208,6 +260,79 @@ describe("SetupManager", () => {
expect(setting.currentSettings().activeConfigurationId).toBe("legacy-couchdb"); expect(setting.currentSettings().activeConfigurationId).toBe("legacy-couchdb");
}); });
it("compatibility: applies legacy defaults when QR data omits ID derivation fields", async () => {
const { manager, setting, dialogManager } = createSetupManager();
const savedKey = "56".repeat(32);
setting.settings = {
...createLegacyRemoteSetting(),
isConfigured: true,
idDerivationVersion: 1,
idDerivationKey: savedKey,
};
const imported = { ...createLegacyRemoteSetting(), isConfigured: true } as Partial<ObsidianLiveSyncSettings>;
delete imported.idDerivationVersion;
delete imported.idDerivationKey;
vi.mocked(decodeSettingsFromQRCodeData).mockReturnValue(imported as ObsidianLiveSyncSettings);
vi.spyOn(setting, "adjustSettings").mockImplementation((settings) => Promise.resolve(settings));
dialogManager.openWithExplicitCancel.mockResolvedValueOnce("cancelled");
await manager.decodeQR("qr-data");
const mergedSettings = vi.mocked(setting.adjustSettings).mock.calls[0][0];
expect(mergedSettings.idDerivationVersion).toBe(0);
expect(mergedSettings.idDerivationKey).toBe("");
expect(setting.currentSettings().idDerivationKey).toBe(savedKey);
});
it("rejects invalid QR settings before applying them", async () => {
const { manager, setting } = createSetupManager();
vi.mocked(decodeSettingsFromQRCodeData).mockImplementationOnce(() => {
throw new Error("Invalid ID derivation key");
});
const applyExternalSettings = vi.spyOn(setting, "applyExternalSettings");
await expect(manager.decodeQR("invalid-qr")).resolves.toBe(false);
expect(applyExternalSettings).not.toHaveBeenCalled();
});
it("requires the normal Fetch choice when ID derivation changes with the Metadata preference", async () => {
const { manager, setting, dialogManager, core } = createSetupManager();
const currentSettings: ObsidianLiveSyncSettings = {
...createLegacyRemoteSetting(),
isConfigured: true,
encrypt: true,
passphrase: "e2ee-passphrase",
usePathObfuscation: true,
encryptInternalMetadata: false,
idDerivationVersion: 0,
idDerivationKey: "",
};
const nextIdKey = "78".repeat(32);
setting.settings = currentSettings;
const applyPartial = vi.spyOn(setting, "applyPartial");
core.confirm = {
askSelectStringDialogue: vi.fn(() =>
Promise.resolve("Enable without rebuilding — update every other device first")
),
};
dialogManager.openWithExplicitCancel
.mockResolvedValueOnce({
...currentSettings,
encryptInternalMetadata: true,
idDerivationVersion: 1,
idDerivationKey: nextIdKey,
})
.mockResolvedValueOnce("existing-user")
.mockResolvedValueOnce("apply");
await manager.onlyE2EEConfiguration(UserMode.Update, currentSettings);
expect(applyPartial).not.toHaveBeenCalled();
expect(core.rebuilder.scheduleFetch).toHaveBeenCalledWith(expect.any(Function));
expect(setting.currentSettings().idDerivationVersion).toBe(1);
expect(setting.currentSettings().idDerivationKey).toBe(nextIdKey);
});
it("reserves Rebuild before saving a new-user configuration", async () => { it("reserves Rebuild before saving a new-user configuration", async () => {
const { manager, setting, dialogManager, core } = createSetupManager(); const { manager, setting, dialogManager, core } = createSetupManager();
setting.settings = { ...setting.currentSettings(), isConfigured: false }; setting.settings = { ...setting.currentSettings(), isConfigured: false };
@@ -13,33 +13,64 @@
E2EEAlgorithms, E2EEAlgorithms,
type EncryptionSettings, type EncryptionSettings,
} from "@vrtmrz/livesync-commonlib/compat/common/types"; } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
deriveIdKey,
deriveOrImportIdKey,
formatIdRecoveryCode,
ID_DERIVATION_VERSION,
ID_RECOVERY_CODE_PREFIX,
} from "@vrtmrz/livesync-commonlib/settings";
import { onMount } from "svelte"; import { onMount } from "svelte";
import type { GuestDialogProps } from "@/modules/services/LiveSyncUI/svelteDialog"; import type { GuestDialogProps } from "@/modules/services/LiveSyncUI/svelteDialog";
import { copyTo, pickEncryptionSettings } from "@vrtmrz/livesync-commonlib/compat/common/utils"; import { copyTo, pickEncryptionSettings } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { TYPE_CANCELLED, type SetupRemoteE2EEResultType } from "./setupDialogTypes"; import {
TYPE_CANCELLED,
type SetupRemoteE2EEInitialData,
type SetupRemoteE2EEResultType,
} from "./setupDialogTypes";
import { $msg as translateMessage } from "@/common/translation"; import { $msg as translateMessage } from "@/common/translation";
type Props = GuestDialogProps<SetupRemoteE2EEResultType, EncryptionSettings>; type Props = GuestDialogProps<SetupRemoteE2EEResultType, SetupRemoteE2EEInitialData>;
type IdConfigurationChoice = "keep" | "random" | "custom";
type IdCustomChoice = "passphrase" | "source" | "recovery";
const { setResult, getInitialData }: Props = $props(); const { setResult, getInitialData }: Props = $props();
let default_encryption: EncryptionSettings = { let default_encryption: EncryptionSettings = {
encrypt: true, encrypt: true,
passphrase: "", passphrase: "",
idDerivationVersion: DEFAULT_SETTINGS.idDerivationVersion,
idDerivationKey: DEFAULT_SETTINGS.idDerivationKey,
E2EEAlgorithm: DEFAULT_SETTINGS.E2EEAlgorithm, E2EEAlgorithm: DEFAULT_SETTINGS.E2EEAlgorithm,
usePathObfuscation: true, usePathObfuscation: true,
encryptInternalMetadata: true, encryptInternalMetadata: true,
idDerivationVersion: 0,
idDerivationKey: "",
}; };
let encryptionSettings = $state<EncryptionSettings>({ ...default_encryption }); let encryptionSettings = $state<EncryptionSettings>({ ...default_encryption });
let newVault = $state(false);
let idConfigurationChoice = $state<IdConfigurationChoice>("keep");
let idCustomChoice = $state<IdCustomChoice>("source");
let idDerivationSource = $state("");
let idDerivationError = $state("");
let recoveryCodeVisible = $state(false);
let recoveryCodeCopied = $state(false);
const idDerivationConfigured = $derived(
encryptionSettings.idDerivationVersion === ID_DERIVATION_VERSION &&
typeof encryptionSettings.idDerivationKey === "string" &&
encryptionSettings.idDerivationKey.length > 0
);
const recoveryCode = $derived.by(() =>
idDerivationConfigured ? formatIdRecoveryCode(encryptionSettings.idDerivationKey) : ""
);
onMount(() => { onMount(() => {
if (getInitialData) { if (getInitialData) {
const initialData = getInitialData(); const initialData = getInitialData();
if (initialData) { if (initialData) {
copyTo(initialData, encryptionSettings); copyTo(initialData.settings, encryptionSettings);
newVault = initialData.newVault;
} }
} }
idConfigurationChoice = !idDerivationConfigured && newVault ? "random" : "keep";
}); });
let e2eeValid = $derived.by(() => { let e2eeValid = $derived.by(() => {
if (!encryptionSettings.encrypt) return true; if (!encryptionSettings.encrypt) return true;
@@ -51,8 +82,75 @@
encryptionSettings.usePathObfuscation encryptionSettings.usePathObfuscation
); );
function commit() { function resetIdDerivationSource() {
setResult(pickEncryptionSettings(encryptionSettings)); idDerivationSource = "";
idDerivationError = "";
}
function toggleEncryption(enabled: boolean) {
encryptionSettings.encrypt = enabled;
if (!enabled) resetIdDerivationSource();
}
function selectIdConfiguration() {
recoveryCodeVisible = false;
recoveryCodeCopied = false;
resetIdDerivationSource();
}
function selectIdCustomSource() {
resetIdDerivationSource();
}
async function copyRecoveryCode() {
try {
await navigator.clipboard.writeText(recoveryCode);
recoveryCodeCopied = true;
} catch {
idDerivationError = translateMessage("The recovery code could not be copied. Select and copy the visible code instead.");
}
}
async function commit() {
idDerivationError = "";
const result = pickEncryptionSettings(encryptionSettings);
if (encryptionSettings.encrypt && idConfigurationChoice !== "keep") {
let source = idDerivationSource;
if (idConfigurationChoice === "random") {
const bytes = crypto.getRandomValues(new Uint8Array(32));
source = Array.from(bytes, (byte) => byte.toString(16).padStart(2, "0")).join("");
} else if (idCustomChoice === "passphrase") {
source = encryptionSettings.passphrase;
}
if (source.length === 0) {
if (!idDerivationConfigured) {
idDerivationError = translateMessage("An ID source is required to enable this option.");
return;
}
} else {
try {
result.idDerivationKey =
idConfigurationChoice === "custom" && idCustomChoice !== "passphrase"
? await importOrDeriveEnteredIdKey(source, idCustomChoice)
: await deriveIdKey(source);
result.idDerivationVersion = ID_DERIVATION_VERSION;
} catch {
idDerivationError = translateMessage("The ID source or recovery code is invalid. Check it and try again.");
return;
}
}
}
idDerivationSource = "";
setResult(result);
}
async function importOrDeriveEnteredIdKey(source: string, choice: IdCustomChoice): Promise<string> {
if (choice === "recovery" && !source.trim().startsWith(ID_RECOVERY_CODE_PREFIX)) {
throw new Error("An ID recovery code is required.");
}
return await deriveOrImportIdKey(source);
} }
</script> </script>
@@ -60,7 +158,11 @@
<DialogHeader title={translateMessage("End-to-End Encryption")} /> <DialogHeader title={translateMessage("End-to-End Encryption")} />
<Guidance>{translateMessage("Please configure your end-to-end encryption settings.")}</Guidance> <Guidance>{translateMessage("Please configure your end-to-end encryption settings.")}</Guidance>
<InputRow label={translateMessage("End-to-End Encryption")}> <InputRow label={translateMessage("End-to-End Encryption")}>
<input type="checkbox" bind:checked={encryptionSettings.encrypt} /> <input
type="checkbox"
checked={encryptionSettings.encrypt}
onchange={(event) => toggleEncryption(event.currentTarget.checked)}
/>
</InputRow> </InputRow>
<InfoNote title={translateMessage("Strongly Recommended")}> <InfoNote title={translateMessage("Strongly Recommended")}>
{translateMessage( {translateMessage(
@@ -95,6 +197,164 @@
</InfoNote> </InfoNote>
{/if} {/if}
<fieldset class="sls-id-choices" disabled={!encryptionSettings.encrypt}>
<legend>{translateMessage("ID generation")}</legend>
<label class="sls-id-choice">
<input
type="radio"
name="id-derivation-choice"
value="keep"
bind:group={idConfigurationChoice}
onchange={selectIdConfiguration}
/>
<div class="sls-id-choice-text">
<span>{translateMessage("Keep current configuration")}</span>
<small class="sls-current-id-configuration">
{#if idDerivationConfigured}
{translateMessage(
encryptionSettings.encrypt
? "Current configuration: a saved ID key is used."
: "Current configuration: the saved ID key is retained while E2EE is off."
)}
{:else}
{translateMessage(
"Current configuration: no ID key is saved. With E2EE enabled, keeping it uses legacy IDs tied to the E2EE passphrase."
)}
{/if}
</small>
</div>
</label>
<label class="sls-id-choice">
<input
type="radio"
name="id-derivation-choice"
value="random"
bind:group={idConfigurationChoice}
onchange={selectIdConfiguration}
/>
<span>{translateMessage("Generate a random ID key")}</span>
</label>
<label class="sls-id-choice">
<input
type="radio"
name="id-derivation-choice"
value="custom"
bind:group={idConfigurationChoice}
onchange={selectIdConfiguration}
/>
<span>{translateMessage("Set an ID key")}</span>
</label>
</fieldset>
{#if encryptionSettings.encrypt && idConfigurationChoice === "keep" && !idDerivationConfigured}
<InfoNote warning>
{translateMessage("Changing the E2EE passphrase changes IDs generated by the legacy configuration.")}
</InfoNote>
{/if}
{#if (encryptionSettings.encrypt && idConfigurationChoice !== "keep") || idDerivationConfigured}
{#if encryptionSettings.encrypt}
<InfoNote>
{translateMessage(
"This uses a saved key for new Chunk IDs and obfuscated Metadata document IDs, so changing the E2EE passphrase does not derive a new key automatically."
)}
</InfoNote>
{/if}
{#if idDerivationConfigured}
<InfoNote title={translateMessage("Configured")}>
{translateMessage("The saved ID key is configured. Its source cannot be shown again.")}
</InfoNote>
<button type="button" onclick={() => (recoveryCodeVisible = !recoveryCodeVisible)}>
{translateMessage(recoveryCodeVisible ? "Hide current recovery code" : "Show current recovery code")}
</button>
{#if recoveryCodeVisible}
<InputRow label={translateMessage("Current ID recovery code")}>
<input type="text" readonly value={recoveryCode} aria-label={translateMessage("Current ID recovery code")} />
<button type="button" onclick={copyRecoveryCode}>{translateMessage("Copy recovery code")}</button>
</InputRow>
{#if recoveryCodeCopied}
<InfoNote>{translateMessage("Recovery code copied.")}</InfoNote>
{/if}
{/if}
{/if}
{#if encryptionSettings.encrypt}
{#if idConfigurationChoice === "custom"}
<fieldset class="sls-id-choices sls-id-custom-choices">
<legend>{translateMessage("How to set the ID key")}</legend>
<label class="sls-id-choice">
<input
type="radio"
name="id-custom-choice"
value="passphrase"
bind:group={idCustomChoice}
onchange={selectIdCustomSource}
/>
<span>{translateMessage("Derive from current E2EE passphrase")}</span>
</label>
<label class="sls-id-choice">
<input
type="radio"
name="id-custom-choice"
value="source"
bind:group={idCustomChoice}
onchange={selectIdCustomSource}
/>
<span>{translateMessage("Enter an ID source")}</span>
</label>
<label class="sls-id-choice">
<input
type="radio"
name="id-custom-choice"
value="recovery"
bind:group={idCustomChoice}
onchange={selectIdCustomSource}
/>
<span>{translateMessage("Import an ID recovery code")}</span>
</label>
</fieldset>
{#if idCustomChoice === "source" || idCustomChoice === "recovery"}
<InputRow
label={translateMessage(idCustomChoice === "source" ? "ID source" : "ID recovery code")}
>
<Password
name="id-derivation-source"
placeholder={translateMessage(
idCustomChoice === "source" ? "Enter an ID source" : "Enter an ID recovery code"
)}
bind:value={idDerivationSource}
/>
</InputRow>
{/if}
{/if}
{#if idDerivationConfigured && idConfigurationChoice !== "keep"}
<InfoNote>
{translateMessage("The displayed recovery code belongs to the current key. Reopen this dialogue after saving to copy the replacement key.")}
</InfoNote>
{/if}
{#if idConfigurationChoice === "custom" && idCustomChoice === "source"}
<InfoNote>
{translateMessage("Choose a long, unpredictable source. It is used once and cannot be shown again after saving. A recovery code can be displayed on this device later. This input also accepts a tagged recovery code.")}
</InfoNote>
{:else if idConfigurationChoice === "custom" && idCustomChoice === "recovery"}
<InfoNote>
{translateMessage("Paste a tagged recovery code from an existing device to restore the same ID key.")}
</InfoNote>
{:else if idConfigurationChoice === "random"}
<InfoNote warning>
{translateMessage("For recovery after losing every device, save the recovery code after setup or choose an ID source you can reproduce.")}
</InfoNote>
{:else if idConfigurationChoice === "custom" && idCustomChoice === "passphrase"}
<InfoNote warning>
{translateMessage(
"The ID key is derived from the current E2EE passphrase and saved separately. Changing that passphrase later does not change the saved ID key. To reduce the risk of guessing that passphrase from known IDs, use a separate, unpredictable ID source instead."
)}
</InfoNote>
{/if}
{#if idDerivationConfigured && idConfigurationChoice === "custom" && idCustomChoice !== "passphrase"}
<InfoNote>{translateMessage("Leave this input empty to keep the saved ID key.")}</InfoNote>
{/if}
{/if}
<InfoNote error visible={idDerivationError !== ""}>{idDerivationError}</InfoNote>
{/if}
<InputRow label="Encrypt internal file Properties"> <InputRow label="Encrypt internal file Properties">
<input <input
type="checkbox" type="checkbox"
@@ -164,4 +424,41 @@
width: auto; width: auto;
min-width: 8em; min-width: 8em;
} }
.sls-id-choices {
border: 0;
display: flex;
flex-direction: column;
gap: 0.35em;
margin: 0;
min-width: 0;
padding: 0;
}
.sls-id-choices legend {
margin-bottom: 0.35em;
}
.sls-id-choices:disabled {
opacity: 0.6;
}
.sls-id-custom-choices {
margin-left: 1.5em;
}
.sls-id-choice {
align-items: flex-start;
display: flex;
gap: 0.5em;
}
.sls-id-choice input[type="radio"] {
flex: none;
margin-top: 0.25em;
}
.sls-id-choice-text {
display: flex;
flex-direction: column;
}
.sls-current-id-configuration {
color: var(--text-muted);
display: block;
font-size: var(--font-ui-smaller);
margin-top: 0.15em;
}
</style> </style>
@@ -110,6 +110,10 @@ export type SetupRemoteResultType = typeof TYPE_COUCHDB | typeof TYPE_BUCKET | t
export type UseSetupURIResultType = typeof TYPE_CANCELLED | ObsidianLiveSyncSettings; export type UseSetupURIResultType = typeof TYPE_CANCELLED | ObsidianLiveSyncSettings;
export type SetupRemoteE2EEResultType = typeof TYPE_CANCELLED | EncryptionSettings; export type SetupRemoteE2EEResultType = typeof TYPE_CANCELLED | EncryptionSettings;
export type SetupRemoteE2EEInitialData = {
settings: EncryptionSettings;
newVault: boolean;
};
export type SetupRemoteBucketResultType = typeof TYPE_CANCELLED | BucketSyncSetting; export type SetupRemoteBucketResultType = typeof TYPE_CANCELLED | BucketSyncSetting;
+4
View File
@@ -15,6 +15,8 @@ import {
runReviewHarnessVaultRoundTrip, runReviewHarnessVaultRoundTrip,
} from "@/features/ReviewHarness/reviewHarnessVaultFixture"; } from "@/features/ReviewHarness/reviewHarnessVaultFixture";
import type { CompatibilityReviewController } from "./compatibilityReview"; import type { CompatibilityReviewController } from "./compatibilityReview";
import { runReviewHarnessIdBenchmark } from "@/features/ReviewHarness/reviewHarnessIdBenchmark";
import { createIdBenchmarkOperations } from "@/features/ReviewHarness/reviewHarnessIdBenchmarkRuntime";
async function runVaultRoundTrip(plugin: ObsidianLiveSyncPlugin): Promise<ReviewHarnessScenarioResult> { async function runVaultRoundTrip(plugin: ObsidianLiveSyncPlugin): Promise<ReviewHarnessScenarioResult> {
const vault = plugin.app.vault; const vault = plugin.app.vault;
@@ -58,6 +60,8 @@ export function useReviewHarness(
getCompatibilityPause: () => compatibilityReview.pendingPause, getCompatibilityPause: () => compatibilityReview.pendingPause,
openCompatibilityReview: () => compatibilityReview.openReview(), openCompatibilityReview: () => compatibilityReview.openReview(),
runVaultRoundTrip: () => runVaultRoundTrip(plugin), runVaultRoundTrip: () => runVaultRoundTrip(plugin),
runIdBenchmark: async () =>
runReviewHarnessIdBenchmark(await createIdBenchmarkOperations(), activeWindow.performance),
readContinuation: () => services.setting.getSmallConfig(REVIEW_HARNESS_STATE_KEY), readContinuation: () => services.setting.getSmallConfig(REVIEW_HARNESS_STATE_KEY),
writeContinuation: (value) => services.setting.setSmallConfig(REVIEW_HARNESS_STATE_KEY, value), writeContinuation: (value) => services.setting.setSmallConfig(REVIEW_HARNESS_STATE_KEY, value),
deleteContinuation: () => services.setting.deleteSmallConfig(REVIEW_HARNESS_STATE_KEY), deleteContinuation: () => services.setting.deleteSmallConfig(REVIEW_HARNESS_STATE_KEY),
+32 -2
View File
@@ -129,6 +129,8 @@ The mobile pass uses Obsidian's `app.emulateMobile(true)`, a 390 by 844 CSS-pixe
`test:e2e:obsidian:review-harness` exercises only the boundaries owned by the opt-in maintainer Harness. It retains a real compatibility pause, uses the fixed Harness restart action to persist a device-local continuation and reload Obsidian, and requires the Harness to delete that state before reopening. It also runs the bounded settings-lifecycle observation, confirms the dedicated Vault fixture root is removed, captures the copied privacy-bounded Markdown report, and checks the Harness layout and touch targets in mobile test mode. Compatibility explanation and persistence details remain owned by `settings-ui`, real P2P transfer remains owned by the dedicated P2P suites, and general Vault reflection remains owned by `vault-reflection`; the Harness test does not duplicate those workflows. `test:e2e:obsidian:review-harness` exercises only the boundaries owned by the opt-in maintainer Harness. It retains a real compatibility pause, uses the fixed Harness restart action to persist a device-local continuation and reload Obsidian, and requires the Harness to delete that state before reopening. It also runs the bounded settings-lifecycle observation, confirms the dedicated Vault fixture root is removed, captures the copied privacy-bounded Markdown report, and checks the Harness layout and touch targets in mobile test mode. Compatibility explanation and persistence details remain owned by `settings-ui`, real P2P transfer remains owned by the dedicated P2P suites, and general Vault reflection remains owned by `vault-reflection`; the Harness test does not duplicate those workflows.
The Harness also measures ID generation with fixed in-memory data on desktop and mobile displays, checks that live settings remain unchanged, and verifies that the copied report includes both per-1,000-ID and per-ID timings, key derivation, and JavaScript heap availability. Mobile test mode verifies the UI and execution path; measure native device performance by running the same Harness on that device.
`test:e2e:obsidian:p2p-pane` starts one configured CouchDB-only session with no P2P profile and separate configured P2P sessions for desktop and mobile. It proves that the command remains registered while the retired command, automatic pane, and ribbon entry without a P2P configuration are absent. For the configured P2P profiles, it verifies that the desktop ribbon is available, the current status command reaches the pane without it opening at start-up, checks its connection control and horizontal layout, and captures unobstructed desktop and mobile screenshots. The mobile session uses a fresh Vault, profile, and Obsidian process, enters `app.emulateMobile(true)` through `lifecycle.beforePluginStart`, and requires the P2P view to belong to the right drawer rather than inheriting desktop workspace state. It deliberately uses no relay or peer: replacement of the active replicator is covered by focused unit tests, the Deno and Compose CLI P2P lifecycle suite covers the headless transport, and `p2p-setup-uri-workflow` owns the visible transfer path between two real Obsidian sessions. `test:e2e:obsidian:p2p-pane` starts one configured CouchDB-only session with no P2P profile and separate configured P2P sessions for desktop and mobile. It proves that the command remains registered while the retired command, automatic pane, and ribbon entry without a P2P configuration are absent. For the configured P2P profiles, it verifies that the desktop ribbon is available, the current status command reaches the pane without it opening at start-up, checks its connection control and horizontal layout, and captures unobstructed desktop and mobile screenshots. The mobile session uses a fresh Vault, profile, and Obsidian process, enters `app.emulateMobile(true)` through `lifecycle.beforePluginStart`, and requires the P2P view to belong to the right drawer rather than inheriting desktop workspace state. It deliberately uses no relay or peer: replacement of the active replicator is covered by focused unit tests, the Deno and Compose CLI P2P lifecycle suite covers the headless transport, and `p2p-setup-uri-workflow` owns the visible transfer path between two real Obsidian sessions.
`test:e2e:obsidian:local-suite` builds the plug-in and, unless `LIVESYNC_CLI_COMMAND` selects an external CLI, the local LiveSync CLI. It then runs discovery, smoke, the onboarding invitation, Svelte dialogue mounting, revision repair, settings UI, the Review Harness, the P2P status pane, Vault reflection, CouchDB upload and manual setup, CLI-to-Obsidian synchronisation, Object Storage upload and Setup URI and QR round trips, P2P Setup URI round-trip, startup scan, provisioned CouchDB Setup URI, two-vault synchronisation, Hidden File Sync, Customisation Sync, internal Metadata Doctor, and setting Markdown export in sequence. Start the local CouchDB, RustFS, and P2P relay fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run. `test:e2e:obsidian:local-suite` builds the plug-in and, unless `LIVESYNC_CLI_COMMAND` selects an external CLI, the local LiveSync CLI. It then runs discovery, smoke, the onboarding invitation, Svelte dialogue mounting, revision repair, settings UI, the Review Harness, the P2P status pane, Vault reflection, CouchDB upload and manual setup, CLI-to-Obsidian synchronisation, Object Storage upload and Setup URI and QR round trips, P2P Setup URI round-trip, startup scan, provisioned CouchDB Setup URI, two-vault synchronisation, Hidden File Sync, Customisation Sync, internal Metadata Doctor, and setting Markdown export in sequence. Start the local CouchDB, RustFS, and P2P relay fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run.
@@ -139,10 +141,16 @@ The same workflow checks the two remote-activity status boundaries. It first hol
`test:e2e:obsidian:couchdb-manual-setup-workflow` follows the visible onboarding path for the first device when no Setup URI is available. It enters end-to-end encryption and CouchDB details, runs the read-only `Check server requirements` step, requires the prepared fixture to pass without applying a server fix, and lets the onboarding connection test create the named database. After Rebuild completes on the first device, it creates an ordinary note, asks that working device to generate a Setup URI for a second device, completes Fetch there, and verifies a bidirectional note round-trip. The workflow captures each decision point and the expanded server-check result; password controls remain visually masked. It uses an E2EE passphrase beginning with `%`, confirms that the saved settings do not contain it in plain text, and checks that Obsidian restores it after restarting with the first Vault. `test:e2e:obsidian:couchdb-manual-setup-workflow` follows the visible onboarding path for the first device when no Setup URI is available. It enters end-to-end encryption and CouchDB details, runs the read-only `Check server requirements` step, requires the prepared fixture to pass without applying a server fix, and lets the onboarding connection test create the named database. After Rebuild completes on the first device, it creates an ordinary note, asks that working device to generate a Setup URI for a second device, completes Fetch there, and verifies a bidirectional note round-trip. The workflow captures each decision point and the expanded server-check result; password controls remain visually masked. It uses an E2EE passphrase beginning with `%`, confirms that the saved settings do not contain it in plain text, and checks that Obsidian restores it after restarting with the first Vault.
The ordinary workflow now checks that all three ID-configuration radio choices are visible, disabled and dimmed while E2EE is off, and fully visible when it is enabled. It also checks that the random key is selected by default for a new Vault, **Keep current configuration** shows its legacy explanation, and the saved key is encrypted locally and transferred by Setup URI. A screenshot of the disabled group is saved as `guide-couchdb-manual-id-generation-disabled.png`. Set `E2E_OBSIDIAN_INDEPENDENT_IDS=true` for the same visible workflow with an explicitly entered, randomly generated source. That variant checks all three nested radio choices, requires a source when no key is saved, retains the saved key when a custom source is empty, rejects an ordinary string in the recovery-code input, restores the same key from a tagged code, verifies that the source is absent from local settings, and checks that both devices compute the same obfuscated document IDs after Setup URI import and Fast Fetch.
If this status workflow fails while Obsidian is running, it writes a full-page screenshot and a JSON snapshot of the status text and counters under `/tmp/obsidian-livesync-e2e`. The dialogue-mount workflow leaves desktop and mobile screenshots for both representative Svelte routes, the Hidden File Sync workflow captures the successfully displayed JSON Resolve dialogue before selecting an option, and the Security Seed reconnect workflow captures each significant application state. The suite therefore records representative evidence without capturing every interaction. Set `E2E_OBSIDIAN_DIAGNOSTICS_DIR` to use another directory. If this status workflow fails while Obsidian is running, it writes a full-page screenshot and a JSON snapshot of the status text and counters under `/tmp/obsidian-livesync-e2e`. The dialogue-mount workflow leaves desktop and mobile screenshots for both representative Svelte routes, the Hidden File Sync workflow captures the successfully displayed JSON Resolve dialogue before selecting an option, and the Security Seed reconnect workflow captures each significant application state. The suite therefore records representative evidence without capturing every interaction. Set `E2E_OBSIDIAN_DIAGNOSTICS_DIR` to use another directory.
The two-Vault workflow verifies that each isolated Vault initialises its missing marker without a compatibility pause. Later process launches reuse the same profile-backed acknowledgement. The Hidden File Sync scenario is narrower: it starts from an explicitly acknowledged marker because it tests consumer-owned hidden-file behaviour, JSON resolution, target filtering, and grouped mobile Notices rather than duplicating the compatibility workflow. After `app.emulateMobile(true)`, its fixture operations use the active DevTools renderer because Obsidian can remove desktop-only CLI commands in mobile mode. The two-Vault workflow verifies that each isolated Vault initialises its missing marker without a compatibility pause. Later process launches reuse the same profile-backed acknowledgement. The Hidden File Sync scenario is narrower: it starts from an explicitly acknowledged marker because it tests consumer-owned hidden-file behaviour, JSON resolution, target filtering, and grouped mobile Notices rather than duplicating the compatibility workflow. After `app.emulateMobile(true)`, its fixture operations use the active DevTools renderer because Obsidian can remove desktop-only CLI commands in mobile mode.
The two-Vault workflow also covers independent ID derivation with two real Obsidian sessions: a note travels in each direction, both devices retain the same obfuscated document IDs, and identical content reuses the same Chunk IDs. Fresh devices with a different ID key or legacy ID configuration must be rejected by ordinary CouchDB replication before any remote document or checkpoint changes. Set `E2E_OBSIDIAN_ONLY_INDEPENDENT_IDS=true` to run that case without the other two-Vault scenarios.
Set `E2E_OBSIDIAN_ONLY_DIFFERENT_CHUNK_ID_KEYS=true` to run the focused case where two devices use different saved ID keys with Path Obfuscation off. It verifies that each device can read the other's note, visible document IDs agree, and writing the same content produces different Chunk IDs.
`test:e2e:obsidian:cli-to-obsidian-sync` is the cross-runtime compatibility check for the official LiveSync CLI and the real Obsidian plug-in. Build the plug-in first, and build the local CLI too when no external CLI command is selected. The script uses E2EE, Path Obfuscation, and the current preferred chunk settings to create and synchronise a note through the CLI, starts real Obsidian with an isolated Vault and profile, synchronises the same CouchDB database, and verifies that the plug-in materialises identical note content. This covers the boundary that CLI-only and plug-in-only round trips do not exercise. `test:e2e:obsidian:cli-to-obsidian-sync` is the cross-runtime compatibility check for the official LiveSync CLI and the real Obsidian plug-in. Build the plug-in first, and build the local CLI too when no external CLI command is selected. The script uses E2EE, Path Obfuscation, and the current preferred chunk settings to create and synchronise a note through the CLI, starts real Obsidian with an isolated Vault and profile, synchronises the same CouchDB database, and verifies that the plug-in materialises identical note content. This covers the boundary that CLI-only and plug-in-only round trips do not exercise.
The isolated Obsidian session starts with its CouchDB settings and device-local compatibility acknowledgement already in place. This keeps the scenario focused on cross-runtime data compatibility; unconfigured start-up and visible CouchDB onboarding are covered by their dedicated workflows. The isolated Obsidian session starts with its CouchDB settings and device-local compatibility acknowledgement already in place. This keeps the scenario focused on cross-runtime data compatibility; unconfigured start-up and visible CouchDB onboarding are covered by their dedicated workflows.
@@ -166,10 +174,15 @@ LIVESYNC_CLI_COMMAND="docker run --rm --network host --user $(id -u):$(id -g) --
`test:e2e:obsidian:minio-upload` reuses the Object Storage variables from `.test.env` or the process environment. It expects a reachable S3-compatible service and starts with isolated Object Storage settings and the device-local compatibility acknowledgement already in place, keeping the scenario focused on upload rather than unconfigured start-up or setup. It confirms those settings through `obsidian-cli eval`, creates a note in real Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written under a unique bucket prefix. Adapter tests separately observe an in-progress SDK command, while this real-runtime workflow verifies the resulting request counters advance and rebalance. `test:e2e:obsidian:minio-upload` reuses the Object Storage variables from `.test.env` or the process environment. It expects a reachable S3-compatible service and starts with isolated Object Storage settings and the device-local compatibility acknowledgement already in place, keeping the scenario focused on upload rather than unconfigured start-up or setup. It confirms those settings through `obsidian-cli eval`, creates a note in real Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written under a unique bucket prefix. Adapter tests separately observe an in-progress SDK command, while this real-runtime workflow verifies the resulting request counters advance and rebalance.
Set `E2E_OBSIDIAN_INDEPENDENT_IDS=true` to run the same upload with E2EE, Path Obfuscation, and a separately derived ID key. The scenario verifies the local document and Chunk ID shapes before the Journal transfer.
Set `E2E_OBSIDIAN_CUSTOM_HTTP_HANDLER=true` when the local Object Storage fixture does not allow browser requests from Obsidian's renderer.
`test:e2e:obsidian:object-storage-setup-uri-workflow` uses the public Commonlib-backed tool to generate the initial Setup URI for a unique Object Storage prefix, completes visible initialisation on the first device, and then asks that working real Obsidian device to create a new Setup URI through the registered command. A second real Obsidian device imports only the device-generated URI. The workflow verifies the A-to-B note through explicit replication, then verifies that the B-to-A note arrives through `syncOnStart` after restarting the first device, without requesting manual replication. It captures the documented onboarding choices, and removes the Object Storage prefix only after both sessions have stopped. The test requires the current version marker and absence of a compatibility pause after Fetch and after restarting the same Vault, without accepting a review automatically. `test:e2e:obsidian:object-storage-setup-uri-workflow` uses the public Commonlib-backed tool to generate the initial Setup URI for a unique Object Storage prefix, completes visible initialisation on the first device, and then asks that working real Obsidian device to create a new Setup URI through the registered command. A second real Obsidian device imports only the device-generated URI. The workflow verifies the A-to-B note through explicit replication, then verifies that the B-to-A note arrives through `syncOnStart` after restarting the first device, without requesting manual replication. It captures the documented onboarding choices, and removes the Object Storage prefix only after both sessions have stopped. The test requires the current version marker and absence of a compatibility pause after Fetch and after restarting the same Vault, without accepting a review automatically.
`test:e2e:obsidian:object-storage-qr-workflow` runs the same Object Storage round trip with QR settings on the second device. It takes the first device's settings, assigns a distinct database suffix in the QR fixture, encodes them with Commonlib's QR encoder, passes the payload to the real QR decoding entry point, and selects **Join this device** in the visible dialogue. Unlike the Setup URI, the QR payload includes a database suffix; explicitly choosing one makes the namespace change independent of Obsidian's initial defaults. Before Fetch begins, the scenario verifies that the imported namespace has its current compatibility marker without a pause. Fetch then selects the receiving device's own suffix when resetting the local database. The test verifies the marker and absence of a pause again after Fetch and after a natural restart. It covers the QR settings and setup flow, without requiring a camera or exercising operating-system URI dispatch. `test:e2e:obsidian:object-storage-qr-workflow` runs the same Object Storage round trip with QR settings on the second device. It takes the first device's settings, assigns a distinct database suffix in the QR fixture, encodes them with Commonlib's QR encoder, passes the payload to the real QR decoding entry point, and selects **Join this device** in the visible dialogue. Unlike the Setup URI, the QR payload includes a database suffix; explicitly choosing one makes the namespace change independent of Obsidian's initial defaults. Before Fetch begins, the scenario verifies that the imported namespace has its current compatibility marker without a pause. Fetch then selects the receiving device's own suffix when resetting the local database. The test verifies the marker and absence of a pause again after Fetch and after a natural restart. It covers the QR settings and setup flow, without requiring a camera or exercising operating-system URI dispatch.
`test:e2e:obsidian:object-storage-compatible-setup-uri-workflow` selects **Compatible (no time limit)** in the real generation dialogue and uses Persistent mode for the bootstrap tool. All three Object Storage sharing scenarios require the generated independent ID key to survive import and natural restarts on both devices, remain encrypted in local settings, and retain document and Chunk IDs during the bidirectional transfer. Before valid setup, they submit an incorrect passphrase and a URI generated in a past window, require the visible rejection, and verify unchanged runtime and persisted settings. Only an isolated fixture worker uses the past clock; Obsidian and the runner use real time.
`test:e2e:obsidian:p2p-setup-uri-workflow` runs two concurrent isolated real Obsidian sessions against the local Compose Nostr relay fixture. The first device imports a generated initial Setup URI and completes its signalling test with zero peers, creates a Setup URI for the second device through the registered command, and remains online while the second device imports it. The second device must select the expected online source before Fetch can rebuild its local database. The workflow accepts each connection request visibly on the receiving device, verifies the initial A-to-B fetch, checks that the menu for the three persistent per-peer actions remains within the viewport, reconnects both P2P sessions in join order, and verifies the B-to-A return journey. Every started session remains tracked until teardown completes. `test:e2e:obsidian:p2p-setup-uri-workflow` runs two concurrent isolated real Obsidian sessions against the local Compose Nostr relay fixture. The first device imports a generated initial Setup URI and completes its signalling test with zero peers, creates a Setup URI for the second device through the registered command, and remains online while the second device imports it. The second device must select the expected online source before Fetch can rebuild its local database. The workflow accepts each connection request visibly on the receiving device, verifies the initial A-to-B fetch, checks that the menu for the three persistent per-peer actions remains within the viewport, reconnects both P2P sessions in join order, and verifies the B-to-A return journey. Every started session remains tracked until teardown completes.
`test:e2e:obsidian:p2p-connection-check` owns the browser-to-Obsidian preflight path. It serves the WebPeer production build from loopback, asks the page to generate a disposable Setup URI using the local relay, starts its browser reference peer, and applies that exact URI through visible onboarding in an isolated empty real Obsidian Vault. After the first successful WebRTC diagnostic appears, it selects the action for another device in the same room, proves that the Setup URI was not regenerated, applies it to a second isolated empty real Obsidian Vault, and requires both the successful total and the baseline number of simultaneous active connections to advance. It captures the result card without Setup URI credentials and does not claim to verify note synchronisation. Run `test:e2e:obsidian:p2p-connection-check:services` to build both production artefacts and let the scenario start and stop the Compose relay. `test:e2e:obsidian:p2p-connection-check` owns the browser-to-Obsidian preflight path. It serves the WebPeer production build from loopback, asks the page to generate a disposable Setup URI using the local relay, starts its browser reference peer, and applies that exact URI through visible onboarding in an isolated empty real Obsidian Vault. After the first successful WebRTC diagnostic appears, it selects the action for another device in the same room, proves that the Setup URI was not regenerated, applies it to a second isolated empty real Obsidian Vault, and requires both the successful total and the baseline number of simultaneous active connections to advance. It captures the result card without Setup URI credentials and does not claim to verify note synchronisation. Run `test:e2e:obsidian:p2p-connection-check:services` to build both production artefacts and let the scenario start and stop the Compose relay.
@@ -210,7 +223,7 @@ This proves in real Obsidian the plug-in behaviour shared by supported platforms
`test:e2e:obsidian:internal-metadata-doctor` reuses the migration fixture and enables encryption through the real Config Doctor dialogues. It checks declining the consultation, skipping the recommendation with a reminder, dismissing the current Doctor version, and accepting the recommendation through **Run Doctor** after dismissal. Each choice is checked against active and persisted settings, with natural restarts of the same Vault and profile verifying reminders and retained choices. A local database sentinel, start-up flag checks, unchanged remote documents, and renderer identity checks detect an unintended automatic Rebuild, Fetch, or restart. The accepted setting then follows the two-device migration and Fast Fetch checks above. This scenario requires CouchDB and is included in `test:e2e:obsidian:local-suite`; run it separately with `npm run test:e2e:obsidian:focused -- internal-metadata-doctor` after starting the CouchDB fixture. `test:e2e:obsidian:internal-metadata-doctor` reuses the migration fixture and enables encryption through the real Config Doctor dialogues. It checks declining the consultation, skipping the recommendation with a reminder, dismissing the current Doctor version, and accepting the recommendation through **Run Doctor** after dismissal. Each choice is checked against active and persisted settings, with natural restarts of the same Vault and profile verifying reminders and retained choices. A local database sentinel, start-up flag checks, unchanged remote documents, and renderer identity checks detect an unintended automatic Rebuild, Fetch, or restart. The accepted setting then follows the two-device migration and Fast Fetch checks above. This scenario requires CouchDB and is included in `test:e2e:obsidian:local-suite`; run it separately with `npm run test:e2e:obsidian:focused -- internal-metadata-doctor` after starting the CouchDB fixture.
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`. `test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`, including both the plaintext ID key and its encrypted local representation.
`test:e2e:obsidian:upgrade-from-stable` is the release-acceptance upgrade workflow. It installs the exact published 0.25.83 artefacts into an isolated Vault, verifies their pinned SHA-256 values, and then replaces only the plug-in artefacts with the current target while retaining the same Vault and isolated Obsidian profile. The first run downloads the old release into the ignored `_testdata/releases` cache; every later run verifies the cached bytes before use. `test:e2e:obsidian:upgrade-from-stable` is the release-acceptance upgrade workflow. It installs the exact published 0.25.83 artefacts into an isolated Vault, verifies their pinned SHA-256 values, and then replaces only the plug-in artefacts with the current target while retaining the same Vault and isolated Obsidian profile. The first run downloads the old release into the ignored `_testdata/releases` cache; every later run verifies the cached bytes before use.
@@ -245,7 +258,24 @@ Or let the wrapper manage both fixtures:
npm run test:e2e:obsidian:local-suite:services npm run test:e2e:obsidian:local-suite:services
``` ```
Useful environment variables: ### Combined setup and security regression checks
Build the current plug-in once, then run these focused scenarios sequentially with their documented fixtures:
| Coverage | Scenario or command |
| --- | --- |
| Time-bound URI, independent key, rejection, restart, and two-way Object Storage transfer | `npm run test:e2e:obsidian:object-storage-setup-uri-workflow` |
| Compatible URI with the same key and transfer checks | `npm run test:e2e:obsidian:object-storage-compatible-setup-uri-workflow` |
| QR import, changed database suffix, key persistence, and two-way transfer | `npm run test:e2e:obsidian:object-storage-qr-workflow` |
| Custom ID source, recovery code, encrypted local storage, and CouchDB Setup URI transfer | `E2E_OBSIDIAN_INDEPENDENT_IDS=true npm run test:e2e:obsidian:couchdb-manual-setup-workflow` |
| Matching IDs and rejection of incompatible document keys before remote writes | `E2E_OBSIDIAN_ONLY_INDEPENDENT_IDS=true npm run test:e2e:obsidian:two-vault-sync` |
| Doctor decline, reminder, dismissal, later acceptance, and mixed internal Metadata | `npm run test:e2e:obsidian:internal-metadata-doctor` |
The setup-tool contract suite also checks ID recovery and explicit legacy IDs in both URI modes. `dialog-mounts` covers the availability dialogue and setup choices on desktop and emulated mobile. These automated scenarios remove the need to repeat every decision path manually during BRAT acceptance.
BRAT acceptance still validates the exact published artefacts: install or update through BRAT, cold-start Obsidian, and exchange one note in each direction. On a physical mobile device, include one Setup URI or QR hand-off and check the displayed instructions, principal controls, and responsiveness. Camera capture, operating-system dispatch, native mobile performance, and the published installation path are outside this local E2E coverage; emulated mobile establishes layout and interaction only.
### Environment variables
- `OBSIDIAN_BINARY`: explicit Obsidian executable path. - `OBSIDIAN_BINARY`: explicit Obsidian executable path.
- `OBSIDIAN_CLI`: explicit companion `obsidian-cli` executable path. - `OBSIDIAN_CLI`: explicit companion `obsidian-cli` executable path.
+51 -6
View File
@@ -73,7 +73,8 @@ export async function enterSetupURI(
port: number, port: number,
mode: "new" | "existing", mode: "new" | "existing",
artifact: SetupArtifact, artifact: SetupArtifact,
captures: SetupCaptureNames captures: SetupCaptureNames,
rejectedArtifacts: readonly SetupArtifact[] = []
): Promise<string> { ): Promise<string> {
await withObsidianPage(port, async (page) => { await withObsidianPage(port, async (page) => {
const invitation = page.locator(".notice").filter({ hasText: "Welcome to Self-hosted LiveSync" }); const invitation = page.locator(".notice").filter({ hasText: "Welcome to Self-hosted LiveSync" });
@@ -101,6 +102,40 @@ export async function enterSetupURI(
const setup = modalByTitle(page, "Enter Setup URI"); const setup = modalByTitle(page, "Enter Setup URI");
await setup.waitFor({ state: "visible", timeout: uiTimeoutMs }); await setup.waitFor({ state: "visible", timeout: uiTimeoutMs });
const settingsSnapshot = () =>
page.evaluate(async () => {
const obsidian = globalThis as typeof globalThis & {
app: {
plugins: {
plugins: Record<
string,
{
core: { services: { setting: { currentSettings(): unknown } } };
loadData(): Promise<unknown>;
}
>;
};
};
};
const plugin = obsidian.app.plugins.plugins["obsidian-livesync"];
return JSON.stringify({
current: plugin.core.services.setting.currentSettings(),
persisted: await plugin.loadData(),
});
});
const before = rejectedArtifacts.length > 0 ? await settingsSnapshot() : undefined;
for (const rejected of rejectedArtifacts) {
await setup.locator('input[placeholder^="obsidian://setuplivesync"]').fill(rejected.setupURI);
await setup.locator('input[name="password"]').fill(rejected.setupPassphrase);
await setup.getByRole("button", { name: "Test Settings and Continue" }).click({ timeout: uiTimeoutMs });
await setup.getByText($msg("Failed to parse Setup-URI."), { exact: false }).waitFor({
state: "visible",
timeout: uiTimeoutMs,
});
if ((await settingsSnapshot()) !== before) {
throw new Error("A rejected Setup URI changed the receiving device's settings.");
}
}
await setup.locator('input[placeholder^="obsidian://setuplivesync"]').fill(artifact.setupURI); await setup.locator('input[placeholder^="obsidian://setuplivesync"]').fill(artifact.setupURI);
await setup.locator('input[name="password"]').fill(artifact.setupPassphrase); await setup.locator('input[name="password"]').fill(artifact.setupPassphrase);
}); });
@@ -120,7 +155,8 @@ export async function enterSetupURI(
export async function generateSetupURIFromDevice( export async function generateSetupURIFromDevice(
port: number, port: number,
setupPassphrase: string, setupPassphrase: string,
captures: SetupCaptureNames captures: SetupCaptureNames,
mode: "ephemeral" | "persistent" = "ephemeral"
): Promise<{ artifact: SetupArtifact; screenshots: string[] }> { ): Promise<{ artifact: SetupArtifact; screenshots: string[] }> {
const opened = await withObsidianPage(port, async (page) => { const opened = await withObsidianPage(port, async (page) => {
return await page.evaluate( return await page.evaluate(
@@ -153,9 +189,15 @@ export async function generateSetupURIFromDevice(
const choice = modalByTitle(page, "Setup URI availability"); const choice = modalByTitle(page, "Setup URI availability");
await choice.waitFor({ state: "visible", timeout: uiTimeoutMs }); await choice.waitFor({ state: "visible", timeout: uiTimeoutMs });
await choice.getByText("Time-bound Setup URIs can be opened until", { exact: false }).waitFor({ await choice.getByText("Time-bound Setup URIs can be opened until", { exact: false }).waitFor({
state: "visible", timeout: uiTimeoutMs, state: "visible",
timeout: uiTimeoutMs,
}); });
await choice.getByRole("button", { name: "Time-bound", exact: true }).click({ timeout: uiTimeoutMs }); await choice
.getByRole("button", {
name: mode === "ephemeral" ? "Time-bound" : "Compatible (no time limit)",
exact: true,
})
.click({ timeout: uiTimeoutMs });
}); });
const resultTitle = "Your Setup URI is ready to be copied"; const resultTitle = "Your Setup URI is ready to be copied";
@@ -407,10 +449,13 @@ export async function finishInitialisation(
while (Date.now() < deadline) { while (Date.now() < deadline) {
const resumeVisible = await withObsidianPage(port, async (page) => { const resumeVisible = await withObsidianPage(port, async (page) => {
const alignedSettingsNotice = page.locator(".modal-container").filter({ const alignedSettingsNotice = page.locator(".modal-container").filter({
hasText: "Your settings differed slightly from the server's. The plug-in has supplemented the incompatible parts with the server settings!", hasText:
"Your settings differed slightly from the server's. The plug-in has supplemented the incompatible parts with the server settings!",
}); });
if (await alignedSettingsNotice.isVisible()) { if (await alignedSettingsNotice.isVisible()) {
await alignedSettingsNotice.getByRole("button", { name: "OK", exact: true }).click({ timeout: uiTimeoutMs }); await alignedSettingsNotice
.getByRole("button", { name: "OK", exact: true })
.click({ timeout: uiTimeoutMs });
} }
return await modalByTitle(page, "Confirmation").filter({ hasText: message }).isVisible(); return await modalByTitle(page, "Confirmation").filter({ hasText: message }).isVisible();
}).catch(() => false); }).catch(() => false);
@@ -2,6 +2,7 @@ import { randomBytes } from "node:crypto";
import { readFile } from "node:fs/promises"; import { readFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { DEVICE_ID_PREFERRED, MILESTONE_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types"; import { DEVICE_ID_PREFERRED, MILESTONE_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { deriveIdKey, formatIdRecoveryCode } from "@vrtmrz/livesync-commonlib/settings";
import { evalObsidianJson } from "../runner/cli.ts"; import { evalObsidianJson } from "../runner/cli.ts";
import { import {
assertCouchDbReachable, assertCouchDbReachable,
@@ -14,7 +15,12 @@ import {
type CouchDbConfig, type CouchDbConfig,
} from "../runner/couchdb.ts"; } from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts"; import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { assertEqual, pushLocalChanges, waitForLocalDatabaseEntry } from "../runner/liveSyncWorkflow.ts"; import {
assertEqual,
assertE2eCompatibilityUnpaused,
pushLocalChanges,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts"; import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { import {
acknowledgeDisabledOptionalFeatures, acknowledgeDisabledOptionalFeatures,
@@ -26,7 +32,6 @@ import {
finishInitialisation, finishInitialisation,
generateSetupURIFromDevice, generateSetupURIFromDevice,
modalByTitle, modalByTitle,
resumeCompatibilityReviewIfShown,
selectRadioOption, selectRadioOption,
continueWithoutRemoteSettings, continueWithoutRemoteSettings,
type SetupArtifact, type SetupArtifact,
@@ -99,7 +104,12 @@ async function captureFailure(session: ObsidianLiveSyncSession, label: string):
} }
} }
async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig, dbName: string): Promise<string[]> { async function enterManualCouchDBSettings(
port: number,
couchDb: CouchDbConfig,
dbName: string,
independentIdSource?: string
): Promise<string[]> {
const screenshots: string[] = []; const screenshots: string[] = [];
await withObsidianPage(port, async (page) => { await withObsidianPage(port, async (page) => {
const invitation = page.locator(".notice").filter({ hasText: "Welcome to Self-hosted LiveSync" }); const invitation = page.locator(".notice").filter({ hasText: "Welcome to Self-hosted LiveSync" });
@@ -134,12 +144,41 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
0, 0,
"The Obfuscate Properties row was present before end-to-end encryption was enabled." "The Obfuscate Properties row was present before end-to-end encryption was enabled."
); );
const disabledIdChoices = encryption.locator("fieldset.sls-id-choices").first();
assertEqual(
await disabledIdChoices.evaluate((element) => element.hasAttribute("disabled")),
true,
"The ID configuration was enabled while E2EE was off."
);
for (const value of ["keep", "random", "custom"]) {
assertEqual(
await disabledIdChoices.locator(`input[value="${value}"]`).isDisabled(),
true,
`The ${value} ID configuration was enabled while E2EE was off.`
);
}
const disabledIdScreenshot = join(
process.env.E2E_OBSIDIAN_DIAGNOSTICS_DIR ?? "/tmp/obsidian-livesync-e2e",
"guide-couchdb-manual-id-generation-disabled.png"
);
await disabledIdChoices.screenshot({ path: disabledIdScreenshot });
screenshots.push(disabledIdScreenshot);
assertEqual(
await disabledIdChoices.evaluate((element) => Number(getComputedStyle(element).opacity) < 1),
true,
"The disabled ID configuration did not look disabled in the default theme."
);
await encryption await encryption
.locator("label.row") .locator("label.row")
.filter({ hasText: "End-to-End Encryption" }) .filter({ hasText: "End-to-End Encryption" })
.locator('input[type="checkbox"]') .locator('input[type="checkbox"]')
.first() .first()
.check({ timeout: uiTimeoutMs }); .check({ timeout: uiTimeoutMs });
assertEqual(
await disabledIdChoices.evaluate((element) => Number(getComputedStyle(element).opacity)),
1,
"The ID configuration remained dimmed after E2EE was enabled."
);
const passphraseInput = encryption.locator('input[name="e2ee-passphrase"]'); const passphraseInput = encryption.locator('input[name="e2ee-passphrase"]');
await passphraseInput.waitFor({ state: "visible", timeout: uiTimeoutMs }); await passphraseInput.waitFor({ state: "visible", timeout: uiTimeoutMs });
await encryption await encryption
@@ -149,7 +188,85 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
.first() .first()
.check({ timeout: uiTimeoutMs }); .check({ timeout: uiTimeoutMs });
await passphraseInput.fill(e2eePassphrase); await passphraseInput.fill(e2eePassphrase);
const passwordToggle = encryption.locator("button.sls-password-toggle"); const idChoices = encryption.locator('input[type="radio"][name="id-derivation-choice"]');
assertEqual(await idChoices.count(), 3, "The three ID configurations were not all shown.");
for (const value of ["keep", "random", "custom"]) {
assertEqual(
await encryption.locator(`input[name="id-derivation-choice"][value="${value}"]`).isVisible(),
true,
`The ${value} ID configuration was not visible.`
);
}
const keepChoice = encryption.locator('input[name="id-derivation-choice"][value="keep"]');
const randomChoice = encryption.locator('input[name="id-derivation-choice"][value="random"]');
const customChoice = encryption.locator('input[name="id-derivation-choice"][value="custom"]');
assertEqual(await randomChoice.isChecked(), true, "The default ID configuration was not random.");
assertEqual(
await encryption.getByText("Keep current configuration", { exact: true }).count(),
1,
"The current-configuration choice was not labelled consistently."
);
assertEqual(
await encryption.getByText("Current configuration: no ID key is saved.", { exact: false }).count(),
1,
"The current legacy configuration was not explained."
);
await keepChoice.check({ timeout: uiTimeoutMs });
assertEqual(
await encryption.getByText("Changing the E2EE passphrase changes IDs", { exact: false }).count(),
1,
"Keeping legacy IDs did not explain the effect of changing the E2EE passphrase."
);
await randomChoice.check({ timeout: uiTimeoutMs });
if (independentIdSource) {
await customChoice.check({ timeout: uiTimeoutMs });
const customChoices = encryption.locator('input[type="radio"][name="id-custom-choice"]');
assertEqual(await customChoices.count(), 3, "The three custom ID inputs were not all shown.");
for (const value of ["passphrase", "source", "recovery"]) {
assertEqual(
await encryption.locator(`input[name="id-custom-choice"][value="${value}"]`).isVisible(),
true,
`The ${value} custom ID input was not visible.`
);
}
const sourceChoice = encryption.locator('input[name="id-custom-choice"][value="source"]');
assertEqual(await sourceChoice.isChecked(), true, "The custom ID input was not selected by default.");
await encryption
.locator('input[name="id-custom-choice"][value="passphrase"]')
.check({ timeout: uiTimeoutMs });
assertEqual(
await encryption.locator('input[name="id-derivation-source"]').count(),
0,
"The E2EE passphrase choice exposed a second source input."
);
await encryption
.locator('input[name="id-custom-choice"][value="recovery"]')
.check({ timeout: uiTimeoutMs });
assertEqual(
await encryption.locator('input[name="id-derivation-source"]').getAttribute("placeholder"),
"Enter an ID recovery code",
"The recovery-code choice did not request a recovery code."
);
const recoveryChoice = encryption.locator('input[name="id-custom-choice"][value="recovery"]');
await sourceChoice.check({ timeout: uiTimeoutMs });
const sourceInput = encryption.locator('input[name="id-derivation-source"]');
await sourceInput.fill("");
await encryption.getByRole("button", { name: "Proceed", exact: true }).click({ timeout: uiTimeoutMs });
assertEqual(
await encryption.getByText("An ID source is required to enable this option.", { exact: false }).count(),
1,
"A first-time independent ID configuration did not require a source."
);
assertEqual(
await encryption.isVisible(),
true,
"The E2EE dialogue closed after a first-time ID source was omitted."
);
await recoveryChoice.check({ timeout: uiTimeoutMs });
await sourceChoice.check({ timeout: uiTimeoutMs });
await sourceInput.fill(independentIdSource);
}
const passwordToggle = passphraseInput.locator("..").locator("button.sls-password-toggle");
await passwordToggle.click({ timeout: uiTimeoutMs }); await passwordToggle.click({ timeout: uiTimeoutMs });
assertEqual( assertEqual(
await passphraseInput.getAttribute("type"), await passphraseInput.getAttribute("type"),
@@ -167,16 +284,13 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
"password", "password",
"Toggling visibility again did not re-mask the passphrase." "Toggling visibility again did not re-mask the passphrase."
); );
assertEqual( assertEqual(await passphraseInput.inputValue(), e2eePassphrase, "Re-masking the passphrase changed its value.");
await passphraseInput.inputValue(),
e2eePassphrase,
"Re-masking the passphrase changed its value."
);
}); });
screenshots.push(await captureGuideDialogue(port, "guide-couchdb-manual-encryption.png", "End-to-End Encryption")); screenshots.push(await captureGuideDialogue(port, "guide-couchdb-manual-encryption.png", "End-to-End Encryption"));
await withObsidianPage(port, async (page) => { await withObsidianPage(port, async (page) => {
const encryption = modalByTitle(page, "End-to-End Encryption"); const encryption = modalByTitle(page, "End-to-End Encryption");
await encryption.getByRole("button", { name: "Proceed", exact: true }).click({ timeout: uiTimeoutMs }); await encryption.getByRole("button", { name: "Proceed", exact: true }).click({ timeout: uiTimeoutMs });
await encryption.waitFor({ state: "hidden", timeout: uiTimeoutMs });
}); });
screenshots.push( screenshots.push(
@@ -288,13 +402,18 @@ async function waitForRemoteEntry(context: RunnerContext, entry: { id: string; c
}); });
} }
async function assertPersistedE2EE(vault: TemporaryVault): Promise<void> { async function assertPersistedE2EE(vault: TemporaryVault, independentIdSource?: string): Promise<void> {
const persisted = JSON.parse( const rawSettings = await readFile(
await readFile(join(vault.path, ".obsidian", "plugins", "obsidian-livesync", "data.json"), "utf8") join(vault.path, ".obsidian", "plugins", "obsidian-livesync", "data.json"),
) as { "utf8"
);
const persisted = JSON.parse(rawSettings) as {
encrypt?: unknown; encrypt?: unknown;
encryptedPassphrase?: unknown; encryptedPassphrase?: unknown;
passphrase?: unknown; passphrase?: unknown;
idDerivationVersion?: unknown;
idDerivationKey?: unknown;
encryptedIdDerivationKey?: unknown;
}; };
assertEqual(persisted.encrypt, true, "Manual CouchDB setup did not persist E2EE as enabled."); assertEqual(persisted.encrypt, true, "Manual CouchDB setup did not persist E2EE as enabled.");
assertEqual(persisted.passphrase, "", "Manual CouchDB setup persisted the E2EE passphrase in plain text."); assertEqual(persisted.passphrase, "", "Manual CouchDB setup persisted the E2EE passphrase in plain text.");
@@ -304,6 +423,14 @@ async function assertPersistedE2EE(vault: TemporaryVault): Promise<void> {
if (JSON.stringify(persisted).includes(e2eePassphrase)) { if (JSON.stringify(persisted).includes(e2eePassphrase)) {
throw new Error("Manual CouchDB setup persisted the E2EE passphrase in plain text."); throw new Error("Manual CouchDB setup persisted the E2EE passphrase in plain text.");
} }
assertEqual(persisted.idDerivationVersion, 1, "The independent ID mode was not persisted.");
assertEqual(persisted.idDerivationKey, "", "The derived ID key was stored in plain text.");
if (typeof persisted.encryptedIdDerivationKey !== "string" || !persisted.encryptedIdDerivationKey) {
throw new Error("The derived ID key was not encrypted in local settings.");
}
if (independentIdSource) {
if (rawSettings.includes(independentIdSource)) throw new Error("The ID source was stored in local settings.");
}
} }
async function assertRestoredE2EEPassphrase(session: ObsidianLiveSyncSession, cliBinary: string): Promise<void> { async function assertRestoredE2EEPassphrase(session: ObsidianLiveSyncSession, cliBinary: string): Promise<void> {
@@ -320,6 +447,126 @@ async function assertRestoredE2EEPassphrase(session: ObsidianLiveSyncSession, cl
assertEqual(restored, true, "The E2EE passphrase was not restored after Obsidian restarted."); assertEqual(restored, true, "The E2EE passphrase was not restored after Obsidian restarted.");
} }
async function assertCurrentIdDerivationKey(
session: ObsidianLiveSyncSession,
cliBinary: string,
expected: string,
context: string
): Promise<void> {
const settings = await evalObsidianJson<{ idDerivationVersion: number; idDerivationKey: string }>(
cliBinary,
[
"(()=>{",
"const settings=app.plugins.plugins['obsidian-livesync'].core.services.setting.currentSettings();",
"return JSON.stringify({idDerivationVersion:settings.idDerivationVersion,idDerivationKey:settings.idDerivationKey});",
"})()",
].join(""),
session.cliEnv
);
assertEqual(settings.idDerivationVersion, 1, `${context}: the independent ID version was not retained.`);
assertEqual(settings.idDerivationKey, expected, `${context}: the saved ID key changed.`);
}
async function assertRecoveryCodeCanBeRevealed(
session: ObsidianLiveSyncSession,
cliBinary: string,
source: string
): Promise<void> {
const port = session.remoteDebuggingPort;
const expected = formatIdRecoveryCode(await deriveIdKey(source));
await withObsidianPage(port, async (page) => {
const settingsNavigator = await openLiveSyncSettings(page, uiTimeoutMs);
const remotePage = await settingsNavigator.openPage("Remote Configuration");
await remotePage
.locator(".setting-item")
.filter({ hasText: "Configure E2EE" })
.getByRole("button", { name: "Configure", exact: true })
.click({ timeout: uiTimeoutMs });
const encryption = modalByTitle(page, "End-to-End Encryption");
await encryption.waitFor({ state: "visible", timeout: uiTimeoutMs });
assertEqual(
await encryption.locator('input[name="id-derivation-choice"][value="keep"]').isChecked(),
true,
"An existing ID key was not selected for reuse."
);
assertEqual(
await encryption.getByText("Current configuration: a saved ID key is used.").count(),
1,
"The saved ID key was not explained."
);
await encryption.getByRole("button", { name: "Show current recovery code" }).click({ timeout: uiTimeoutMs });
assertEqual(
await encryption.getByRole("textbox", { name: "Current ID recovery code" }).inputValue(),
expected,
"The displayed recovery code did not contain the saved ID key."
);
await encryption.locator('input[name="id-derivation-choice"][value="custom"]').check({ timeout: uiTimeoutMs });
await encryption.locator('input[name="id-custom-choice"][value="recovery"]').check({ timeout: uiTimeoutMs });
const recoveryInput = encryption.locator('input[name="id-derivation-source"]');
await recoveryInput.fill("not-a-recovery-code");
await encryption.getByRole("button", { name: "Proceed" }).click({ timeout: uiTimeoutMs });
assertEqual(
await encryption.getByText("The ID source or recovery code is invalid.", { exact: false }).count(),
1,
"The recovery-code choice accepted an ordinary source string."
);
await recoveryInput.fill(expected);
await encryption.getByRole("button", { name: "Proceed" }).click({ timeout: uiTimeoutMs });
await encryption.waitFor({ state: "hidden", timeout: uiTimeoutMs });
assertEqual(
await modalByTitle(page, "Mostly Complete: Decision Required").count(),
0,
"Recovering the existing ID key opened a new setup decision."
);
});
const expectedIdKey = await deriveIdKey(source);
await assertCurrentIdDerivationKey(session, cliBinary, expectedIdKey, "Recovering the saved ID key");
await withObsidianPage(port, async (page) => {
const settingsNavigator = await openLiveSyncSettings(page, uiTimeoutMs);
const remotePage = await settingsNavigator.openPage("Remote Configuration");
await remotePage
.locator(".setting-item")
.filter({ hasText: "Configure E2EE" })
.getByRole("button", { name: "Configure", exact: true })
.click({ timeout: uiTimeoutMs });
const encryption = modalByTitle(page, "End-to-End Encryption");
await encryption.waitFor({ state: "visible", timeout: uiTimeoutMs });
await encryption.locator('input[name="id-derivation-choice"][value="custom"]').check({ timeout: uiTimeoutMs });
await encryption.locator('input[name="id-custom-choice"][value="source"]').check({ timeout: uiTimeoutMs });
const sourceInput = encryption.locator('input[name="id-derivation-source"]');
await sourceInput.fill("");
assertEqual(await sourceInput.inputValue(), "", "The independent ID source input was not empty.");
assertEqual(
await encryption.getByText("Leave this input empty to keep the saved ID key.", { exact: true }).count(),
1,
"The configured ID source did not explain that an empty input keeps the saved ID key."
);
await encryption.getByRole("button", { name: "Proceed", exact: true }).click({ timeout: uiTimeoutMs });
await encryption.waitFor({ state: "hidden", timeout: uiTimeoutMs });
assertEqual(
await modalByTitle(page, "Mostly Complete: Decision Required").count(),
0,
"Keeping the saved ID key after an empty source opened a new setup decision."
);
});
await assertCurrentIdDerivationKey(session, cliBinary, expectedIdKey, "Saving an empty custom ID source");
}
async function readDocumentId(cliBinary: string, environment: NodeJS.ProcessEnv, path: string): Promise<string> {
return await evalObsidianJson<string>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"return JSON.stringify(await core.services.path.path2id(path));",
"})()",
].join(""),
environment
);
}
async function setRemotePreferredE2EEDisabled(context: RunnerContext): Promise<void> { async function setRemotePreferredE2EEDisabled(context: RunnerContext): Promise<void> {
const milestone = await fetchCouchDbDocument(context.couchDb, context.dbName, MILESTONE_DOCID); const milestone = await fetchCouchDbDocument(context.couchDb, context.dbName, MILESTONE_DOCID);
const tweakValues = milestone.tweak_values; const tweakValues = milestone.tweak_values;
@@ -430,6 +677,10 @@ async function main(): Promise<void> {
}; };
const screenshots: string[] = []; const screenshots: string[] = [];
let secondDeviceArtifact: SetupArtifact | undefined; let secondDeviceArtifact: SetupArtifact | undefined;
const independentIdSource =
process.env.E2E_OBSIDIAN_INDEPENDENT_IDS === "true" ? randomBytes(32).toString("base64url") : undefined;
let firstEntryId: string | undefined;
let returnEntryId: string | undefined;
try { try {
await assertCouchDbReachable(couchDb); await assertCouchDbReachable(couchDb);
@@ -440,23 +691,29 @@ async function main(): Promise<void> {
let session = await startUnconfiguredSession(context, vaultA); let session = await startUnconfiguredSession(context, vaultA);
try { try {
screenshots.push(...(await enterManualCouchDBSettings(session.remoteDebuggingPort, couchDb, dbName))); screenshots.push(
...(await enterManualCouchDBSettings(session.remoteDebuggingPort, couchDb, dbName, independentIdSource))
);
screenshots.push(await captureAndStartInitialisation(session.remoteDebuggingPort, "new", captures)); screenshots.push(await captureAndStartInitialisation(session.remoteDebuggingPort, "new", captures));
screenshots.push(await confirmRebuild(session.remoteDebuggingPort, captures)); screenshots.push(await confirmRebuild(session.remoteDebuggingPort, captures));
screenshots.push(await continueWithoutRemoteSettings(session.remoteDebuggingPort, captures)); screenshots.push(await continueWithoutRemoteSettings(session.remoteDebuggingPort, captures));
screenshots.push(await acknowledgeDisabledOptionalFeatures(session.remoteDebuggingPort, captures)); screenshots.push(await acknowledgeDisabledOptionalFeatures(session.remoteDebuggingPort, captures));
const state = await finishInitialisation(session.remoteDebuggingPort, context.cliBinary, session.cliEnv); const state = await finishInitialisation(session.remoteDebuggingPort, context.cliBinary, session.cliEnv);
await resumeCompatibilityReviewIfShown(session.remoteDebuggingPort); await assertE2eCompatibilityUnpaused(context.cliBinary, session.cliEnv, session.remoteDebuggingPort);
assertEqual(state.activeConfigurationId !== "", true, "Manual CouchDB setup did not activate a profile."); assertEqual(state.activeConfigurationId !== "", true, "Manual CouchDB setup did not activate a profile.");
assertEqual( assertEqual(
state.remoteConfigurationCount, state.remoteConfigurationCount,
1, 1,
"Manual CouchDB setup did not persist exactly one remote profile." "Manual CouchDB setup did not persist exactly one remote profile."
); );
await assertPersistedE2EE(vaultA); await assertPersistedE2EE(vaultA, independentIdSource);
if (independentIdSource) {
await assertRecoveryCodeCanBeRevealed(session, context.cliBinary, independentIdSource);
}
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, notePath, noteContent); await writeNoteViaObsidian(context.cliBinary, session.cliEnv, notePath, noteContent);
const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, notePath); const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, notePath);
firstEntryId = entry.id;
await pushLocalChanges(context.cliBinary, session.cliEnv); await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForRemoteEntry(context, entry); await waitForRemoteEntry(context, entry);
} catch (error) { } catch (error) {
@@ -478,10 +735,13 @@ async function main(): Promise<void> {
await acknowledgeDisabledOptionalFeatures(session.remoteDebuggingPort, e2eeRebuildCaptures) await acknowledgeDisabledOptionalFeatures(session.remoteDebuggingPort, e2eeRebuildCaptures)
); );
await finishInitialisation(session.remoteDebuggingPort, context.cliBinary, session.cliEnv); await finishInitialisation(session.remoteDebuggingPort, context.cliBinary, session.cliEnv);
await resumeCompatibilityReviewIfShown(session.remoteDebuggingPort); await assertE2eCompatibilityUnpaused(context.cliBinary, session.cliEnv, session.remoteDebuggingPort);
await assertPersistedE2EE(vaultA); await assertPersistedE2EE(vaultA, independentIdSource);
const rebuiltEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, notePath); const rebuiltEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, notePath);
if (independentIdSource) {
assertEqual(rebuiltEntry.id, firstEntryId, "Rebuild changed the configured document ID.");
}
await waitForRemoteEntry(context, rebuiltEntry); await waitForRemoteEntry(context, rebuiltEntry);
await assertRemoteEntryEncrypted(context, rebuiltEntry, notePath, noteContent); await assertRemoteEntryEncrypted(context, rebuiltEntry, notePath, noteContent);
await assertRemotePreferredE2EE(context, true); await assertRemotePreferredE2EE(context, true);
@@ -511,12 +771,21 @@ async function main(): Promise<void> {
screenshots.push(await captureAndStartInitialisation(session.remoteDebuggingPort, "existing", captures)); screenshots.push(await captureAndStartInitialisation(session.remoteDebuggingPort, "existing", captures));
screenshots.push(...(await confirmFastFetch(session.remoteDebuggingPort, captures))); screenshots.push(...(await confirmFastFetch(session.remoteDebuggingPort, captures)));
await finishInitialisation(session.remoteDebuggingPort, context.cliBinary, session.cliEnv); await finishInitialisation(session.remoteDebuggingPort, context.cliBinary, session.cliEnv);
await resumeCompatibilityReviewIfShown(session.remoteDebuggingPort); await assertE2eCompatibilityUnpaused(context.cliBinary, session.cliEnv, session.remoteDebuggingPort);
await assertPersistedE2EE(vaultB, independentIdSource);
await pushLocalChanges(context.cliBinary, session.cliEnv); await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForVaultFile(vaultB, notePath, noteContent); await waitForVaultFile(vaultB, notePath, noteContent);
if (independentIdSource) {
assertEqual(
await readDocumentId(context.cliBinary, session.cliEnv, notePath),
firstEntryId,
"The Setup URI did not restore the document ID key on the second device."
);
}
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, returnNotePath, returnNoteContent); await writeNoteViaObsidian(context.cliBinary, session.cliEnv, returnNotePath, returnNoteContent);
const returnEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, returnNotePath); const returnEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, returnNotePath);
returnEntryId = returnEntry.id;
await pushLocalChanges(context.cliBinary, session.cliEnv); await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForRemoteEntry(context, returnEntry); await waitForRemoteEntry(context, returnEntry);
} catch (error) { } catch (error) {
@@ -528,9 +797,16 @@ async function main(): Promise<void> {
session = await startUnconfiguredSession(context, vaultA); session = await startUnconfiguredSession(context, vaultA);
try { try {
await resumeCompatibilityReviewIfShown(session.remoteDebuggingPort); await assertE2eCompatibilityUnpaused(context.cliBinary, session.cliEnv, session.remoteDebuggingPort);
await pushLocalChanges(context.cliBinary, session.cliEnv); await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForVaultFile(vaultA, returnNotePath, returnNoteContent); await waitForVaultFile(vaultA, returnNotePath, returnNoteContent);
if (independentIdSource) {
assertEqual(
await readDocumentId(context.cliBinary, session.cliEnv, returnNotePath),
returnEntryId,
"The first device did not retain the shared document ID key."
);
}
} catch (error) { } catch (error) {
await captureFailure(session, "return-journey"); await captureFailure(session, "return-journey");
throw error; throw error;
+4
View File
@@ -35,6 +35,10 @@ const testSteps: Step[] = [
name: "Object Storage Setup URI workflow", name: "Object Storage Setup URI workflow",
args: ["run", "test:e2e:obsidian:object-storage-setup-uri-workflow"], args: ["run", "test:e2e:obsidian:object-storage-setup-uri-workflow"],
}, },
{
name: "Object Storage Compatible Setup URI workflow",
args: ["run", "test:e2e:obsidian:object-storage-compatible-setup-uri-workflow"],
},
{ {
name: "Object Storage QR workflow", name: "Object Storage QR workflow",
args: ["run", "test:e2e:obsidian:object-storage-qr-workflow"], args: ["run", "test:e2e:obsidian:object-storage-qr-workflow"],
+41 -4
View File
@@ -15,6 +15,8 @@
* Separate successes would not prove that those observations belonged to the * Separate successes would not prove that those observations belonged to the
* same upload. * same upload.
*/ */
import { randomBytes } from "node:crypto";
import { deriveIdKey } from "@vrtmrz/livesync-commonlib/settings";
import { evalObsidianJson } from "../runner/cli.ts"; import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts"; import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { import {
@@ -33,6 +35,7 @@ import {
listObjectStorageObjects, listObjectStorageObjects,
loadObjectStorageConfig, loadObjectStorageConfig,
makeUniqueBucketPrefix, makeUniqueBucketPrefix,
readObjectStorageJson,
} from "../runner/objectStorage.ts"; } from "../runner/objectStorage.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts"; import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts"; import { createTemporaryVault } from "../runner/vault.ts";
@@ -41,6 +44,8 @@ import { REMOTE_ACTIVITY_EXPECTED_STATE, waitForRemoteActivityState } from "../r
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000"; process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/minio-upload.md"; const notePath = "E2E/minio-upload.md";
const useIndependentIds = process.env.E2E_OBSIDIAN_INDEPENDENT_IDS === "true";
const useCustomRequestHandler = process.env.E2E_OBSIDIAN_CUSTOM_HTTP_HANDLER === "true";
const noteContent = [ const noteContent = [
"# Object Storage upload from real Obsidian", "# Object Storage upload from real Obsidian",
"", "",
@@ -127,10 +132,23 @@ async function main(): Promise<void> {
}); });
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv); await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureObjectStorage(cli.binary, session.cliEnv, { const configured = await configureObjectStorage(
...objectStorage, cli.binary,
bucketPrefix, session.cliEnv,
}); { ...objectStorage, bucketPrefix },
{
...(useIndependentIds
? {
encrypt: true,
usePathObfuscation: true,
passphrase: randomBytes(32).toString("base64url"),
idDerivationVersion: 1,
idDerivationKey: await deriveIdKey(randomBytes(32).toString("base64url")),
}
: {}),
...(useCustomRequestHandler ? { useCustomRequestHandler: true } : {}),
}
);
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv); await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured."); assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
assertEqual(configured.remoteType, "MINIO", "Remote type was not Object Storage."); assertEqual(configured.remoteType, "MINIO", "Remote type was not Object Storage.");
@@ -145,6 +163,16 @@ async function main(): Promise<void> {
REMOTE_ACTIVITY_EXPECTED_STATE.idle REMOTE_ACTIVITY_EXPECTED_STATE.idle
); );
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv); const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
if (useIndependentIds) {
if (
!/^f:[0-9a-f]{64}$/u.test(localEntry.id) ||
localEntry.children.some((child) => !/^h:\+[0-9a-f]{64}$/u.test(child))
) {
throw new Error(
`The real Obsidian Journal upload did not use independent document and Chunk IDs (document length ${localEntry.id.length}, Chunk lengths ${localEntry.children.map((child) => child.length).join(",")}).`
);
}
}
await pushLocalChanges(cli.binary, session.cliEnv); await pushLocalChanges(cli.binary, session.cliEnv);
const activityAfterUpload = await waitForRemoteActivityState( const activityAfterUpload = await waitForRemoteActivityState(
session.remoteDebuggingPort, session.remoteDebuggingPort,
@@ -160,6 +188,15 @@ async function main(): Promise<void> {
); );
const keys = await waitForObjectStorageObjects(bucketPrefix); const keys = await waitForObjectStorageObjects(bucketPrefix);
if (useIndependentIds) {
const milestone = await readObjectStorageJson<{ encrypted_id_derivation_proof?: string }>(
objectStorage,
`${bucketPrefix}_00000000-milestone.json`
);
if (!milestone.encrypted_id_derivation_proof) {
throw new Error("The Journal milestone did not retain an encrypted ID agreement proof.");
}
}
console.log( console.log(
`Uploaded ${localEntry.path} through Journal Sync to ${objectStorage.bucket}/${bucketPrefix} (${keys.length} object(s)); tracked requests: ${activityAfterUpload.requestCount - activityBeforeUpload.requestCount}` `Uploaded ${localEntry.path} through Journal Sync to ${objectStorage.bucket}/${bucketPrefix} (${keys.length} object(s)); tracked requests: ${activityAfterUpload.requestCount - activityBeforeUpload.requestCount}`
@@ -3,13 +3,14 @@ import { randomBytes } from "node:crypto";
import { readFile } from "node:fs/promises"; import { readFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { promisify } from "node:util"; import { promisify } from "node:util";
import { Worker } from "node:worker_threads";
import { encodeSettingsToQRCodeData } from "@vrtmrz/livesync-commonlib/compat/API/processSetting"; import { encodeSettingsToQRCodeData } from "@vrtmrz/livesync-commonlib/compat/API/processSetting";
import { decodeSettingsFromSetupURI } from "@vrtmrz/livesync-commonlib/setup-uri";
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types"; import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { evalObsidianJson } from "../runner/cli.ts"; import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts"; import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { import {
assertEqual, assertEqual,
assertE2eCompatibilityMarker,
assertE2eCompatibilityUnpaused, assertE2eCompatibilityUnpaused,
pushLocalChanges, pushLocalChanges,
type ConfiguredSettings, type ConfiguredSettings,
@@ -53,11 +54,17 @@ process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "90000";
const execFileAsync = promisify(execFile); const execFileAsync = promisify(execFile);
const useCustomRequestHandler = process.argv.includes("--custom-http-handler"); const useCustomRequestHandler = process.argv.includes("--custom-http-handler");
const useQRCode = process.argv.includes("--qr"); const useQRCode = process.argv.includes("--qr");
const uriMode = process.argv.includes("--compatible") ? "persistent" : "ephemeral";
const captures: SetupCaptureNames = useQRCode const captures: SetupCaptureNames = useQRCode
? { scenario: "object-storage-qr", guide: "object-storage-qr-setup" } ? { scenario: "object-storage-qr", guide: "object-storage-qr-setup" }
: useCustomRequestHandler : uriMode === "persistent"
? { scenario: "object-storage-custom-http-handler-setup-uri", guide: "object-storage-custom-http-handler-setup" } ? { scenario: "object-storage-compatible-uri", guide: "object-storage-compatible-setup" }
: { scenario: "object-storage-setup-uri", guide: "object-storage-setup" }; : useCustomRequestHandler
? {
scenario: "object-storage-custom-http-handler-setup-uri",
guide: "object-storage-custom-http-handler-setup",
}
: { scenario: "object-storage-setup-uri", guide: "object-storage-setup" };
const noteFromFirst = "E2E/object-storage/from-first.md"; const noteFromFirst = "E2E/object-storage/from-first.md";
const noteFromSecond = "E2E/object-storage/from-second.md"; const noteFromSecond = "E2E/object-storage/from-second.md";
const firstContent = const firstContent =
@@ -123,12 +130,63 @@ async function generateBootstrapSetupURI(
...(useCustomRequestHandler ? { use_custom_request_handler: "true" } : {}), ...(useCustomRequestHandler ? { use_custom_request_handler: "true" } : {}),
passphrase: randomBytes(24).toString("base64url"), passphrase: randomBytes(24).toString("base64url"),
uri_passphrase: setupPassphrase, uri_passphrase: setupPassphrase,
uri_mode: uriMode,
}); });
const setupURI = output.split(/\r?\n/u).find((line) => line.startsWith("obsidian://setuplivesync?settings=")); const setupURI = output.split(/\r?\n/u).find((line) => line.startsWith("obsidian://setuplivesync?settings="));
if (!setupURI) throw new Error("The public Setup URI generator did not emit an Object Storage Setup URI."); if (!setupURI) throw new Error("The public Setup URI generator did not emit an Object Storage Setup URI.");
return { setupURI, setupPassphrase }; return { setupURI, setupPassphrase };
} }
async function expiredSetupURI(settings: ObsidianLiveSyncSettings, setupPassphrase: string): Promise<SetupArtifact> {
// Only this fixture worker uses a past clock; Obsidian and the runner keep real time.
const worker = new Worker(
`
const { parentPort, workerData } = require('node:worker_threads');
Date.now = () => Date.UTC(2024, 0, 1);
import('@vrtmrz/livesync-commonlib/setup-uri').then(async ({ encodeTimeBoundSetupURI }) => {
const result = await encodeTimeBoundSetupURI(workerData.settings, workerData.setupPassphrase);
parentPort.postMessage(result.uri);
});
`,
{ eval: true, workerData: { settings, setupPassphrase } }
);
try {
const setupURI = await new Promise<string>((resolve, reject) => {
worker.once("message", resolve);
worker.once("error", reject);
worker.once("exit", (code) => reject(new Error(`The expired URI fixture exited with ${code}.`)));
});
return { setupURI, setupPassphrase };
} finally {
await worker.terminate();
}
}
async function assertIndependentIdKey(
context: RunnerContext,
session: ObsidianLiveSyncSession,
vault: TemporaryVault,
expectedKey: string
): Promise<void> {
const matches = await evalObsidianJson<boolean>(
context.cliBinary,
`(() => {
const settings = app.plugins.plugins['obsidian-livesync'].core.services.setting.currentSettings();
return JSON.stringify(settings.encrypt && settings.usePathObfuscation &&
settings.idDerivationVersion === 1 && settings.idDerivationKey === ${JSON.stringify(expectedKey)});
})()`,
session.cliEnv
);
assertEqual(matches, true, "The device did not retain the shared independent ID key and Path Obfuscation.");
const raw = await readFile(join(vault.path, ".obsidian/plugins/obsidian-livesync/data.json"), "utf8");
const saved = JSON.parse(raw) as Record<string, unknown>;
assertEqual(saved.idDerivationVersion, 1, "The independent ID version was not saved.");
assertEqual(saved.idDerivationKey, "", "The ID key was saved in plain text.");
if (!saved.encryptedIdDerivationKey || raw.includes(expectedKey)) {
throw new Error("The shared ID key was not encrypted in local settings.");
}
}
async function startSession( async function startSession(
context: RunnerContext, context: RunnerContext,
vault: TemporaryVault, vault: TemporaryVault,
@@ -297,6 +355,18 @@ async function main(): Promise<void> {
const objectStorage = await loadObjectStorageConfig(); const objectStorage = await loadObjectStorageConfig();
const bucketPrefix = makeUniqueBucketPrefix("setup-uri-workflow"); const bucketPrefix = makeUniqueBucketPrefix("setup-uri-workflow");
const bootstrapArtifact = await generateBootstrapSetupURI(objectStorage, bucketPrefix, useCustomRequestHandler); const bootstrapArtifact = await generateBootstrapSetupURI(objectStorage, bucketPrefix, useCustomRequestHandler);
const bootstrapSettings = await decodeSettingsFromSetupURI(
bootstrapArtifact.setupURI,
bootstrapArtifact.setupPassphrase
);
if (!bootstrapSettings || bootstrapSettings.idDerivationVersion !== 1 || !bootstrapSettings.idDerivationKey) {
throw new Error("The public Setup URI generator did not configure an independent ID key.");
}
const idKey = bootstrapSettings.idDerivationKey;
const rejectedArtifacts = [
{ ...bootstrapArtifact, setupPassphrase: "incorrect-setup-passphrase" },
await expiredSetupURI(bootstrapSettings as ObsidianLiveSyncSettings, bootstrapArtifact.setupPassphrase),
];
const vaultA = await createTemporaryVault(); const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault(); const vaultB = await createTemporaryVault();
const [portA, portB] = sessionPorts(); const [portA, portB] = sessionPorts();
@@ -308,13 +378,14 @@ async function main(): Promise<void> {
console.log(`Temporary Object Storage target: ${objectStorage.bucket}/${bucketPrefix}`); console.log(`Temporary Object Storage target: ${objectStorage.bucket}/${bucketPrefix}`);
const sessionA = await startSession(context, vaultA, portA); const sessionA = await startSession(context, vaultA, portA);
screenshots.push(await enterSetupURI(portA, "new", bootstrapArtifact, captures)); screenshots.push(await enterSetupURI(portA, "new", bootstrapArtifact, captures, rejectedArtifacts));
screenshots.push(await captureAndStartInitialisation(portA, "new", captures)); screenshots.push(await captureAndStartInitialisation(portA, "new", captures));
screenshots.push(await confirmRebuild(portA, captures)); screenshots.push(await confirmRebuild(portA, captures));
screenshots.push(await continueWithoutRemoteSettings(portA, captures)); screenshots.push(await continueWithoutRemoteSettings(portA, captures));
screenshots.push(await acknowledgeDisabledOptionalFeatures(portA, captures)); screenshots.push(await acknowledgeDisabledOptionalFeatures(portA, captures));
const firstState = await finishInitialisation(portA, context.cliBinary, sessionA.cliEnv); const firstState = await finishInitialisation(portA, context.cliBinary, sessionA.cliEnv);
await assertE2eCompatibilityUnpaused(context.cliBinary, sessionA.cliEnv, portA); await assertE2eCompatibilityUnpaused(context.cliBinary, sessionA.cliEnv, portA);
await assertIndependentIdKey(context, sessionA, vaultA, idKey);
assertEqual( assertEqual(
firstState.endpoint, firstState.endpoint,
objectStorage.endpoint, objectStorage.endpoint,
@@ -337,9 +408,22 @@ async function main(): Promise<void> {
); );
await writeNote(context.cliBinary, sessionA.cliEnv, noteFromFirst, firstContent); await writeNote(context.cliBinary, sessionA.cliEnv, noteFromFirst, firstContent);
const firstEntry = await waitForLocalDatabaseEntry(context.cliBinary, sessionA.cliEnv, noteFromFirst);
if (
!/^f:[0-9a-f]{64}$/u.test(firstEntry.id) ||
firstEntry.children.length === 0 ||
firstEntry.children.some((id) => !/^h:\+[0-9a-f]{64}$/u.test(id))
) {
throw new Error("The source note did not use independent document and Chunk IDs.");
}
await pushLocalChanges(context.cliBinary, sessionA.cliEnv); await pushLocalChanges(context.cliBinary, sessionA.cliEnv);
await waitForObjectStorageData(objectStorage, bucketPrefix); await waitForObjectStorageData(objectStorage, bucketPrefix);
const generated = await generateSetupURIFromDevice(portA, randomBytes(24).toString("base64url"), captures); const generated = await generateSetupURIFromDevice(
portA,
randomBytes(24).toString("base64url"),
captures,
uriMode
);
if (generated.artifact.setupURI === bootstrapArtifact.setupURI) { if (generated.artifact.setupURI === bootstrapArtifact.setupURI) {
throw new Error("The first device returned the bootstrap Setup URI instead of generating a new one."); throw new Error("The first device returned the bootstrap Setup URI instead of generating a new one.");
} }
@@ -377,7 +461,7 @@ async function main(): Promise<void> {
await stopSession(context, sessionA); await stopSession(context, sessionA);
const sessionB = await startSession(context, vaultB, portB); const sessionB = await startSession(context, vaultB, portB);
const initialMarker = await assertE2eCompatibilityMarker(context.cliBinary, sessionB.cliEnv); const initialMarker = await assertE2eCompatibilityUnpaused(context.cliBinary, sessionB.cliEnv, portB);
if (qrSettings) { if (qrSettings) {
assertEqual( assertEqual(
initialMarker.additionalSuffix === `-${qrSettings.additionalSuffixOfDatabaseName}`, initialMarker.additionalSuffix === `-${qrSettings.additionalSuffixOfDatabaseName}`,
@@ -412,6 +496,7 @@ async function main(): Promise<void> {
screenshots.push(...(await confirmFastFetch(portB, captures))); screenshots.push(...(await confirmFastFetch(portB, captures)));
const secondState = await finishInitialisation(portB, context.cliBinary, sessionB.cliEnv); const secondState = await finishInitialisation(portB, context.cliBinary, sessionB.cliEnv);
const fetchedMarker = await assertE2eCompatibilityUnpaused(context.cliBinary, sessionB.cliEnv, portB); const fetchedMarker = await assertE2eCompatibilityUnpaused(context.cliBinary, sessionB.cliEnv, portB);
await assertIndependentIdKey(context, sessionB, vaultB, idKey);
assertEqual( assertEqual(
secondState.endpoint, secondState.endpoint,
objectStorage.endpoint, objectStorage.endpoint,
@@ -429,6 +514,13 @@ async function main(): Promise<void> {
); );
await pushLocalChanges(context.cliBinary, sessionB.cliEnv); await pushLocalChanges(context.cliBinary, sessionB.cliEnv);
await waitForPathContent(vaultB, noteFromFirst, firstContent); await waitForPathContent(vaultB, noteFromFirst, firstContent);
const importedEntry = await waitForLocalDatabaseEntry(context.cliBinary, sessionB.cliEnv, noteFromFirst);
assertEqual(importedEntry.id, firstEntry.id, "Import changed the obfuscated document ID.");
assertEqual(
JSON.stringify(importedEntry.children),
JSON.stringify(firstEntry.children),
"Import changed the Chunk IDs."
);
screenshots.push( screenshots.push(
await captureNote( await captureNote(
portB, portB,
@@ -439,10 +531,12 @@ async function main(): Promise<void> {
); );
await writeNote(context.cliBinary, sessionB.cliEnv, noteFromSecond, secondContent); await writeNote(context.cliBinary, sessionB.cliEnv, noteFromSecond, secondContent);
const secondEntry = await waitForLocalDatabaseEntry(context.cliBinary, sessionB.cliEnv, noteFromSecond);
await pushLocalChanges(context.cliBinary, sessionB.cliEnv); await pushLocalChanges(context.cliBinary, sessionB.cliEnv);
await stopSession(context, sessionB); await stopSession(context, sessionB);
const returningSessionB = await startSession(context, vaultB, portB); const returningSessionB = await startSession(context, vaultB, portB);
await waitForLiveSyncCoreReady(context.cliBinary, returningSessionB.cliEnv); await waitForLiveSyncCoreReady(context.cliBinary, returningSessionB.cliEnv);
await assertIndependentIdKey(context, returningSessionB, vaultB, idKey);
const restartedMarker = await assertE2eCompatibilityUnpaused( const restartedMarker = await assertE2eCompatibilityUnpaused(
context.cliBinary, context.cliBinary,
returningSessionB.cliEnv, returningSessionB.cliEnv,
@@ -459,10 +553,22 @@ async function main(): Promise<void> {
const returningSessionA = await startSession(context, vaultA, portA); const returningSessionA = await startSession(context, vaultA, portA);
await waitForLiveSyncCoreReady(context.cliBinary, returningSessionA.cliEnv); await waitForLiveSyncCoreReady(context.cliBinary, returningSessionA.cliEnv);
await assertE2eCompatibilityUnpaused(context.cliBinary, returningSessionA.cliEnv, portA); await assertE2eCompatibilityUnpaused(context.cliBinary, returningSessionA.cliEnv, portA);
await assertIndependentIdKey(context, returningSessionA, vaultA, idKey);
// Deliberately omit manual replication here. Object Storage reports // Deliberately omit manual replication here. Object Storage reports
// Continuous as not applicable, so startup scheduling must honour the // Continuous as not applicable, so startup scheduling must honour the
// retained syncOnStart setting by running an unattended OneShot. // retained syncOnStart setting by running an unattended OneShot.
await waitForPathContent(vaultA, noteFromSecond, secondContent); await waitForPathContent(vaultA, noteFromSecond, secondContent);
const returnedEntry = await waitForLocalDatabaseEntry(
context.cliBinary,
returningSessionA.cliEnv,
noteFromSecond
);
assertEqual(returnedEntry.id, secondEntry.id, "The return journey changed the obfuscated document ID.");
assertEqual(
JSON.stringify(returnedEntry.children),
JSON.stringify(secondEntry.children),
"The return journey changed the Chunk IDs."
);
screenshots.push( screenshots.push(
await captureNote( await captureNote(
portA, portA,
+112 -54
View File
@@ -167,7 +167,8 @@ async function captureReadinessFailure(
async function openHarness(): Promise<void> { async function openHarness(): Promise<void> {
const opened = await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => { const opened = await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
return await page.evaluate( return await page.evaluate(
(commandId) => (globalThis as ReviewHarnessTestGlobal).app?.commands?.executeCommandById(commandId) === true, (commandId) =>
(globalThis as ReviewHarnessTestGlobal).app?.commands?.executeCommandById(commandId) === true,
"obsidian-livesync:open-review-harness" "obsidian-livesync:open-review-harness"
); );
}); });
@@ -206,11 +207,54 @@ async function runAutomaticScenarios(): Promise<void> {
}); });
} }
async function runIdBenchmark(): Promise<void> {
await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
const snapshotSettings = () =>
page.evaluate(() => {
const plugin = (globalThis as ReviewHarnessTestGlobal).app?.plugins?.plugins["obsidian-livesync"] as {
core: { services: { setting: { currentSettings(): unknown } } };
};
return JSON.stringify(plugin.core.services.setting.currentSettings());
});
const before = await snapshotSettings();
const harness = page.locator('[data-testid="review-harness"]');
await harness
.locator('[data-testid="review-harness-run-id-generation-performance"]')
.click({ timeout: uiTimeoutMs });
const result = harness.locator('[data-testid="review-harness-result-id-generation-performance"]');
await result.getByText("Passed:", { exact: false }).waitFor({ state: "visible", timeout: uiTimeoutMs * 4 });
const observations = await result.locator("li").allTextContents();
for (const label of [
"Chunk IDs, 256 B",
"Chunk IDs, 4096 B",
"Chunk IDs, 32768 B",
"Obfuscated document IDs",
]) {
for (const mode of ["legacy", "independent"]) {
if (
!observations.some(
(line) =>
line.startsWith(`${label}, ${mode}: 1000 IDs total median=`) && line.includes("; per ID=")
)
) {
throw new Error(`Missing benchmark timing and units: ${label}, ${mode}`);
}
}
}
if (
!observations.some((line) => line.startsWith("ID key derivation at save time:")) ||
!observations.some((line) => line.startsWith("JavaScript heap:"))
) {
throw new Error("The benchmark did not report derivation and heap observations.");
}
if ((await snapshotSettings()) !== before) throw new Error("The benchmark changed the live settings.");
await assertNoHorizontalOverflow(page, harness, { label: "ID benchmark results" });
});
}
async function runVaultFixture(): Promise<string> { async function runVaultFixture(): Promise<string> {
await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => { await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
await page await page.locator('[data-testid="review-harness-run-vault-round-trip"]').click({ timeout: uiTimeoutMs });
.locator('[data-testid="review-harness-run-vault-round-trip"]')
.click({ timeout: uiTimeoutMs });
const confirmation = page.locator(".modal-container").filter({ const confirmation = page.locator(".modal-container").filter({
has: page.getByText("Review Harness: Vault fixture access", { exact: true }), has: page.getByText("Review Harness: Vault fixture access", { exact: true }),
}); });
@@ -273,27 +317,22 @@ async function restartAndResumeHarness(): Promise<string> {
}); });
await keepCompatibilityPaused(); await keepCompatibilityPaused();
await waitForHarness(); await waitForHarness();
return await captureObsidianDialogue( return await captureObsidianDialogue(obsidianRemoteDebuggingPort(), "review-harness-resumed.png", async (page) => {
obsidianRemoteDebuggingPort(), const harness = page.locator('[data-testid="review-harness"]');
"review-harness-resumed.png", await harness
async (page) => { .locator('[data-testid="review-harness-resumed"]')
const harness = page.locator('[data-testid="review-harness"]'); .waitFor({ state: "visible", timeout: uiTimeoutMs });
await harness const continuationRemoved = await page.evaluate((stateKey) => {
.locator('[data-testid="review-harness-resumed"]') const plugin = (globalThis as ReviewHarnessTestGlobal).app?.plugins?.plugins["obsidian-livesync"];
.waitFor({ state: "visible", timeout: uiTimeoutMs }); if (typeof plugin !== "object" || plugin === null || !("core" in plugin)) {
const continuationRemoved = await page.evaluate((stateKey) => { throw new Error("Self-hosted LiveSync is unavailable after restart.");
const plugin = (globalThis as ReviewHarnessTestGlobal).app?.plugins?.plugins["obsidian-livesync"]; }
if (typeof plugin !== "object" || plugin === null || !("core" in plugin)) { const core = (plugin as { core: { services: { setting: { getSmallConfig(key: string): string } } } }).core;
throw new Error("Self-hosted LiveSync is unavailable after restart."); return core.services.setting.getSmallConfig(stateKey) === "";
} }, REVIEW_HARNESS_STATE_KEY);
const core = (plugin as { core: { services: { setting: { getSmallConfig(key: string): string } } } }) if (!continuationRemoved) throw new Error("The one-shot continuation was not removed before use.");
.core; await assertNoHorizontalOverflow(page, harness, { label: "resumed Review Harness" });
return core.services.setting.getSmallConfig(stateKey) === ""; });
}, REVIEW_HARNESS_STATE_KEY);
if (!continuationRemoved) throw new Error("The one-shot continuation was not removed before use.");
await assertNoHorizontalOverflow(page, harness, { label: "resumed Review Harness" });
}
);
} }
async function completeResumedCompatibilityStep(): Promise<void> { async function completeResumedCompatibilityStep(): Promise<void> {
@@ -332,9 +371,7 @@ async function copyAndReadReport(): Promise<string> {
undefined, undefined,
{ timeout: uiTimeoutMs } { timeout: uiTimeoutMs }
); );
return await page.evaluate( return await page.evaluate(() => (globalThis as ReviewHarnessTestGlobal).reviewHarnessCopiedReport ?? "");
() => (globalThis as ReviewHarnessTestGlobal).reviewHarnessCopiedReport ?? ""
);
}); });
} }
@@ -348,35 +385,36 @@ async function verifyMobileHarness(): Promise<string> {
if (typeof plugin !== "object" || plugin === null || !("core" in plugin)) { if (typeof plugin !== "object" || plugin === null || !("core" in plugin)) {
throw new Error("Self-hosted LiveSync is unavailable in mobile test mode."); throw new Error("Self-hosted LiveSync is unavailable in mobile test mode.");
} }
const core = (plugin as { const core = (
core: { services: { API: { showWindow(type: string): Promise<void> } } }; plugin as {
}).core; core: { services: { API: { showWindow(type: string): Promise<void> } } };
}
).core;
await core.services.API.showWindow(viewType); await core.services.API.showWindow(viewType);
}, "self-hosted-livesync-review-harness"); }, "self-hosted-livesync-review-harness");
}); });
return await captureObsidianDialogue( await runIdBenchmark();
obsidianRemoteDebuggingPort(), return await captureObsidianDialogue(obsidianRemoteDebuggingPort(), "review-harness-mobile.png", async (page) => {
"review-harness-mobile.png", const harness = page.locator('[data-testid="review-harness"]');
async (page) => { await harness.waitFor({ state: "visible", timeout: uiTimeoutMs });
const harness = page.locator('[data-testid="review-harness"]'); await harness.getByRole("heading", { name: "Self-hosted LiveSync review harness" }).scrollIntoViewIfNeeded();
await harness.waitFor({ state: "visible", timeout: uiTimeoutMs }); await assertNoHorizontalOverflow(page, harness, { label: "mobile Review Harness" });
await assertNoHorizontalOverflow(page, harness, { label: "mobile Review Harness" }); const heading = harness.getByRole("heading", { name: "Self-hosted LiveSync review harness" });
const heading = harness.getByRole("heading", { name: "Self-hosted LiveSync review harness" }); await assertLocatorWithinSafeArea(page, heading, {
await assertLocatorWithinSafeArea(page, heading, { label: "mobile Review Harness heading",
label: "mobile Review Harness heading", safeAreaInsets: iPhoneSafeArea,
safeAreaInsets: iPhoneSafeArea, });
for (const testId of [
"review-harness-run-automatic",
"review-harness-run-full",
"review-harness-copy-report",
"review-harness-run-id-generation-performance",
]) {
await assertLocatorHasMinimumTouchTarget(page, harness.locator(`[data-testid="${testId}"]`), {
label: testId,
}); });
for (const testId of [
"review-harness-run-automatic",
"review-harness-run-full",
"review-harness-copy-report",
]) {
await assertLocatorHasMinimumTouchTarget(page, harness.locator(`[data-testid="${testId}"]`), {
label: testId,
});
}
} }
); });
} }
async function main(): Promise<void> { async function main(): Promise<void> {
@@ -392,6 +430,7 @@ async function main(): Promise<void> {
vault, vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000), startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
pluginData: { pluginData: {
// Config Doctor is covered by settings-ui; this fixture exercises the Harness.
doctorProcessedVersion: DoctorRegulation.version, doctorProcessedVersion: DoctorRegulation.version,
settingVersion: CURRENT_SETTING_VERSION, settingVersion: CURRENT_SETTING_VERSION,
isConfigured: true, isConfigured: true,
@@ -442,15 +481,34 @@ async function main(): Promise<void> {
const vaultConfirmationScreenshot = await runVaultFixture(); const vaultConfirmationScreenshot = await runVaultFixture();
const resumedScreenshot = await restartAndResumeHarness(); const resumedScreenshot = await restartAndResumeHarness();
await completeResumedCompatibilityStep(); await completeResumedCompatibilityStep();
await runIdBenchmark();
const report = await copyAndReadReport(); const report = await copyAndReadReport();
if (!report.includes("## Self-hosted LiveSync Review Harness report")) { if (!report.includes("## Self-hosted LiveSync Review Harness report")) {
throw new Error("The copied Review Harness report was not Markdown evidence."); throw new Error("The copied Review Harness report was not Markdown evidence.");
} }
for (const forbidden of [vault.name, REVIEW_HARNESS_FIXTURE_ROOT]) { for (const expected of [
if (report.includes(forbidden)) throw new Error(`The Review Harness report exposed local state: ${forbidden}`); "1000 IDs total median=",
"; per ID=",
"ID key derivation at save time:",
"JavaScript heap:",
]) {
if (!report.includes(expected)) throw new Error(`Missing copied benchmark observation: ${expected}`);
}
for (const forbidden of [
vault.name,
REVIEW_HARNESS_FIXTURE_ROOT,
"ab".repeat(32),
"Self-hosted LiveSync ID benchmark passphrase",
"Self-hosted LiveSync ID benchmark source",
]) {
if (report.includes(forbidden))
throw new Error(`The Review Harness report exposed local state: ${forbidden}`);
} }
const mobileScreenshot = await verifyMobileHarness(); const mobileScreenshot = await verifyMobileHarness();
const outputDirectory = process.env.E2E_OBSIDIAN_DIAGNOSTICS_DIR ?? "/tmp/obsidian-livesync-e2e";
await mkdir(outputDirectory, { recursive: true });
await writeFile(join(outputDirectory, "review-harness-report.md"), report, "utf8");
console.log( console.log(
`Review Harness passed one-shot, fixture, report, and mobile checks. Screenshots: ${[ `Review Harness passed one-shot, fixture, report, and mobile checks. Screenshots: ${[
initialScreenshot, initialScreenshot,
+1
View File
@@ -21,6 +21,7 @@ const focusedScenarios = new Set([
"cli-to-obsidian-sync", "cli-to-obsidian-sync",
"minio-upload", "minio-upload",
"object-storage-setup-uri-workflow", "object-storage-setup-uri-workflow",
"object-storage-compatible-setup-uri-workflow",
"object-storage-qr-workflow", "object-storage-qr-workflow",
"object-storage-custom-http-handler-setup-uri-workflow", "object-storage-custom-http-handler-setup-uri-workflow",
"p2p-setup-uri-workflow", "p2p-setup-uri-workflow",
@@ -1,5 +1,6 @@
import { readFile } from "node:fs/promises"; import { readFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { deriveIdKey } from "@vrtmrz/livesync-commonlib/settings";
import { evalObsidianJson } from "../runner/cli.ts"; import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts"; import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { assertEqual } from "../runner/liveSyncWorkflow.ts"; import { assertEqual } from "../runner/liveSyncWorkflow.ts";
@@ -34,7 +35,11 @@ async function waitForFileContaining(
throw new Error(`Timed out waiting for setting Markdown: ${fullPath}\nLast error: ${String(lastError)}`); throw new Error(`Timed out waiting for setting Markdown: ${fullPath}\nLast error: ${String(lastError)}`);
} }
async function configureSettingMarkdown(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> { async function configureSettingMarkdown(
cliBinary: string,
env: NodeJS.ProcessEnv,
idDerivationKey: string
): Promise<void> {
await evalObsidianJson<unknown>( await evalObsidianJson<unknown>(
cliBinary, cliBinary,
[ [
@@ -46,6 +51,8 @@ async function configureSettingMarkdown(cliBinary: string, env: NodeJS.ProcessEn
"couchDB_USER:'e2e-user',", "couchDB_USER:'e2e-user',",
"couchDB_PASSWORD:'e2e-password',", "couchDB_PASSWORD:'e2e-password',",
"passphrase:'e2e-passphrase',", "passphrase:'e2e-passphrase',",
"idDerivationVersion:1,",
`idDerivationKey:${JSON.stringify(idDerivationKey)},`,
"showVerboseLog:true,", "showVerboseLog:true,",
"},true);", "},true);",
"await core.services.setting.saveSettingData();", "await core.services.setting.saveSettingData();",
@@ -64,6 +71,7 @@ async function main(): Promise<void> {
} }
const vault = await createTemporaryVault(); const vault = await createTemporaryVault();
const idDerivationKey = await deriveIdKey("setting-markdown-export-independent-id-key-fixture");
let session: ObsidianLiveSyncSession | undefined; let session: ObsidianLiveSyncSession | undefined;
try { try {
console.log(`Using Obsidian executable: ${binary}`); console.log(`Using Obsidian executable: ${binary}`);
@@ -77,19 +85,39 @@ async function main(): Promise<void> {
}); });
// The export is available while an unconfigured Vault remains outside // The export is available while an unconfigured Vault remains outside
// application readiness; the session helper has already loaded the plug-in. // application readiness; the session helper has already loaded the plug-in.
await configureSettingMarkdown(cli.binary, session.cliEnv); await configureSettingMarkdown(cli.binary, session.cliEnv, idDerivationKey);
const content = await waitForFileContaining(vault.path, settingPath, [ const content = await waitForFileContaining(vault.path, settingPath, [
(value) => value.includes("````yaml:livesync-setting"), (value) => value.includes("````yaml:livesync-setting"),
(value) => value.includes(`settingSyncFile: ${settingPath}`), (value) => value.includes(`settingSyncFile: ${settingPath}`),
(value) => value.includes("showVerboseLog: true"), (value) => value.includes("showVerboseLog: true"),
]); ]);
const persisted = JSON.parse(
await readFile(join(vault.path, ".obsidian", "plugins", "obsidian-livesync", "data.json"), "utf-8")
) as {
idDerivationVersion?: unknown;
idDerivationKey?: unknown;
encryptedIdDerivationKey?: unknown;
};
assertEqual(persisted.idDerivationVersion, 1, "The independent ID key fixture was not persisted.");
assertEqual(persisted.idDerivationKey, "", "The independent ID key was stored in plain text locally.");
const encryptedIdDerivationKey = persisted.encryptedIdDerivationKey;
if (typeof encryptedIdDerivationKey !== "string" || encryptedIdDerivationKey.length === 0) {
throw new Error("The independent ID key fixture was not saved in encrypted local settings.");
}
assertEqual( assertEqual(
content.includes("couchDB_PASSWORD: e2e-password"), content.includes("couchDB_PASSWORD: e2e-password"),
false, false,
"Credential leaked into setting Markdown." "Credential leaked into setting Markdown."
); );
assertEqual(content.includes("passphrase: e2e-passphrase"), false, "Passphrase leaked into setting Markdown."); assertEqual(content.includes("passphrase: e2e-passphrase"), false, "Passphrase leaked into setting Markdown.");
assertEqual(content.includes(idDerivationKey), false, "Plaintext ID key leaked into setting Markdown.");
assertEqual(
content.includes(encryptedIdDerivationKey),
false,
"Encrypted ID key leaked into setting Markdown."
);
console.log(`Generated setting Markdown without credentials: ${settingPath}`); console.log(`Generated setting Markdown without credentials: ${settingPath}`);
} finally { } finally {
+232 -21
View File
@@ -1,11 +1,16 @@
import { mkdir, readFile, rename as renameFilesystemPath, rm, writeFile } from "node:fs/promises"; import { mkdir, readFile, rename as renameFilesystemPath, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path"; import { dirname, join } from "node:path";
import { SALT_OF_PASSPHRASE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { encryptString } from "@vrtmrz/livesync-commonlib/compat/encryption/stringEncryption";
import { deriveIdKey } from "@vrtmrz/livesync-commonlib/settings";
import { CENTRAL_COMPATIBILITY_REJECTION_REASONS } from "@vrtmrz/livesync-commonlib/replication";
import { evalObsidianJson } from "../runner/cli.ts"; import { evalObsidianJson } from "../runner/cli.ts";
import { import {
assertCouchDbReachable, assertCouchDbReachable,
createCouchDbDatabase, createCouchDbDatabase,
deleteCouchDbDatabase, deleteCouchDbDatabase,
fetchAllCouchDbDocs, fetchAllCouchDbDocs,
fetchCouchDbLocalDocs,
loadCouchDbConfig, loadCouchDbConfig,
makeUniqueDatabaseName, makeUniqueDatabaseName,
waitForCouchDbDocs, waitForCouchDbDocs,
@@ -18,6 +23,7 @@ import {
assertE2eCompatibilityUnpaused, assertE2eCompatibilityUnpaused,
configureCouchDb, configureCouchDb,
createE2eCouchDbPluginData, createE2eCouchDbPluginData,
createE2eObsidianDeviceLocalState,
prepareRemote, prepareRemote,
pushLocalChanges, pushLocalChanges,
waitForLiveSyncCoreReady, waitForLiveSyncCoreReady,
@@ -439,7 +445,8 @@ async function renameNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv,
async function startConfiguredSession( async function startConfiguredSession(
context: RunnerContext, context: RunnerContext,
vault: TemporaryVault, vault: TemporaryVault,
overrides: Record<string, unknown> = {} overrides: Record<string, unknown> = {},
persistedOverrides: Record<string, unknown> = overrides
): Promise<ObsidianLiveSyncSession> { ): Promise<ObsidianLiveSyncSession> {
const couchDbSettings = { const couchDbSettings = {
uri: context.couchDb.uri, uri: context.couchDb.uri,
@@ -452,7 +459,7 @@ async function startConfiguredSession(
cliBinary: context.cliBinary, cliBinary: context.cliBinary,
vault, vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000), startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
pluginData: createE2eCouchDbPluginData(couchDbSettings, overrides), pluginData: createE2eCouchDbPluginData(couchDbSettings, persistedOverrides),
}); });
context.activeSessions.add(session); context.activeSessions.add(session);
try { try {
@@ -961,6 +968,193 @@ async function runEncryptedRoundTrip(
console.log("Two-vault encrypted note synchronisation round-tripped."); console.log("Two-vault encrypted note synchronisation round-tripped.");
} }
async function runIndependentIdRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const source = "real-obsidian-e2e-independent-id-source";
const key = await deriveIdKey(source);
const content = "# Shared content with an independent ID key.\n";
const pathA = "E2E/independent-ids/from-a.md";
const pathB = "E2E/independent-ids/from-b.md";
const overrides = {
encrypt: true,
passphrase: "real-obsidian-e2e-independent-passphrase",
usePathObfuscation: true,
E2EEAlgorithm: "v2",
idDerivationVersion: 1,
idDerivationKey: key,
};
const persistedOverrides = {
...overrides,
idDerivationKey: "",
encryptedIdDerivationKey: await encryptString(key, `*${SALT_OF_PASSPHRASE}`),
};
let session = await startConfiguredSession(context, vaultA, overrides, persistedOverrides);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, pathA, content);
const entryA = await uploadNote(context, session, pathA);
if (entryA.children.length === 0) throw new Error("Independent ID mode produced no Chunks.");
await stopTrackedSession(context, session);
session = await startConfiguredSession(context, vaultB, overrides, persistedOverrides);
await syncAndApply(context, session);
await waitForPathContent(vaultB.path, pathA, (received) => received === content);
const receivedA = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, pathA);
assertEqual(receivedA.id, entryA.id, "The second device did not preserve the obfuscated document ID.");
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, pathB, content);
const entryB = await uploadNote(context, session, pathB);
assertEqual(
JSON.stringify(entryB.children),
JSON.stringify(entryA.children),
"The second device did not reuse the same content-derived Chunk IDs."
);
await stopTrackedSession(context, session);
session = await startConfiguredSession(context, vaultA, overrides, persistedOverrides);
await syncAndApply(context, session);
await waitForPathContent(vaultA.path, pathB, (received) => received === content);
const receivedB = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, pathB);
assertEqual(receivedB.id, entryB.id, "The first device did not preserve the return document ID.");
await stopTrackedSession(context, session);
console.log("Two real Obsidian devices shared independent document and Chunk IDs in both directions.");
for (const [label, candidateKey] of [
["different key", "cd".repeat(32)],
["legacy IDs", ""],
] as const) {
const remoteBefore = await fetchAllCouchDbDocs(context.couchDb, context.dbName);
const checkpointsBefore = await fetchCouchDbLocalDocs(context.couchDb, context.dbName);
const rejectedVault = await createTemporaryVault();
let rejectedSession: ObsidianLiveSyncSession | undefined;
try {
rejectedSession = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
vault: rejectedVault,
localStorageEntries: createE2eObsidianDeviceLocalState(rejectedVault.name),
pluginData: createE2eCouchDbPluginData(
{ ...context.couchDb, dbName: context.dbName },
{
...overrides,
idDerivationVersion: candidateKey ? 1 : 0,
idDerivationKey: "",
encryptedIdDerivationKey: candidateKey
? await encryptString(candidateKey, `*${SALT_OF_PASSPHRASE}`)
: "",
}
),
});
context.activeSessions.add(rejectedSession);
await waitForLiveSyncCoreReady(context.cliBinary, rejectedSession.cliEnv);
const unsentPath = "E2E/independent-ids/rejected.md";
await writeNoteViaObsidian(context.cliBinary, rejectedSession.cliEnv, unsentPath, content);
await waitForLocalDatabaseEntry(context.cliBinary, rejectedSession.cliEnv, unsentPath);
const attempt = await evalObsidianJson<{ admitted: boolean; reason: string; replicated: boolean }>(
context.cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const replicator=core.services.replicator.getActiveReplicator();",
"const settings=core.services.setting.currentSettings();",
"let reason='';",
"const connection=await replicator.checkReplicationConnectivity(settings,false,false,false,false,undefined,(decision)=>{reason=decision.reason??'';});",
"if(connection) await connection.close();",
"const replicated=await core.services.replication.replicate(true);",
"return JSON.stringify({admitted:!!connection,reason,replicated:!!replicated});",
"})()",
].join(""),
rejectedSession.cliEnv
);
assertEqual(attempt.admitted, false, `CouchDB admitted ${label} for obfuscated document IDs.`);
assertEqual(
attempt.reason,
CENTRAL_COMPATIBILITY_REJECTION_REASONS.ID_DERIVATION_MISMATCH,
`CouchDB rejected ${label} for an unrelated reason.`
);
assertEqual(attempt.replicated, false, `Ordinary replication accepted ${label}.`);
assertEqual(
await pathExists(rejectedVault.path, pathA),
false,
"A rejected device received a remote note."
);
await stopTrackedSession(context, rejectedSession);
rejectedSession = undefined;
assertEqual(
JSON.stringify(await fetchAllCouchDbDocs(context.couchDb, context.dbName)),
JSON.stringify(remoteBefore),
`A rejected ${label} connection changed remote documents.`
);
assertEqual(
JSON.stringify(await fetchCouchDbLocalDocs(context.couchDb, context.dbName)),
JSON.stringify(checkpointsBefore),
`A rejected ${label} connection changed remote checkpoints.`
);
} finally {
if (rejectedSession) await stopTrackedSession(context, rejectedSession);
await rejectedVault.dispose();
}
}
console.log("Ordinary CouchDB replication rejected different and legacy document ID keys without remote writes.");
}
async function runDifferentChunkIdKeysRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const keyA = await deriveIdKey("real-obsidian-e2e-chunk-source-a");
const keyB = await deriveIdKey("real-obsidian-e2e-chunk-source-b");
const content = "# Shared content with different Chunk ID keys.\n";
const pathA = "E2E/chunk-id-keys/from-a.md";
const pathB = "E2E/chunk-id-keys/from-b.md";
const commonSettings = {
encrypt: true,
passphrase: "real-obsidian-e2e-chunk-passphrase",
usePathObfuscation: false,
E2EEAlgorithm: "v2",
idDerivationVersion: 1,
};
const settingsFor = (key: string) => ({ ...commonSettings, idDerivationKey: key });
const persistedSettingsFor = async (key: string) => ({
...settingsFor(key),
idDerivationKey: "",
encryptedIdDerivationKey: await encryptString(key, `*${SALT_OF_PASSPHRASE}`),
});
const persistedA = await persistedSettingsFor(keyA);
const persistedB = await persistedSettingsFor(keyB);
let session = await startConfiguredSession(context, vaultA, settingsFor(keyA), persistedA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, pathA, content);
const entryA = await uploadNote(context, session, pathA);
if (entryA.children.length === 0) throw new Error("The first device produced no Chunks.");
await stopTrackedSession(context, session);
session = await startConfiguredSession(context, vaultB, settingsFor(keyB), persistedB);
await syncAndApply(context, session);
await waitForPathContent(vaultB.path, pathA, (received) => received === content);
const receivedA = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, pathA);
assertEqual(receivedA.id, entryA.id, "The second device changed the visible document ID.");
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, pathB, content);
const entryB = await uploadNote(context, session, pathB);
if (entryB.children.length === 0) throw new Error("The second device produced no Chunks.");
if (JSON.stringify(entryB.children) === JSON.stringify(entryA.children)) {
throw new Error("Different ID keys unexpectedly generated the same Chunk IDs.");
}
await stopTrackedSession(context, session);
session = await startConfiguredSession(context, vaultA, settingsFor(keyA), persistedA);
await syncAndApply(context, session);
await waitForPathContent(vaultA.path, pathB, (received) => received === content);
const receivedB = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, pathB);
assertEqual(receivedB.id, entryB.id, "The first device changed the return document ID.");
await stopTrackedSession(context, session);
console.log("Two real Obsidian devices exchanged notes with different Chunk ID keys and visible document paths.");
}
async function runMarkdownAutoMerge( async function runMarkdownAutoMerge(
context: RunnerContext, context: RunnerContext,
vaultA: TemporaryVault, vaultA: TemporaryVault,
@@ -1085,10 +1279,9 @@ async function runConflictTimeStorageOperations(
showMergeDialogOnlyOnActive: true, showMergeDialogOnlyOnActive: true,
handleFilenameCaseSensitive: false, handleFilenameCaseSensitive: false,
}; };
const baseContent = Object.fromEntries(paths.map((path) => [path, `# Conflict operation\n\nBase for ${path}.\n`])) as Record< const baseContent = Object.fromEntries(
(typeof paths)[number], paths.map((path) => [path, `# Conflict operation\n\nBase for ${path}.\n`])
string ) as Record<(typeof paths)[number], string>;
>;
const leftContent = Object.fromEntries( const leftContent = Object.fromEntries(
paths.map((path) => [path, `${baseContent[path]}\nEdit made on Vault A.\n`]) paths.map((path) => [path, `${baseContent[path]}\nEdit made on Vault A.\n`])
) as Record<(typeof paths)[number], string>; ) as Record<(typeof paths)[number], string>;
@@ -1129,7 +1322,9 @@ async function runConflictTimeStorageOperations(
const initialBranchRevisions = new Map<string, Set<string>>(); const initialBranchRevisions = new Map<string, Set<string>>();
for (const path of paths) { for (const path of paths) {
const state = await waitForFileConflict(context.cliBinary, session.cliEnv, path); const state = await waitForFileConflict(context.cliBinary, session.cliEnv, path);
const displayedBranch = state.branches.find((branch) => branch.content === rightContent[path] && !branch.deleted); const displayedBranch = state.branches.find(
(branch) => branch.content === rightContent[path] && !branch.deleted
);
if (!displayedBranch) { if (!displayedBranch) {
throw new Error(`Could not identify the branch displayed by Vault B: ${path}; ${JSON.stringify(state)}`); throw new Error(`Could not identify the branch displayed by Vault B: ${path}; ${JSON.stringify(state)}`);
} }
@@ -1170,12 +1365,7 @@ async function runConflictTimeStorageOperations(
"A conflict-time deletion did not extend the displayed revision." "A conflict-time deletion did not extend the displayed revision."
); );
await renameNoteViaObsidian( await renameNoteViaObsidian(context.cliBinary, session.cliEnv, conflictCaseFromPath, conflictCaseToPath);
context.cliBinary,
session.cliEnv,
conflictCaseFromPath,
conflictCaseToPath
);
const caseRenamedBranch = await waitForConflictBranch( const caseRenamedBranch = await waitForConflictBranch(
context.cliBinary, context.cliBinary,
session.cliEnv, session.cliEnv,
@@ -1216,12 +1406,7 @@ async function runConflictTimeStorageOperations(
"A conflict-time case-only rename did not record the new displayed revision." "A conflict-time case-only rename did not record the new displayed revision."
); );
await renameNoteViaObsidian( await renameNoteViaObsidian(context.cliBinary, session.cliEnv, conflictRenameFromPath, conflictRenameToPath);
context.cliBinary,
session.cliEnv,
conflictRenameFromPath,
conflictRenameToPath
);
const renamedTarget = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, conflictRenameToPath); const renamedTarget = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, conflictRenameToPath);
const renamedSourceDeletion = await waitForConflictBranch( const renamedSourceDeletion = await waitForConflictBranch(
context.cliBinary, context.cliBinary,
@@ -1367,10 +1552,13 @@ async function main(): Promise<void> {
const couchDb = await loadCouchDbConfig(); const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync"); const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync");
const encryptedDbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync-e2ee"); const encryptedDbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync-e2ee");
const independentDbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync-independent-ids");
const vaultA = await createTemporaryVault(); const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault(); const vaultB = await createTemporaryVault();
const encryptedVaultA = await createTemporaryVault(); const encryptedVaultA = await createTemporaryVault();
const encryptedVaultB = await createTemporaryVault(); const encryptedVaultB = await createTemporaryVault();
const independentVaultA = await createTemporaryVault();
const independentVaultB = await createTemporaryVault();
const context: RunnerContext = { const context: RunnerContext = {
binary, binary,
cliBinary: cli.binary, cliBinary: cli.binary,
@@ -1385,11 +1573,19 @@ async function main(): Promise<void> {
dbName: encryptedDbName, dbName: encryptedDbName,
activeSessions: new Set(), activeSessions: new Set(),
}; };
const independentContext: RunnerContext = {
binary,
cliBinary: cli.binary,
couchDb,
dbName: independentDbName,
activeSessions: new Set(),
};
try { try {
await assertCouchDbReachable(couchDb); await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName); await createCouchDbDatabase(couchDb, dbName);
await createCouchDbDatabase(couchDb, encryptedDbName); await createCouchDbDatabase(couchDb, encryptedDbName);
await createCouchDbDatabase(couchDb, independentDbName);
console.log(`Using Obsidian executable: ${binary}`); console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`); console.log(`Temporary vault A: ${vaultA.path}`);
@@ -1398,11 +1594,13 @@ async function main(): Promise<void> {
console.log(`Temporary encrypted CouchDB database: ${encryptedDbName}`); console.log(`Temporary encrypted CouchDB database: ${encryptedDbName}`);
const onlyParentCaseDeletion = process.env.E2E_OBSIDIAN_ONLY_PARENT_CASE_DELETION === "true"; const onlyParentCaseDeletion = process.env.E2E_OBSIDIAN_ONLY_PARENT_CASE_DELETION === "true";
const onlyIndependentIds = process.env.E2E_OBSIDIAN_ONLY_INDEPENDENT_IDS === "true";
const onlyDifferentChunkIdKeys = process.env.E2E_OBSIDIAN_ONLY_DIFFERENT_CHUNK_ID_KEYS === "true";
if (onlyParentCaseDeletion) { if (onlyParentCaseDeletion) {
await runParentCaseDeletionProtection(context, vaultA, vaultB); await runParentCaseDeletionProtection(context, vaultA, vaultB);
} }
const onlyConflictOperations = process.env.E2E_OBSIDIAN_ONLY_CONFLICT_OPERATIONS === "true"; const onlyConflictOperations = process.env.E2E_OBSIDIAN_ONLY_CONFLICT_OPERATIONS === "true";
if (!onlyParentCaseDeletion && !onlyConflictOperations) { if (!onlyParentCaseDeletion && !onlyConflictOperations && !onlyIndependentIds && !onlyDifferentChunkIdKeys) {
await runCreateUpdateDelete(context, vaultA, vaultB); await runCreateUpdateDelete(context, vaultA, vaultB);
await runRename(context, vaultA, vaultB); await runRename(context, vaultA, vaultB);
await runCaseOnlyRename(context, vaultA, vaultB); await runCaseOnlyRename(context, vaultA, vaultB);
@@ -1416,17 +1614,27 @@ async function main(): Promise<void> {
) { ) {
await runConflictTimeStorageOperations(context, vaultA, vaultB); await runConflictTimeStorageOperations(context, vaultA, vaultB);
} }
if (!onlyParentCaseDeletion && !onlyConflictOperations) { if (!onlyParentCaseDeletion && !onlyConflictOperations && !onlyIndependentIds && !onlyDifferentChunkIdKeys) {
await runTargetMismatch(context, vaultA, vaultB); await runTargetMismatch(context, vaultA, vaultB);
await runEncryptedRoundTrip(encryptedContext, encryptedVaultA, encryptedVaultB); await runEncryptedRoundTrip(encryptedContext, encryptedVaultA, encryptedVaultB);
} }
if (!onlyParentCaseDeletion && !onlyConflictOperations) {
if (onlyDifferentChunkIdKeys) {
await runDifferentChunkIdKeysRoundTrip(independentContext, independentVaultA, independentVaultB);
} else {
await runIndependentIdRoundTrip(independentContext, independentVaultA, independentVaultB);
}
}
} finally { } finally {
await stopTrackedSessions(context); await stopTrackedSessions(context);
await stopTrackedSessions(encryptedContext); await stopTrackedSessions(encryptedContext);
await stopTrackedSessions(independentContext);
await vaultA.dispose(); await vaultA.dispose();
await vaultB.dispose(); await vaultB.dispose();
await encryptedVaultA.dispose(); await encryptedVaultA.dispose();
await encryptedVaultB.dispose(); await encryptedVaultB.dispose();
await independentVaultA.dispose();
await independentVaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") { if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => { await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error); console.warn(error instanceof Error ? error.message : error);
@@ -1434,6 +1642,9 @@ async function main(): Promise<void> {
await deleteCouchDbDatabase(couchDb, encryptedDbName).catch((error: unknown) => { await deleteCouchDbDatabase(couchDb, encryptedDbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error); console.warn(error instanceof Error ? error.message : error);
}); });
await deleteCouchDbDatabase(couchDb, independentDbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
} }
} }
} }
+12
View File
@@ -16,6 +16,9 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
#### New Feature #### New Feature
- An optional saved ID key can generate encrypted Chunk IDs and obfuscated Metadata document IDs independently of the current E2EE passphrase.
- New Vaults use a random key by default; existing Vaults keep their current ID configuration by default. You can also derive a key from the current E2EE passphrase, enter a separate source, or import a recovery code. The source is not retained; the saved key can be revealed locally as a recovery code.
- The saved key stays in place when the E2EE passphrase changes or E2EE is turned off. Share it with another device through a protected Setup URI. Changing document IDs on an existing remote requires the usual Rebuild and Fetch procedure.
- We can now keep the file properties used by Hidden File Sync and Customisation Sync private in CouchDB. - We can now keep the file properties used by Hidden File Sync and Customisation Sync private in CouchDB.
- **Encrypt internal file Properties** extends E2EE V2 and Property Encryption to their paths, times, sizes, and Chunk references. - **Encrypt internal file Properties** extends E2EE V2 and Property Encryption to their paths, times, sizes, and Chunk references.
- Existing configurations keep this preference disabled. New Vaults enable it for use when the required encryption settings are active. - Existing configurations keep this preference disabled. New Vaults enable it for use when the required encryption settings are active.
@@ -23,6 +26,9 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
- We can now see which unsupported feature prevents a client from synchronising with CouchDB. - We can now see which unsupported feature prevents a client from synchronising with CouchDB.
- Clients check the features required by the remote before transferring data or resetting the local database for Fast Fetch. Receiving an unsupported requirement also stops active replication. - Clients check the features required by the remote before transferring data or resetting the local database for Fast Fetch. Receiving an unsupported requirement also stops active replication.
- We can now compare ID generation performance on a desktop or mobile device through **Open review harness**, available with the developers' debug tools enabled.
- The copied report includes legacy and independent ID timings and, where available, approximate JavaScript heap samples. The measurement uses fixed test data and keeps our Vault and settings unchanged.
#### Fixed #### Fixed
- We can now keep using an E2EE passphrase beginning with `%` after restarting Obsidian. (#1221) - We can now keep using an E2EE passphrase beginning with `%` after restarting Obsidian. (#1221)
@@ -30,6 +36,12 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
### Setup ### Setup
#### New Feature
- We can now share a Setup URI with a displayed time limit, or choose **Compatible** for reuse without a time limit.
- **Time-bound** uses the current fixed seven-day UTC window, so the displayed end may be less than seven days away. Compatible retains the existing URI format; receiving devices still need to support the shared settings.
- The time condition applies when opening the URI. It does not revoke imported credentials or prevent reuse after rolling the device clock back.
#### Improved #### Improved
- We can now distinguish the three Setup URI and QR code choices by their short labels and icons: initialise or overwrite the remote, join this device, or apply settings only. - We can now distinguish the three Setup URI and QR code choices by their short labels and icons: initialise or overwrite the remote, join this device, or apply settings only.
+7
View File
@@ -35,6 +35,13 @@ Deno.test("generates a current self-hosted Setup URI through the published Commo
const decoded = await decodeSettingsFromSetupURI(setupURI, "setup-secret"); const decoded = await decodeSettingsFromSetupURI(setupURI, "setup-secret");
assert(decoded, "Commonlib could not decode the generated Setup URI"); assert(decoded, "Commonlib could not decode the generated Setup URI");
const effectiveSettings = { ...DEFAULT_SETTINGS, ...decoded }; const effectiveSettings = { ...DEFAULT_SETTINGS, ...decoded };
const recoveryCode = stdout.match(/sls-id-v1:[0-9a-f]{64}/u)?.[0];
assert(recoveryCode, "the generator did not print an ID recovery code");
assert(
(effectiveSettings as typeof effectiveSettings & { idDerivationKey?: string }).idDerivationKey ===
recoveryCode.slice("sls-id-v1:".length),
"the CouchDB Setup URI did not contain the generated ID key",
);
assert( assert(
effectiveSettings.isConfigured, effectiveSettings.isConfigured,
"the CouchDB Setup URI left the imported device unconfigured", "the CouchDB Setup URI left the imported device unconfigured",
+2
View File
@@ -31,6 +31,8 @@ Authentication and other non-retryable HTTP failures stop immediately. Network a
The existing `flyio/generate_setupuri.ts` path remains a CouchDB-only compatibility wrapper for the Fly.io deployment script. The existing `flyio/generate_setupuri.ts` path remains a CouchDB-only compatibility wrapper for the Fly.io deployment script.
The generator creates a fresh random ID key by default and includes it in the encrypted Setup URI. It prints a tagged `sls-id-v1:` recovery code. Set `id_recovery_code` to that code when generating another URI for the same Vault; a new run without it creates a different key. This restores only the ID key: reuse the original connection details too. For P2P, provide the original `p2p_room_id` and `p2p_passphrase` because omitted values are generated afresh. Set `id_mode=legacy` to generate a URI with the previous ID behaviour. `id_mode=legacy` and `id_recovery_code` cannot be combined. Keep the recovery code private and retain it if every device might be lost.
### CouchDB ### CouchDB
```sh ```sh
+99
View File
@@ -34,6 +34,22 @@ Deno.test("generates an Object Storage Setup URI with a selected S3 profile", as
); );
assert(decoded, "Commonlib could not decode the Object Storage Setup URI"); assert(decoded, "Commonlib could not decode the Object Storage Setup URI");
const effective = { ...DEFAULT_SETTINGS, ...decoded }; const effective = { ...DEFAULT_SETTINGS, ...decoded };
const recoveryCode = generated.idRecoveryCode;
assert(
typeof recoveryCode === "string" && recoveryCode.startsWith("sls-id-v1:"),
"the generator did not return an ID recovery code",
);
assert(
(effective as typeof effective & { idDerivationVersion?: number })
.idDerivationVersion === 1,
"the Setup URI did not enable independent IDs",
);
assert(
(effective as typeof effective & { idDerivationKey?: string })
.idDerivationKey ===
recoveryCode.slice("sls-id-v1:".length),
"the Setup URI did not contain the generated ID key",
);
assert( assert(
effective.isConfigured, effective.isConfigured,
"the Setup URI left the imported device unconfigured", "the Setup URI left the imported device unconfigured",
@@ -82,6 +98,12 @@ Deno.test("generates a random-room P2P Setup URI without copying a device identi
); );
assert(decoded, "Commonlib could not decode the P2P Setup URI"); assert(decoded, "Commonlib could not decode the P2P Setup URI");
const effective = { ...DEFAULT_SETTINGS, ...decoded }; const effective = { ...DEFAULT_SETTINGS, ...decoded };
assert(
(effective as typeof effective & { idDerivationKey?: string })
.idDerivationKey ===
generated.idRecoveryCode?.slice("sls-id-v1:".length),
"the P2P Setup URI did not contain the generated ID key",
);
assert( assert(
/^\d{3}-\d{3}-\d{3}-[a-z0-9]{3}$/.test(effective.P2P_roomID), /^\d{3}-\d{3}-\d{3}-[a-z0-9]{3}$/.test(effective.P2P_roomID),
"Commonlib did not generate the expected random room ID", "Commonlib did not generate the expected random room ID",
@@ -159,3 +181,80 @@ Deno.test("rejects an unknown Setup URI mode", async () => {
} }
assert(rejected, "the generator accepted an unknown Setup URI mode"); assert(rejected, "the generator accepted an unknown Setup URI mode");
}); });
for (const mode of ["ephemeral", "persistent"] as const) {
Deno.test(`preserves ID recovery and explicit legacy IDs in ${mode} URIs`, async () => {
const environment = {
remote_type: "p2p",
uri_mode: mode,
passphrase: "vault-secret",
uri_passphrase: "setup-secret",
};
const first = await generateSetupURI(environment);
const second = await generateSetupURI({
...environment,
id_recovery_code: first.idRecoveryCode,
});
const independentlyGenerated = await generateSetupURI(environment);
assert(
second.idRecoveryCode === first.idRecoveryCode,
"the recovery code changed on repeat generation",
);
assert(
independentlyGenerated.idRecoveryCode !== first.idRecoveryCode,
"the default ID key was reused",
);
const repeatedSettings = await decodeSettingsFromSetupURI(
second.setupURI,
second.setupPassphrase,
);
assert(repeatedSettings, "the repeated Setup URI could not be decoded");
assert(
(repeatedSettings as typeof repeatedSettings & {
idDerivationKey?: string;
}).idDerivationKey ===
first.idRecoveryCode?.slice("sls-id-v1:".length),
"the recovery code did not restore the original ID key",
);
const legacy = await generateSetupURI({
...environment,
id_mode: "legacy",
});
const decoded = await decodeSettingsFromSetupURI(
legacy.setupURI,
legacy.setupPassphrase,
);
assert(decoded, "the legacy Setup URI could not be decoded");
assert(
legacy.idRecoveryCode === undefined,
"legacy mode returned an ID recovery code",
);
assert(
(decoded as typeof decoded & { idDerivationVersion?: number })
.idDerivationVersion !== 1,
"legacy mode enabled independent IDs",
);
let rejected = false;
try {
await generateSetupURI({
...environment,
id_recovery_code: "sls-id-v1:wrong",
});
} catch {
rejected = true;
}
assert(rejected, "an invalid recovery code was accepted");
rejected = false;
try {
await generateSetupURI({
...environment,
id_mode: "legacy",
id_recovery_code: first.idRecoveryCode,
});
} catch {
rejected = true;
}
assert(rejected, "legacy mode silently ignored a recovery code");
});
}
+44
View File
@@ -23,6 +23,42 @@ export interface GeneratedSetupURI {
setupPassphrase: string; setupPassphrase: string;
mode: TimeBoundSetupURIMode; mode: TimeBoundSetupURIMode;
usableUntil: number | null; usableUntil: number | null;
idRecoveryCode?: string;
}
const ID_RECOVERY_CODE_PREFIX = "sls-id-v1:";
const ID_RECOVERY_CODE_PATTERN = /^sls-id-v1:([0-9a-f]{64})$/u;
function generateRandomIdKey(): string {
const bytes = crypto.getRandomValues(new Uint8Array(32));
return Array.from(bytes, (byte) => byte.toString(16).padStart(2, "0")).join(
"",
);
}
function configureIdDerivation(
settings: ObsidianLiveSyncSettings,
environment: SetupGeneratorEnvironment,
): string | undefined {
const mode = environment.id_mode?.trim().toLowerCase() || "random";
if (mode !== "random" && mode !== "legacy") {
throw new Error("id_mode must be random or legacy");
}
const suppliedCode = environment.id_recovery_code?.trim();
if (mode === "legacy") {
if (suppliedCode) {
throw new Error("id_recovery_code cannot be used with id_mode=legacy");
}
return undefined;
}
const key = suppliedCode
? ID_RECOVERY_CODE_PATTERN.exec(suppliedCode)?.[1]
: generateRandomIdKey();
if (!key) {
throw new Error("id_recovery_code must be a valid sls-id-v1 recovery code");
}
Object.assign(settings, { idDerivationVersion: 1, idDerivationKey: key });
return `${ID_RECOVERY_CODE_PREFIX}${key}`;
} }
function requireValue( function requireValue(
@@ -179,6 +215,7 @@ export async function generateSetupURI(
generateSecret(); generateSecret();
const mode = parseSetupURIMode(environment); const mode = parseSetupURIMode(environment);
const { remoteType, settings } = createSetupSettings(environment); const { remoteType, settings } = createSetupSettings(environment);
const idRecoveryCode = configureIdDerivation(settings, environment);
const { uri, usableUntil } = await encodeTimeBoundSetupURI( const { uri, usableUntil } = await encodeTimeBoundSetupURI(
settings, settings,
setupPassphrase, setupPassphrase,
@@ -200,6 +237,7 @@ export async function generateSetupURI(
setupPassphrase, setupPassphrase,
mode, mode,
usableUntil, usableUntil,
idRecoveryCode,
}; };
} }
@@ -230,6 +268,12 @@ export async function runSetupURIGenerator(
generated.setupPassphrase, generated.setupPassphrase,
); );
console.log("This passphrase is never shown again, so store it safely."); console.log("This passphrase is never shown again, so store it safely.");
if (generated.idRecoveryCode) {
console.log("ID recovery code:", generated.idRecoveryCode);
console.log(
"Use id_recovery_code with this value and reuse the same remote settings when generating another Setup URI for the same Vault.",
);
}
console.log(generated.setupURI); console.log(generated.setupURI);
} }