mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-09-18 16:47:05 +00:00
Compare commits
25
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e8bb05e884 | ||
|
|
c60323e11f | ||
|
|
ce4eb7f9ac | ||
|
|
1c8db1a2b2 | ||
|
|
f449292792 | ||
|
|
dee5b69689 | ||
|
|
1044dca94f | ||
|
|
63c2811d47 | ||
|
|
00abaf2668 | ||
|
|
ef99cd8499 | ||
|
|
85a12e311c | ||
|
|
db805fd70c | ||
|
|
2d00fcfb47 | ||
|
|
64063b4a2c | ||
|
|
d1405f2fb4 | ||
|
|
98284005c9 | ||
|
|
cf9d671b8a | ||
|
|
1ba61a5052 | ||
|
|
11a07b26af | ||
|
|
a565070809 | ||
|
|
ab55eb5aff | ||
|
|
93bc161f20 | ||
|
|
ba297d1233 | ||
|
|
28884c3fd2 | ||
|
|
7200cb5aff |
@@ -121,6 +121,9 @@ jobs:
|
||||
node-version: '24.x'
|
||||
cache: 'npm'
|
||||
|
||||
- name: Verify clean installation with npm 10
|
||||
run: npx --yes npm@10.9.4 ci --ignore-scripts --no-audit --no-fund
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
|
||||
@@ -29,18 +29,7 @@ npm run build
|
||||
|
||||
#### Community Review dependency installation
|
||||
|
||||
Community Review installs dependencies independently before applying type-aware source rules. A successful installation with the npm version bundled with the repository's current Node.js CI does not prove that the lockfile is accepted by the scanner's npm version.
|
||||
|
||||
After changing `package.json`, a workspace manifest, or `package-lock.json`, verify both installation paths:
|
||||
|
||||
```bash
|
||||
npm ci --ignore-scripts
|
||||
npx --yes npm@10.9.2 ci --ignore-scripts
|
||||
```
|
||||
|
||||
The npm 10.9.2 command is the current project-side compatibility check for the Community Review installation path. Update this check when the scanner runtime changes.
|
||||
|
||||
If Community Review reports widespread TypeScript `error` types across unrelated external packages, confirm that dependency installation completed successfully before changing source imports, declarations, or lint rules. An installation failure can make every unresolved external type appear as downstream unsafe-type findings.
|
||||
After changing a dependency manifest or lockfile, follow the [npm 10 clean-installation check](test/README.md#npm-10-clean-installation-check) before the normal source and unit checks. The test guide records the command used by CI and the distinction between installation failures and source diagnostics.
|
||||
|
||||
### Commands
|
||||
|
||||
@@ -77,6 +66,8 @@ To facilitate development and testing, the build process can automatically copy
|
||||
|
||||
### Testing Infrastructure
|
||||
|
||||
See the [test procedures](test/README.md) for clean-installation checks, local validation commands, and links to each runtime suite.
|
||||
|
||||
- **Vitest**:
|
||||
- **Unit Tests** (`vitest.config.unit.ts`): Unit tests run in Node.js (excluding harnesses and integration tests). Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`). Executed via `npm run test:unit`.
|
||||
- **Integration Tests** (`vitest.config.integration.ts`): Tests run in Node.js against a real CouchDB instance. Integration tests should be `*.integration.spec.ts` or `*.integration.test.ts` and placed alongside the implementation file (e.g., `StreamingFetch.integration.spec.ts`). Executed via `npm run test:integration`.
|
||||
@@ -189,15 +180,17 @@ steps required to add a built-in provider.
|
||||
|
||||
Commonlib owns one stable `LiveSyncP2PService`, its `P2PRoomSessionOwner`, and the replaceable Trystero room session. Host commands, event handlers, and views consume the focused transport, connection-probe admission, directory, peer-admission, transfer, change-relay, configuration, and diagnostic views returned by the service feature. They must not retain the deprecated compatibility Replicator as an ordinary service locator, close Trystero-owned raw peers, or install another Trystero transport generation at the application root. The exact implemented ownership and shutdown boundaries are recorded in Commonlib's [P2P transport lifecycle](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/p2p-transport-lifecycle.md) design document.
|
||||
|
||||
The [TURN connection settings design](docs/design_docs/renewable_turn_credentials.md) describes how the host prepares temporary ICE credentials in a connection-only settings copy. It covers room reuse and expiry, replication continuation, profile persistence and sharing, and report redaction.
|
||||
|
||||
### Conflict Merge Policy
|
||||
|
||||
Markdown conflict auto-merge should behave like a conservative three-way merge. The guiding rule is to merge changes when they touch non-overlapping regions, and to keep a manual conflict when the edits overlap semantically.
|
||||
|
||||
When in doubt, prefer the safer outcome: preserve data, keep the conflict visible, and ask the user rather than silently discarding content or choosing one side.
|
||||
|
||||
The detailed contract is documented in [Conflict resolution and revision provenance](docs/specs_conflict_resolution.md). Determine the merge base by intersecting the exact `available` revision IDs from both leaf histories and selecting the nearest shared revision. Do not infer ancestry from revision generation numbers. When a remote resolution reaches a Vault which still contains the exact content of a deleted losing branch, treat that content as known synchronised history so the resolution can be reflected without recreating the conflict.
|
||||
The detailed contract is documented in [Conflict resolution and revision provenance](docs/specs_conflict_resolution.md). Determine the merge base by intersecting the exact `available` revision IDs from both leaf histories and selecting the nearest shared revision. Do not infer ancestry from revision generation numbers. An unchanged file is recognised by comparing its bytes with its exact device-local file-reflection provenance, including when that revision belongs to a deleted losing branch.
|
||||
|
||||
File operations made while a conflict is active must use the device-local file-reflection provenance injected into `ServiceFileHandlerBase`. Treat its exact revision as authoritative; use byte equality only to reconstruct a missing record when exactly one available revision matches. If branch identity remains unknown, preserve data and leave the conflict visible. Do not hide key-value database readiness behind an implicit wait: maintained hosts open it through the sequential settings lifecycle before file events or replication begin.
|
||||
Ordinary file saves and incoming reflection use that provenance even before a conflict exists. An unchanged stale file must not become a child of the current winner; a genuine edit extends the recorded revision. Without a readable recorded base, compare only current live leaves to avoid duplicate content. Otherwise, preserve the file as a fresh independent root under the same document ID, leaving ancestry unknown. Historical byte equality cannot distinguish an unchanged file from an intentional revert. Explicit reconciliation, deletion, and rename retain their separate contracts. Do not hide key-value database readiness behind an implicit wait: maintained hosts open it through the sequential settings lifecycle before file events or replication begin.
|
||||
|
||||
- If one side deletes a line and the other side leaves that same line unchanged, treat it as a safe deletion. The deleted line must not be reintroduced into the merged result.
|
||||
- If one side inserts new content in a different region while the other side deletes an unchanged old region, preserve the insertion and the deletion.
|
||||
@@ -207,6 +200,8 @@ File operations made while a conflict is active must use the device-local file-r
|
||||
|
||||
This policy is intentionally aligned with the conflict checkboxes and compatibility settings: automatic merge should remove avoidable prompts, but it must not silently choose between overlapping user intentions.
|
||||
|
||||
The [multiple-device conflict test procedure](test/README.md#multiple-device-conflict-regression-tests) documents the five CouchDB-backed cases, execution steps, expected results, and coverage boundaries.
|
||||
|
||||
### File Structure Conventions
|
||||
|
||||
- **Platform-specific code**: Use `.platform.ts` suffix (replaced with `.obsidian.ts` in production builds via esbuild)
|
||||
|
||||
@@ -46,7 +46,7 @@ LiveSync will expose a separate `Connection path` choice:
|
||||
- `Automatic` retains normal ICE selection and is the default.
|
||||
- `TURN relay only` supplies `iceTransportPolicy: 'relay'` and prevents direct or server-reflexive candidates from being selected.
|
||||
|
||||
`TURN relay only` is enabled only when at least one syntactically valid `turn:` or `turns:` URL is configured. If the last valid TURN URL is removed while relay-only mode is selected, the dialogue restores `Automatic` and displays a concise explanation.
|
||||
`TURN relay only` is enabled when a managed TURN provider is selected or at least one syntactically valid manual `turn:` or `turns:` URL is configured. If neither is available while relay-only mode is selected, the dialogue restores `Automatic` and displays a concise explanation. Selecting a managed provider does not itself force relay use; `Automatic` retains normal ICE selection.
|
||||
|
||||
The route policy is an ordinary P2P profile property. It is retained in P2P connection strings and encrypted Setup URIs so that an imported compatibility profile has reproducible transport behaviour.
|
||||
|
||||
@@ -60,7 +60,15 @@ The first settings revision retains the existing storage and dialogue contract o
|
||||
|
||||
A future interface may present the existing comma-separated value as ordered `turn:` and `turns:` URL rows without changing its serialised representation. A structured list of multiple credential profiles is deferred until a provider or self-hosted use case requires different credentials in the same P2P profile.
|
||||
|
||||
Static long-term credentials are the supported first stage. Managed providers may return short-lived credentials, but LiveSync must not store a provider API token or a Coturn shared authentication secret. A future managed-credential design needs a separately trusted HTTPS endpoint, expiry handling, refresh behaviour, failure reporting, and a clear Setup URI policy. It is not represented as another static password field.
|
||||
Static long-term credentials remain supported. For managed credentials, a host preparation hook requests ICE settings and places them on a connection-only copy of `P2PSyncSetting`. Service-specific requests and validation belong under `src/integrations/`; Commonlib consumes that copy and owns room reuse, expiry checks, and replacement. It has no provider catalogue or source factory.
|
||||
|
||||
A user-supplied provider API token is persisted as a sensitive P2P profile setting and included in encrypted Setup URI sharing, so that participating devices can use the same configuration without repeated token entry. Existing profile-URI encryption covers the saved token; its flat runtime projection is omitted from persistence. Reports and logs redact the complete provider configuration and issued credentials, including inactive profiles and settings projections. Coturn's server-side shared authentication secret remains outside client settings.
|
||||
|
||||
Issued short-lived TURN credentials and their expiry remain in memory. The existing room reuse decision checks both the effective connection settings and credential validity. When reconciliation finds expired credentials, it uses the normal room retirement and replacement path with newly acquired credentials. Replacement may cancel an in-progress transfer; the next replication attempt uses stored checkpoints and revision comparison to retain received progress. Whether that next attempt starts automatically follows the existing synchronisation policy.
|
||||
|
||||
Time passing alone does not trigger acquisition or disconnection. This design adds no renewal timer, per-peer acquisition hook, configuration update on raw peers, or credential-driven ICE restart. Internal peer reconnection within an unchanged room does not guarantee fresh issuance. Acquisition failure is reported without changing the selected provider or route policy. See [TURN connection settings](../design_docs/renewable_turn_credentials.md) for the preparation hook, persistence and sharing formats, room replacement, and verified replication continuation behaviour.
|
||||
|
||||
Relay-only validation accepts a valid managed TURN configuration as well as the existing manual URL list. Failure to acquire usable TURN entries keeps relay-only mode selected and reports the connection failure; it does not restore `Automatic` silently.
|
||||
|
||||
### TURN allocation check and route diagnostics
|
||||
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
---
|
||||
date: 2026-09-16
|
||||
commonlib-version: "0.1.25"
|
||||
self-hosted-livesync-version: "1.0.28"
|
||||
status: unreleased
|
||||
---
|
||||
|
||||
# TURN credentials in P2P connection settings
|
||||
|
||||
## Purpose
|
||||
|
||||
This design addresses [Issue #1182](https://github.com/vrtmrz/obsidian-livesync/issues/1182)
|
||||
by acquiring temporary TURN credentials on the device before opening a P2P room.
|
||||
The [P2P transport compatibility ADR](../adr/2026_08_p2p_transport_compatibility.md)
|
||||
records the connection and persistence policy.
|
||||
|
||||
LiveSync prepares a connection copy of `P2PSyncSetting`. Commonlib owns the room
|
||||
lifecycle and consumes the resulting ICE settings. Service-specific HTTP and
|
||||
validation remain under `src/integrations/`; Commonlib has no provider catalogue
|
||||
or versioned acquisition descriptor. Cloudflare is the first optional integration.
|
||||
Manual TURN configuration remains available without a provider account.
|
||||
|
||||
## Settings and ownership
|
||||
|
||||
| Setting | Meaning | Lifetime |
|
||||
| --- | --- | --- |
|
||||
| `P2P_managedType` | Provider identifier; `CF` selects Cloudflare | P2P profile |
|
||||
| `P2P_managedId` | Provider key identifier; Cloudflare TURN Key ID | P2P profile |
|
||||
| `P2P_managedToken` | Provider API token used to request credentials | P2P profile |
|
||||
| `P2P_iceServers` | Prepared `RTCIceServer[]` | One room connection |
|
||||
| `P2P_iceServersExpiresAt` | Absolute expiry in Unix milliseconds | One room connection |
|
||||
|
||||
The first three values use ordinary ConnStr query parameters `managedType`,
|
||||
`managedId`, and `token`. The existing `appId` parameter continues to identify the
|
||||
P2P application. Commonlib reads and writes the three scalar values so profile
|
||||
editing and activation preserve them. The host interprets the provider identifier.
|
||||
An absent identifier selects the existing manual fields; an unsupported identifier
|
||||
produces an explicit error when a connection is requested.
|
||||
|
||||
Keep `P2P_turnServers`, `P2P_turnUsername`, and `P2P_turnCredential` for manual
|
||||
configuration. Issuance does not overwrite them. Retain the complete ICE array:
|
||||
individual entries can contain different credentials or STUN-only URLs.
|
||||
|
||||
## Host preparation
|
||||
|
||||
The optional `prepareP2PSettings(settings, signal)` composition hook receives a
|
||||
snapshot of requested P2P settings. LiveSync supplies the same preparation function
|
||||
to Obsidian, CLI, WebApp, and WebPeer using each host's HTTP adapter.
|
||||
|
||||
For a managed selection, the function validates the provider inputs, requests
|
||||
credentials, and returns a connection copy:
|
||||
|
||||
```typescript
|
||||
return {
|
||||
...settings,
|
||||
P2P_iceServers: iceServers,
|
||||
P2P_iceServersExpiresAt: expiresAt,
|
||||
};
|
||||
```
|
||||
|
||||
Commonlib takes the prepared ICE fields into its session snapshot and passes that
|
||||
snapshot through `ReplicatorHostEnv.settings`. The hook does not change the
|
||||
requested room identity, persist settings, own replication, or schedule renewal.
|
||||
Its HTTP request must settle on cancellation and has a bounded deadline. The room
|
||||
owner also stops waiting for preparation when the connection request is retired.
|
||||
An explicitly managed configuration requires a preparation hook and usable ICE
|
||||
credentials; acquisition failure does not select a fallback provider or route.
|
||||
Managed credential acquisition is independent of the connection path. `Automatic`
|
||||
retains normal ICE selection, including direct candidates; only `TURN relay only`
|
||||
forces relay use. Acquisition must still succeed before opening a managed room
|
||||
when `Automatic` is selected.
|
||||
|
||||
## Room reuse and expiry
|
||||
|
||||
The active connection settings hold the issued credentials. They are the only
|
||||
credential cache. The existing room reuse decision checks:
|
||||
|
||||
1. whether the requested database and connection settings still match; and
|
||||
2. whether the active connection's credentials have enough remaining lifetime.
|
||||
|
||||
The static connection signature includes the provider type, key ID, and token.
|
||||
It excludes the generated ICE array and expiry. Comparing the prepared and stored
|
||||
settings directly would incorrectly trigger issuance on every reconciliation.
|
||||
|
||||
When reuse is unavailable, the owner retires the existing room, obtains a fresh
|
||||
connection copy, and opens its replacement. It checks settings, room demand,
|
||||
cancellation, and expiry again before publishing the replacement. A late result
|
||||
cannot reopen a closed room or apply credentials requested for different settings.
|
||||
Explicit reconnection acquires fresh credentials. Closing the room releases its
|
||||
credential references. Preserve a 30-second connection-establishment margin.
|
||||
|
||||
Reconciliation runs at existing connection, settings, and lifecycle boundaries.
|
||||
Time passing alone does not trigger acquisition or disconnection. There is no
|
||||
renewal timer, per-peer acquisition, raw WebRTC configuration update, ICE restart,
|
||||
or general retry mechanism. Trystero's internal peer reconnection within an
|
||||
unchanged room uses that room's existing configuration.
|
||||
|
||||
Normal retirement may cancel an in-progress transfer. A later replication attempt
|
||||
uses stored checkpoints and revision comparison to retain received progress.
|
||||
An unfinished network message may be sent again. Whether another attempt starts
|
||||
automatically continues to follow the existing synchronisation policy.
|
||||
|
||||
## Persistence, sharing, and privacy
|
||||
|
||||
Persist provider values only inside the selected P2P profile URI. Flat values in
|
||||
runtime settings are a projection restored by profile activation. Profile edits
|
||||
update that URI explicitly. General settings saves do not rebuild a P2P profile
|
||||
from unrelated flat settings. Flat-settings migration creates and selects its
|
||||
P2P profile once, independently of the selected main remote.
|
||||
|
||||
Existing whole-profile encryption covers the saved API token. The default mode
|
||||
uses the existing built-in key; a user-supplied configuration passphrase has its
|
||||
existing protection semantics. Failure to encrypt a managed profile leaves the
|
||||
previous saved data intact. No separate encrypted-token field is added. A draft
|
||||
containing provider credentials but no Group ID remains unsaved.
|
||||
|
||||
Setup URIs and ordinary settings QR codes already contain `remoteConfigurations`.
|
||||
The provider values travel inside that profile URI, including inactive profiles.
|
||||
Omit their duplicate flat projections from sharing. No new URI scheme, encoded QR
|
||||
slot, or encryption envelope is needed. Setup URIs retain passphrase encryption;
|
||||
QR codes retain their unencrypted format and 'FOR YOUR EYES ONLY' display.
|
||||
|
||||
Issued ICE credentials and expiry appear only in connection copies. Remove both
|
||||
runtime fields at save, import, and sharing boundaries, including
|
||||
`TrysteroReplicator.getAllConfig`, which starts from the session settings.
|
||||
Incoming settings cannot install an issued credential override. Reports omit
|
||||
runtime ICE fields, redact provider values, and retain scheme-only profile URIs.
|
||||
Logs use safe errors and omit request headers, raw responses, and connection
|
||||
signatures. Ordinary plaintext in process memory is permitted.
|
||||
|
||||
Markdown settings omit managed provider values and the profile collection with
|
||||
its selections. If that group is omitted during import, preserve the corresponding
|
||||
local P2P connection values as well as the profiles. This prevents combining an
|
||||
imported room with the local provider token or overwriting the saved profile.
|
||||
|
||||
## Cloudflare integration
|
||||
|
||||
The UI presents `Manual` and `Managed (Cloudflare)`, with `TURN Key ID` and a masked
|
||||
`TURN Key API Token` input for Cloudflare. It requires no account ID, custom
|
||||
endpoint, SDK, credential broker, or renewal interval setting.
|
||||
|
||||
The provider function uses Cloudflare's
|
||||
[credential-generation endpoint](https://developers.cloudflare.com/realtime/turn/generate-credentials/)
|
||||
and converts its response into ICE servers. The implementation requests a fixed
|
||||
24-hour lifetime and derives local expiry from the clock before the request starts.
|
||||
This lifetime applies to the issued TURN credentials, not the provider API token.
|
||||
There is currently no setting to change it.
|
||||
|
||||
The HTTP boundary uses the injected standard fetch adapter with cancellation,
|
||||
a 15-second deadline, refused redirects, omitted cookies, and disabled caching.
|
||||
It bounds the response to 32 KiB, 16 ICE entries, and 32 URLs, and validates URLs
|
||||
and complete TURN credentials. These are local implementation limits. Keep this
|
||||
validation at the provider boundary instead of repeating it in Commonlib.
|
||||
|
||||
The token is supplied and shared by the user on their devices. The provider
|
||||
function sends the key ID, API token, and requested lifetime; it has no need for
|
||||
Vault data, the Group ID, or the Vault passphrase.
|
||||
|
||||
## Setup and verification
|
||||
|
||||
The Setup connection test remains a signalling check. A separately owned trial
|
||||
uses signalling-only settings and performs no managed TURN issuance. The existing
|
||||
active-relay admission rule still applies. Success does not verify the API token,
|
||||
TURN allocation, or document transfer. Actual room connections use the preparation
|
||||
hook and preserve the selected route policy on failure.
|
||||
|
||||
Focused tests cover provider validation and cancellation, room reuse and expiry,
|
||||
late results after configuration changes or closure, migration without duplicate
|
||||
profiles, Markdown import through save/reload, safe acquisition failures, and
|
||||
exclusion of runtime credentials from storage and sharing.
|
||||
|
||||
Validate Commonlib as an exact packed artefact before testing its LiveSync
|
||||
consumer. Verify the changed settings and restart boundary in real Obsidian.
|
||||
Previously observed provider issuance and relay synchronisation do not establish
|
||||
expiry-driven reconnection for a revised build. Fresh TURN allocation after
|
||||
expiry, mobile runtimes, and cross-network behaviour require their own runtime
|
||||
verification; a surviving Trystero shared peer is not evidence of new allocation.
|
||||
+37
-4
@@ -18,7 +18,7 @@ flowchart LR
|
||||
The signalling relay and TURN server have different roles:
|
||||
|
||||
- The **signalling relay** is required for peer discovery and connection negotiation. LiveSync uses Nostr-compatible WebSocket relays for this role. The relay does not store or transfer Vault contents.
|
||||
- A **TURN server** is an optional fallback. WebRTC uses it to relay the encrypted peer connection only when the devices cannot establish a direct path through their networks.
|
||||
- A **TURN server** is an optional fallback. WebRTC uses it to relay the encrypted peer connection when the devices cannot establish a direct path through their networks, or whenever **TURN relay only** is selected.
|
||||
|
||||
## The project's public signalling relay
|
||||
|
||||
@@ -40,16 +40,49 @@ Both settings contain server addresses, but they are not interchangeable.
|
||||
| Setting | Required | Carries Vault contents | Purpose |
|
||||
| --- | --- | --- | --- |
|
||||
| **Signalling relay URLs** | Yes | No | Finds peers and exchanges the information needed to establish WebRTC connections. |
|
||||
| **TURN server URLs** | Only when direct WebRTC connectivity fails | Encrypted WebRTC traffic | Relays traffic between peers when NAT or firewall rules prevent a direct path. |
|
||||
| **TURN server URLs** | When direct WebRTC connectivity fails or **TURN relay only** is selected | Encrypted WebRTC traffic | Relays traffic between peers when NAT or firewall rules prevent a direct path. |
|
||||
|
||||
A TURN provider cannot read LiveSync's encrypted Vault contents, but it can observe connection metadata and traffic volume. Use a provider you trust. The project does not operate an official TURN service.
|
||||
WebRTC encrypts data between the devices, including when it passes through TURN. The TURN provider cannot read the transferred data, but it can observe network addresses and traffic volume. This transport encryption also applies when LiveSync's optional database encryption is disabled. The project does not operate an official TURN service.
|
||||
|
||||
## TURN credentials
|
||||
|
||||
In **TURN configuration**, select **Manual** to enter your own TURN server URLs,
|
||||
username, and credential, or select **Managed (Cloudflare)** to enter a **TURN Key ID** and
|
||||
**TURN Key API Token**. Cloudflare is optional; the project does not require a
|
||||
particular TURN provider or operate a credential broker. See Cloudflare's
|
||||
[credential instructions](https://developers.cloudflare.com/realtime/turn/generate-credentials/)
|
||||
for creating a TURN key and its API token.
|
||||
|
||||
The API token is saved with the P2P profile and included when sharing settings
|
||||
through an existing Setup URI or QR code. Setup URIs retain their existing
|
||||
passphrase encryption. QR codes retain their existing unencrypted format and
|
||||
'FOR YOUR EYES ONLY' display. Missing provider settings use the ordinary manual
|
||||
configuration defaults. Receiving clients need support for the selected provider
|
||||
to acquire its temporary TURN credentials.
|
||||
Markdown settings omit the connection profile group when it contains a managed
|
||||
TURN provider, including inactive profiles, and importing those omitted settings
|
||||
preserves this device's existing profiles. Diagnostic reports redact provider settings. The existing profile-URI
|
||||
encryption also covers the saved token.
|
||||
|
||||
Each device requests temporary TURN credentials when opening a new room.
|
||||
An existing room reuses its credentials while they remain valid. Cloudflare credentials have a requested lifetime
|
||||
of 24 hours and remain in memory only. Expiry is checked when LiveSync next
|
||||
reconciles the room connection. If necessary, it replaces the room and obtains
|
||||
new credentials. There is no periodic renewal: if a long-lived room cannot
|
||||
reconnect after credentials expire, disconnect and open the connection again.
|
||||
|
||||
Room replacement may interrupt replication. The next synchronisation keeps
|
||||
received Metadata and Chunks, resumes from its saved checkpoint, and compares
|
||||
revisions to fetch missing data. An unfinished network message can be sent
|
||||
again. Automatic synchronisation follows the existing peer rules; after an
|
||||
interrupted manual operation, use **Replicate now** again.
|
||||
|
||||
## Connection compatibility profiles
|
||||
|
||||
`P2P Configuration` includes a separate `Connection compatibility` section. Its defaults preserve the existing transport behaviour:
|
||||
|
||||
- **P2P message size** defaults to **Standard**. **Reduced**, **Conservative**, and **Maximum compatibility** progressively limit outgoing P2P messages when a network path appears to drop larger WebRTC messages. This is not a Vault Chunk size or an IP MTU. Smaller values add framing and processing overhead.
|
||||
- **Connection path** defaults to **Automatic**, which lets WebRTC select a viable direct or TURN-relayed path. **TURN relay only** forces the encrypted connection through TURN and is available only when the profile contains at least one valid `turn:` or `turns:` URL.
|
||||
- **Connection path** defaults to **Automatic**, which lets WebRTC select a viable direct or TURN-relayed path. **TURN relay only** forces the encrypted connection through TURN and is available when the profile contains a valid manual TURN URL or a configured TURN provider.
|
||||
|
||||
The sending device controls its outgoing message size. Select the same conservative preset on every device which may send across the constrained path. Existing devices do not receive the choice retrospectively merely because another device changed it.
|
||||
|
||||
|
||||
@@ -2,6 +2,18 @@
|
||||
|
||||
This document contains earlier published releases from the 1.0 line of the [current Self-hosted LiveSync release history](../../updates.md). Beta and release-candidate builds published before 1.0.0 are recorded in the [1.0 preview history](1.0-previews.md). Earlier release lines continue in the [0.25 history](0.25.md) and the [legacy history](legacy.md).
|
||||
|
||||
## 1.0.23
|
||||
|
||||
2nd September, 2026
|
||||
|
||||
I am sorry to make this release while several pull requests are still awaiting merge, but I believe that the safeguards provided by this work are significant, so I have decided to release it. I will merge the remaining pull requests in turn. Thank you for bearing with me while I have been less active recently.
|
||||
|
||||
### Synchronisation and storage
|
||||
|
||||
#### Fixed
|
||||
|
||||
- **Sync now** once again keeps routine progress quiet, while still opening recovery dialogues when a decision is required. Repeated OneShot Sync requests received while an earlier attempt is running are now ignored instead of starting overlapping work.
|
||||
|
||||
## 1.0.22
|
||||
|
||||
1st September, 2026
|
||||
|
||||
+20
-1
@@ -485,6 +485,25 @@ Setting key: P2P_AutoBroadcast
|
||||
|
||||
When enabled, this device notifies connected peers after a local change. The notification contains no Vault data. A receiving peer fetches the change only when it follows this device.
|
||||
|
||||
#### TURN configuration
|
||||
|
||||
Setting key: P2P_managedType
|
||||
|
||||
Select **Manual** for the existing TURN server fields, or **Managed (Cloudflare)** for a
|
||||
TURN Key ID and TURN Key API Token. The API token is persisted with the profile
|
||||
and included in Setup URI and QR code sharing. Issued temporary credentials are
|
||||
kept in memory only. Reports redact the provider settings. See
|
||||
[TURN credentials](p2p.md#turn-credentials) for sharing, expiry, and reconnect
|
||||
behaviour.
|
||||
|
||||
#### TURN Key ID and TURN Key API Token
|
||||
|
||||
Setting keys: P2P_managedId, P2P_managedToken
|
||||
|
||||
These fields appear when **Managed (Cloudflare)** is selected. Enter the TURN key's ID and
|
||||
its dedicated API token. The token field is masked. No account ID, custom
|
||||
endpoint, or renewal interval is required.
|
||||
|
||||
#### TURN Server URLs (comma-separated)
|
||||
|
||||
Setting key: P2P_turnServers
|
||||
@@ -515,7 +534,7 @@ The sender controls the size of its outgoing messages. Select the same conservat
|
||||
|
||||
Setting key: P2P_connectionPath
|
||||
|
||||
**Automatic** lets WebRTC select a viable direct or TURN-relayed path and is the default. **TURN relay only** forces `iceTransportPolicy: 'relay'` and is available only when the profile contains at least one valid `turn:` or `turns:` URL. Removing the last valid TURN URL while relay-only mode is selected restores **Automatic** and displays a Notice.
|
||||
**Automatic** lets WebRTC select a viable direct or TURN-relayed path and is the default. **TURN relay only** forces `iceTransportPolicy: 'relay'` and is available when the profile contains a valid manual TURN URL or a configured TURN credential source. Removing the manual TURN configuration while relay-only mode is selected restores **Automatic** and displays a Notice. A selected credential source which cannot supply credentials prevents the connection from opening; it does not change the connection path.
|
||||
|
||||
This choice belongs to the P2P profile and is retained in P2P connection strings and encrypted Setup URIs. Separate profiles may use the same Group ID and credentials with different compatibility choices; only the selected P2P profile is active.
|
||||
|
||||
|
||||
@@ -28,24 +28,27 @@ The modifiers defined under [Revision](glossary.md#revision) describe independen
|
||||
| The Vault displays conflict leaf `C` | `W` | `C` | `C` |
|
||||
| The database advances before Vault reflection | new winner `W2` | previous revision `R`, while the Vault is unchanged | `R` |
|
||||
| A local edit of displayed revision `R` is pending | independent | none, or a coincidental content match | `R`, as the branch which the edit must extend |
|
||||
| Provenance is missing and exactly one revision fits | independent | `M` | none, then `M` after safe reconstruction |
|
||||
| Provenance is missing and exactly one current non-deleted leaf fits | independent | `M` | none, then `M` after safe reconstruction |
|
||||
| Provenance is missing and several revisions fit | independent | every matching revision | none |
|
||||
| A logical-deletion winner agrees with an absent file | deleted winner `D` | `D`, and possibly other logical-deletion revisions | none; an absent file retains no displayed provenance |
|
||||
|
||||
At most one revision is the winner, more than one revision can be Vault-matching, and at most one revision can be displayed for a path on one device. A displayed revision may stop matching the Vault while a local edit is pending, but its branch identity remains authoritative until that edit is stored or the relationship is safely reconstructed.
|
||||
|
||||
## Implemented 1.0 guarantees
|
||||
## File saving and reflection guarantees
|
||||
|
||||
- Automatic text and structured-data merge uses the nearest `available` revision ID which is present in both leaf histories.
|
||||
- Missing or compacted history stops conservative automatic merge instead of guessing a base.
|
||||
- A receiving Vault file which exactly matches any available revision in the document tree is treated as previously synchronised content. This includes an ancestor below a deleted losing leaf.
|
||||
- A receiving Vault file whose bytes do not match any available revision is preserved as an unsynchronised local change.
|
||||
- A Vault file which still matches its exact recorded revision is unchanged. An ordinary save does not append those stale bytes to a newer database revision; a newer, unconflicted database result is reflected through the existing file-reflection path.
|
||||
- A file which differs from its readable recorded revision is an edit of that revision, even if its bytes match another historical revision. Saving and incoming overwrite protection use the same rule.
|
||||
- Without a readable recorded revision, current non-deleted leaves are checked for duplicate content. If none matches, the file is preserved as a fresh independent root under the same document ID. Its unknown ancestry cannot supply a three-way merge base.
|
||||
- File bytes, rather than path, size, modification time, or revision generation, determine whether content is known.
|
||||
- Three or more current versions are reviewed one pair at a time in a deterministic order, with each completed pair committed before the next pair is read.
|
||||
- Each device records the exact revision most recently reflected in each Vault file. An edit, deletion, or case-only rename made while a conflict is active extends that displayed branch rather than the deterministic database winner.
|
||||
- Each device records the exact revision most recently reflected in each Vault file. An ordinary edit extends that displayed branch even before a conflict exists. Conflict-time deletion and case-only rename retain their separate displayed-branch contracts.
|
||||
- A cross-path rename stores the target before logically deleting only the displayed source branch.
|
||||
|
||||
The all-branch history check prevents a resolved conflict from being recreated merely because the receiving Vault still contains the known losing version. If the user has edited that version again, its bytes differ and the overwrite guard preserves it.
|
||||
The recorded revision can belong to a deleted losing branch. If its readable body still matches the Vault, the propagated resolution can be reflected without recreating the conflict. A historical byte match without that record does not establish that the file is unchanged: it may be an intentional revert. Existing Vaults can lack records, so an upgrade, reset, or unavailable old body can expose additional conflicts requiring review.
|
||||
|
||||
The explicit **Always overwrite with a newer file** option retains its existing modification-time policy. An independent branch prevents an inferred three-way merge; it does not disable the user's selected conflict-resolution option. Metadata and Chunks retain their existing format, and matching chunks can be shared between branches.
|
||||
|
||||
## Resolution patterns
|
||||
|
||||
@@ -55,8 +58,10 @@ The all-branch history check prevents a resolved conflict from being recreated m
|
||||
| Text or structured data has an available shared base and non-overlapping changes | Perform a conservative three-way merge. |
|
||||
| One side deletes content which the other leaves unchanged | Preserve the deletion. |
|
||||
| One side deletes content which the other modifies | Ask the user. |
|
||||
| A receiving file matches a revision available anywhere in the tree | Apply the propagated database result. |
|
||||
| A receiving file matches no available revision | Preserve it and ask the user. |
|
||||
| A receiving file matches its exact readable recorded revision | Apply the propagated database result under the existing conflict policy. |
|
||||
| A receiving file differs from its readable recorded revision | Preserve the edit as a child of that exact revision. |
|
||||
| Provenance is unknown and no current non-deleted leaf matches the file | Preserve a fresh independent branch for conflict resolution. |
|
||||
| Provenance is unknown and current non-deleted leaves already hold the file bytes | Avoid duplicate storage; infer provenance only for a unique match. |
|
||||
| A required body or shared ancestor is missing or compacted | Ask the user. |
|
||||
| Binary contents differ | Prefer an explicit user selection; semantic merge is unavailable. |
|
||||
|
||||
@@ -138,13 +143,17 @@ LiveSync composes Commonlib's injected `FileReflectionProvenance` with its local
|
||||
path -> { revision, observedStorageMtime? }
|
||||
```
|
||||
|
||||
`revision` identifies the exact database revision which most recently produced the displayed Vault file. `observedStorageMtime` is the raw local modification time observed after reflection. It is not rounded, combined with another device's value, or used as proof of branch identity. No content hash is persisted.
|
||||
`revision` identifies the exact database revision most recently saved from or reflected in this device's Vault. It is the base for subsequent local edits, rather than a certificate that the current file still contains those bytes. `observedStorageMtime` is the raw local modification time of the saved snapshot or the file observed after reflection. It is not rounded, combined with another device's value, or used as proof of branch identity. No content hash is persisted.
|
||||
|
||||
The record changes only after a successful database-to-Vault reflection or Vault-to-database write. Reading a file does not change it. The recorded revision remains authoritative even if the user edits the file to bytes which equal another branch; otherwise content equality could silently move the edit to a branch which was not displayed.
|
||||
|
||||
Saving and reflection for the same Metadata document run one at a time, including the final provenance update. An ordinary save holds one captured file body and its base until the database write completes. An edit made while that save is running belongs to the next operation; the save does not reread the file to prove that it remained unchanged. Different files retain their existing concurrency limits, and the handler acquires the lock before loading a file body from storage. Conflict checking runs after the lock is released so that an immediate resolution can safely call the file handler again. The host queues count document-lock waiters against their concurrency limits, so a burst for one document can temporarily delay unrelated files.
|
||||
|
||||
The common lock does not stop Obsidian edits, external filesystem writes, or replication into the database. Incoming overwrite and deletion protection still checks current storage. Pending events restored at startup retain bounded rechecks because they run before file watching begins and cannot rely on another change notification.
|
||||
|
||||
LiveSync creates the namespaced store handle during service composition, before the key-value database is open. The sequential `onSettingLoaded` lifecycle opens that database before Vault scanning, watching, or replication starts. Store operations do not wait for implicit readiness: a lifecycle violation fails promptly, avoiding an indefinite or self-referential initialisation wait. Local database reset is a transient unavailable boundary, after which scanning reconstructs derived state.
|
||||
|
||||
When no record exists, LiveSync may reconstruct the displayed revision only if the current Vault bytes match exactly one available revision body. No match, or identical content in multiple revisions, cannot prove branch identity.
|
||||
For ordinary saves and incoming reflection, a missing or unreadable recorded base permits reconstruction only from exactly one matching current non-deleted leaf. Matching several current leaves avoids duplicate storage but does not identify a displayed branch. No current match creates an independent branch, even when an older ancestor has the same bytes. Deletion and rename retain their existing provenance-recovery contracts.
|
||||
|
||||
## Operations while a conflict exists
|
||||
|
||||
@@ -231,7 +240,7 @@ If the user renames `draft.md` to `published.md`, LiveSync stores `published.md`
|
||||
|
||||
### A remote resolution reaches a device which still shows the losing content
|
||||
|
||||
Android may resolve a conflict and continue editing while Mac still shows the losing revision. When Mac receives the resolved tree, LiveSync searches every available branch and recognises Mac's unchanged bytes as content which was already synchronised below the deleted losing leaf. It can apply Android's resolution without asking Mac to resolve the same unchanged conflict again.
|
||||
Android may resolve a conflict and continue editing while Mac still shows the losing revision. When Mac receives the resolved tree, LiveSync compares Mac's bytes with the exact revision recorded for its Vault. If that body remains readable and matches, it can apply Android's resolution without asking Mac to resolve the same unchanged conflict again.
|
||||
|
||||
If the user edited the file on Mac before the resolution arrived, the bytes no longer match that historical revision. LiveSync preserves the Mac edit as an unsynchronised conflict instead of overwriting it.
|
||||
|
||||
@@ -243,15 +252,15 @@ The first decision has already changed the ordinary revision tree. On restart, L
|
||||
|
||||
### The device-local record is missing
|
||||
|
||||
A local-database reset removes revision provenance. On the next scan, if the Vault file matches exactly one available revision, LiveSync can reconstruct which branch was displayed and continue from it. If the bytes match multiple revisions, or no available revision, the branch remains unproved.
|
||||
A local-database reset removes revision provenance. When an ordinary save or incoming reflection examines the file, exactly one matching current non-deleted leaf can reconstruct the record. Multiple current matches prevent duplicate storage but leave branch identity unproved. A match only in past history is insufficient; differing current content is preserved as an independent branch. An unchanged-time scan alone does not guarantee that a record is created.
|
||||
|
||||
In that unproved state, an edit is retained as another manual-resolution branch. A deletion leaves all existing branches intact. A cross-path rename stores the target but leaves every source branch for review. The result can require an extra decision, but it does not discard data by guessing the winner.
|
||||
If no current non-deleted leaf contains the file bytes, an ordinary save retains them as another independent branch. An unproven deletion leaves all existing branches intact. A cross-path rename stores the target but leaves every unproven source branch for review. The result can require an extra decision, but it does not discard data by guessing the winner.
|
||||
|
||||
### Start-up or reset overlaps a provenance operation
|
||||
|
||||
LiveSync creates the provenance handle during composition, then opens its backing store during the sequential settings lifecycle before starting scans, watchers, or replication. If the store cannot open, start-up stops rather than leaving file processing waiting indefinitely.
|
||||
|
||||
During reset, the store can be temporarily unavailable. A racing provenance lookup fails promptly and follows the same conservative missing-record behaviour. After reopen, scanning can reconstruct a record when one exact revision body matches the Vault file.
|
||||
During reset, the store can be temporarily unavailable. A racing provenance lookup fails promptly and follows the same conservative missing-record behaviour. After reopen, ordinary saving or reflection can reconstruct a record from a unique matching current non-deleted leaf.
|
||||
|
||||
## Unsafe shortcuts
|
||||
|
||||
@@ -260,6 +269,7 @@ Do not:
|
||||
- infer a common ancestor from generation numbers alone;
|
||||
- assume that the PouchDB winner is the version currently displayed in the Vault;
|
||||
- replace recorded displayed provenance merely because current bytes match another branch;
|
||||
- classify a file as unchanged solely because it matches an ancestor somewhere in history;
|
||||
- discard local content when revision-history lookup fails;
|
||||
- infer revision identity from path, size, modification time, or content hash without a revision ID;
|
||||
- select the newest modification time unless the user has explicitly chosen that destructive policy; or
|
||||
@@ -267,7 +277,9 @@ Do not:
|
||||
|
||||
## Verification
|
||||
|
||||
Commonlib's real-PouchDB and injected-boundary unit tests cover unequal branch lengths, exact shared ancestry, deterministic ordering of multiple current leaves, a sensible stage followed by reconstruction of a manual pair, content below a deleted losing leaf, recorded and reconstructed branch identity, ambiguous matches, conflict-time editing, missing-body preservation when parent metadata is available, refusal to invent a parent for a generation-one revision, logical deletion, case-only rename, cross-path rename, and safe unproven fallbacks.
|
||||
LiveSync also exercises three and four independently editing devices through real CouchDB, using the installed Commonlib package and the CLI conflict-resolution command dispatcher. These tests check unchanged losing files before and after resolution, genuine edits on a losing branch, missing provenance, compacted bases, independent-root deduplication, and propagation of the selected result. See the [multiple-device regression procedure and coverage boundaries](../test/README.md#multiple-device-conflict-regression-tests).
|
||||
|
||||
Commonlib owns the real-PouchDB and injected-boundary tests for revision ancestry, content preservation, provenance, independent branches, and repeated file events. LiveSync owns persistent host composition and actual Obsidian restart coverage. The focused `test:e2e:obsidian:stale-file-restart` scenario advances the local DB while old Vault bytes remain, persists pending file events, and restarts the same isolated profile. It requires an unchanged recorded file to reflect the DB without a new revision, an unknown file to remain on an independent branch alongside the DB content, and repeated processing after provenance loss to leave those branches unchanged. It uses real local storage and startup processing; transport replication and mobile lifecycle coverage are separate.
|
||||
|
||||
LiveSync's optional real-Obsidian two-Vault checks have two scopes. `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` resolves and edits a Markdown conflict, propagates it to a Vault which still displays the deleted losing content, and requires one current result to remain. `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` edits, deletes, case-renames, and cross-path-renames files while conflicts remain active; it verifies the parent revision of each resulting branch, replicates those exact trees, and confirms that the other conflict branches remain intact.
|
||||
|
||||
|
||||
@@ -52,6 +52,8 @@ export default defineConfig(
|
||||
"obsidianmd/rule-custom-message": "off",
|
||||
"no-console": "warn",
|
||||
"obsidianmd/no-unsupported-api": "error",
|
||||
// Treat direct globalThis access as an error so the CI gate rejects it.
|
||||
"obsidianmd/no-global-this": "error",
|
||||
// Keep legacy type-safety debt visible while reserving errors for directory-review blockers.
|
||||
"@typescript-eslint/no-unsafe-argument": "warn",
|
||||
"@typescript-eslint/no-unsafe-assignment": "warn",
|
||||
|
||||
@@ -99,6 +99,13 @@ export default defineConfig([
|
||||
...ImportAliasRules("."),
|
||||
},
|
||||
},
|
||||
{
|
||||
files: ["src/integrations/**/*.ts"],
|
||||
rules: {
|
||||
// External-service integrations also run in Node and do not own window UI.
|
||||
"obsidianmd/no-global-this": "off",
|
||||
},
|
||||
},
|
||||
{
|
||||
files: ["src/apps/**/*.ts"],
|
||||
rules: {
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"id": "obsidian-livesync",
|
||||
"name": "Self-hosted LiveSync",
|
||||
"version": "1.0.28",
|
||||
"version": "1.0.29",
|
||||
"minAppVersion": "1.7.2",
|
||||
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
|
||||
"author": "vorotamoroz",
|
||||
|
||||
Generated
+9
-9
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "obsidian-livesync",
|
||||
"version": "1.0.28",
|
||||
"version": "1.0.29",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "obsidian-livesync",
|
||||
"version": "1.0.28",
|
||||
"version": "1.0.29",
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
"src/apps/cli",
|
||||
@@ -23,7 +23,7 @@
|
||||
"@smithy/types": "^4.14.3",
|
||||
"@smithy/util-retry": "^4.4.5",
|
||||
"@vrtmrz/browser-ui-kit": "0.1.0",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.24",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.26",
|
||||
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
|
||||
"@vrtmrz/ui-interactions": "0.1.2",
|
||||
"diff-match-patch": "^1.0.5",
|
||||
@@ -4567,9 +4567,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vrtmrz/livesync-commonlib": {
|
||||
"version": "0.1.24",
|
||||
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.24.tgz",
|
||||
"integrity": "sha512-gOXKo3ptEUYDkjLd5PGq2vAhOSvxOqW6xE7YWo9Y8ienglfYBz8R3eZ0I/JruvwZltH2B7Bmi41pHMjmRJQe3Q==",
|
||||
"version": "0.1.26",
|
||||
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.26.tgz",
|
||||
"integrity": "sha512-AVJky976PP1M+g18im6tJ1eKSq82uN73vkS98WWZWvMJ1UDz6O8YRVWa9dkFakAVXecdxxxLkiD45g5HTRnn6Q==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.808.0",
|
||||
@@ -12665,7 +12665,7 @@
|
||||
},
|
||||
"src/apps/cli": {
|
||||
"name": "self-hosted-livesync-cli",
|
||||
"version": "1.0.28-cli",
|
||||
"version": "1.0.29-cli",
|
||||
"dependencies": {
|
||||
"chokidar": "^4.0.0",
|
||||
"minimatch": "^10.2.5",
|
||||
@@ -12690,7 +12690,7 @@
|
||||
},
|
||||
"src/apps/webapp": {
|
||||
"name": "livesync-webapp",
|
||||
"version": "1.0.28-webapp",
|
||||
"version": "1.0.29-webapp",
|
||||
"dependencies": {
|
||||
"octagonal-wheels": "^0.1.54"
|
||||
},
|
||||
@@ -12702,7 +12702,7 @@
|
||||
}
|
||||
},
|
||||
"src/apps/webpeer": {
|
||||
"version": "1.0.28-webpeer",
|
||||
"version": "1.0.29-webpeer",
|
||||
"dependencies": {
|
||||
"octagonal-wheels": "^0.1.54"
|
||||
},
|
||||
|
||||
+5
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "obsidian-livesync",
|
||||
"version": "1.0.28",
|
||||
"version": "1.0.29",
|
||||
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
|
||||
"main": "main.js",
|
||||
"type": "module",
|
||||
@@ -23,7 +23,7 @@
|
||||
"prettyNoWrite": "prettier --config ./.prettierrc.mjs \"**/*.js\" \"**/*.ts\" \"**/*.json\" ",
|
||||
"precheck:compatibility": "npm run build",
|
||||
"check:compatibility": "node utils/check-compatibility.js --file main.js --ios 15",
|
||||
"check": "npm run tsc-check && npm run tsc-check:apps && npm run lint && npm run lint:community -- --quiet && npm run lint:community:tools && npm run svelte-check && npm run check:compatibility",
|
||||
"check": "npm run tsc-check && npm run tsc-check:apps && npm run lint && npm run lint:community && npm run lint:community:tools && npm run svelte-check && npm run check:compatibility",
|
||||
"i18n:bake": "npm run i18n:yaml2json && npm run i18n:bakejson && npm run i18n:format",
|
||||
"i18n:bakejson": "tsx _tools/bakei18n.ts",
|
||||
"i18n:format": "prettier --config .prettierrc.mjs --write --log-level error 'src/common/messagesJson/*.json' 'src/common/messages/*.ts'",
|
||||
@@ -78,6 +78,8 @@
|
||||
"test:e2e:obsidian:p2p-connection-check:services": "npm run test:e2e:obsidian:p2p-connection-check -- --manage-p2p",
|
||||
"test:e2e:obsidian:partial-startup-file-failure": "tsx test/e2e-obsidian/scripts/partial-startup-file-failure.ts",
|
||||
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
|
||||
"test:e2e:obsidian:stale-file-restart": "tsx test/e2e-obsidian/scripts/stale-file-restart.ts",
|
||||
"test:e2e:obsidian:folder-batch": "tsx test/e2e-obsidian/scripts/folder-batch.ts",
|
||||
"test:e2e:obsidian:setup-uri-workflow": "tsx test/e2e-obsidian/scripts/setup-uri-workflow.ts",
|
||||
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
|
||||
"test:e2e:obsidian:security-seed-reconnect": "tsx test/e2e-obsidian/scripts/security-seed-reconnect.ts",
|
||||
@@ -181,7 +183,7 @@
|
||||
"@smithy/types": "^4.14.3",
|
||||
"@smithy/util-retry": "^4.4.5",
|
||||
"@vrtmrz/browser-ui-kit": "0.1.0",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.24",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.26",
|
||||
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
|
||||
"@vrtmrz/ui-interactions": "0.1.2",
|
||||
"diff-match-patch": "^1.0.5",
|
||||
|
||||
@@ -1,105 +1,67 @@
|
||||
<script lang="ts">
|
||||
import { onMount } from "svelte";
|
||||
import { upsertRemoteConfigurationInPlace } from "@vrtmrz/livesync-commonlib/remote-configurations";
|
||||
import { REMOTE_P2P } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { P2PSyncSetting } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
|
||||
import type { P2PReplicatorPaneHost } from "@/features/P2PSync/P2PReplicator/P2PReplicatorPaneHost";
|
||||
import TurnConfiguration from "@/features/P2PSync/TurnConfiguration.svelte";
|
||||
import { validateManagedTurnSettings } from "@/integrations/turnSettings";
|
||||
|
||||
interface Props {
|
||||
host: P2PReplicatorPaneHost;
|
||||
}
|
||||
|
||||
let { host }: Props = $props();
|
||||
let { host }: { host: P2PReplicatorPaneHost } = $props();
|
||||
const currentSettings = () => host.services.setting.currentSettings() as P2PSyncSetting;
|
||||
const initialSettings = currentSettings();
|
||||
|
||||
let savedTurnServers = $state(initialSettings.P2P_turnServers);
|
||||
let savedTurnUsername = $state(initialSettings.P2P_turnUsername);
|
||||
let savedTurnCredential = $state(initialSettings.P2P_turnCredential);
|
||||
let turnServers = $state(initialSettings.P2P_turnServers);
|
||||
let turnUsername = $state(initialSettings.P2P_turnUsername);
|
||||
let turnCredential = $state(initialSettings.P2P_turnCredential);
|
||||
|
||||
const isTurnServersModified = $derived(turnServers !== savedTurnServers);
|
||||
const isTurnUsernameModified = $derived(turnUsername !== savedTurnUsername);
|
||||
const isTurnCredentialModified = $derived(turnCredential !== savedTurnCredential);
|
||||
const isModified = $derived(
|
||||
isTurnServersModified || isTurnUsernameModified || isTurnCredentialModified
|
||||
);
|
||||
function turnSettings(settings: P2PSyncSetting) {
|
||||
return {
|
||||
P2P_roomID: settings.P2P_roomID,
|
||||
P2P_turnServers: settings.P2P_turnServers,
|
||||
P2P_turnUsername: settings.P2P_turnUsername,
|
||||
P2P_turnCredential: settings.P2P_turnCredential,
|
||||
P2P_managedType: settings.P2P_managedType,
|
||||
P2P_managedId: settings.P2P_managedId,
|
||||
P2P_managedToken: settings.P2P_managedToken,
|
||||
};
|
||||
}
|
||||
let draft = $state(turnSettings(currentSettings()));
|
||||
let saved = $state(JSON.stringify(turnSettings(currentSettings())));
|
||||
const isModified = $derived(JSON.stringify(draft) !== saved);
|
||||
const sourceError = $derived(validateManagedTurnSettings(draft));
|
||||
const sourceNeedsRoom = $derived(!!draft.P2P_managedType && (draft.P2P_roomID ?? "").trim() === "");
|
||||
|
||||
function loadSettings(settings: P2PSyncSetting): void {
|
||||
savedTurnServers = settings.P2P_turnServers;
|
||||
savedTurnUsername = settings.P2P_turnUsername;
|
||||
savedTurnCredential = settings.P2P_turnCredential;
|
||||
turnServers = savedTurnServers;
|
||||
turnUsername = savedTurnUsername;
|
||||
turnCredential = savedTurnCredential;
|
||||
const next = turnSettings(settings);
|
||||
draft = next;
|
||||
saved = JSON.stringify(next);
|
||||
}
|
||||
|
||||
onMount(() =>
|
||||
host.services.context.events.onEvent("setting-saved", (settings) => {
|
||||
loadSettings(settings as P2PSyncSetting);
|
||||
})
|
||||
);
|
||||
onMount(() => host.services.context.events.onEvent("setting-saved", () => loadSettings(currentSettings())));
|
||||
|
||||
async function save(): Promise<void> {
|
||||
await host.services.setting.applyPartial(
|
||||
{
|
||||
P2P_turnServers: turnServers,
|
||||
P2P_turnUsername: turnUsername,
|
||||
P2P_turnCredential: turnCredential,
|
||||
},
|
||||
true
|
||||
);
|
||||
if (sourceError || sourceNeedsRoom) return;
|
||||
const values = $state.snapshot(draft);
|
||||
await host.services.setting.updateSettings((settings) => {
|
||||
const next = { ...settings, ...values, remoteConfigurations: { ...settings.remoteConfigurations } };
|
||||
const profileId = settings.P2P_ActiveRemoteConfigurationId ||
|
||||
(settings.remoteType === REMOTE_P2P ? settings.activeConfigurationId : "");
|
||||
const selected = next.remoteConfigurations[profileId];
|
||||
if (selected?.uri.startsWith("sls+p2p://")) {
|
||||
upsertRemoteConfigurationInPlace(next, "p2p", { id: profileId, activateForP2P: true });
|
||||
} else if (values.P2P_managedType) {
|
||||
upsertRemoteConfigurationInPlace(next, "p2p", { activateForP2P: true });
|
||||
}
|
||||
return next;
|
||||
}, true);
|
||||
loadSettings(currentSettings());
|
||||
}
|
||||
|
||||
function revert(): void {
|
||||
turnServers = savedTurnServers;
|
||||
turnUsername = savedTurnUsername;
|
||||
turnCredential = savedTurnCredential;
|
||||
}
|
||||
</script>
|
||||
|
||||
<section class="browser-p2p-transport-settings">
|
||||
<details>
|
||||
<summary>Optional TURN server settings</summary>
|
||||
<p>
|
||||
Configure TURN only when a direct peer-to-peer connection cannot be established.
|
||||
</p>
|
||||
<label class:is-dirty={isTurnServersModified}>
|
||||
<span>TURN Server URLs (comma-separated)</span>
|
||||
<input
|
||||
type="text"
|
||||
placeholder="turn:turn.example.com:3478"
|
||||
bind:value={turnServers}
|
||||
autocomplete="off"
|
||||
spellcheck="false"
|
||||
autocorrect="off"
|
||||
/>
|
||||
</label>
|
||||
<label class:is-dirty={isTurnUsernameModified}>
|
||||
<span>TURN Username</span>
|
||||
<input
|
||||
type="text"
|
||||
placeholder="Enter TURN username"
|
||||
bind:value={turnUsername}
|
||||
autocomplete="off"
|
||||
/>
|
||||
</label>
|
||||
<label class:is-dirty={isTurnCredentialModified}>
|
||||
<span>TURN Credential</span>
|
||||
<input
|
||||
type="password"
|
||||
placeholder="Enter TURN credential"
|
||||
bind:value={turnCredential}
|
||||
autocomplete="new-password"
|
||||
/>
|
||||
</label>
|
||||
<p>Configure TURN only when a direct peer-to-peer connection cannot be established.</p>
|
||||
<TurnConfiguration bind:settings={draft} />
|
||||
<div class="actions">
|
||||
<button type="button" class="button mod-cta" disabled={!isModified} onclick={save}>
|
||||
<button type="button" class="button mod-cta" disabled={!isModified || !!sourceError || sourceNeedsRoom} onclick={save}>
|
||||
Save TURN settings
|
||||
</button>
|
||||
<button type="button" class="button" disabled={!isModified} onclick={revert}>
|
||||
<button type="button" class="button" disabled={!isModified} onclick={() => loadSettings(currentSettings())}>
|
||||
Revert TURN settings
|
||||
</button>
|
||||
</div>
|
||||
@@ -107,27 +69,7 @@
|
||||
</section>
|
||||
|
||||
<style>
|
||||
.browser-p2p-transport-settings {
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
p {
|
||||
margin: 0.75rem 0;
|
||||
}
|
||||
label {
|
||||
display: grid;
|
||||
gap: 0.25rem;
|
||||
margin: 0.75rem 0;
|
||||
}
|
||||
label.is-dirty {
|
||||
background-color: var(--background-modifier-error);
|
||||
}
|
||||
input {
|
||||
box-sizing: border-box;
|
||||
width: 100%;
|
||||
}
|
||||
.actions {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
.browser-p2p-transport-settings { margin-bottom: 1rem; }
|
||||
p { margin: 0.75rem 0; }
|
||||
.actions { display: flex; flex-wrap: wrap; gap: 0.5rem; }
|
||||
</style>
|
||||
|
||||
@@ -419,6 +419,30 @@ describe("runCommand abnormal cases", () => {
|
||||
expect(appliedSettings.useIndexedDBAdapter).toBe(false);
|
||||
});
|
||||
|
||||
it("setup imports managed TURN through the existing encrypted URI", async () => {
|
||||
const core = createCoreMock();
|
||||
const profiles = {
|
||||
turn: { id: "turn", name: "TURN", isEncrypted: false,
|
||||
uri: "sls+p2p://room?managedType=CF&managedId=turn-key&token=private-token" },
|
||||
};
|
||||
const passphrase = "setup-passphrase";
|
||||
const setupURI = await processSetting.encodeSettingsToSetupURI(
|
||||
{
|
||||
...DEFAULT_SETTINGS,
|
||||
remoteConfigurations: profiles,
|
||||
},
|
||||
passphrase
|
||||
);
|
||||
expect(setupURI.startsWith(configURIBase)).toBe(true);
|
||||
expect(setupURI).not.toContain("private-token");
|
||||
core.services.context.standardIo.prompt.mockResolvedValue(passphrase);
|
||||
await runCommand(makeOptions("setup", [setupURI]), { ...context, core });
|
||||
expect(core.services.setting.applyExternalSettings).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ remoteConfigurations: profiles }),
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
it("setup rejects encoded URI when passphrase is wrong", async () => {
|
||||
const core = createCoreMock();
|
||||
const setupURI = await createSetupURI("correct-passphrase");
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
||||
import { NodeServiceContext, NodeServiceHub } from "./services/NodeServiceHub";
|
||||
import { configureNodeLocalStorage, ensureGlobalNodeLocalStorage } from "./services/NodeLocalStorage";
|
||||
import { LiveSyncBaseCore, type StartupDatabaseOptions } from "@/LiveSyncBaseCore";
|
||||
@@ -524,7 +525,9 @@ export async function main(
|
||||
useOfflineScanner(core);
|
||||
}
|
||||
// Register P2P replicator feature.
|
||||
p2pReplicator = useP2PReplicatorFeature(core);
|
||||
p2pReplicator = useP2PReplicatorFeature(core, undefined, undefined, {
|
||||
prepareP2PSettings: useP2PSettingsPreparation(core.services.API.webCompatFetch.bind(core.services.API)),
|
||||
});
|
||||
// Add target filter to prevent internal files are handled
|
||||
core.services.vault.isTargetFile.addHandler(async (target) => {
|
||||
const targetPath = stripAllPrefixes(getPathFromUXFileInfo(target));
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "self-hosted-livesync-cli",
|
||||
"private": true,
|
||||
"version": "1.0.28-cli",
|
||||
"version": "1.0.29-cli",
|
||||
"main": "dist/index.cjs",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -7,6 +7,8 @@
|
||||
# 3. DB-deleted file → NOT restored to storage (UPDATE STORAGE skip)
|
||||
# 4. Both, storage newer → DB updated (SYNC: STORAGE → DB)
|
||||
# 5. Both, DB newer → storage updated (SYNC: DB → STORAGE)
|
||||
# 6. Compatibility mode → omitted vault-path works
|
||||
# 7. Unknown local origin → conflict preserved, deduplicated, and resolved
|
||||
#
|
||||
# Not covered (require precise mtime control or artificial conflict injection):
|
||||
# - Both, equal mtime → no-op (EVEN)
|
||||
@@ -43,7 +45,8 @@ cli_test_init_settings_file "$SETTINGS_FILE"
|
||||
# isConfigured=true is required for mirror (canProceedScan checks this)
|
||||
cli_test_mark_settings_configured "$SETTINGS_FILE"
|
||||
|
||||
# Enable writeDocumentsIfConflicted to resolve unsynced conflicts during mirror
|
||||
# Allow incoming DB content to be reflected when conflicts exist (Case 5).
|
||||
# This does not resolve conflicts or authorise overwriting DB content.
|
||||
node -e '
|
||||
const fs = require("fs");
|
||||
const file = process.argv[1];
|
||||
@@ -181,6 +184,11 @@ echo "=== Case 4: storage newer → DB updated (Separated Paths) ==="
|
||||
# Seed DB with old content (mtime ≈ now)
|
||||
printf 'old content\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put test/sync-storage-newer.md
|
||||
|
||||
# Establish the file's recorded base before making an ordinary local edit.
|
||||
# A direct put followed by unrelated local content has unknown provenance.
|
||||
run_mirror_test
|
||||
cli_test_assert_equal "old content" "$(cat "$VAULT_DIR/test/sync-storage-newer.md")" "Case 4 base was not reflected"
|
||||
|
||||
# Write new content to storage with a timestamp 1 hour in the future
|
||||
printf 'new content\n' > "$VAULT_DIR/test/sync-storage-newer.md"
|
||||
touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/test/sync-storage-newer.md"
|
||||
@@ -188,6 +196,8 @@ touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/test/sync-storage-n
|
||||
run_mirror_test
|
||||
|
||||
DB_RESULT_FILE="$WORK_DIR/case4-pull.txt"
|
||||
CASE4_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info test/sync-storage-newer.md)"
|
||||
cli_test_assert_equal "N/A" "$(printf '%s' "$CASE4_INFO" | cli_test_json_string_field_from_stdin conflicts)" "Ordinary local edit unexpectedly created a conflict"
|
||||
run_cli "$DB_DIR" --settings "$DB_SETTINGS" pull test/sync-storage-newer.md "$DB_RESULT_FILE"
|
||||
if cmp -s "$VAULT_DIR/test/sync-storage-newer.md" "$DB_RESULT_FILE"; then
|
||||
assert_pass "DB updated to match newer storage file"
|
||||
@@ -238,6 +248,55 @@ else
|
||||
assert_fail "Compatibility mode failed to sync file into DB"
|
||||
fi
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Case 7: Unknown local origin must preserve both contents, regardless of mtime
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "=== Case 7: unknown local origin → preserve and resolve conflict ==="
|
||||
|
||||
UNKNOWN_PATH="test/unknown-origin.md"
|
||||
printf 'original DB content\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put "$UNKNOWN_PATH"
|
||||
printf 'unrelated local content\n' > "$VAULT_DIR/$UNKNOWN_PATH"
|
||||
touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/$UNKNOWN_PATH"
|
||||
run_mirror_test
|
||||
|
||||
UNKNOWN_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info "$UNKNOWN_PATH")"
|
||||
WINNER="$(printf '%s' "$UNKNOWN_INFO" | cli_test_json_string_field_from_stdin revision)"
|
||||
CONFLICT="$(printf '%s' "$UNKNOWN_INFO" | cli_test_json_string_field_from_stdin conflicts)"
|
||||
if [[ ! "$WINNER" =~ ^1-[[:xdigit:]]+$ || ! "$CONFLICT" =~ ^1-[[:xdigit:]]+$ || "$WINNER" == "$CONFLICT" ]]; then
|
||||
echo "[FAIL] Expected two independent non-deleted root revisions: $UNKNOWN_INFO" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Do not assume which randomly identified root PouchDB selects as the winner.
|
||||
LOCAL_REV=""
|
||||
DB_REV=""
|
||||
for revision in "$WINNER" "$CONFLICT"; do
|
||||
CONTENT="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" cat-rev "$UNKNOWN_PATH" "$revision" | cli_test_sanitise_cat_stdout)"
|
||||
case "$CONTENT" in
|
||||
'unrelated local content') LOCAL_REV="$revision" ;;
|
||||
'original DB content') DB_REV="$revision" ;;
|
||||
*) echo "[FAIL] Unexpected content for $revision: $CONTENT" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
[[ -n "$LOCAL_REV" && -n "$DB_REV" ]] || { echo "[FAIL] Both contents must remain readable" >&2; exit 1; }
|
||||
|
||||
# Force another ordinary save of the same unknown bytes, even if incoming
|
||||
# reflection replaced the file under writeDocumentsIfConflicted.
|
||||
printf 'unrelated local content\n' > "$VAULT_DIR/$UNKNOWN_PATH"
|
||||
touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/$UNKNOWN_PATH"
|
||||
run_mirror_test
|
||||
REPEATED_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info "$UNKNOWN_PATH")"
|
||||
cli_test_assert_equal "$WINNER" "$(printf '%s' "$REPEATED_INFO" | cli_test_json_string_field_from_stdin revision)" "Repeated mirror changed the winning revision"
|
||||
cli_test_assert_equal "$CONFLICT" "$(printf '%s' "$REPEATED_INFO" | cli_test_json_string_field_from_stdin conflicts)" "Repeated mirror created another conflict"
|
||||
|
||||
run_cli "$DB_DIR" --vault "$VAULT_DIR" --settings "$DB_SETTINGS" resolve "$UNKNOWN_PATH" "$LOCAL_REV"
|
||||
RESOLVED_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info "$UNKNOWN_PATH")"
|
||||
cli_test_assert_equal "N/A" "$(printf '%s' "$RESOLVED_INFO" | cli_test_json_string_field_from_stdin conflicts)" "CLI resolve left a conflict"
|
||||
cli_test_assert_equal "$LOCAL_REV" "$(printf '%s' "$RESOLVED_INFO" | cli_test_json_string_field_from_stdin revision)" "CLI resolve selected the wrong revision"
|
||||
cli_test_assert_equal "unrelated local content" "$(cat "$VAULT_DIR/$UNKNOWN_PATH")" "CLI resolve did not reflect the selected content"
|
||||
assert_pass "Unknown local content was preserved, deduplicated, and resolved through the CLI"
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Summary
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
* 4. Both, storage newer -> DB updated (SYNC: STORAGE -> DB)
|
||||
* 5. Both, DB newer -> storage updated (SYNC: DB -> STORAGE)
|
||||
* 6. Compatibility mode -> omitted vault-path works (same DB + vault path)
|
||||
* 7. Unknown local origin -> conflict preserved, deduplicated, and resolved
|
||||
*
|
||||
* No external services are required.
|
||||
*
|
||||
@@ -18,9 +19,9 @@
|
||||
* deno test -A test-mirror.ts
|
||||
*/
|
||||
|
||||
import { assert } from "@std/assert";
|
||||
import { assert, assertEquals } from "@std/assert";
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { runCliOrFail } from "./helpers/cli.ts";
|
||||
import { runCliOrFail, runCliWithInputOrFail } from "./helpers/cli.ts";
|
||||
import { initSettingsFile, markSettingsConfigured } from "./helpers/settings.ts";
|
||||
|
||||
Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
|
||||
@@ -130,6 +131,10 @@ Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
|
||||
await Deno.writeTextFile(seedFile, "old content\n");
|
||||
await dbRun("push", seedFile, "test/sync-storage-newer.md");
|
||||
|
||||
// Reflect the shared base into the actual Vault before editing it.
|
||||
await runMirror();
|
||||
assertEquals(await Deno.readTextFile(workDir.join("vault", "test", "sync-storage-newer.md")), "old content\n");
|
||||
|
||||
// Write new content to storage with a timestamp 1 hour in the future
|
||||
const storageFile = workDir.join("vault", "test", "sync-storage-newer.md");
|
||||
await Deno.writeTextFile(storageFile, "new content\n");
|
||||
@@ -138,6 +143,8 @@ Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
|
||||
await runMirror();
|
||||
|
||||
const resultFile = workDir.join("case4-pull.txt");
|
||||
const info = JSON.parse(await dbRun("info", "test/sync-storage-newer.md"));
|
||||
assertEquals(info.conflicts, "N/A", "An ordinary local edit must not create a conflict");
|
||||
await dbRun("pull", "test/sync-storage-newer.md", resultFile);
|
||||
const storageContent = await Deno.readTextFile(storageFile);
|
||||
const pulledContent = await Deno.readTextFile(resultFile);
|
||||
@@ -184,6 +191,47 @@ Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
|
||||
assert(pulled === "compat-content\n", `Compatibility mode failed to sync file into DB (got: '${pulled}')`);
|
||||
console.log("[PASS] case 6: compatibility mode works");
|
||||
});
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Case 7: unknown local origin must preserve both contents regardless of mtime.
|
||||
// This deliberately uses put: push would record a file provenance entry.
|
||||
// -------------------------------------------------------------------
|
||||
await t.step("case 7: unknown local content is preserved, deduplicated, and resolved", async () => {
|
||||
const path = "test/unknown-origin.md";
|
||||
const storageFile = workDir.join("vault", "test", "unknown-origin.md");
|
||||
await runCliWithInputOrFail("original DB content\n", dbDir, "--settings", dbSettings, "put", path);
|
||||
const writeUnknownFile = async () => {
|
||||
await Deno.writeTextFile(storageFile, "unrelated local content\n");
|
||||
await Deno.utime(storageFile, new Date(), new Date(Date.now() + 3600_000));
|
||||
};
|
||||
await writeUnknownFile();
|
||||
await runMirror();
|
||||
|
||||
const info = JSON.parse(await dbRun("info", path));
|
||||
assert(/^1-[\da-f]+$/.test(info.revision), "Expected an independent winning root");
|
||||
assert(/^1-[\da-f]+$/.test(info.conflicts), "Expected exactly one independent conflicting root");
|
||||
assert(info.revision !== info.conflicts, "Expected two distinct revisions");
|
||||
const contents = new Map<string, string>();
|
||||
for (const revision of [info.revision, info.conflicts]) {
|
||||
contents.set(await dbRun("cat-rev", path, revision), revision);
|
||||
}
|
||||
assertEquals([...contents.keys()].sort(), ["original DB content\n", "unrelated local content\n"]);
|
||||
|
||||
// Re-submit identical local bytes even if incoming reflection replaced
|
||||
// the file under writeDocumentsIfConflicted; no third branch is needed.
|
||||
await writeUnknownFile();
|
||||
await runMirror();
|
||||
const repeated = JSON.parse(await dbRun("info", path));
|
||||
assertEquals(repeated.revision, info.revision);
|
||||
assertEquals(repeated.conflicts, info.conflicts);
|
||||
|
||||
const localRevision = contents.get("unrelated local content\n")!;
|
||||
await runCliOrFail(dbDir, "--vault", vaultDir, "--settings", dbSettings, "resolve", path, localRevision);
|
||||
const resolved = JSON.parse(await dbRun("info", path));
|
||||
assertEquals(resolved.conflicts, "N/A");
|
||||
assertEquals(resolved.revision, localRevision);
|
||||
assertEquals(await Deno.readTextFile(storageFile), "unrelated local content\n");
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
||||
/** Browser runtime for Self-hosted LiveSync over the File System Access API. */
|
||||
|
||||
import { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
|
||||
@@ -217,7 +218,9 @@ export class WebAppRuntime {
|
||||
useRedFlagFeatures(core);
|
||||
useCheckRemoteSize(core);
|
||||
useRemoteConfiguration(core);
|
||||
this.p2p = useP2PReplicatorFeature(core);
|
||||
this.p2p = useP2PReplicatorFeature(core, undefined, undefined, {
|
||||
prepareP2PSettings: useP2PSettingsPreparation(core.services.API.webCompatFetch.bind(core.services.API)),
|
||||
});
|
||||
this.paneHost = {
|
||||
services: core.services,
|
||||
p2p: this.p2p,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "livesync-webapp",
|
||||
"private": true,
|
||||
"version": "1.0.28-webapp",
|
||||
"version": "1.0.29-webapp",
|
||||
"type": "module",
|
||||
"description": "Browser-based Self-hosted LiveSync using FileSystem API",
|
||||
"scripts": {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "webpeer",
|
||||
"private": true,
|
||||
"version": "1.0.28-webpeer",
|
||||
"version": "1.0.29-webpeer",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
||||
import { type P2PSyncSetting, SETTING_KEY_P2P_DEVICE_NAME } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { compatGlobal } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
||||
import { EVENT_LAYOUT_READY } from "@vrtmrz/livesync-commonlib/compat/events/coreEvents";
|
||||
@@ -70,9 +71,8 @@ export class WebPeerRuntime {
|
||||
isScheduled: () => this.restartScheduled,
|
||||
},
|
||||
});
|
||||
this.p2p = useP2PReplicatorFeature({
|
||||
services: this.services,
|
||||
serviceModules: {},
|
||||
this.p2p = useP2PReplicatorFeature({ services: this.services, serviceModules: {} }, undefined, undefined, {
|
||||
prepareP2PSettings: useP2PSettingsPreparation(this.services.API.webCompatFetch.bind(this.services.API)),
|
||||
});
|
||||
this.p2pLogCollector = new P2PLogCollector(this.events);
|
||||
this.paneHost = {
|
||||
|
||||
@@ -7,6 +7,26 @@
|
||||
* remove it from this map in the same change.
|
||||
*/
|
||||
export const liveSyncProvisionalEnglishMessages = {
|
||||
"Configure TURN when a direct connection cannot be established or when you select TURN relay only.":
|
||||
"Configure TURN when a direct connection cannot be established or when you select TURN relay only.",
|
||||
"TURN configuration": "TURN configuration",
|
||||
Manual: "Manual",
|
||||
"Managed (Cloudflare)": "Managed (Cloudflare)",
|
||||
"TURN Key ID": "TURN Key ID",
|
||||
"TURN Key API Token": "TURN Key API Token",
|
||||
"Unsupported TURN configuration": "Unsupported TURN configuration",
|
||||
"The API token is saved with this profile and included in Setup URI and QR code sharing. Temporary TURN credentials are kept in memory only.":
|
||||
"The API token is saved with this profile and included in Setup URI and QR code sharing. Temporary TURN credentials are kept in memory only.",
|
||||
"TURN relay only requires a TURN server or a configured credential source under Advanced Settings.":
|
||||
"TURN relay only requires a TURN server or a configured credential source under Advanced Settings.",
|
||||
"TURN relay only requires TURN configuration. Connection path has been restored to Automatic.":
|
||||
"TURN relay only requires TURN configuration. Connection path has been restored to Automatic.",
|
||||
"Enter a TURN Key ID.": "Enter a TURN Key ID.",
|
||||
"TURN Key ID contains unsupported characters.": "TURN Key ID contains unsupported characters.",
|
||||
"Enter a TURN Key API Token.": "Enter a TURN Key API Token.",
|
||||
"TURN Key API Token must use Bearer token syntax.": "TURN Key API Token must use Bearer token syntax.",
|
||||
"The selected TURN configuration is not supported.": "The selected TURN configuration is not supported.",
|
||||
|
||||
"Setup Complete: Preparing to Fetch from Another Device": "Setup Complete: Preparing to Fetch from Another Device",
|
||||
"The P2P connection has been configured successfully. The initial synchronisation data must now be fetched from an online source device.":
|
||||
"The P2P connection has been configured successfully. The initial synchronisation data must now be fetched from an online source device.",
|
||||
@@ -28,8 +48,8 @@ export const liveSyncProvisionalEnglishMessages = {
|
||||
"The project's public signalling relay is a best-effort convenience operated by the project author. It does not store Vault contents, but signalling metadata may be visible to the relay. Availability and log retention are not guaranteed. You can replace it with your own Nostr-compatible relay.",
|
||||
"Learn more about P2P connections": "Learn more about P2P connections",
|
||||
"Learn more about signalling and TURN": "Learn more about signalling and TURN",
|
||||
"TURN relays the encrypted WebRTC connection only when a direct path cannot be established. A TURN provider cannot read encrypted Vault contents, but it can observe connection metadata and traffic volume. Use a provider you trust.":
|
||||
"TURN relays the encrypted WebRTC connection only when a direct path cannot be established. A TURN provider cannot read encrypted Vault contents, but it can observe connection metadata and traffic volume. Use a provider you trust.",
|
||||
"WebRTC encrypts data between your devices, including when it passes through TURN. The TURN provider cannot read the transferred data. It can see network addresses and traffic volume.":
|
||||
"WebRTC encrypts data between your devices, including when it passes through TURN. The TURN provider cannot read the transferred data. It can see network addresses and traffic volume.",
|
||||
"Connection compatibility": "Connection compatibility",
|
||||
"P2P message size": "P2P message size",
|
||||
Standard: "Standard",
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { redactTurnSettingsForReport } from "./turnSettingsPrivacy";
|
||||
import { REMOTE_COUCHDB, REMOTE_MINIO } from "@vrtmrz/livesync-commonlib/compat/common/models/setting.const";
|
||||
import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { generateCredentialObject } from "@vrtmrz/livesync-commonlib/compat/replication/httplib";
|
||||
@@ -67,6 +68,7 @@ export async function generateReport(settings: ObsidianLiveSyncSettings, core: L
|
||||
delete pluginConfig[key as keyof ObsidianLiveSyncSettings];
|
||||
}
|
||||
|
||||
redactTurnSettingsForReport(pluginConfig);
|
||||
pluginConfig.couchDB_DBNAME = REDACTED;
|
||||
pluginConfig.couchDB_PASSWORD = REDACTED;
|
||||
const scheme = pluginConfig.couchDB_URI.startsWith("http:")
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import { REMOTE_P2P } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
|
||||
import { generateReport } from "./reportTool";
|
||||
|
||||
vi.mock("./utils", () => ({ requestToCouchDBWithCredentials: vi.fn() }));
|
||||
vi.mock("@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions", () => ({
|
||||
compatGlobal: { origin: "test", navigator: { userAgent: "test" } },
|
||||
}));
|
||||
|
||||
describe("TURN credentials in diagnostic reports", () => {
|
||||
it("redacts provider tokens in all profiles and runtime credentials", async () => {
|
||||
const token = "private+token/with=symbols";
|
||||
const provider = { P2P_managedType: "CF", P2P_managedId: "private-key", P2P_managedToken: token };
|
||||
const settings = {
|
||||
...DEFAULT_SETTINGS,
|
||||
remoteType: REMOTE_P2P,
|
||||
...provider,
|
||||
P2P_iceServers: [{ urls: "turn:example.test", username: "issued-user", credential: "issued-password" }],
|
||||
P2P_iceServersExpiresAt: 123456789,
|
||||
remoteConfigurations: {
|
||||
inactive: {
|
||||
id: "inactive",
|
||||
name: "Inactive TURN",
|
||||
isEncrypted: false,
|
||||
uri: `sls+p2p://room?managedType=CF&managedId=private-key&token=${encodeURIComponent(token)}`,
|
||||
},
|
||||
},
|
||||
};
|
||||
const core = { services: { vault: { isStorageInsensitive: () => false } } } as unknown as LiveSyncBaseCore;
|
||||
const report = await generateReport(settings, core);
|
||||
const text = JSON.stringify(report);
|
||||
expect(text).not.toContain(token);
|
||||
expect(text).not.toContain(encodeURIComponent(token));
|
||||
expect(text).not.toContain("private-key");
|
||||
expect(report.pluginConfig.remoteConfigurations.inactive.uri).toBe("sls+p2p://");
|
||||
expect(settings.P2P_managedToken).toBe(token);
|
||||
expect(text).not.toMatch(/issued-user|issued-password|P2P_iceServers/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,63 @@
|
||||
import {
|
||||
hasManagedP2PTurnConfiguration,
|
||||
type ObsidianLiveSyncSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { pickP2PSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import { CLOUDFLARE_TURN_TYPE } from "@/integrations/cloudflare/settings";
|
||||
|
||||
/** Include inactive profiles when deciding whether Markdown would disclose provider settings. */
|
||||
export function hasManagedTurnSettings(settings: Partial<ObsidianLiveSyncSettings>): boolean {
|
||||
return (
|
||||
hasManagedP2PTurnConfiguration(settings) ||
|
||||
Object.values(settings.remoteConfigurations ?? {}).some(({ uri }) => {
|
||||
if (!uri.startsWith("sls+p2p://")) return false;
|
||||
const queryStart = uri.indexOf("?");
|
||||
return (
|
||||
queryStart >= 0 && new URLSearchParams(uri.slice(queryStart + 1).split("#", 1)[0]).has("managedType")
|
||||
);
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
/** Reports retain a recognised provider label and omit issued credentials. */
|
||||
export function redactTurnSettingsForReport(settings: Partial<ObsidianLiveSyncSettings>): void {
|
||||
if (settings.P2P_managedType) {
|
||||
settings.P2P_managedType =
|
||||
settings.P2P_managedType === CLOUDFLARE_TURN_TYPE ? CLOUDFLARE_TURN_TYPE : "redacted";
|
||||
}
|
||||
if (settings.P2P_managedId !== undefined) settings.P2P_managedId = "redacted";
|
||||
if (settings.P2P_managedToken !== undefined) settings.P2P_managedToken = "redacted";
|
||||
delete settings.P2P_iceServers;
|
||||
delete settings.P2P_iceServersExpiresAt;
|
||||
}
|
||||
|
||||
/** Managed connection profiles are shared through Setup URIs and QR codes. */
|
||||
export function omitManagedTurnProfilesFromMarkdown(settings: Partial<ObsidianLiveSyncSettings>): void {
|
||||
delete settings.P2P_iceServers;
|
||||
delete settings.P2P_iceServersExpiresAt;
|
||||
if (!hasManagedTurnSettings(settings)) return;
|
||||
delete settings.P2P_managedType;
|
||||
delete settings.P2P_managedId;
|
||||
delete settings.P2P_managedToken;
|
||||
delete settings.remoteConfigurations;
|
||||
delete settings.activeConfigurationId;
|
||||
delete settings.P2P_ActiveRemoteConfigurationId;
|
||||
}
|
||||
|
||||
/** Preserve the complete connection when Markdown omits its profile group. */
|
||||
export function preserveManagedTurnProfilesOnMarkdownImport(
|
||||
incoming: Partial<ObsidianLiveSyncSettings>,
|
||||
current: ObsidianLiveSyncSettings,
|
||||
merged: ObsidianLiveSyncSettings
|
||||
): void {
|
||||
if (
|
||||
!hasManagedTurnSettings(current) ||
|
||||
incoming.remoteConfigurations !== undefined ||
|
||||
incoming.P2P_managedType !== undefined
|
||||
)
|
||||
return;
|
||||
merged.remoteConfigurations = structuredClone(current.remoteConfigurations);
|
||||
merged.activeConfigurationId = current.activeConfigurationId;
|
||||
merged.P2P_ActiveRemoteConfigurationId = current.P2P_ActiveRemoteConfigurationId;
|
||||
Object.assign(merged, pickP2PSyncSettings(current));
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
DEFAULT_SETTINGS,
|
||||
REMOTE_P2P,
|
||||
type ObsidianLiveSyncSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import {
|
||||
SettingService,
|
||||
type SettingServiceDependencies,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/services/base/SettingService";
|
||||
import { ServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
|
||||
import { ConnectionStringParser } from "@vrtmrz/livesync-commonlib/compat/common/ConnectionString";
|
||||
import {
|
||||
hasManagedTurnSettings,
|
||||
omitManagedTurnProfilesFromMarkdown,
|
||||
preserveManagedTurnProfilesOnMarkdownImport,
|
||||
redactTurnSettingsForReport,
|
||||
} from "./turnSettingsPrivacy";
|
||||
|
||||
class MemorySettingService extends SettingService {
|
||||
readonly items = new Map<string, string>();
|
||||
saved?: ObsidianLiveSyncSettings;
|
||||
protected setItem(key: string, value: string) {
|
||||
this.items.set(key, value);
|
||||
}
|
||||
protected getItem(key: string) {
|
||||
return this.items.get(key) ?? "";
|
||||
}
|
||||
protected deleteItem(key: string) {
|
||||
this.items.delete(key);
|
||||
}
|
||||
protected saveData(settings: ObsidianLiveSyncSettings) {
|
||||
this.saved = structuredClone(settings);
|
||||
return Promise.resolve();
|
||||
}
|
||||
protected loadData() {
|
||||
return Promise.resolve(this.saved);
|
||||
}
|
||||
}
|
||||
|
||||
function configuredSettings() {
|
||||
return {
|
||||
...DEFAULT_SETTINGS,
|
||||
P2P_managedType: "CF",
|
||||
P2P_managedId: "private-key-id",
|
||||
P2P_managedToken: "private-token",
|
||||
remoteConfigurations: {
|
||||
managed: {
|
||||
id: "managed",
|
||||
name: "Managed TURN",
|
||||
isEncrypted: false,
|
||||
uri: "sls+p2p://room?managedType=CF&managedId=private-key-id&token=private-token",
|
||||
},
|
||||
},
|
||||
activeConfigurationId: "central",
|
||||
P2P_ActiveRemoteConfigurationId: "managed",
|
||||
};
|
||||
}
|
||||
|
||||
describe("managed TURN settings privacy", () => {
|
||||
it("preserves the active managed room through Markdown import, save, and reload", async () => {
|
||||
const current = {
|
||||
...configuredSettings(),
|
||||
remoteType: REMOTE_P2P,
|
||||
activeConfigurationId: "managed",
|
||||
P2P_roomID: "local-room",
|
||||
P2P_relays: "wss://local-relay.example.test",
|
||||
P2P_passphrase: "local-passphrase",
|
||||
};
|
||||
const originalURI = ConnectionStringParser.serialize({ type: "p2p", settings: current });
|
||||
current.remoteConfigurations.managed.uri = originalURI;
|
||||
const service = new MemorySettingService(new ServiceContext(), {
|
||||
APIService: {
|
||||
getSystemVaultName: () => "test-vault",
|
||||
getAppID: () => "test-app",
|
||||
addLog: () => undefined,
|
||||
confirm: { askString: async () => "" },
|
||||
} as unknown as SettingServiceDependencies["APIService"],
|
||||
});
|
||||
service.settings = structuredClone(current);
|
||||
const incoming: Partial<ObsidianLiveSyncSettings> = {
|
||||
P2P_roomID: "imported-room",
|
||||
P2P_relays: "wss://imported-relay.example.test",
|
||||
P2P_passphrase: "imported-passphrase",
|
||||
};
|
||||
const merged = { ...structuredClone(DEFAULT_SETTINGS), ...incoming };
|
||||
preserveManagedTurnProfilesOnMarkdownImport(incoming, current, merged);
|
||||
await service.applyExternalSettings(merged, true);
|
||||
const saved = service.saved!.remoteConfigurations.managed;
|
||||
const uri = saved.isEncrypted ? await service.decryptConfigurationItem(saved.uri, "*") : saved.uri;
|
||||
expect(uri).toBe(originalURI);
|
||||
expect(service.settings.P2P_roomID).toBe("local-room");
|
||||
await service.loadSettings();
|
||||
expect(service.settings.P2P_roomID).toBe("local-room");
|
||||
});
|
||||
|
||||
it("redacts provider fields and issued credentials, including unknown integrations", () => {
|
||||
const settings = configuredSettings();
|
||||
settings.P2P_managedType = "private-token";
|
||||
redactTurnSettingsForReport(settings);
|
||||
expect([settings.P2P_managedType, settings.P2P_managedId, settings.P2P_managedToken]).toEqual([
|
||||
"redacted",
|
||||
"redacted",
|
||||
"redacted",
|
||||
]);
|
||||
});
|
||||
|
||||
it("omits the whole managed profile group from Markdown, including inactive sources", () => {
|
||||
const settings = configuredSettings();
|
||||
settings.P2P_managedType = "";
|
||||
expect(hasManagedTurnSettings(settings)).toBe(true);
|
||||
omitManagedTurnProfilesFromMarkdown(settings);
|
||||
expect(JSON.stringify(settings)).not.toMatch(/private-token|private-key-id|sls\+p2p/);
|
||||
expect(settings).not.toHaveProperty("remoteConfigurations");
|
||||
expect(settings).not.toHaveProperty("activeConfigurationId");
|
||||
expect(settings).not.toHaveProperty("P2P_ActiveRemoteConfigurationId");
|
||||
});
|
||||
|
||||
it("preserves existing profiles and both selections when Markdown omits the group", () => {
|
||||
const current = configuredSettings();
|
||||
const incoming = { ...DEFAULT_SETTINGS };
|
||||
delete (incoming as Partial<typeof incoming>).remoteConfigurations;
|
||||
delete (incoming as Partial<typeof incoming>).P2P_managedType;
|
||||
const merged = { ...DEFAULT_SETTINGS, ...incoming };
|
||||
preserveManagedTurnProfilesOnMarkdownImport(incoming, current, merged);
|
||||
expect(merged.remoteConfigurations).toEqual(current.remoteConfigurations);
|
||||
expect(merged.remoteConfigurations).not.toBe(current.remoteConfigurations);
|
||||
expect(merged.P2P_managedToken).toEqual(current.P2P_managedToken);
|
||||
expect(merged.activeConfigurationId).toBe("central");
|
||||
expect(merged.P2P_ActiveRemoteConfigurationId).toBe("managed");
|
||||
});
|
||||
|
||||
it("retains the manual-only Markdown contract", () => {
|
||||
const settings = { ...DEFAULT_SETTINGS };
|
||||
const before = structuredClone(settings);
|
||||
omitManagedTurnProfilesFromMarkdown(settings);
|
||||
expect(settings).toEqual(before);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,69 @@
|
||||
<script lang="ts">
|
||||
import type { P2PConnectionInfo } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { CLOUDFLARE_TURN_TYPE } from "@/integrations/cloudflare/settings";
|
||||
import { validateManagedTurnSettings } from "@/integrations/turnSettings";
|
||||
import { translateLiveSyncMessage as translate, translateIfAvailable } from "@/common/translation";
|
||||
|
||||
type TurnSettings = Pick<P2PConnectionInfo, "P2P_turnServers" | "P2P_turnUsername" | "P2P_turnCredential" | "P2P_managedType" | "P2P_managedId" | "P2P_managedToken">;
|
||||
let { settings = $bindable() }: { settings: TurnSettings } = $props();
|
||||
const managedType = $derived(settings.P2P_managedType ?? "");
|
||||
const error = $derived(validateManagedTurnSettings(settings));
|
||||
|
||||
function selectProvider(type: string) {
|
||||
settings.P2P_managedType = type || undefined;
|
||||
settings.P2P_managedId = type ? "" : undefined;
|
||||
settings.P2P_managedToken = type ? "" : undefined;
|
||||
}
|
||||
</script>
|
||||
|
||||
<div class="turn-configuration">
|
||||
<label>
|
||||
<span>{translate("TURN configuration")}</span>
|
||||
<select aria-label={translate("TURN configuration")} name="p2p-turn-source" value={managedType} onchange={(event) => selectProvider(event.currentTarget.value)}>
|
||||
<option value="">{translate("Manual")}</option>
|
||||
<option value={CLOUDFLARE_TURN_TYPE}>{translate("Managed (Cloudflare)")}</option>
|
||||
{#if managedType !== "" && managedType !== CLOUDFLARE_TURN_TYPE}
|
||||
<option value={managedType} disabled>{translate("Unsupported TURN configuration")}</option>
|
||||
{/if}
|
||||
</select>
|
||||
</label>
|
||||
{#if managedType === ""}
|
||||
<label>
|
||||
<span>{translate("TURN Server URLs (comma-separated)")}</span>
|
||||
<textarea name="p2p-turn-servers" rows="3" placeholder="turn:turn.example.com:3478"
|
||||
bind:value={settings.P2P_turnServers} autocapitalize="off" spellcheck="false"></textarea>
|
||||
</label>
|
||||
<label>
|
||||
<span>{translate("TURN Username")}</span>
|
||||
<input type="text" name="p2p-turn-username" placeholder={translate("Enter TURN username")} bind:value={settings.P2P_turnUsername}
|
||||
autocomplete="off" autocapitalize="off" spellcheck="false" />
|
||||
</label>
|
||||
<label>
|
||||
<span>{translate("TURN Credential")}</span>
|
||||
<input type="password" name="p2p-turn-credential" placeholder={translate("Enter TURN credential")} bind:value={settings.P2P_turnCredential}
|
||||
autocomplete="new-password" />
|
||||
</label>
|
||||
{:else if managedType === CLOUDFLARE_TURN_TYPE}
|
||||
<label>
|
||||
<span>{translate("TURN Key ID")}</span>
|
||||
<input type="text" name="p2p-turn-turnKeyId" bind:value={settings.P2P_managedId}
|
||||
autocomplete="off" autocapitalize="off" spellcheck="false" />
|
||||
</label>
|
||||
<label>
|
||||
<span>{translate("TURN Key API Token")}</span>
|
||||
<input type="password" name="p2p-turn-apiToken" bind:value={settings.P2P_managedToken}
|
||||
autocomplete="new-password" autocapitalize="off" spellcheck="false" />
|
||||
</label>
|
||||
<p>{translate("The API token is saved with this profile and included in Setup URI and QR code sharing. Temporary TURN credentials are kept in memory only.")}</p>
|
||||
{/if}
|
||||
{#if error}
|
||||
<p role="status" class="turn-error">{translateIfAvailable(error)}</p>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
<style>
|
||||
label { display: grid; gap: 0.25rem; margin: 0.75rem 0; }
|
||||
input, textarea, select { box-sizing: border-box; width: 100%; }
|
||||
p { font-size: var(--font-ui-small, 0.9rem); }
|
||||
.turn-error { color: var(--text-error, #b33); }
|
||||
</style>
|
||||
@@ -0,0 +1,49 @@
|
||||
/** The provider identifier persisted in a P2P profile for Cloudflare TURN. */
|
||||
export const CLOUDFLARE_TURN_TYPE = "CF" as const;
|
||||
|
||||
/** The lifetime requested from Cloudflare for each issued credential set. */
|
||||
export const CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS = 86_400 as const;
|
||||
|
||||
/** The Cloudflare TURN credential-generation endpoint. */
|
||||
export const CLOUDFLARE_TURN_CREDENTIAL_ENDPOINT = "https://rtc.live.cloudflare.com/v1/turn/keys" as const;
|
||||
|
||||
/** A Cloudflare TURN configuration. */
|
||||
export interface CloudflareTurnConfiguration {
|
||||
readonly turnKeyId: string;
|
||||
readonly apiToken: string;
|
||||
}
|
||||
|
||||
// TURN Key IDs are inserted into one fixed URL path. Keep the accepted set
|
||||
// deliberately narrower than URI escaping so a configuration cannot alter
|
||||
// the request path or add a query string.
|
||||
const TURN_KEY_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._~-]{0,255}$/;
|
||||
|
||||
// RFC 6750's b64token grammar, including optional trailing padding. This
|
||||
// also excludes whitespace and control characters from the Authorization
|
||||
// header without exposing the token in a validation message.
|
||||
const BEARER_TOKEN_PATTERN = /^[A-Za-z0-9._~+/-]+={0,2}$/;
|
||||
const MAX_BEARER_TOKEN_LENGTH = 4_096;
|
||||
|
||||
/**
|
||||
* Returns a safe validation message for a Cloudflare TURN configuration.
|
||||
* The result never includes the supplied Key ID or API token.
|
||||
*/
|
||||
export function validateCloudflareTurnConfiguration(value: CloudflareTurnConfiguration): string | undefined {
|
||||
const turnKeyId = value.turnKeyId;
|
||||
if (typeof turnKeyId !== "string" || turnKeyId.length === 0) {
|
||||
return "Enter a TURN Key ID.";
|
||||
}
|
||||
if (!TURN_KEY_ID_PATTERN.test(turnKeyId)) {
|
||||
return "TURN Key ID contains unsupported characters.";
|
||||
}
|
||||
|
||||
const apiToken = value.apiToken;
|
||||
if (typeof apiToken !== "string" || apiToken.length === 0) {
|
||||
return "Enter a TURN Key API Token.";
|
||||
}
|
||||
if (apiToken.length > MAX_BEARER_TOKEN_LENGTH || !BEARER_TOKEN_PATTERN.test(apiToken)) {
|
||||
return "TURN Key API Token must use Bearer token syntax.";
|
||||
}
|
||||
|
||||
return undefined;
|
||||
}
|
||||
@@ -0,0 +1,364 @@
|
||||
import {
|
||||
CLOUDFLARE_TURN_CREDENTIAL_ENDPOINT,
|
||||
CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS,
|
||||
type CloudflareTurnConfiguration,
|
||||
validateCloudflareTurnConfiguration,
|
||||
} from "./settings";
|
||||
import { compatGlobal, type CompatTimeoutHandle } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
||||
|
||||
/** Fetch-compatible function supplied by the host composition. */
|
||||
export type CloudflareTurnFetch = (input: string | Request, init?: RequestInit) => Promise<Response>;
|
||||
|
||||
export interface CloudflareTurnDependencies {
|
||||
readonly fetch: CloudflareTurnFetch;
|
||||
readonly now?: () => number;
|
||||
readonly requestDeadlineMs?: number;
|
||||
}
|
||||
|
||||
export const CLOUDFLARE_TURN_REQUEST_DEADLINE_MS = 15_000 as const;
|
||||
export const CLOUDFLARE_TURN_MAX_RESPONSE_BYTES = 32 * 1024;
|
||||
export const CLOUDFLARE_TURN_MAX_ICE_SERVER_ENTRIES = 16 as const;
|
||||
export const CLOUDFLARE_TURN_MAX_ICE_SERVER_URLS = 32 as const;
|
||||
export const CLOUDFLARE_TURN_MIN_REMAINING_LIFETIME_MS = 30_000 as const;
|
||||
|
||||
type TurnFailureCode = "configuration" | "authentication" | "unavailable" | "invalid-response";
|
||||
|
||||
const FAILURE_MESSAGES: Record<TurnFailureCode, string> = {
|
||||
configuration: "The Cloudflare TURN configuration is invalid.",
|
||||
authentication: "The Cloudflare TURN credential request was not authorised.",
|
||||
unavailable: "The Cloudflare TURN service is unavailable.",
|
||||
"invalid-response": "The Cloudflare TURN service returned an invalid response.",
|
||||
};
|
||||
|
||||
function credentialFailure(code: TurnFailureCode, retryable: boolean): Error {
|
||||
return Object.assign(new Error(FAILURE_MESSAGES[code]), { code, retryable });
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function abortError(): Error {
|
||||
try {
|
||||
return new DOMException("The operation was aborted.", "AbortError");
|
||||
} catch {
|
||||
const error = new Error("The operation was aborted.");
|
||||
error.name = "AbortError";
|
||||
return error;
|
||||
}
|
||||
}
|
||||
|
||||
function throwIfAborted(signal: AbortSignal): void {
|
||||
if (signal.aborted) {
|
||||
throw abortError();
|
||||
}
|
||||
}
|
||||
|
||||
function isControlCharacter(value: string): boolean {
|
||||
return Array.from(value).some((character) => {
|
||||
const code = character.charCodeAt(0);
|
||||
return code <= 0x1f || code === 0x7f;
|
||||
});
|
||||
}
|
||||
|
||||
function isPort(value: string): boolean {
|
||||
if (!/^\d{1,5}$/.test(value)) return false;
|
||||
const port = Number(value);
|
||||
return port >= 1 && port <= 65_535;
|
||||
}
|
||||
|
||||
function isHost(value: string): boolean {
|
||||
return value.length > 0 && /^[A-Za-z0-9._-]+$/.test(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* Validates the URL forms accepted by WebRTC's ICE server configuration.
|
||||
* TURN URLs may carry only the standard transport query parameter; userinfo,
|
||||
* paths, fragments, and arbitrary query values are not accepted.
|
||||
*/
|
||||
export function isSupportedIceServerUrl(value: string): boolean {
|
||||
if (value.length === 0 || value.length > 2_048 || isControlCharacter(value)) return false;
|
||||
const schemeMatch = /^(stun|stuns|turn|turns):(.+)$/i.exec(value);
|
||||
if (!schemeMatch) return false;
|
||||
|
||||
const remainder = schemeMatch[2];
|
||||
const queryIndex = remainder.indexOf("?");
|
||||
const authority = queryIndex >= 0 ? remainder.slice(0, queryIndex) : remainder;
|
||||
const query = queryIndex >= 0 ? remainder.slice(queryIndex + 1) : "";
|
||||
if (authority.length === 0 || authority.includes("/") || authority.includes("#") || authority.includes("@")) {
|
||||
return false;
|
||||
}
|
||||
if (authority.includes("%")) return false;
|
||||
|
||||
if (authority.startsWith("[")) {
|
||||
const closingBracket = authority.indexOf("]");
|
||||
if (closingBracket < 0) return false;
|
||||
const host = authority.slice(1, closingBracket);
|
||||
if (!/^[0-9A-Fa-f:.]+$/.test(host) || !host.includes(":")) return false;
|
||||
const suffix = authority.slice(closingBracket + 1);
|
||||
if (suffix !== "" && (!suffix.startsWith(":") || !isPort(suffix.slice(1)))) return false;
|
||||
} else {
|
||||
const colonIndex = authority.lastIndexOf(":");
|
||||
const host = colonIndex >= 0 ? authority.slice(0, colonIndex) : authority;
|
||||
if (!isHost(host) || (colonIndex >= 0 && !isPort(authority.slice(colonIndex + 1)))) return false;
|
||||
// IPv6 literals must use brackets so a colon cannot be interpreted as
|
||||
// an ambiguous port separator.
|
||||
if (colonIndex >= 0 && host.includes(":")) return false;
|
||||
}
|
||||
|
||||
if (query.length === 0) return true;
|
||||
const queryParts = query.split("&");
|
||||
return queryParts.length === 1 && /^transport=(udp|tcp)$/i.test(queryParts[0]);
|
||||
}
|
||||
|
||||
function isTurnUrl(value: string): boolean {
|
||||
return /^(turn|turns):/i.test(value);
|
||||
}
|
||||
|
||||
function isCredential(value: unknown): value is string {
|
||||
return typeof value === "string" && value.length > 0 && value.length <= 4_096 && !isControlCharacter(value);
|
||||
}
|
||||
|
||||
function normaliseIceServers(value: unknown): readonly RTCIceServer[] {
|
||||
if (!isRecord(value) || !Array.isArray(value.iceServers)) {
|
||||
throw credentialFailure("invalid-response", false);
|
||||
}
|
||||
if (value.iceServers.length === 0 || value.iceServers.length > CLOUDFLARE_TURN_MAX_ICE_SERVER_ENTRIES) {
|
||||
throw credentialFailure("invalid-response", false);
|
||||
}
|
||||
|
||||
const servers: RTCIceServer[] = [];
|
||||
let urlCount = 0;
|
||||
let hasTurnServer = false;
|
||||
|
||||
for (const candidate of value.iceServers) {
|
||||
if (!isRecord(candidate)) throw credentialFailure("invalid-response", false);
|
||||
const rawUrls = candidate.urls;
|
||||
const urls =
|
||||
typeof rawUrls === "string"
|
||||
? [rawUrls]
|
||||
: Array.isArray(rawUrls) && rawUrls.every((url): url is string => typeof url === "string")
|
||||
? [...rawUrls]
|
||||
: undefined;
|
||||
if (!urls || urls.length === 0) throw credentialFailure("invalid-response", false);
|
||||
|
||||
urlCount += urls.length;
|
||||
if (urlCount > CLOUDFLARE_TURN_MAX_ICE_SERVER_URLS || urls.some((url) => !isSupportedIceServerUrl(url))) {
|
||||
throw credentialFailure("invalid-response", false);
|
||||
}
|
||||
|
||||
const turnEntry = urls.some(isTurnUrl);
|
||||
hasTurnServer ||= turnEntry;
|
||||
const normalised: RTCIceServer = { urls };
|
||||
if (turnEntry) {
|
||||
if (!isCredential(candidate.username) || !isCredential(candidate.credential)) {
|
||||
throw credentialFailure("invalid-response", false);
|
||||
}
|
||||
normalised.username = candidate.username;
|
||||
normalised.credential = candidate.credential;
|
||||
}
|
||||
servers.push(normalised);
|
||||
}
|
||||
|
||||
if (!hasTurnServer) throw credentialFailure("invalid-response", false);
|
||||
return Object.freeze(servers);
|
||||
}
|
||||
|
||||
class BoundedResponseError extends Error {
|
||||
constructor(readonly kind: "too-large" | "invalid-length" | "read-failed") {
|
||||
super(kind);
|
||||
}
|
||||
}
|
||||
|
||||
async function readResponseBody(response: Response): Promise<string> {
|
||||
const contentLength = response.headers.get("content-length");
|
||||
if (contentLength !== null) {
|
||||
const declaredLength = Number(contentLength);
|
||||
if (!Number.isFinite(declaredLength) || declaredLength < 0) {
|
||||
throw new BoundedResponseError("invalid-length");
|
||||
}
|
||||
if (declaredLength > CLOUDFLARE_TURN_MAX_RESPONSE_BYTES) {
|
||||
throw new BoundedResponseError("too-large");
|
||||
}
|
||||
}
|
||||
|
||||
if (!response.body) {
|
||||
try {
|
||||
const text = await response.text();
|
||||
if (new TextEncoder().encode(text).byteLength > CLOUDFLARE_TURN_MAX_RESPONSE_BYTES) {
|
||||
throw new BoundedResponseError("too-large");
|
||||
}
|
||||
return text;
|
||||
} catch (error) {
|
||||
if (error instanceof BoundedResponseError) throw error;
|
||||
throw new BoundedResponseError("read-failed");
|
||||
}
|
||||
}
|
||||
|
||||
const reader = response.body.getReader();
|
||||
const chunks: Uint8Array[] = [];
|
||||
let totalBytes = 0;
|
||||
try {
|
||||
while (true) {
|
||||
const result = await reader.read();
|
||||
if (result.done) break;
|
||||
totalBytes += result.value.byteLength;
|
||||
if (totalBytes > CLOUDFLARE_TURN_MAX_RESPONSE_BYTES) {
|
||||
try {
|
||||
await reader.cancel();
|
||||
} catch {
|
||||
// The response is already invalid because it exceeded the
|
||||
// bound; cancellation failure must not change the safe
|
||||
// classification or expose a host-specific error.
|
||||
}
|
||||
throw new BoundedResponseError("too-large");
|
||||
}
|
||||
chunks.push(result.value);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof BoundedResponseError) throw error;
|
||||
throw new BoundedResponseError("read-failed");
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
|
||||
const bytes = new Uint8Array(totalBytes);
|
||||
let offset = 0;
|
||||
for (const chunk of chunks) {
|
||||
bytes.set(chunk, offset);
|
||||
offset += chunk.byteLength;
|
||||
}
|
||||
return new TextDecoder().decode(bytes);
|
||||
}
|
||||
|
||||
function classifyHttpFailure(status: number): Error {
|
||||
if (status === 401 || status === 403) {
|
||||
return credentialFailure("authentication", false);
|
||||
}
|
||||
if (status === 408 || status === 429 || status >= 500) {
|
||||
return credentialFailure("unavailable", true);
|
||||
}
|
||||
return credentialFailure("unavailable", false);
|
||||
}
|
||||
|
||||
function parseResponseBody(body: string): readonly RTCIceServer[] {
|
||||
let value: unknown;
|
||||
try {
|
||||
value = JSON.parse(body) as unknown;
|
||||
} catch {
|
||||
throw credentialFailure("invalid-response", false);
|
||||
}
|
||||
return normaliseIceServers(value);
|
||||
}
|
||||
|
||||
/** Acquire one temporary ICE configuration for a new room connection. */
|
||||
export async function acquireCloudflareTurnCredentials(
|
||||
configuration: CloudflareTurnConfiguration,
|
||||
dependencies: CloudflareTurnDependencies,
|
||||
signal: AbortSignal
|
||||
): Promise<{ iceServers: readonly RTCIceServer[]; expiresAt: number }> {
|
||||
if (validateCloudflareTurnConfiguration(configuration)) throw credentialFailure("configuration", false);
|
||||
const now = dependencies.now ?? Date.now;
|
||||
const requestDeadlineMs = dependencies.requestDeadlineMs ?? CLOUDFLARE_TURN_REQUEST_DEADLINE_MS;
|
||||
throwIfAborted(signal);
|
||||
const requestStartedAt = now();
|
||||
if (!Number.isFinite(requestStartedAt)) {
|
||||
throw credentialFailure("unavailable", true);
|
||||
}
|
||||
|
||||
const requestController = new AbortController();
|
||||
let cancelledByCaller = false;
|
||||
let rejectCaller: ((reason?: unknown) => void) | undefined;
|
||||
const callerAbort = new Promise<never>((_resolve, reject) => {
|
||||
rejectCaller = reject;
|
||||
});
|
||||
let timedOut = false;
|
||||
const onAbort = () => {
|
||||
cancelledByCaller = true;
|
||||
requestController.abort();
|
||||
rejectCaller?.(abortError());
|
||||
};
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
if (signal.aborted) {
|
||||
signal.removeEventListener("abort", onAbort);
|
||||
requestController.abort();
|
||||
throw abortError();
|
||||
}
|
||||
let timeoutId: CompatTimeoutHandle | undefined;
|
||||
const deadline = new Promise<never>((_resolve, reject) => {
|
||||
timeoutId = compatGlobal.setTimeout(() => {
|
||||
timedOut = true;
|
||||
requestController.abort();
|
||||
reject(credentialFailure("unavailable", true));
|
||||
}, requestDeadlineMs);
|
||||
});
|
||||
|
||||
const cleanup = () => {
|
||||
if (timeoutId !== undefined) compatGlobal.clearTimeout(timeoutId);
|
||||
signal.removeEventListener("abort", onAbort);
|
||||
};
|
||||
|
||||
const endpoint = `${CLOUDFLARE_TURN_CREDENTIAL_ENDPOINT}/${configuration.turnKeyId}/credentials/generate-ice-servers`;
|
||||
let response: Response;
|
||||
try {
|
||||
response = await Promise.race([
|
||||
dependencies.fetch(endpoint, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${configuration.apiToken}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({ ttl: CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS }),
|
||||
signal: requestController.signal,
|
||||
redirect: "error",
|
||||
credentials: "omit",
|
||||
cache: "no-store",
|
||||
}),
|
||||
callerAbort,
|
||||
deadline,
|
||||
]);
|
||||
} catch {
|
||||
cleanup();
|
||||
if (cancelledByCaller || signal.aborted) throw abortError();
|
||||
if (timedOut) throw credentialFailure("unavailable", true);
|
||||
throw credentialFailure("unavailable", true);
|
||||
}
|
||||
|
||||
if (cancelledByCaller || signal.aborted) {
|
||||
cleanup();
|
||||
throw abortError();
|
||||
}
|
||||
if (timedOut || requestController.signal.aborted) {
|
||||
cleanup();
|
||||
throw credentialFailure("unavailable", true);
|
||||
}
|
||||
if (response.status !== 201) {
|
||||
cleanup();
|
||||
throw classifyHttpFailure(response.status);
|
||||
}
|
||||
|
||||
let body: string;
|
||||
try {
|
||||
body = await Promise.race([readResponseBody(response), callerAbort, deadline]);
|
||||
} catch (error) {
|
||||
cleanup();
|
||||
if (cancelledByCaller || signal.aborted) throw abortError();
|
||||
if (timedOut) throw credentialFailure("unavailable", true);
|
||||
if (error instanceof BoundedResponseError && error.kind === "read-failed") {
|
||||
throw credentialFailure("unavailable", true);
|
||||
}
|
||||
throw credentialFailure("invalid-response", false);
|
||||
}
|
||||
|
||||
try {
|
||||
throwIfAborted(signal);
|
||||
const iceServers = parseResponseBody(body);
|
||||
const expiresAt = requestStartedAt + CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS * 1_000;
|
||||
if (!Number.isFinite(expiresAt) || expiresAt <= now() + CLOUDFLARE_TURN_MIN_REMAINING_LIFETIME_MS) {
|
||||
throw credentialFailure("invalid-response", false);
|
||||
}
|
||||
return { iceServers, expiresAt };
|
||||
} finally {
|
||||
cleanup();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
CLOUDFLARE_TURN_MAX_RESPONSE_BYTES,
|
||||
CLOUDFLARE_TURN_REQUEST_DEADLINE_MS,
|
||||
acquireCloudflareTurnCredentials,
|
||||
} from "./turnCredentials";
|
||||
import {
|
||||
CLOUDFLARE_TURN_CREDENTIAL_ENDPOINT,
|
||||
CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS,
|
||||
validateCloudflareTurnConfiguration,
|
||||
} from "./settings";
|
||||
|
||||
const configuration = {
|
||||
turnKeyId: "key-123",
|
||||
apiToken: "token_abc-123",
|
||||
} as const;
|
||||
|
||||
function response(body: unknown, status = 201): Response {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
function validBody() {
|
||||
return {
|
||||
iceServers: [
|
||||
{
|
||||
urls: ["turn:relay.example.test:3478?transport=udp", "turns:relay.example.test:5349"],
|
||||
username: "turn-user",
|
||||
credential: "turn-password",
|
||||
},
|
||||
{ urls: "stun:stun.example.test:3478" },
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
describe("Cloudflare TURN credentials", () => {
|
||||
it("requests the fixed endpoint with the bearer token and TTL", async () => {
|
||||
const now = 1_000_000;
|
||||
let requestUrl: string | Request | undefined;
|
||||
let requestInit: RequestInit | undefined;
|
||||
const fetch = vi.fn(async (input: string | Request, init?: RequestInit) => {
|
||||
requestUrl = input;
|
||||
requestInit = init;
|
||||
return response(validBody());
|
||||
});
|
||||
const dependencies = { fetch, now: () => now };
|
||||
|
||||
const result = await acquireCloudflareTurnCredentials(
|
||||
configuration,
|
||||
dependencies,
|
||||
new AbortController().signal
|
||||
);
|
||||
|
||||
expect(requestUrl).toBe(`${CLOUDFLARE_TURN_CREDENTIAL_ENDPOINT}/key-123/credentials/generate-ice-servers`);
|
||||
expect(requestInit).toMatchObject({
|
||||
method: "POST",
|
||||
redirect: "error",
|
||||
credentials: "omit",
|
||||
cache: "no-store",
|
||||
body: JSON.stringify({ ttl: CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS }),
|
||||
});
|
||||
expect(new Headers(requestInit?.headers).get("authorization")).toBe("Bearer token_abc-123");
|
||||
expect(new Headers(requestInit?.headers).get("content-type")).toBe("application/json");
|
||||
expect(requestInit?.signal).toBeInstanceOf(AbortSignal);
|
||||
expect(result.iceServers).toHaveLength(2);
|
||||
expect(result.expiresAt).toBe(now + CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS * 1_000);
|
||||
});
|
||||
|
||||
it("rejects malformed, oversized, and STUN-only responses without exposing secrets", async () => {
|
||||
const cases: Array<{ body: unknown; expectedCode: string }> = [
|
||||
{ body: { iceServers: [] }, expectedCode: "invalid-response" },
|
||||
{ body: { iceServers: [{ urls: "turn:relay.example.test:3478" }] }, expectedCode: "invalid-response" },
|
||||
{ body: { iceServers: [{ urls: "stun:stun.example.test:3478" }] }, expectedCode: "invalid-response" },
|
||||
];
|
||||
for (const testCase of cases) {
|
||||
const dependencies = {
|
||||
fetch: vi.fn(async () => response(testCase.body)),
|
||||
now: () => 1_000_000,
|
||||
};
|
||||
const error = await acquireCloudflareTurnCredentials(
|
||||
configuration,
|
||||
dependencies,
|
||||
new AbortController().signal
|
||||
).catch((reason: unknown) => reason);
|
||||
expect(error).toMatchObject({ code: testCase.expectedCode });
|
||||
expect(String(error)).not.toContain(configuration.apiToken);
|
||||
expect(String(error)).not.toContain(configuration.turnKeyId);
|
||||
}
|
||||
|
||||
const oversized = "x".repeat(CLOUDFLARE_TURN_MAX_RESPONSE_BYTES + 1);
|
||||
const dependencies = {
|
||||
fetch: vi.fn(async () => new Response(oversized, { status: 201 })),
|
||||
now: () => 1_000_000,
|
||||
};
|
||||
const error = await acquireCloudflareTurnCredentials(
|
||||
configuration,
|
||||
dependencies,
|
||||
new AbortController().signal
|
||||
).catch((reason: unknown) => reason);
|
||||
expect(error).toMatchObject({ code: "invalid-response" });
|
||||
});
|
||||
|
||||
it("classifies authentication and transient provider failures", async () => {
|
||||
const authDependencies = {
|
||||
fetch: vi.fn(async () => response({}, 401)),
|
||||
};
|
||||
await expect(
|
||||
acquireCloudflareTurnCredentials(configuration, authDependencies, new AbortController().signal)
|
||||
).rejects.toMatchObject({
|
||||
code: "authentication",
|
||||
retryable: false,
|
||||
});
|
||||
|
||||
const transientDependencies = {
|
||||
fetch: vi.fn(async () => response({}, 503)),
|
||||
};
|
||||
await expect(
|
||||
acquireCloudflareTurnCredentials(configuration, transientDependencies, new AbortController().signal)
|
||||
).rejects.toMatchObject({
|
||||
code: "unavailable",
|
||||
retryable: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("propagates caller cancellation and turns a deadline into an unavailable failure", async () => {
|
||||
const controller = new AbortController();
|
||||
const fetch = vi.fn((_input: string | Request, init?: RequestInit) => {
|
||||
return new Promise<Response>((_resolve, reject) => {
|
||||
init?.signal?.addEventListener("abort", () => reject(new DOMException("aborted", "AbortError")), {
|
||||
once: true,
|
||||
});
|
||||
});
|
||||
});
|
||||
const dependencies = { fetch };
|
||||
const cancelled = acquireCloudflareTurnCredentials(configuration, dependencies, controller.signal);
|
||||
controller.abort();
|
||||
await expect(cancelled).rejects.toMatchObject({ name: "AbortError" });
|
||||
|
||||
vi.useFakeTimers();
|
||||
const timedDependencies = { fetch };
|
||||
const timed = acquireCloudflareTurnCredentials(configuration, timedDependencies, new AbortController().signal);
|
||||
const assertion = expect(timed).rejects.toMatchObject({ code: "unavailable", retryable: true });
|
||||
await vi.advanceTimersByTimeAsync(CLOUDFLARE_TURN_REQUEST_DEADLINE_MS);
|
||||
await assertion;
|
||||
});
|
||||
|
||||
it("rejects an issuance which has no usable remaining lifetime", async () => {
|
||||
let now = 1_000_000;
|
||||
const dependencies = {
|
||||
fetch: vi.fn(async () => {
|
||||
now += CLOUDFLARE_TURN_CREDENTIAL_TTL_SECONDS * 1_000;
|
||||
return response(validBody());
|
||||
}),
|
||||
now: () => now,
|
||||
};
|
||||
await expect(
|
||||
acquireCloudflareTurnCredentials(configuration, dependencies, new AbortController().signal)
|
||||
).rejects.toMatchObject({
|
||||
code: "invalid-response",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("Cloudflare TURN input validation", () => {
|
||||
it("rejects unsafe key IDs and malformed bearer credentials", () => {
|
||||
expect(
|
||||
validateCloudflareTurnConfiguration({ turnKeyId: "key/id", apiToken: configuration.apiToken })
|
||||
).toContain("unsupported characters");
|
||||
expect(validateCloudflareTurnConfiguration({ ...configuration, apiToken: "token with spaces" })).toContain(
|
||||
"Bearer token syntax"
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,14 @@
|
||||
import type { P2PConnectionInfo } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { CLOUDFLARE_TURN_TYPE, validateCloudflareTurnConfiguration } from "./cloudflare/settings";
|
||||
|
||||
/** Validate provider inputs without requesting credentials. */
|
||||
export function validateManagedTurnSettings(settings: Partial<P2PConnectionInfo>): string | undefined {
|
||||
if (settings.P2P_managedType === undefined || settings.P2P_managedType === "") return undefined;
|
||||
if (settings.P2P_managedType !== CLOUDFLARE_TURN_TYPE) {
|
||||
return "The selected TURN configuration is not supported.";
|
||||
}
|
||||
return validateCloudflareTurnConfiguration({
|
||||
turnKeyId: settings.P2P_managedId ?? "",
|
||||
apiToken: settings.P2P_managedToken ?? "",
|
||||
});
|
||||
}
|
||||
+3
-1
@@ -1,3 +1,4 @@
|
||||
import { useP2PSettingsPreparation } from "@/serviceFeatures/useP2PSettingsPreparation";
|
||||
import { getLanguage, Notice, Plugin, type App, type PluginManifest } from "./deps";
|
||||
import { setGetLanguage } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
||||
setGetLanguage(getLanguage);
|
||||
@@ -182,7 +183,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
|
||||
const replicator = useP2PReplicatorFeature(
|
||||
core,
|
||||
(_compatibilityReplicator, p2p) => createInteractiveP2PReplication(p2p),
|
||||
createOpenRebuildUI(this.app)
|
||||
createOpenRebuildUI(this.app),
|
||||
{ prepareP2PSettings: useP2PSettingsPreparation(core.services.API.webCompatFetch.bind(core.services.API)) }
|
||||
);
|
||||
setupManager.registerP2PSetupConnectionProbe(replicator.connectionProbe);
|
||||
useP2PReplicatorCommands(core, replicator);
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
import {
|
||||
hasManagedTurnSettings,
|
||||
omitManagedTurnProfilesFromMarkdown,
|
||||
preserveManagedTurnProfilesOnMarkdownImport,
|
||||
} from "@/common/turnSettingsPrivacy";
|
||||
// import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser";
|
||||
import { isObjectDifferent } from "octagonal-wheels/object";
|
||||
import { EVENT_SETTING_SAVED, eventHub } from "@/common/events";
|
||||
@@ -129,6 +134,7 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractModule {
|
||||
|
||||
let settingToApply = { ...DEFAULT_SETTINGS } as ObsidianLiveSyncSettings;
|
||||
settingToApply = { ...settingToApply, ...newSetting };
|
||||
preserveManagedTurnProfilesOnMarkdownImport(newSetting, this.settings, settingToApply);
|
||||
if (!settingToApply?.writeCredentialsForSettingSync) {
|
||||
//New setting does not contains credentials.
|
||||
settingToApply.couchDB_USER = this.settings.couchDB_USER;
|
||||
@@ -208,11 +214,18 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractModule {
|
||||
delete saveData.couchDB_CustomHeaders;
|
||||
delete saveData.bucketCustomHeaders;
|
||||
}
|
||||
omitManagedTurnProfilesFromMarkdown(saveData);
|
||||
return saveData;
|
||||
}
|
||||
|
||||
async saveSettingToMarkdown(filename: string) {
|
||||
const saveData = this.generateSettingForMarkdown();
|
||||
if (hasManagedTurnSettings(this.settings)) {
|
||||
this._log(
|
||||
"Share TURN provider credentials through an encrypted Setup URI. Connection profiles are omitted from Markdown settings.",
|
||||
LOG_LEVEL_INFO
|
||||
);
|
||||
}
|
||||
const file = await this.core.storageAccess.isExists(filename);
|
||||
|
||||
if (!file) {
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
<script lang="ts">
|
||||
import TurnConfiguration from "@/features/P2PSync/TurnConfiguration.svelte";
|
||||
import { validateManagedTurnSettings } from "@/integrations/turnSettings";
|
||||
// import { delay } from "octagonal-wheels/promises";
|
||||
import DialogHeader from "@/modules/services/LiveSyncUI/components/DialogHeader.svelte";
|
||||
import Guidance from "@/modules/services/LiveSyncUI/components/Guidance.svelte";
|
||||
@@ -15,7 +17,7 @@
|
||||
P2PMessageSizePresets,
|
||||
PREFERRED_BASE,
|
||||
RemoteTypes,
|
||||
hasValidP2PTurnServerUrl,
|
||||
hasP2PTurnConfiguration,
|
||||
normaliseP2PConnectionPath,
|
||||
normaliseP2PMaxWirePayloadBytes,
|
||||
type EntryDoc,
|
||||
@@ -27,7 +29,6 @@
|
||||
import { TrysteroReplicator } from "@vrtmrz/livesync-commonlib/compat/replication/trystero/TrysteroReplicator";
|
||||
import type { ReplicatorHostEnv } from "@vrtmrz/livesync-commonlib/compat/replication/trystero/types";
|
||||
import {
|
||||
copyTo,
|
||||
generateP2PRoomId,
|
||||
pickP2PSyncSettings,
|
||||
type SimpleStore,
|
||||
@@ -51,7 +52,7 @@
|
||||
const context = getDialogContext();
|
||||
let error = $state("");
|
||||
let connectionPathResetNotice = $state(false);
|
||||
const hasValidTurnServer = $derived(hasValidP2PTurnServerUrl(syncSetting.P2P_turnServers ?? ""));
|
||||
const hasValidTurnServer = $derived(hasP2PTurnConfiguration(syncSetting));
|
||||
type Props = GuestDialogProps<SetupRemoteP2PResultType, SetupRemoteP2PInitialData>;
|
||||
|
||||
const { setResult, getInitialData }: Props = $props();
|
||||
@@ -61,7 +62,7 @@
|
||||
connectionProbe = initialData?.connectionProbe;
|
||||
const initialSettings = initialData?.settings;
|
||||
if (initialSettings) {
|
||||
copyTo(initialSettings, syncSetting);
|
||||
syncSetting = pickP2PSyncSettings(initialSettings);
|
||||
}
|
||||
const initialPeerName = (initialSettings?.P2P_DevicePeerName ?? "").trim();
|
||||
if (initialPeerName !== "") {
|
||||
@@ -100,12 +101,14 @@
|
||||
async function checkConnection() {
|
||||
try {
|
||||
processing = true;
|
||||
const sourceError = validateManagedTurnSettings(syncSetting);
|
||||
if (sourceError) return sourceError;
|
||||
const trialRemoteSetting = generateSetting();
|
||||
const admission = connectionProbe;
|
||||
if (!admission) {
|
||||
throw new Error("The P2P Setup connection probe is not available.");
|
||||
}
|
||||
const result = await coordinateP2PSetupConnectionProbe(admission, trialRemoteSetting, async () => {
|
||||
const result = await coordinateP2PSetupConnectionProbe(admission, trialRemoteSetting, async (signallingSettings) => {
|
||||
const map = new Map<string, unknown>();
|
||||
const store = {
|
||||
get: (key: string) => {
|
||||
@@ -133,7 +136,7 @@
|
||||
const env: ReplicatorHostEnv = {
|
||||
events: context.context.events,
|
||||
translate: context.context.translate,
|
||||
settings: trialRemoteSetting,
|
||||
settings: signallingSettings,
|
||||
processReplicatedDocs: async (_docs: PouchDB.Core.ExistingDocument<EntryDoc>[]) => {
|
||||
return;
|
||||
},
|
||||
@@ -204,6 +207,8 @@
|
||||
}
|
||||
}
|
||||
function commit() {
|
||||
error = validateManagedTurnSettings(syncSetting) ?? "";
|
||||
if (error) return;
|
||||
const setting = pickP2PSyncSettings(generateSetting());
|
||||
setResult(setting);
|
||||
}
|
||||
@@ -215,7 +220,8 @@
|
||||
syncSetting.P2P_relays.trim() !== "" &&
|
||||
syncSetting.P2P_roomID.trim() !== "" &&
|
||||
syncSetting.P2P_passphrase.trim() !== "" &&
|
||||
(syncSetting.P2P_DevicePeerName ?? "").trim() !== ""
|
||||
(syncSetting.P2P_DevicePeerName ?? "").trim() !== "" &&
|
||||
validateManagedTurnSettings(syncSetting) === undefined
|
||||
);
|
||||
});
|
||||
</script>
|
||||
@@ -339,24 +345,24 @@
|
||||
</InputRow>
|
||||
<InfoNote>
|
||||
{translateMessage(
|
||||
"TURN relay only is available when at least one valid TURN server URL is configured under Advanced Settings."
|
||||
"TURN relay only requires a TURN server or a configured credential source under Advanced Settings."
|
||||
)}
|
||||
</InfoNote>
|
||||
<InfoNote notice visible={connectionPathResetNotice}>
|
||||
{translateMessage(
|
||||
"TURN relay only requires at least one valid TURN server URL. Connection path has been restored to Automatic."
|
||||
"TURN relay only requires TURN configuration. Connection path has been restored to Automatic."
|
||||
)}
|
||||
</InfoNote>
|
||||
</ExtraItems>
|
||||
<ExtraItems title={translateMessage("Advanced Settings")}>
|
||||
<InfoNote>
|
||||
{translateMessage(
|
||||
"TURN server settings are only necessary if you are behind a strict NAT or firewall that prevents direct P2P connections. In most cases, you can leave these fields blank."
|
||||
"Configure TURN when a direct connection cannot be established or when you select TURN relay only."
|
||||
)}
|
||||
</InfoNote>
|
||||
<InfoNote warning>
|
||||
<InfoNote>
|
||||
{translateMessage(
|
||||
"TURN relays the encrypted WebRTC connection only when a direct path cannot be established. A TURN provider cannot read encrypted Vault contents, but it can observe connection metadata and traffic volume. Use a provider you trust."
|
||||
"WebRTC encrypts data between your devices, including when it passes through TURN. The TURN provider cannot read the transferred data. It can see network addresses and traffic volume."
|
||||
)}
|
||||
<a
|
||||
href="https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/p2p.md#signalling-relay-and-turn-server"
|
||||
@@ -364,34 +370,7 @@
|
||||
rel="noopener noreferrer">{translateMessage("Learn more about signalling and TURN")}</a
|
||||
>.
|
||||
</InfoNote>
|
||||
<InputRow label={translateMessage("TURN Server URLs (comma-separated)")}>
|
||||
<textarea
|
||||
name="p2p-turn-servers"
|
||||
placeholder="turn:turn.example.com:3478,turn:turn.example.com:443"
|
||||
autocapitalize="off"
|
||||
spellcheck="false"
|
||||
bind:value={syncSetting.P2P_turnServers}
|
||||
rows="5"
|
||||
></textarea>
|
||||
</InputRow>
|
||||
<InputRow label={translateMessage("TURN Username")}>
|
||||
<input
|
||||
type="text"
|
||||
name="p2p-turn-username"
|
||||
placeholder={translateMessage("Enter TURN username")}
|
||||
autocorrect="off"
|
||||
autocapitalize="off"
|
||||
spellcheck="false"
|
||||
bind:value={syncSetting.P2P_turnUsername}
|
||||
/>
|
||||
</InputRow>
|
||||
<InputRow label={translateMessage("TURN Credential")}>
|
||||
<Password
|
||||
name="p2p-turn-credential"
|
||||
placeholder={translateMessage("Enter TURN credential")}
|
||||
bind:value={syncSetting.P2P_turnCredential}
|
||||
/>
|
||||
</InputRow>
|
||||
<TurnConfiguration bind:settings={syncSetting} />
|
||||
</ExtraItems>
|
||||
<InfoNote error visible={error !== ""}>
|
||||
{error}
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
<script lang="ts">
|
||||
import { configURIBase } from "@/common/types";
|
||||
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import DialogHeader from "@/modules/services/LiveSyncUI/components/DialogHeader.svelte";
|
||||
import Guidance from "@/modules/services/LiveSyncUI/components/Guidance.svelte";
|
||||
import Decision from "@/modules/services/LiveSyncUI/components/Decision.svelte";
|
||||
@@ -10,7 +9,7 @@
|
||||
import Password from "@/modules/services/LiveSyncUI/components/Password.svelte";
|
||||
|
||||
import { onMount } from "svelte";
|
||||
import { decryptString } from "@vrtmrz/livesync-commonlib/compat/encryption/stringEncryption";
|
||||
import { decodeSettingsFromSetupURI } from "@vrtmrz/livesync-commonlib/compat/API/processSetting";
|
||||
import type { GuestDialogProps } from "@/modules/services/LiveSyncUI/svelteDialog";
|
||||
import { TYPE_CANCELLED, type UseSetupURIResultType } from "./setupDialogTypes";
|
||||
import { $msg as translateMessage } from "@/common/translation";
|
||||
@@ -30,7 +29,7 @@
|
||||
}
|
||||
});
|
||||
|
||||
const seemsValid = $derived.by(() => setupURI.startsWith(configURIBase));
|
||||
const seemsValid = $derived(setupURI.startsWith(configURIBase));
|
||||
async function processSetupURI() {
|
||||
error = "";
|
||||
if (!seemsValid) return;
|
||||
@@ -39,11 +38,8 @@
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const settingPieces = setupURI.substring(configURIBase.length);
|
||||
const encodedConfig = decodeURIComponent(settingPieces);
|
||||
const newConf = (await JSON.parse(
|
||||
await decryptString(encodedConfig, passphrase)
|
||||
)) as ObsidianLiveSyncSettings;
|
||||
const newConf = await decodeSettingsFromSetupURI(setupURI.trim(), passphrase);
|
||||
if (!newConf) throw new Error("Invalid Setup URI settings");
|
||||
setResult(newConf);
|
||||
// Logger("Settings imported successfully", LOG_LEVEL_NOTICE);
|
||||
return;
|
||||
|
||||
@@ -3,6 +3,7 @@ import {
|
||||
type P2PConnectionProbeAdmission,
|
||||
type P2PConnectionProbeSettings,
|
||||
} from "@vrtmrz/livesync-commonlib/p2p";
|
||||
import { P2PConnectionPaths, type P2PSyncSetting } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
|
||||
export type P2PSetupConnectionProbeResult =
|
||||
| { readonly ok: true }
|
||||
@@ -20,12 +21,25 @@ export interface P2PSetupConnectionProbe {
|
||||
}
|
||||
|
||||
/** Interpret the stable P2P owner's admission without constructing transport eagerly. */
|
||||
export async function coordinateP2PSetupConnectionProbe(
|
||||
export async function coordinateP2PSetupConnectionProbe<T extends P2PConnectionProbeSettings>(
|
||||
admission: P2PConnectionProbeAdmission,
|
||||
trialSettings: P2PConnectionProbeSettings,
|
||||
runOwnedTrial: () => Promise<P2PSetupConnectionProbeResult>
|
||||
trialSettings: T,
|
||||
runOwnedTrial: (settings: T) => Promise<P2PSetupConnectionProbeResult>
|
||||
): Promise<P2PSetupConnectionProbeResult> {
|
||||
const settlement = await admission.run(trialSettings, runOwnedTrial);
|
||||
const settlement = await admission.run(trialSettings, () => {
|
||||
// This trial checks signalling only; TURN allocation belongs to an actual connection.
|
||||
const settings: T & Partial<P2PSyncSetting> = { ...trialSettings };
|
||||
delete settings.P2P_managedType;
|
||||
delete settings.P2P_managedId;
|
||||
delete settings.P2P_managedToken;
|
||||
delete settings.P2P_iceServers;
|
||||
delete settings.P2P_iceServersExpiresAt;
|
||||
settings.P2P_turnServers = "";
|
||||
settings.P2P_turnUsername = "";
|
||||
settings.P2P_turnCredential = "";
|
||||
settings.P2P_connectionPath = P2PConnectionPaths.Automatic;
|
||||
return runOwnedTrial(settings);
|
||||
});
|
||||
if (settlement.status === "observed-active") return { ok: true };
|
||||
if (settlement.status === "blocked") {
|
||||
return {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { ACTIVE_P2P_RELAY_BINDING_CONFLICT, type P2PConnectionProbeAdmission } from "@vrtmrz/livesync-commonlib/p2p";
|
||||
import { DEFAULT_SETTINGS, P2PConnectionPaths } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import {
|
||||
coordinateP2PSetupConnectionProbe,
|
||||
probeP2PSetupConnection,
|
||||
@@ -7,6 +8,40 @@ import {
|
||||
} from "./p2pSetupConnectionProbe";
|
||||
|
||||
describe("P2P setup connection probe", () => {
|
||||
it("constructs a signalling-only trial when the draft selects managed TURN", async () => {
|
||||
const settings = {
|
||||
...DEFAULT_SETTINGS,
|
||||
P2P_managedType: "CF",
|
||||
P2P_managedId: "test-key",
|
||||
P2P_managedToken: "test-token",
|
||||
P2P_iceServers: [
|
||||
{ urls: "turn:temporary.example.test", username: "issued-user", credential: "issued-password" },
|
||||
],
|
||||
P2P_iceServersExpiresAt: 123456789,
|
||||
P2P_turnServers: "turn:unused.example.test:3478",
|
||||
P2P_turnUsername: "unused-user",
|
||||
P2P_turnCredential: "unused-password",
|
||||
P2P_connectionPath: P2PConnectionPaths.Relay,
|
||||
};
|
||||
const admission: P2PConnectionProbeAdmission = {
|
||||
run: async (_settings, trial) => ({ status: "trial", result: await trial() }),
|
||||
};
|
||||
const result = await coordinateP2PSetupConnectionProbe(admission, settings, async (trial = settings) => {
|
||||
expect(trial.P2P_managedType).toBeUndefined();
|
||||
expect(trial.P2P_managedToken).toBeUndefined();
|
||||
expect(trial.P2P_iceServers).toBeUndefined();
|
||||
expect(trial.P2P_iceServersExpiresAt).toBeUndefined();
|
||||
expect(trial.P2P_turnServers).toBe("");
|
||||
expect(trial.P2P_turnUsername).toBe("");
|
||||
expect(trial.P2P_turnCredential).toBe("");
|
||||
expect(trial.P2P_connectionPath).toBe(P2PConnectionPaths.Automatic);
|
||||
return { ok: true };
|
||||
});
|
||||
expect(result).toEqual({ ok: true });
|
||||
expect(settings.P2P_managedToken).toBe("test-token");
|
||||
expect(settings.P2P_connectionPath).toBe(P2PConnectionPaths.Relay);
|
||||
});
|
||||
|
||||
it("uses a compatible active signalling connection without constructing a trial", async () => {
|
||||
const runOwnedTrial = vi.fn(async (): Promise<P2PSetupConnectionProbeResult> => ({ ok: true }));
|
||||
const admission: P2PConnectionProbeAdmission = {
|
||||
|
||||
@@ -83,6 +83,40 @@ function setup(options: SetupOptions = {}) {
|
||||
}
|
||||
|
||||
describe("ReplicateResultProcessor", () => {
|
||||
it("resumes another document after in-flight updates to one document fill the application slots", async () => {
|
||||
const hotGate = promiseWithResolvers<boolean>();
|
||||
const { processor, processSynchroniseResult } = setup({
|
||||
processSynchroniseResult: async (entry) => {
|
||||
if ((entry as { _id: string })._id === "hot-queue") return await hotGate.promise;
|
||||
return true;
|
||||
},
|
||||
});
|
||||
try {
|
||||
for (let index = 1; index <= 10; index++) {
|
||||
// A queued duplicate is coalesced; a new notification for a document
|
||||
// already being processed can occupy another application slot.
|
||||
processor.enqueueAll([note("hot-queue")]);
|
||||
await vi.waitFor(() => expect(processor["_processingChanges"]).toHaveLength(index));
|
||||
}
|
||||
processor.enqueueAll([note("unrelated-queue")]);
|
||||
await vi.waitFor(() => {
|
||||
expect(processor["_semaphore"].waiting).toBeGreaterThan(0);
|
||||
expect(processSynchroniseResult).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
expect(processor["_queuedChanges"].map((entry) => entry._id)).toEqual(["unrelated-queue"]);
|
||||
} finally {
|
||||
hotGate.resolve(true);
|
||||
await vi.waitFor(() => {
|
||||
expect(processor["_processingChanges"]).toHaveLength(0);
|
||||
expect(processor["_queuedChanges"]).toHaveLength(0);
|
||||
});
|
||||
}
|
||||
expect(processSynchroniseResult).toHaveBeenCalledTimes(11);
|
||||
expect(processSynchroniseResult.mock.calls.some(([entry]) =>
|
||||
(entry as { _id: string })._id === "unrelated-queue"
|
||||
)).toBe(true);
|
||||
});
|
||||
|
||||
it("suspends result application while the application is not ready", () => {
|
||||
const { isReady, processor } = setup({ applicationReady: false });
|
||||
|
||||
|
||||
@@ -9,7 +9,8 @@ import { fireAndForget } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import type { SetupFeatureHost } from "./types";
|
||||
|
||||
export async function encodeSetupSettingsAsQR(host: SetupFeatureHost) {
|
||||
const settingString = encodeSettingsToQRCodeData(host.services.setting.currentSettings());
|
||||
const settings = host.services.setting.currentSettings();
|
||||
const settingString = encodeSettingsToQRCodeData(settings);
|
||||
const result = encodeQR(settingString, OutputFormat.SVG);
|
||||
if (result === "") {
|
||||
return "";
|
||||
|
||||
@@ -3,6 +3,9 @@ import { EVENT_REQUEST_SHOW_SETUP_QR } from "@vrtmrz/livesync-commonlib/compat/e
|
||||
import { createServiceContext } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { encodeSetupSettingsAsQR, useSetupQRCodeFeature } from "./qrCode";
|
||||
import { encodeQR, encodeSettingsToQRCodeData } from "@vrtmrz/livesync-commonlib/compat/API/processSetting";
|
||||
import { copySetupURI } from "./setupUri";
|
||||
|
||||
vi.mock("./setupUri", () => ({ copySetupURI: vi.fn() }));
|
||||
|
||||
vi.mock("@vrtmrz/livesync-commonlib/compat/API/processSetting", () => {
|
||||
return {
|
||||
@@ -15,6 +18,32 @@ vi.mock("@vrtmrz/livesync-commonlib/compat/API/processSetting", () => {
|
||||
});
|
||||
|
||||
describe("setupObsidian/qrCode", () => {
|
||||
it("shows managed TURN settings and inactive profiles through the ordinary QR dialogue", async () => {
|
||||
const settings = {
|
||||
remoteConfigurations: {
|
||||
managed: { uri: "sls+p2p://room?managedType=CF&managedId=turn-key&token=private-token" },
|
||||
},
|
||||
};
|
||||
const confirmWithMessage = vi.fn();
|
||||
const translate = vi.fn(() => "qr-message");
|
||||
const host = {
|
||||
services: {
|
||||
API: { addLog: vi.fn() },
|
||||
context: createServiceContext({ translate }),
|
||||
setting: { currentSettings: () => settings },
|
||||
UI: { confirm: { confirmWithMessage } },
|
||||
},
|
||||
} as any;
|
||||
vi.mocked(encodeSettingsToQRCodeData).mockReturnValue("encoded-settings");
|
||||
vi.mocked(encodeQR).mockReturnValue("<svg/>");
|
||||
|
||||
expect(await encodeSetupSettingsAsQR(host)).toBe("<svg/>");
|
||||
expect(encodeSettingsToQRCodeData).toHaveBeenCalledWith(settings);
|
||||
expect(translate).toHaveBeenCalledWith("Setup.QRCode", { qr_image: "<svg/>" });
|
||||
expect(confirmWithMessage).toHaveBeenCalledWith("Settings QR Code", "qr-message", ["OK"], "OK");
|
||||
expect(copySetupURI).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
vi.clearAllMocks();
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { P2PSyncSetting } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { acquireCloudflareTurnCredentials, type CloudflareTurnFetch } from "@/integrations/cloudflare/turnCredentials";
|
||||
import { validateManagedTurnSettings } from "@/integrations/turnSettings";
|
||||
|
||||
/** Prepare a connection copy using the host's HTTP adapter. */
|
||||
export function useP2PSettingsPreparation(fetch: CloudflareTurnFetch) {
|
||||
return async (settings: Readonly<P2PSyncSetting>, signal: AbortSignal): Promise<P2PSyncSetting> => {
|
||||
const error = validateManagedTurnSettings(settings);
|
||||
if (error) throw new Error(error);
|
||||
if (!settings.P2P_managedType) return { ...settings };
|
||||
const { iceServers, expiresAt } = await acquireCloudflareTurnCredentials(
|
||||
{ turnKeyId: settings.P2P_managedId ?? "", apiToken: settings.P2P_managedToken ?? "" },
|
||||
{ fetch },
|
||||
signal
|
||||
);
|
||||
return { ...settings, P2P_iceServers: iceServers, P2P_iceServersExpiresAt: expiresAt };
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { useP2PSettingsPreparation } from "./useP2PSettingsPreparation";
|
||||
|
||||
const managed = {
|
||||
...DEFAULT_SETTINGS,
|
||||
P2P_managedType: "CF",
|
||||
P2P_managedId: "key-123",
|
||||
P2P_managedToken: "test-token",
|
||||
};
|
||||
|
||||
describe("host preparation of P2P settings", () => {
|
||||
it("puts issued ICE credentials on a connection copy without changing saved inputs", async () => {
|
||||
const iceServers = [
|
||||
{ urls: ["turn:relay.example.test:3478"], username: "issued-user", credential: "issued-password" },
|
||||
];
|
||||
const fetch = vi.fn(async () => new Response(JSON.stringify({ iceServers }), { status: 201 }));
|
||||
const before = structuredClone(managed);
|
||||
const settings = await useP2PSettingsPreparation(fetch)(managed, new AbortController().signal);
|
||||
expect(settings.P2P_iceServers).toEqual(iceServers);
|
||||
expect(settings.P2P_iceServersExpiresAt).toBeGreaterThan(Date.now());
|
||||
expect(managed).toEqual(before);
|
||||
expect(settings).not.toBe(managed);
|
||||
expect(fetch).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("keeps manual settings and rejects an unsupported provider without HTTP requests", async () => {
|
||||
const fetch = vi.fn();
|
||||
const prepare = useP2PSettingsPreparation(fetch);
|
||||
await expect(prepare(DEFAULT_SETTINGS, new AbortController().signal)).resolves.toEqual(DEFAULT_SETTINGS);
|
||||
await expect(prepare({ ...managed, P2P_managedType: "unknown" }, new AbortController().signal)).rejects.toThrow(
|
||||
"not supported"
|
||||
);
|
||||
await expect(
|
||||
prepare({ ...managed, P2P_managedToken: "invalid token" }, new AbortController().signal)
|
||||
).rejects.toThrow("Bearer token syntax");
|
||||
expect(fetch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("propagates a safe acquisition failure without using the manual TURN fields", async () => {
|
||||
const fetch = vi.fn(async () => new Response(null, { status: 401 }));
|
||||
const prepare = useP2PSettingsPreparation(fetch);
|
||||
await expect(
|
||||
prepare({ ...managed, P2P_turnServers: "turn:manual.example.test" }, new AbortController().signal)
|
||||
).rejects.toThrow("not authorised");
|
||||
expect(fetch).toHaveBeenCalledOnce();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,334 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import PouchDB from "pouchdb-core";
|
||||
import MemoryAdapter from "pouchdb-adapter-memory";
|
||||
import HttpAdapter from "pouchdb-adapter-http";
|
||||
import replication from "pouchdb-replication";
|
||||
import type { EntryDoc, FilePathWithPrefix, UXFileInfo } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { compareMTime, createTextBlob, readContent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import { createLiveSyncEventHub } from "@vrtmrz/livesync-commonlib/context";
|
||||
import { LiveSyncLocalDB, type LiveSyncLocalDBEnv } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
|
||||
import {
|
||||
ServiceDatabaseFileAccessBase,
|
||||
type ServiceDatabaseFileAccessDependencies,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/ServiceDatabaseFileAccessBase";
|
||||
import type { ServiceFileHandlerDependencies } from "@vrtmrz/livesync-commonlib/compat/serviceModules/ServiceFileHandlerBase";
|
||||
import { ServiceFileHandler } from "./FileHandler";
|
||||
import {
|
||||
createConflictResolutionOperations,
|
||||
type ConflictResolutionOperationsDependencies,
|
||||
} from "@/serviceFeatures/conflictResolution/operations";
|
||||
import { runCommand } from "@/apps/cli/commands/runCommand";
|
||||
import type { CLICommandContext } from "@/apps/cli/commands/types";
|
||||
|
||||
PouchDB.plugin(MemoryAdapter).plugin(HttpAdapter).plugin(replication);
|
||||
const path = "multi-device.txt" as FilePathWithPrefix;
|
||||
const old = "Original content\n";
|
||||
const oldTime = 1_000_000;
|
||||
class TestHandler extends ServiceFileHandler {}
|
||||
|
||||
function makeFile(body: string, mtime = oldTime): UXFileInfo {
|
||||
return {
|
||||
name: path,
|
||||
path,
|
||||
stat: { type: "file", ctime: oldTime, mtime, size: new Blob([body]).size },
|
||||
body: createTextBlob(body),
|
||||
};
|
||||
}
|
||||
|
||||
async function makeDevice(name: string) {
|
||||
const db = new PouchDB<EntryDoc>(name, { adapter: "memory" });
|
||||
const reflection = new Map<FilePathWithPrefix, { revision: string; observedStorageMtime?: number }>();
|
||||
let storage = makeFile(old);
|
||||
const settings = { ...DEFAULT_SETTINGS, useOnlyLocalChunk: true, writeDocumentsIfConflicted: false };
|
||||
const setting = { currentSettings: () => settings };
|
||||
const pathService = {
|
||||
path2id: (value: string) => Promise.resolve(value),
|
||||
id2path: (id: string, entry?: { path?: string }) => entry?.path ?? id,
|
||||
getPath: (entry: { path: FilePathWithPrefix }) => entry.path,
|
||||
compareFileFreshness: (file: UXFileInfo, entry: { mtime: number }) =>
|
||||
compareMTime(file.stat.mtime, entry.mtime),
|
||||
markChangesAreSame: vi.fn(),
|
||||
};
|
||||
const events = createLiveSyncEventHub();
|
||||
const API = { addLog: vi.fn() };
|
||||
const localDatabase = new LiveSyncLocalDB(name, {
|
||||
services: {
|
||||
API,
|
||||
setting,
|
||||
path: pathService,
|
||||
context: { events },
|
||||
database: { createPouchDBInstance: () => db },
|
||||
databaseEvents: {
|
||||
onDatabaseInitialisation: () => Promise.resolve(true),
|
||||
onDatabaseHasReady: () => Promise.resolve(true),
|
||||
onCloseDatabase: () => Promise.resolve(true),
|
||||
onUnloadDatabase: () => Promise.resolve(true),
|
||||
},
|
||||
replicator: { onCloseActiveReplication: () => Promise.resolve(true) },
|
||||
},
|
||||
} as unknown as LiveSyncLocalDBEnv);
|
||||
await expect(localDatabase.initializeDatabase()).resolves.toBe(true);
|
||||
const storageAccess = {
|
||||
getStub: () => Promise.resolve(storage),
|
||||
getFileStub: () => Promise.resolve(storage),
|
||||
readStubContent: () => Promise.resolve(storage),
|
||||
ensureDir: () => Promise.resolve(true),
|
||||
writeFileAuto: vi.fn((_path: string, body: string, times: { mtime: number }) => {
|
||||
storage = makeFile(body, times.mtime);
|
||||
return Promise.resolve(true);
|
||||
}),
|
||||
stat: () => Promise.resolve(storage.stat),
|
||||
touched: () => Promise.resolve(),
|
||||
triggerFileEvent: vi.fn(),
|
||||
};
|
||||
const conflict = { queueCheckFor: vi.fn(), queueCheckForIfOpen: vi.fn() };
|
||||
const services = {
|
||||
API,
|
||||
path: pathService,
|
||||
setting,
|
||||
events,
|
||||
database: { localDatabase },
|
||||
vault: { isTargetFile: () => Promise.resolve(true), isFileSizeTooLarge: () => false },
|
||||
storageAccess,
|
||||
conflict,
|
||||
fileReflectionProvenance: {
|
||||
get: (value: FilePathWithPrefix) => Promise.resolve(reflection.get(value)),
|
||||
set: (value: FilePathWithPrefix, record: { revision: string }) => {
|
||||
reflection.set(value, record);
|
||||
return Promise.resolve();
|
||||
},
|
||||
delete: (value: FilePathWithPrefix) => {
|
||||
reflection.delete(value);
|
||||
return Promise.resolve();
|
||||
},
|
||||
},
|
||||
fileProcessing: { processFileEvent: { addHandler: vi.fn() } },
|
||||
replication: { processSynchroniseResult: { addHandler: vi.fn() } },
|
||||
} as unknown as ServiceFileHandlerDependencies & ServiceDatabaseFileAccessDependencies;
|
||||
const access = new ServiceDatabaseFileAccessBase(services);
|
||||
(services as ServiceFileHandlerDependencies).databaseFileAccess = access;
|
||||
const handler = new TestHandler(services);
|
||||
return {
|
||||
db,
|
||||
localDatabase,
|
||||
access,
|
||||
handler,
|
||||
conflict,
|
||||
reflection,
|
||||
storageAccess,
|
||||
settings,
|
||||
services,
|
||||
getStorage: () => storage,
|
||||
setStorage: (file: UXFileInfo) => {
|
||||
storage = file;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
type Device = Awaited<ReturnType<typeof makeDevice>>;
|
||||
|
||||
function requiredEnvironment(name: "hostname" | "username" | "password"): string {
|
||||
const value = process.env[name];
|
||||
if (!value) throw new Error(`Missing integration-test environment variable: ${name}`);
|
||||
return value;
|
||||
}
|
||||
|
||||
/** Read every non-deleted leaf, including branches which are not the winner. */
|
||||
async function leaves(db: PouchDB.Database<EntryDoc>) {
|
||||
const docs = await db.get(path, { open_revs: "all", revs: true });
|
||||
return docs
|
||||
.flatMap((result) => ("ok" in result && !result.ok._deleted ? [result.ok] : []))
|
||||
.sort((left, right) => left._rev.localeCompare(right._rev));
|
||||
}
|
||||
|
||||
async function revisionContent(device: Device, rev: string) {
|
||||
const entry = await device.access.fetchEntry(path, rev, true);
|
||||
if (!entry) throw new Error(`Missing content for ${rev}`);
|
||||
return readContent(entry);
|
||||
}
|
||||
|
||||
/** Exercise the real CLI dispatcher and conflict operations with the fixture's real DB services. */
|
||||
async function resolveFromCLI(device: Device, keep: string) {
|
||||
const operations = createConflictResolutionOperations({
|
||||
events: device.services.events,
|
||||
databaseFileAccess: device.access,
|
||||
fileHandler: device.handler,
|
||||
log: vi.fn(),
|
||||
} as unknown as ConflictResolutionOperationsDependencies);
|
||||
const context = {
|
||||
databasePath: "/fixture",
|
||||
vaultPath: "/fixture",
|
||||
core: {
|
||||
services: {
|
||||
context: { standardIo: { writeStdout: vi.fn(), writeStderr: vi.fn() } },
|
||||
control: { activated: Promise.resolve() },
|
||||
conflict: { resolveByDeletingRevision: operations.resolveByDeletingRevision },
|
||||
},
|
||||
serviceModules: { databaseFileAccess: device.access, fileHandler: device.handler },
|
||||
},
|
||||
} as unknown as CLICommandContext;
|
||||
await expect(runCommand({ command: "resolve", commandArgs: [path, keep] }, context)).resolves.toBe(true);
|
||||
}
|
||||
|
||||
describe("file provenance across multiple devices and real CouchDB", () => {
|
||||
const databases: PouchDB.Database<EntryDoc>[] = [];
|
||||
const owners: LiveSyncLocalDB[] = [];
|
||||
afterEach(async () => {
|
||||
for (const owner of owners.splice(0)) {
|
||||
owner.offRemoteChunkFetchedHandler?.();
|
||||
await owner.managers.teardownManagers();
|
||||
}
|
||||
const results = await Promise.allSettled(databases.splice(0).map((db) => db.destroy()));
|
||||
for (const result of results) if (result.status === "rejected") throw result.reason;
|
||||
});
|
||||
|
||||
/** Replication deliberately precedes file reflection, modelling a delayed storage event. */
|
||||
async function conflictedDevices(count: number) {
|
||||
const name = `livesync-provenance-${crypto.randomUUID()}`;
|
||||
const remote = new PouchDB<EntryDoc>(`${requiredEnvironment("hostname").replace(/\/+$/u, "")}/${name}`, {
|
||||
adapter: "http",
|
||||
auth: { username: requiredEnvironment("username"), password: requiredEnvironment("password") },
|
||||
});
|
||||
databases.push(remote);
|
||||
await remote.info();
|
||||
const devices: Device[] = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
const device = await makeDevice(`${name}-${i}`);
|
||||
devices.push(device);
|
||||
owners.push(device.localDatabase);
|
||||
databases.push(device.db);
|
||||
}
|
||||
const root = await devices[0].access.storeWithBaseRevision(makeFile(old), undefined, true);
|
||||
if (!root) throw new Error("Could not create the shared original revision");
|
||||
await devices[0].db.replicate.to(remote);
|
||||
const revisions: string[] = [];
|
||||
for (const [index, device] of devices.entries()) {
|
||||
await device.db.replicate.from(remote);
|
||||
device.reflection.set(path, { revision: root });
|
||||
// All devices edit while disconnected. Equal mtimes rule out timestamp-based detection.
|
||||
device.setStorage(makeFile(`Edited on device ${index}\n`));
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
revisions.push((await device.db.get(path))._rev);
|
||||
}
|
||||
// Reverse upload order so the fixture does not rely on the first writer winning.
|
||||
for (const device of [...devices].reverse()) await device.db.replicate.to(remote);
|
||||
for (const device of devices) {
|
||||
await device.db.replicate.from(remote);
|
||||
expect((await leaves(device.db)).map((doc) => doc._rev)).toEqual([...revisions].sort());
|
||||
expect(await Promise.all(revisions.map((rev) => revisionContent(device, rev)))).toEqual(
|
||||
devices.map((_, index) => `Edited on device ${index}\n`)
|
||||
);
|
||||
}
|
||||
expect(await leaves(remote)).toHaveLength(count);
|
||||
return { devices, remote, root, revisions };
|
||||
}
|
||||
|
||||
async function resolveAndReplicate(f: Awaited<ReturnType<typeof conflictedDevices>>) {
|
||||
const winner = (await f.devices[0].db.get(path))._rev;
|
||||
const keepIndex = f.revisions.findIndex((rev) => rev !== winner);
|
||||
const keep = f.revisions[keepIndex];
|
||||
const content = await revisionContent(f.devices[0], keep);
|
||||
await resolveFromCLI(f.devices[0], keep);
|
||||
expect((await leaves(f.devices[0].db)).map((doc) => doc._rev)).toEqual([keep]);
|
||||
expect(await f.devices[0].getStorage().body.text()).toBe(content);
|
||||
expect(f.devices[0].reflection.get(path)?.revision).toBe(keep);
|
||||
await f.devices[0].db.replicate.to(f.remote);
|
||||
for (const device of f.devices) await device.db.replicate.from(f.remote);
|
||||
return { keep, content };
|
||||
}
|
||||
|
||||
it.each([3, 4])(
|
||||
"does not resurrect unchanged files before or after CLI resolution with %i editing devices",
|
||||
async (count) => {
|
||||
const f = await conflictedDevices(count);
|
||||
for (const [index, device] of f.devices.entries()) {
|
||||
const before = (await device.db.info()).update_seq;
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
expect((await device.db.info()).update_seq).toBe(before);
|
||||
expect(await device.getStorage().body.text()).toBe(`Edited on device ${index}\n`);
|
||||
expect(device.conflict.queueCheckFor).toHaveBeenCalled();
|
||||
}
|
||||
const { keep, content } = await resolveAndReplicate(f);
|
||||
for (const device of f.devices) {
|
||||
const before = (await device.db.info()).update_seq;
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
expect((await device.db.info()).update_seq).toBe(before);
|
||||
expect(await device.getStorage().body.text()).toBe(content);
|
||||
expect(device.reflection.get(path)?.revision).toBe(keep);
|
||||
await device.db.replicate.to(f.remote);
|
||||
}
|
||||
for (const device of f.devices) {
|
||||
await device.db.replicate.from(f.remote);
|
||||
expect((await leaves(device.db)).map((doc) => doc._rev)).toEqual([keep]);
|
||||
}
|
||||
expect((await leaves(f.remote)).map((doc) => doc._rev)).toEqual([keep]);
|
||||
},
|
||||
60_000
|
||||
);
|
||||
|
||||
it("preserves a real edit made on a losing device after three-way resolution", async () => {
|
||||
const f = await conflictedDevices(3);
|
||||
const { keep, content } = await resolveAndReplicate(f);
|
||||
const index = f.revisions.findIndex((rev, i) => i > 0 && rev !== keep);
|
||||
const device = f.devices[index];
|
||||
const edit = `${await device.getStorage().body.text()}A further offline edit\n`;
|
||||
device.setStorage(makeFile(edit));
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
const editedRevision = device.reflection.get(path)!.revision;
|
||||
const edited = await device.db.get(path, { rev: editedRevision, revs: true });
|
||||
expect(edited._revisions?.ids[1]).toBe(f.revisions[index].split("-")[1]);
|
||||
await device.db.replicate.to(f.remote);
|
||||
for (const peer of f.devices) {
|
||||
await peer.db.replicate.from(f.remote);
|
||||
expect((await leaves(peer.db)).map((doc) => doc._rev)).toEqual([keep, editedRevision].sort());
|
||||
expect(await revisionContent(peer, keep)).toBe(content);
|
||||
expect(await revisionContent(peer, editedRevision)).toBe(edit);
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
it.each(["missing record", "compacted base"] as const)(
|
||||
"preserves uncertain storage as one independent conflict with four devices: %s",
|
||||
async (reason) => {
|
||||
const f = await conflictedDevices(4);
|
||||
const { keep, content } = await resolveAndReplicate(f);
|
||||
const index = f.revisions.findIndex((rev, i) => i > 0 && rev !== keep);
|
||||
const device = f.devices[index];
|
||||
if (reason === "missing record") {
|
||||
device.reflection.delete(path);
|
||||
// Matching an old ancestor must not be mistaken for an unchanged current branch.
|
||||
device.setStorage(makeFile(old));
|
||||
} else {
|
||||
await device.db.compact();
|
||||
await expect(device.db.get(path, { rev: f.revisions[index] })).rejects.toMatchObject({ status: 404 });
|
||||
}
|
||||
const uncertainContent = await device.getStorage().body.text();
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
const independent = device.reflection.get(path)!.revision;
|
||||
expect(independent).toMatch(/^1-/u);
|
||||
expect(independent).not.toBe(f.root);
|
||||
expect((await device.db.get(path, { rev: independent, revs: true }))._revisions?.ids).toHaveLength(1);
|
||||
const before = (await device.db.info()).update_seq;
|
||||
device.reflection.delete(path);
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
expect((await device.db.info()).update_seq).toBe(before);
|
||||
await device.db.replicate.to(f.remote);
|
||||
for (const peer of f.devices) {
|
||||
await peer.db.replicate.from(f.remote);
|
||||
expect((await leaves(peer.db)).map((doc) => doc._rev)).toEqual([keep, independent].sort());
|
||||
expect(await revisionContent(peer, keep)).toBe(content);
|
||||
expect(await revisionContent(peer, independent)).toBe(uncertainContent);
|
||||
}
|
||||
// The CLI must also accept the independent root as the selected conflict.
|
||||
await resolveFromCLI(f.devices[0], independent);
|
||||
await f.devices[0].db.replicate.to(f.remote);
|
||||
for (const peer of f.devices) {
|
||||
await peer.db.replicate.from(f.remote);
|
||||
expect((await leaves(peer.db)).map((doc) => doc._rev)).toEqual([independent]);
|
||||
}
|
||||
expect(await f.devices[0].getStorage().body.text()).toBe(uncertainContent);
|
||||
},
|
||||
60_000
|
||||
);
|
||||
});
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
# Test procedures
|
||||
|
||||
Run the commands below from the repository root. Test ownership and source layout are described in the [development guide](../devs.md#testing-infrastructure).
|
||||
|
||||
## npm 10 clean-installation check
|
||||
|
||||
Run this check after changing `package.json`, a workspace manifest, or `package-lock.json`, including a Commonlib dependency update. A successful installation with the npm version bundled with Node.js does not prove that npm 10 accepts the lockfile.
|
||||
|
||||
The following sequence matches the installation steps in [unit-ci](../.github/workflows/unit-ci.yml). Use Node.js 24, as configured in that workflow:
|
||||
|
||||
```bash
|
||||
npx --yes npm@10.9.4 ci --ignore-scripts --no-audit --no-fund
|
||||
npm ci
|
||||
```
|
||||
|
||||
Both commands must complete successfully without changing the lockfile. The first checks npm 10 lockfile compatibility; the second prepares the normal development installation, including lifecycle scripts, for the source checks and tests below. Keep the pinned npm version and command here aligned with CI. This project-side check does not establish the runtime version used by the external Community Review service or replace its authenticated review result.
|
||||
|
||||
If Community Review reports widespread TypeScript `error` types across unrelated external packages, first confirm that dependency installation completed successfully. An installation failure can leave external types unresolved and produce misleading source warnings.
|
||||
|
||||
## Source and unit checks
|
||||
|
||||
Run broad checks sequentially. These examples bound the Node.js heap and Vitest workers for machines with limited memory:
|
||||
|
||||
```bash
|
||||
NODE_OPTIONS=--max-old-space-size=3072 npm run check
|
||||
NODE_OPTIONS=--max-old-space-size=3072 npm run test:unit -- --maxWorkers=1
|
||||
```
|
||||
|
||||
`npm run check` includes TypeScript, ESLint, the Community rules, Svelte checks, a production build, and bundle compatibility checks. Inspect installation and source-check failures before interpreting later test results. Add the relevant service or runtime suite for the boundary changed:
|
||||
|
||||
| Boundary | Procedure |
|
||||
| --- | --- |
|
||||
| Multiple-device file conflicts and stale-file protection | [CouchDB procedure below](#multiple-device-conflict-regression-tests) |
|
||||
| Obsidian startup, file watching, persistence, and UI | [Real Obsidian E2E](e2e-obsidian/README.md) |
|
||||
| CLI subprocesses, filesystem workflows, and P2P | [CLI Deno tests](../src/apps/cli/testdeno/test_dev_deno.md) and [test authoring](../src/apps/cli/testdeno/CONTRIBUTING_TESTS.md) |
|
||||
| WebApp, WebPeer, and browser interoperability | [Browser application tests](browser-apps/README.md) |
|
||||
|
||||
## Community Review checks and CI confirmation
|
||||
|
||||
Before requesting review or merging source or dependency changes, run the project-side Community checks after dependency installation:
|
||||
|
||||
```bash
|
||||
NODE_OPTIONS=--max-old-space-size=3072 npm run lint:community
|
||||
NODE_OPTIONS=--max-old-space-size=3072 npm run lint:community:tools
|
||||
```
|
||||
|
||||
The source check uses the official `eslint-plugin-obsidianmd` rules with the repository's [Community configuration](../eslint.community.config.mjs). Run it without `--quiet` so warnings remain visible. Review new warnings as well as errors, and distinguish existing warnings from those introduced by the change. A successful exit alone does not establish that the source has no warnings. The tooling check requires zero warnings.
|
||||
|
||||
The [unit-ci workflow](../.github/workflows/unit-ci.yml), in its `Unit Tests` job, runs the npm 10 installation check and then `npm run check`. That script includes `lint:community` and `lint:community:tools`, so source warnings are visible in the CI log as well as during local checks. Source errors fail the gate; source warnings remain visible for review without failing it. The explicit commands above can run these checks independently of the full source-check sequence.
|
||||
|
||||
After pushing, confirm that the `Unit Tests` job passed for the exact commit being reviewed, including its `Verify clean installation with npm 10` and `Run source checks` steps. For service-backed changes, also confirm the integration-test job and the relevant runtime checks. A successful run for an earlier commit does not validate later changes.
|
||||
|
||||
The local checks and CI use the project's installed rule versions and configured file scope. Record the authenticated external Community Review result separately when that review is required; the project-side checks do not replace it. When the external review reports additional findings, retain its relevant output and investigate differences in installation, type resolution, scope, or rules.
|
||||
|
||||
## CLI mirror regression tests
|
||||
|
||||
Run the native CLI subprocess suite after building the CLI:
|
||||
|
||||
```bash
|
||||
NODE_OPTIONS=--max-old-space-size=3072 npm run build --workspace self-hosted-livesync-cli
|
||||
cd src/apps/cli/testdeno
|
||||
deno test -A --no-check test-mirror.ts
|
||||
```
|
||||
|
||||
The expected result is seven passing steps. These cover storage-only and database-only files, database deletion, an ordinary local edit, incoming database content, an omitted Vault path, and local content with unknown provenance.
|
||||
|
||||
For an ordinary local edit, first reflect the database content into the Vault, confirm its bytes, and then edit that file. The next `mirror` must store the edit without a conflict. For unknown provenance, use `put` to seed only the database and independently create different local content with a newer modification time. The next `mirror` must preserve two independent non-deleted revisions. Read both with `cat-rev`, submit the same local bytes again to check that no additional revision appears, and use `resolve` to select the local content. Confirm that the conflict is gone and the selected content is reflected into the Vault.
|
||||
|
||||
`put` deliberately bypasses file provenance, whereas `push` records it. Substituting one for the other changes the scenario. The tests enable `writeDocumentsIfConflicted` for incoming reflection; this setting does not authorise an ordinary save to replace unrelated database content or resolve the conflict. Do not assume which independent root PouchDB selects as the winner.
|
||||
|
||||
The [CLI Docker workflow](../.github/workflows/cli-docker.yml) runs the corresponding Bash suite. From the repository root, build and check that path with:
|
||||
|
||||
```bash
|
||||
NODE_OPTIONS=--max-old-space-size=3072 npm run build:docker --workspace self-hosted-livesync-cli
|
||||
npm run test:e2e:docker:mirror --workspace self-hosted-livesync-cli
|
||||
```
|
||||
|
||||
The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Deno and Bash suites.
|
||||
|
||||
## Multiple-device conflict regression tests
|
||||
|
||||
### Preparation and execution
|
||||
|
||||
The suite is [FileHandler.multidevice.integration.spec.ts](../src/serviceModules/FileHandler.multidevice.integration.spec.ts). It exercises the installed Commonlib package through LiveSync's shared file handler, the CLI `resolve` command dispatcher, and the shared conflict-resolution operations.
|
||||
|
||||
Use a disposable CouchDB service. To use the repository's Docker fixture, set the following values in `.test.env` and ensure `.env` exists. The fixture uses the container name `couchdb-test` and host port `5989`:
|
||||
|
||||
```dotenv
|
||||
hostname=http://127.0.0.1:5989/
|
||||
username=admin
|
||||
password=testpassword
|
||||
```
|
||||
|
||||
Start the fixture, then run the focused suite:
|
||||
|
||||
```bash
|
||||
npm run test:docker-couchdb:start
|
||||
NODE_OPTIONS=--max-old-space-size=3072 npm run test:integration -- src/serviceModules/FileHandler.multidevice.integration.spec.ts --maxWorkers=1
|
||||
```
|
||||
|
||||
The expected result is five passing tests. Each case creates a uniquely named remote database and removes it during teardown. Stop the fixture after the run, including when the test command fails:
|
||||
|
||||
```bash
|
||||
npm run test:docker-couchdb:stop
|
||||
```
|
||||
|
||||
When using an already running disposable CouchDB service, configure its endpoint and credentials in `.test.env` and run only the test command. The start and stop commands manage the repository's Docker fixture. This suite needs no Object Storage, P2P relay, or Obsidian application. It is also discovered by the existing integration-test CI job.
|
||||
|
||||
### Scenarios and expected results
|
||||
|
||||
Each simulated device owns a separate real PouchDB database, `LiveSyncLocalDB` managers, file content, and provenance record. Devices first share one revision, edit while disconnected, and then replicate their Metadata and Chunks through real CouchDB. File reflection is deliberately delayed after replication to reproduce the interval in which the database has advanced but the file still contains older content. All edits use equal modification times, so the tests require revision and content checks rather than timestamp ordering.
|
||||
|
||||
| Case | Setup and action | Required result |
|
||||
| --- | --- | --- |
|
||||
| Three editing devices | Replicate three conflicting edits, reprocess unchanged files, select a non-winning revision through CLI `resolve`, and replicate the resolution before reprocessing the other files. | All three contents are initially readable on every replica. Unchanged saves make no database writes. After resolution, all files and replicas converge to the selected revision without creating revisions or restoring conflicts. |
|
||||
| Four editing devices | Repeat the same sequence with four independently edited branches. | All four contents are initially preserved, and the same unchanged-save and convergence guarantees hold. |
|
||||
| A genuine edit on a losing device | After three-way resolution reaches its DB, a device adds content to its still-unreflected losing file, then saves and replicates it. | The new revision extends that device's recorded branch. The selected result and the new edit remain readable on every replica. |
|
||||
| Missing provenance | After four-way resolution, remove a losing device's provenance record and set its file to the original historical ancestor's content. Save, remove provenance again, and repeat the save. | The file becomes one fresh independent root, distinct from the historical root. Both contents remain readable after replication, and repeated saves create no duplicates. CLI `resolve` can select the independent root and propagate its resolution. |
|
||||
| Compacted base | After four-way resolution, compact a losing device's local DB and confirm that its recorded revision body is unavailable. Save the remaining file, then repeat after removing provenance. | The file is preserved as one independent conflict rather than discarded. Both contents remain readable on every replica, repeated saves create no duplicates, and CLI `resolve` can select the independent root and propagate its resolution. |
|
||||
|
||||
### Coverage boundaries
|
||||
|
||||
This is a service integration test, not a CLI subprocess or Obsidian runtime test. The file and provenance stores are in-memory fixtures; automatic conflict callbacks are observed without running interactive or automatic merge policies. PouchDB revision creation, chunk storage and retrieval, local compaction, CouchDB replication, the CLI command dispatcher, and its resolution operations are real.
|
||||
|
||||
Use the CLI and real-Obsidian procedures linked above for argument parsing, persistent host stores, file watchers, and dialogues. In particular, the real-Obsidian `stale-file-restart` scenario exercises persisted pending events and restart, and `folder-batch` exercises bulk Vault rename and deletion. These scenarios do not simulate a mobile operating system suspending the application.
|
||||
@@ -27,7 +27,7 @@ const runtimeMocks = vi.hoisted(() => {
|
||||
};
|
||||
|
||||
const serviceHub = {
|
||||
API: { addLog },
|
||||
API: { addLog, webCompatFetch: vi.fn() },
|
||||
appLifecycle: { isReady, markIsReady },
|
||||
control: { onLoad, onReady, onUnload },
|
||||
databaseEvents: { onDatabaseInitialised },
|
||||
|
||||
@@ -52,6 +52,37 @@ Deno.test({
|
||||
assertEquals(await page.getByPlaceholder("Enter TURN username").inputValue(), "browser-turn-user");
|
||||
assertEquals(await page.getByPlaceholder("Enter TURN credential").inputValue(), "browser-turn-credential");
|
||||
assertEquals(await page.getByRole("button", { name: "Connect", exact: true }).isVisible(), true);
|
||||
|
||||
await page.getByLabel("TURN configuration", { exact: true }).selectOption("CF");
|
||||
assertEquals(await saveTurn.isDisabled(), true);
|
||||
await page.getByLabel("TURN Key ID", { exact: true }).fill("browser-turn-key");
|
||||
const tokenField = page.getByLabel("TURN Key API Token", { exact: true });
|
||||
assertEquals(await tokenField.getAttribute("type"), "password");
|
||||
await tokenField.fill("browser-api-token");
|
||||
await saveTurn.click();
|
||||
await waitFor(async () => await saveTurn.isDisabled(), "WebPeer did not save its managed TURN profile");
|
||||
assertEquals(await page.getByPlaceholder("anything-you-like").inputValue(), "browser-e2e-room");
|
||||
await page.reload();
|
||||
await page.getByRole("heading", { name: "Peer to Peer Replicator", exact: true }).waitFor();
|
||||
assertEquals(await page.getByPlaceholder("anything-you-like").inputValue(), "browser-e2e-room");
|
||||
await page.getByText("Optional TURN server settings", { exact: true }).click();
|
||||
assertEquals(await page.getByLabel("TURN configuration", { exact: true }).inputValue(), "CF");
|
||||
assertEquals(await page.getByLabel("TURN Key ID", { exact: true }).inputValue(), "browser-turn-key");
|
||||
assertEquals(
|
||||
await page.getByLabel("TURN Key API Token", { exact: true }).inputValue(),
|
||||
"browser-api-token"
|
||||
);
|
||||
await page.getByPlaceholder("iphone-16").fill("browser-e2e-peer-renamed");
|
||||
await save.click();
|
||||
await waitFor(async () => await save.isDisabled(), "WebPeer did not save its updated device name");
|
||||
assertEquals(await page.getByLabel("TURN configuration", { exact: true }).inputValue(), "CF");
|
||||
assertEquals(
|
||||
await page.getByLabel("TURN Key API Token", { exact: true }).inputValue(),
|
||||
"browser-api-token"
|
||||
);
|
||||
await page.getByLabel("TURN configuration", { exact: true }).selectOption("");
|
||||
assertEquals(await page.getByPlaceholder("Enter TURN username").inputValue(), "browser-turn-user");
|
||||
assertEquals(await page.getByPlaceholder("Enter TURN credential").inputValue(), "browser-turn-credential");
|
||||
assertNoPageFailures();
|
||||
} finally {
|
||||
await browser.close();
|
||||
|
||||
@@ -75,11 +75,17 @@ After changing plug-in source, use the focused wrapper rather than invoking a sc
|
||||
```bash
|
||||
npm run test:e2e:obsidian:focused -- settings-ui
|
||||
npm run test:e2e:obsidian:focused -- two-vault-sync
|
||||
npm run test:e2e:obsidian:focused -- stale-file-restart
|
||||
npm run test:e2e:obsidian:focused -- folder-batch
|
||||
npm run test:e2e:obsidian:focused -- security-seed-reconnect
|
||||
```
|
||||
|
||||
The wrapper accepts only maintained real-Obsidian scenario names; run it with `--help` for the current list. It deliberately does not manage CouchDB, Object Storage, or the P2P signalling relay. Start the required fixture first, or use the complete service-managed suite.
|
||||
|
||||
`folder-batch` needs no remote service. It creates 24 notes in nested folders, renames and deletes the parent through the Obsidian Vault API, and checks descendant events, content, Chunks, deletion markers, and provenance. A note outside the parent must remain writable.
|
||||
|
||||
`stale-file-restart` needs no remote service. It advances the local database while old Vault bytes remain, persists pending storage events, and restarts the same isolated Vault and profile. It checks that an unchanged file with exact provenance receives the newer database content without creating a revision, that unknown-origin content is preserved on a fresh independent branch, and that losing provenance and processing the file again does not duplicate or automatically merge that branch. The database advance and pending snapshot are controlled fixtures; startup processing, persistence, file reflection, and conflict checking run in real Obsidian. The scenario does not simulate a mobile operating system suspending the application.
|
||||
|
||||
The principal entry points are:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import { createE2eObsidianDeviceLocalState, waitForLiveSyncCoreReady } from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
|
||||
const originalRoot = "batch/original";
|
||||
const renamedRoot = "batch/renamed";
|
||||
const outsidePath = "batch/outside.md";
|
||||
const folders = ["alpha", "alpha/deep", "beta"];
|
||||
const notes = Array.from({ length: 24 }, (_, index) => ({
|
||||
relativePath: `${folders[index % folders.length]}/note-${index}.md`,
|
||||
body: `# Descendant ${index}\n\nThis body must survive a parent folder rename.\n`,
|
||||
}));
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked: ${cli.checked.join(", ")}`);
|
||||
const cliBinary = cli.binary;
|
||||
const vault = await createTemporaryVault("obsidian-livesync-folder-batch-");
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
try {
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary,
|
||||
vault,
|
||||
pluginData: {
|
||||
doctorProcessedVersion: "1.0.0",
|
||||
isConfigured: true,
|
||||
liveSync: false,
|
||||
remoteType: "",
|
||||
couchDB_URI: "http://127.0.0.1:5984",
|
||||
couchDB_DBNAME: "folder-batch",
|
||||
notifyThresholdOfRemoteStorageSize: -1,
|
||||
periodicReplication: false,
|
||||
syncOnStart: false,
|
||||
syncOnSave: false,
|
||||
syncOnFileOpen: false,
|
||||
syncOnEditorSave: false,
|
||||
syncAfterMerge: false,
|
||||
useEden: false,
|
||||
},
|
||||
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||
const result = await evalObsidianJson<{ descendants: number; renamed: number; deleted: number }>(
|
||||
cliBinary,
|
||||
`(async()=>{
|
||||
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||
const provenance=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
|
||||
const notes=${JSON.stringify(notes)};
|
||||
const originalRoot=${JSON.stringify(originalRoot)};
|
||||
const renamedRoot=${JSON.stringify(renamedRoot)};
|
||||
const outsidePath=${JSON.stringify(outsidePath)};
|
||||
const renamed=new Set(), deleted=new Set();
|
||||
const refs=[
|
||||
app.vault.on('rename',(file,oldPath)=>{
|
||||
if(file.stat) renamed.add(oldPath+' -> '+file.path);
|
||||
}),
|
||||
app.vault.on('delete',(file)=>{if(file.stat) deleted.add(file.path);}),
|
||||
];
|
||||
const meta=(path)=>core.localDatabase.getDBEntryMeta(path,{conflicts:true},true);
|
||||
const isDeleted=(entry)=>entry && (entry.deleted || entry._deleted);
|
||||
const getContent=(entry)=>Array.isArray(entry.data)?entry.data.join(''):entry.data;
|
||||
|
||||
async function liveErrors(path,body){
|
||||
const errors=[];
|
||||
const file=app.vault.getAbstractFileByPath(path);
|
||||
const entry=await meta(path);
|
||||
if(!file?.stat || file.path!==path || await app.vault.read(file)!==body)
|
||||
errors.push('Vault content: '+path);
|
||||
if(!entry || isDeleted(entry) || entry.path!==path || !entry.children.length){
|
||||
errors.push('DB metadata: '+path);
|
||||
}else{
|
||||
const loaded=await core.localDatabase.getDBEntry(path,{rev:entry._rev},false,true,true);
|
||||
if(!loaded || getContent(loaded)!==body) errors.push('DB content: '+path);
|
||||
if(entry._conflicts?.length) errors.push('Unexpected conflict: '+path);
|
||||
if((await provenance.get(path))?.revision!==entry._rev)
|
||||
errors.push('Provenance: '+path);
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
async function deletedErrors(path){
|
||||
const errors=[];
|
||||
const entry=await meta(path);
|
||||
if(app.vault.getAbstractFileByPath(path)) errors.push('File remains: '+path);
|
||||
if(!isDeleted(entry)) errors.push('Missing tombstone: '+path);
|
||||
if(entry?._conflicts?.length) errors.push('Deletion conflict: '+path);
|
||||
if(await provenance.get(path)) errors.push('Old provenance remains: '+path);
|
||||
return errors;
|
||||
}
|
||||
async function waitFor(phase,check){
|
||||
const deadline=Date.now()+20000;
|
||||
let errors=[];
|
||||
do{
|
||||
await core.services.fileProcessing.commitPendingFileEvents();
|
||||
errors=await check();
|
||||
if(!errors.length) return;
|
||||
await new Promise(resolve=>setTimeout(resolve,50));
|
||||
}while(Date.now()<deadline);
|
||||
throw new Error(phase+': '+errors.slice(0,8).join('; '));
|
||||
}
|
||||
const liveBatch=(root)=>Promise.all(notes.map(note=>
|
||||
liveErrors(root+'/'+note.relativePath,note.body))).then(results=>results.flat());
|
||||
const deletedBatch=(root)=>Promise.all(notes.map(note=>
|
||||
deletedErrors(root+'/'+note.relativePath))).then(results=>results.flat());
|
||||
|
||||
try{
|
||||
await app.vault.createFolder('batch');
|
||||
await app.vault.createFolder(originalRoot);
|
||||
for(const folder of ${JSON.stringify(folders)})
|
||||
await app.vault.createFolder(originalRoot+'/'+folder);
|
||||
await Promise.all(notes.map(note=>app.vault.create(originalRoot+'/'+note.relativePath,note.body)));
|
||||
await app.vault.create(outsidePath,'Outside note');
|
||||
await waitFor('Initial batch',async()=>[
|
||||
...await liveBatch(originalRoot), ...await liveErrors(outsidePath,'Outside note'),
|
||||
]);
|
||||
const originalIds=await Promise.all(notes.map(async note=>(await meta(originalRoot+'/'+note.relativePath))._id));
|
||||
|
||||
// Rename the parent once: Obsidian must emit every descendant event.
|
||||
await app.vault.rename(app.vault.getAbstractFileByPath(originalRoot),renamedRoot);
|
||||
await waitFor('Renamed batch',async()=>[
|
||||
...await liveBatch(renamedRoot), ...await deletedBatch(originalRoot),
|
||||
...await liveErrors(outsidePath,'Outside note'),
|
||||
]);
|
||||
for(const [index,note] of notes.entries()){
|
||||
const from=originalRoot+'/'+note.relativePath, to=renamedRoot+'/'+note.relativePath;
|
||||
if(!renamed.has(from+' -> '+to)) throw new Error('Missing descendant rename: '+from);
|
||||
if((await meta(to))._id===originalIds[index]) throw new Error('Rename reused the source ID: '+to);
|
||||
}
|
||||
|
||||
// Delete the parent once, without synthesising individual file events.
|
||||
await app.vault.delete(app.vault.getAbstractFileByPath(renamedRoot),true);
|
||||
await waitFor('Deleted batch',async()=>[
|
||||
...await deletedBatch(renamedRoot), ...await deletedBatch(originalRoot),
|
||||
...await liveErrors(outsidePath,'Outside note'),
|
||||
]);
|
||||
for(const note of notes){
|
||||
const path=renamedRoot+'/'+note.relativePath;
|
||||
if(!deleted.has(path)) throw new Error('Missing descendant deletion: '+path);
|
||||
}
|
||||
if(app.vault.getAbstractFileByPath(renamedRoot)) throw new Error('Deleted folder remains');
|
||||
await app.vault.modify(app.vault.getAbstractFileByPath(outsidePath),'Outside note updated');
|
||||
await waitFor('Outside update',()=>liveErrors(outsidePath,'Outside note updated'));
|
||||
return JSON.stringify({descendants:notes.length,renamed:renamed.size,deleted:deleted.size});
|
||||
}finally{
|
||||
for(const ref of refs) app.vault.offref(ref);
|
||||
}
|
||||
})()`,
|
||||
session.cliEnv
|
||||
);
|
||||
console.log(
|
||||
`Folder batch: ${result.descendants} descendants persisted, renamed, and deleted; ` +
|
||||
`${result.renamed} rename and ${result.deleted} delete events observed; outside note remained writable.`
|
||||
);
|
||||
} finally {
|
||||
if (session) await session.app.stop();
|
||||
await vault.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -25,6 +25,8 @@ const focusedScenarios = new Set([
|
||||
"p2p-setup-uri-workflow",
|
||||
"partial-startup-file-failure",
|
||||
"startup-scan",
|
||||
"stale-file-restart",
|
||||
"folder-batch",
|
||||
"setup-uri-workflow",
|
||||
"two-vault-sync",
|
||||
"security-seed-reconnect",
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
createE2eObsidianDeviceLocalState,
|
||||
waitForLiveSyncCoreReady,
|
||||
waitForLocalDatabaseEntry,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
const paths = ["stale-known.md", "stale-unknown.md"];
|
||||
const oldContent = "# Note\nKeep\n\nTail\n\nFooter\n";
|
||||
const newContent = oldContent.replace(
|
||||
"Footer\n",
|
||||
Array.from({ length: 50 }, (_, index) => `Remote addition ${index}\n`).join("") + "Footer\n"
|
||||
);
|
||||
|
||||
type Branch = { rev: string; content: string; history: string[] };
|
||||
type FileState = { path: string; content: string; rev: string; branches: Branch[]; provenance: string | null };
|
||||
|
||||
async function readState(cliBinary: string, env: NodeJS.ProcessEnv): Promise<FileState[]> {
|
||||
return await evalObsidianJson<FileState[]>(
|
||||
cliBinary,
|
||||
`(async()=>{
|
||||
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||
const store=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
|
||||
const states=[];
|
||||
for(const path of ${JSON.stringify(paths)}){
|
||||
const meta=await core.localDatabase.getDBEntryMeta(path,{conflicts:true},true);
|
||||
const branches=[];
|
||||
for(const rev of [meta._rev,...(meta._conflicts??[])]){
|
||||
const entry=await core.localDatabase.getDBEntry(path,{rev,revs:true},false,true,true);
|
||||
const raw=await core.localDatabase.getRaw(meta._id,{rev,revs:true});
|
||||
branches.push({rev,content:Array.isArray(entry.data)?entry.data.join(''):entry.data,
|
||||
history:raw._revisions.ids.map((id,i)=>(raw._revisions.start-i)+'-'+id)});
|
||||
}
|
||||
const file=app.vault.getAbstractFileByPath(path);
|
||||
states.push({path,content:await app.vault.read(file),rev:meta._rev,branches,
|
||||
provenance:(await store.get(path))?.revision??null});
|
||||
}
|
||||
return JSON.stringify(states);
|
||||
})()`,
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked: ${cli.checked.join(", ")}`);
|
||||
const cliBinary = cli.binary;
|
||||
const vault = await createTemporaryVault("obsidian-livesync-stale-file-");
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
try {
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary,
|
||||
vault,
|
||||
pluginData: {
|
||||
doctorProcessedVersion: "1.0.0",
|
||||
isConfigured: true,
|
||||
liveSync: false,
|
||||
remoteType: "",
|
||||
couchDB_URI: "http://127.0.0.1:5984",
|
||||
couchDB_DBNAME: "stale-file-restart",
|
||||
notifyThresholdOfRemoteStorageSize: -1,
|
||||
periodicReplication: false,
|
||||
syncAfterMerge: false,
|
||||
syncOnEditorSave: false,
|
||||
syncOnFileOpen: false,
|
||||
syncOnSave: false,
|
||||
syncOnStart: false,
|
||||
disableMarkdownAutoMerge: false,
|
||||
resolveConflictsByNewerFile: false,
|
||||
checkConflictOnlyOnOpen: true,
|
||||
showMergeDialogOnlyOnActive: true,
|
||||
},
|
||||
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||
await evalObsidianJson(
|
||||
cliBinary,
|
||||
`(async()=>{
|
||||
for(const path of ${JSON.stringify(paths)}) await app.vault.create(path,${JSON.stringify(oldContent)});
|
||||
return JSON.stringify(true);
|
||||
})()`,
|
||||
session.cliEnv
|
||||
);
|
||||
for (const path of paths) await waitForLocalDatabaseEntry(cliBinary, session.cliEnv, path);
|
||||
|
||||
// Drain real Vault events before creating a persisted pending-event fixture.
|
||||
// The DB advances without reflecting it in the Vault, as on an offline device.
|
||||
const fixture = await evalObsidianJson<{ current: string[]; original: string[] }>(
|
||||
cliBinary,
|
||||
`(async()=>{
|
||||
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||
const store=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
|
||||
await core.services.fileProcessing.commitPendingFileEvents();
|
||||
const snapshot=[], current=[], original=[];
|
||||
for(const [index,path] of ${JSON.stringify(paths)}.entries()){
|
||||
const meta=await core.localDatabase.getDBEntryMeta(path,{},true);
|
||||
const file=await core.storageAccess.getFileStub(path);
|
||||
const data=new Blob([${JSON.stringify(newContent)}],{type:'text/plain'});
|
||||
const result=await core.localDatabase.putDBEntry({...meta,data,mtime:file.stat.mtime+60000,
|
||||
size:data.size,children:[]},false,meta._rev);
|
||||
if(!result?.ok) throw new Error('Could not advance '+path);
|
||||
current.push(result.rev); original.push(meta._rev);
|
||||
if(index===0) await store.set(path,{revision:meta._rev,observedStorageMtime:file.stat.mtime});
|
||||
else await store.delete(path);
|
||||
snapshot.push({type:'CHANGED',key:'CHANGED-'+path,args:{file}});
|
||||
}
|
||||
await core.kvDB.set('storage-event-manager-snapshot',snapshot);
|
||||
return JSON.stringify({current,original});
|
||||
})()`,
|
||||
session.cliEnv
|
||||
);
|
||||
await session.app.stop();
|
||||
session = undefined;
|
||||
|
||||
session = await startObsidianLiveSyncSession({ binary, cliBinary, vault });
|
||||
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
|
||||
const [known, unknown] = await readState(cliBinary, session.cliEnv);
|
||||
assertEqual(known.rev, fixture.current[0], "An unchanged stale file created a revision during restart.");
|
||||
assertEqual(known.branches.length, 1, "An unchanged stale file created a conflict.");
|
||||
assertEqual(known.content, newContent, "The newer DB content was not reflected after suppressing the save.");
|
||||
assertEqual(known.provenance, fixture.current[0], "The reflected revision was not recorded.");
|
||||
assertEqual(unknown.branches.length, 2, "Unknown local content was not preserved as a conflict.");
|
||||
assertEqual(unknown.content, oldContent, "Unknown local content was overwritten.");
|
||||
const independent = unknown.branches.find((branch) => branch.content === oldContent);
|
||||
if (!independent) throw new Error("The old local content is missing from the current branches.");
|
||||
assertEqual(independent.history.length, 1, "Unknown content was attached to an inferred ancestor.");
|
||||
if (independent.rev === fixture.original[1]) throw new Error("The historical root was reused.");
|
||||
if (!unknown.branches.some((branch) => branch.content === newContent)) {
|
||||
throw new Error("The remote additions were lost.");
|
||||
}
|
||||
|
||||
await evalObsidianJson(
|
||||
cliBinary,
|
||||
`(async()=>{
|
||||
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||
const path=${JSON.stringify(paths[1])};
|
||||
await core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1').delete(path);
|
||||
if(!await core.fileHandler.storeFileToDB(path)) throw new Error('Repeated save failed');
|
||||
await app.workspace.getLeaf(false).openFile(app.vault.getAbstractFileByPath(${JSON.stringify(paths[0])}));
|
||||
await core.services.conflict.resolve(path);
|
||||
return JSON.stringify(true);
|
||||
})()`,
|
||||
session.cliEnv
|
||||
);
|
||||
const [, repeated] = await readState(cliBinary, session.cliEnv);
|
||||
assertEqual(
|
||||
repeated.branches
|
||||
.map((branch) => branch.rev)
|
||||
.sort()
|
||||
.join(","),
|
||||
unknown.branches
|
||||
.map((branch) => branch.rev)
|
||||
.sort()
|
||||
.join(","),
|
||||
"Losing provenance and reprocessing added or auto-merged a branch."
|
||||
);
|
||||
await evalObsidianJson(
|
||||
cliBinary,
|
||||
`(async()=>{
|
||||
const core=app.plugins.plugins['obsidian-livesync'].core;
|
||||
core.settings.resolveConflictsByNewerFile=true;
|
||||
await core.services.conflict.resolve(${JSON.stringify(paths[1])});
|
||||
return JSON.stringify(true);
|
||||
})()`,
|
||||
session.cliEnv
|
||||
);
|
||||
const [, resolved] = await readState(cliBinary, session.cliEnv);
|
||||
assertEqual(resolved.branches.length, 1, "The explicit newer-file option did not resolve the conflict.");
|
||||
assertEqual(resolved.content, newContent, "The newer-file option did not reflect the newer DB version.");
|
||||
console.log(
|
||||
"Stale-file restart: known content reflected; unknown content preserved without duplicate branches; explicit newer-file resolution retained."
|
||||
);
|
||||
} finally {
|
||||
if (session) await session.app.stop();
|
||||
await vault.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -997,14 +997,26 @@ async function runMarkdownAutoMerge(
|
||||
|
||||
session = await startConfiguredSession(context, vaultA, conflictOverrides);
|
||||
const baseOnA = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, conflictPath);
|
||||
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, left, baseOnA.rev);
|
||||
await writeVaultFile(vaultA.path, conflictPath, left);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, left);
|
||||
const storedLeft = await waitForConflictBranch(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
conflictPath,
|
||||
(branch) => branch.content === left
|
||||
);
|
||||
assertEqual(storedLeft.parentRev, baseOnA.rev, "Vault A's edit did not extend its displayed base.");
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await stopTrackedSession(context, session);
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, conflictOverrides);
|
||||
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, right, baseOnB.rev);
|
||||
await writeVaultFile(vaultB.path, conflictPath, right);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, right);
|
||||
const storedRight = await waitForConflictBranch(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
conflictPath,
|
||||
(branch) => branch.content === right
|
||||
);
|
||||
assertEqual(storedRight.parentRev, baseOnB.rev, "Vault B's edit did not extend its displayed base.");
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
const conflict = await waitForFileConflict(context.cliBinary, session.cliEnv, conflictPath);
|
||||
const leftBranch = conflict.branches.find((branch) => branch.content === left);
|
||||
@@ -1028,8 +1040,18 @@ async function runMarkdownAutoMerge(
|
||||
);
|
||||
|
||||
const afterResolution = `${merged.trimEnd()}\n\nPost-resolution edit on B.\n`;
|
||||
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, afterResolution, mergedRev);
|
||||
await writeVaultFile(vaultB.path, conflictPath, afterResolution);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, afterResolution);
|
||||
const storedAfterResolution = await waitForConflictBranch(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
conflictPath,
|
||||
(branch) => branch.content === afterResolution
|
||||
);
|
||||
assertEqual(
|
||||
storedAfterResolution.parentRev,
|
||||
mergedRev,
|
||||
"The post-resolution edit did not extend the merged revision."
|
||||
);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await stopTrackedSession(context, session);
|
||||
|
||||
|
||||
+24
-15
@@ -12,10 +12,31 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Fixed
|
||||
## 1.0.29
|
||||
|
||||
- CLI: daemon and mirror now scan the Vault during database initialisation, following the Obsidian startup sequence. The daemon completes this scan before replication; mirror runs the scan once and still exits with an error if any file cannot be processed.
|
||||
- CLI: file enumeration now includes current files even after individual path lookups or earlier scans.
|
||||
16th September, 2026
|
||||
|
||||
Unusually for this project, I have added a feature that relies on a particular infrastructure provider. I made this choice for the convenience it offers.
|
||||
|
||||
### Peer-to-peer synchronisation
|
||||
|
||||
#### New Feature
|
||||
|
||||
- P2P synchronisation now supports **Managed (Cloudflare)** TURN to help devices connect when a direct connection is unavailable. Enter your TURN Key ID and API token, and LiveSync obtains temporary TURN credentials automatically. (#1182)
|
||||
|
||||
- Managed TURN settings are saved with your encrypted P2P profile and included when you share it through a Setup URI or QR code.
|
||||
- Your API token is omitted from generated reports.
|
||||
|
||||
### Command-line tool
|
||||
|
||||
#### Fixed
|
||||
|
||||
- The CLI daemon now synchronises files already present at start-up and picks up edits and deletions made while it was stopped.
|
||||
- CLI Vault scans no longer miss files after an earlier scan or file lookup. This incorporates an adapted version of the fix proposed in PR #1188. Thank you to @YakupEmreYerli for the fix and regression tests, and to @nsanitas for the detailed report and analysis in #1143!
|
||||
|
||||
### Miscellaneous
|
||||
|
||||
In general, I would prefer to avoid features that depend on a particular service. Still, I think there is room for them when they are entirely optional, clearly explained, and maintainable. Even then, I would want open alternatives to remain available. I will write more about this principle separately.
|
||||
|
||||
## 1.0.28
|
||||
|
||||
@@ -101,15 +122,3 @@ For now, I am addressing the issues I can resolve first. I hope this helps.
|
||||
#### Fixed
|
||||
|
||||
- The systemd installer now finds the repository root correctly, installs every generated bundle chunk and required production dependency, checks the installed command before activation, and reports success only when the service remains active.
|
||||
|
||||
## 1.0.23
|
||||
|
||||
2nd September, 2026
|
||||
|
||||
I am sorry to make this release while several pull requests are still awaiting merge, but I believe that the safeguards provided by this work are significant, so I have decided to release it. I will merge the remaining pull requests in turn. Thank you for bearing with me while I have been less active recently.
|
||||
|
||||
### Synchronisation and storage
|
||||
|
||||
#### Fixed
|
||||
|
||||
- **Sync now** once again keeps routine progress quiet, while still opening recovery dialogues when a decision is required. Repeated OneShot Sync requests received while an earlier attempt is running are now ignored instead of starting overlapping work.
|
||||
|
||||
+2
-1
@@ -39,5 +39,6 @@
|
||||
"1.0.24": "1.7.2",
|
||||
"1.0.26": "1.7.2",
|
||||
"1.0.27": "1.7.2",
|
||||
"1.0.28": "1.7.2"
|
||||
"1.0.28": "1.7.2",
|
||||
"1.0.29": "1.7.2"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user