mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-09-21 18:17:05 +00:00
Merge main into stale-file protection integration
This commit is contained in:
vorotamoroz
@@ -46,7 +46,7 @@ LiveSync will expose a separate `Connection path` choice:
|
||||
- `Automatic` retains normal ICE selection and is the default.
|
||||
- `TURN relay only` supplies `iceTransportPolicy: 'relay'` and prevents direct or server-reflexive candidates from being selected.
|
||||
|
||||
`TURN relay only` is enabled only when at least one syntactically valid `turn:` or `turns:` URL is configured. If the last valid TURN URL is removed while relay-only mode is selected, the dialogue restores `Automatic` and displays a concise explanation.
|
||||
`TURN relay only` is enabled when a managed TURN provider is selected or at least one syntactically valid manual `turn:` or `turns:` URL is configured. If neither is available while relay-only mode is selected, the dialogue restores `Automatic` and displays a concise explanation. Selecting a managed provider does not itself force relay use; `Automatic` retains normal ICE selection.
|
||||
|
||||
The route policy is an ordinary P2P profile property. It is retained in P2P connection strings and encrypted Setup URIs so that an imported compatibility profile has reproducible transport behaviour.
|
||||
|
||||
@@ -60,7 +60,15 @@ The first settings revision retains the existing storage and dialogue contract o
|
||||
|
||||
A future interface may present the existing comma-separated value as ordered `turn:` and `turns:` URL rows without changing its serialised representation. A structured list of multiple credential profiles is deferred until a provider or self-hosted use case requires different credentials in the same P2P profile.
|
||||
|
||||
Static long-term credentials are the supported first stage. Managed providers may return short-lived credentials, but LiveSync must not store a provider API token or a Coturn shared authentication secret. A future managed-credential design needs a separately trusted HTTPS endpoint, expiry handling, refresh behaviour, failure reporting, and a clear Setup URI policy. It is not represented as another static password field.
|
||||
Static long-term credentials remain supported. For managed credentials, a host preparation hook requests ICE settings and places them on a connection-only copy of `P2PSyncSetting`. Service-specific requests and validation belong under `src/integrations/`; Commonlib consumes that copy and owns room reuse, expiry checks, and replacement. It has no provider catalogue or source factory.
|
||||
|
||||
A user-supplied provider API token is persisted as a sensitive P2P profile setting and included in encrypted Setup URI sharing, so that participating devices can use the same configuration without repeated token entry. Existing profile-URI encryption covers the saved token; its flat runtime projection is omitted from persistence. Reports and logs redact the complete provider configuration and issued credentials, including inactive profiles and settings projections. Coturn's server-side shared authentication secret remains outside client settings.
|
||||
|
||||
Issued short-lived TURN credentials and their expiry remain in memory. The existing room reuse decision checks both the effective connection settings and credential validity. When reconciliation finds expired credentials, it uses the normal room retirement and replacement path with newly acquired credentials. Replacement may cancel an in-progress transfer; the next replication attempt uses stored checkpoints and revision comparison to retain received progress. Whether that next attempt starts automatically follows the existing synchronisation policy.
|
||||
|
||||
Time passing alone does not trigger acquisition or disconnection. This design adds no renewal timer, per-peer acquisition hook, configuration update on raw peers, or credential-driven ICE restart. Internal peer reconnection within an unchanged room does not guarantee fresh issuance. Acquisition failure is reported without changing the selected provider or route policy. See [TURN connection settings](../design_docs/renewable_turn_credentials.md) for the preparation hook, persistence and sharing formats, room replacement, and verified replication continuation behaviour.
|
||||
|
||||
Relay-only validation accepts a valid managed TURN configuration as well as the existing manual URL list. Failure to acquire usable TURN entries keeps relay-only mode selected and reports the connection failure; it does not restore `Automatic` silently.
|
||||
|
||||
### TURN allocation check and route diagnostics
|
||||
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
---
|
||||
date: 2026-09-16
|
||||
commonlib-version: "0.1.25"
|
||||
self-hosted-livesync-version: "1.0.28"
|
||||
status: unreleased
|
||||
---
|
||||
|
||||
# TURN credentials in P2P connection settings
|
||||
|
||||
## Purpose
|
||||
|
||||
This design addresses [Issue #1182](https://github.com/vrtmrz/obsidian-livesync/issues/1182)
|
||||
by acquiring temporary TURN credentials on the device before opening a P2P room.
|
||||
The [P2P transport compatibility ADR](../adr/2026_08_p2p_transport_compatibility.md)
|
||||
records the connection and persistence policy.
|
||||
|
||||
LiveSync prepares a connection copy of `P2PSyncSetting`. Commonlib owns the room
|
||||
lifecycle and consumes the resulting ICE settings. Service-specific HTTP and
|
||||
validation remain under `src/integrations/`; Commonlib has no provider catalogue
|
||||
or versioned acquisition descriptor. Cloudflare is the first optional integration.
|
||||
Manual TURN configuration remains available without a provider account.
|
||||
|
||||
## Settings and ownership
|
||||
|
||||
| Setting | Meaning | Lifetime |
|
||||
| --- | --- | --- |
|
||||
| `P2P_managedType` | Provider identifier; `CF` selects Cloudflare | P2P profile |
|
||||
| `P2P_managedId` | Provider key identifier; Cloudflare TURN Key ID | P2P profile |
|
||||
| `P2P_managedToken` | Provider API token used to request credentials | P2P profile |
|
||||
| `P2P_iceServers` | Prepared `RTCIceServer[]` | One room connection |
|
||||
| `P2P_iceServersExpiresAt` | Absolute expiry in Unix milliseconds | One room connection |
|
||||
|
||||
The first three values use ordinary ConnStr query parameters `managedType`,
|
||||
`managedId`, and `token`. The existing `appId` parameter continues to identify the
|
||||
P2P application. Commonlib reads and writes the three scalar values so profile
|
||||
editing and activation preserve them. The host interprets the provider identifier.
|
||||
An absent identifier selects the existing manual fields; an unsupported identifier
|
||||
produces an explicit error when a connection is requested.
|
||||
|
||||
Keep `P2P_turnServers`, `P2P_turnUsername`, and `P2P_turnCredential` for manual
|
||||
configuration. Issuance does not overwrite them. Retain the complete ICE array:
|
||||
individual entries can contain different credentials or STUN-only URLs.
|
||||
|
||||
## Host preparation
|
||||
|
||||
The optional `prepareP2PSettings(settings, signal)` composition hook receives a
|
||||
snapshot of requested P2P settings. LiveSync supplies the same preparation function
|
||||
to Obsidian, CLI, WebApp, and WebPeer using each host's HTTP adapter.
|
||||
|
||||
For a managed selection, the function validates the provider inputs, requests
|
||||
credentials, and returns a connection copy:
|
||||
|
||||
```typescript
|
||||
return {
|
||||
...settings,
|
||||
P2P_iceServers: iceServers,
|
||||
P2P_iceServersExpiresAt: expiresAt,
|
||||
};
|
||||
```
|
||||
|
||||
Commonlib takes the prepared ICE fields into its session snapshot and passes that
|
||||
snapshot through `ReplicatorHostEnv.settings`. The hook does not change the
|
||||
requested room identity, persist settings, own replication, or schedule renewal.
|
||||
Its HTTP request must settle on cancellation and has a bounded deadline. The room
|
||||
owner also stops waiting for preparation when the connection request is retired.
|
||||
An explicitly managed configuration requires a preparation hook and usable ICE
|
||||
credentials; acquisition failure does not select a fallback provider or route.
|
||||
Managed credential acquisition is independent of the connection path. `Automatic`
|
||||
retains normal ICE selection, including direct candidates; only `TURN relay only`
|
||||
forces relay use. Acquisition must still succeed before opening a managed room
|
||||
when `Automatic` is selected.
|
||||
|
||||
## Room reuse and expiry
|
||||
|
||||
The active connection settings hold the issued credentials. They are the only
|
||||
credential cache. The existing room reuse decision checks:
|
||||
|
||||
1. whether the requested database and connection settings still match; and
|
||||
2. whether the active connection's credentials have enough remaining lifetime.
|
||||
|
||||
The static connection signature includes the provider type, key ID, and token.
|
||||
It excludes the generated ICE array and expiry. Comparing the prepared and stored
|
||||
settings directly would incorrectly trigger issuance on every reconciliation.
|
||||
|
||||
When reuse is unavailable, the owner retires the existing room, obtains a fresh
|
||||
connection copy, and opens its replacement. It checks settings, room demand,
|
||||
cancellation, and expiry again before publishing the replacement. A late result
|
||||
cannot reopen a closed room or apply credentials requested for different settings.
|
||||
Explicit reconnection acquires fresh credentials. Closing the room releases its
|
||||
credential references. Preserve a 30-second connection-establishment margin.
|
||||
|
||||
Reconciliation runs at existing connection, settings, and lifecycle boundaries.
|
||||
Time passing alone does not trigger acquisition or disconnection. There is no
|
||||
renewal timer, per-peer acquisition, raw WebRTC configuration update, ICE restart,
|
||||
or general retry mechanism. Trystero's internal peer reconnection within an
|
||||
unchanged room uses that room's existing configuration.
|
||||
|
||||
Normal retirement may cancel an in-progress transfer. A later replication attempt
|
||||
uses stored checkpoints and revision comparison to retain received progress.
|
||||
An unfinished network message may be sent again. Whether another attempt starts
|
||||
automatically continues to follow the existing synchronisation policy.
|
||||
|
||||
## Persistence, sharing, and privacy
|
||||
|
||||
Persist provider values only inside the selected P2P profile URI. Flat values in
|
||||
runtime settings are a projection restored by profile activation. Profile edits
|
||||
update that URI explicitly. General settings saves do not rebuild a P2P profile
|
||||
from unrelated flat settings. Flat-settings migration creates and selects its
|
||||
P2P profile once, independently of the selected main remote.
|
||||
|
||||
Existing whole-profile encryption covers the saved API token. The default mode
|
||||
uses the existing built-in key; a user-supplied configuration passphrase has its
|
||||
existing protection semantics. Failure to encrypt a managed profile leaves the
|
||||
previous saved data intact. No separate encrypted-token field is added. A draft
|
||||
containing provider credentials but no Group ID remains unsaved.
|
||||
|
||||
Setup URIs and ordinary settings QR codes already contain `remoteConfigurations`.
|
||||
The provider values travel inside that profile URI, including inactive profiles.
|
||||
Omit their duplicate flat projections from sharing. No new URI scheme, encoded QR
|
||||
slot, or encryption envelope is needed. Setup URIs retain passphrase encryption;
|
||||
QR codes retain their unencrypted format and 'FOR YOUR EYES ONLY' display.
|
||||
|
||||
Issued ICE credentials and expiry appear only in connection copies. Remove both
|
||||
runtime fields at save, import, and sharing boundaries, including
|
||||
`TrysteroReplicator.getAllConfig`, which starts from the session settings.
|
||||
Incoming settings cannot install an issued credential override. Reports omit
|
||||
runtime ICE fields, redact provider values, and retain scheme-only profile URIs.
|
||||
Logs use safe errors and omit request headers, raw responses, and connection
|
||||
signatures. Ordinary plaintext in process memory is permitted.
|
||||
|
||||
Markdown settings omit managed provider values and the profile collection with
|
||||
its selections. If that group is omitted during import, preserve the corresponding
|
||||
local P2P connection values as well as the profiles. This prevents combining an
|
||||
imported room with the local provider token or overwriting the saved profile.
|
||||
|
||||
## Cloudflare integration
|
||||
|
||||
The UI presents `Manual` and `Managed (Cloudflare)`, with `TURN Key ID` and a masked
|
||||
`TURN Key API Token` input for Cloudflare. It requires no account ID, custom
|
||||
endpoint, SDK, credential broker, or renewal interval setting.
|
||||
|
||||
The provider function uses Cloudflare's
|
||||
[credential-generation endpoint](https://developers.cloudflare.com/realtime/turn/generate-credentials/)
|
||||
and converts its response into ICE servers. The implementation requests a fixed
|
||||
24-hour lifetime and derives local expiry from the clock before the request starts.
|
||||
This lifetime applies to the issued TURN credentials, not the provider API token.
|
||||
There is currently no setting to change it.
|
||||
|
||||
The HTTP boundary uses the injected standard fetch adapter with cancellation,
|
||||
a 15-second deadline, refused redirects, omitted cookies, and disabled caching.
|
||||
It bounds the response to 32 KiB, 16 ICE entries, and 32 URLs, and validates URLs
|
||||
and complete TURN credentials. These are local implementation limits. Keep this
|
||||
validation at the provider boundary instead of repeating it in Commonlib.
|
||||
|
||||
The token is supplied and shared by the user on their devices. The provider
|
||||
function sends the key ID, API token, and requested lifetime; it has no need for
|
||||
Vault data, the Group ID, or the Vault passphrase.
|
||||
|
||||
## Setup and verification
|
||||
|
||||
The Setup connection test remains a signalling check. A separately owned trial
|
||||
uses signalling-only settings and performs no managed TURN issuance. The existing
|
||||
active-relay admission rule still applies. Success does not verify the API token,
|
||||
TURN allocation, or document transfer. Actual room connections use the preparation
|
||||
hook and preserve the selected route policy on failure.
|
||||
|
||||
Focused tests cover provider validation and cancellation, room reuse and expiry,
|
||||
late results after configuration changes or closure, migration without duplicate
|
||||
profiles, Markdown import through save/reload, safe acquisition failures, and
|
||||
exclusion of runtime credentials from storage and sharing.
|
||||
|
||||
Validate Commonlib as an exact packed artefact before testing its LiveSync
|
||||
consumer. Verify the changed settings and restart boundary in real Obsidian.
|
||||
Previously observed provider issuance and relay synchronisation do not establish
|
||||
expiry-driven reconnection for a revised build. Fresh TURN allocation after
|
||||
expiry, mobile runtimes, and cross-network behaviour require their own runtime
|
||||
verification; a surviving Trystero shared peer is not evidence of new allocation.
|
||||
+37
-4
@@ -18,7 +18,7 @@ flowchart LR
|
||||
The signalling relay and TURN server have different roles:
|
||||
|
||||
- The **signalling relay** is required for peer discovery and connection negotiation. LiveSync uses Nostr-compatible WebSocket relays for this role. The relay does not store or transfer Vault contents.
|
||||
- A **TURN server** is an optional fallback. WebRTC uses it to relay the encrypted peer connection only when the devices cannot establish a direct path through their networks.
|
||||
- A **TURN server** is an optional fallback. WebRTC uses it to relay the encrypted peer connection when the devices cannot establish a direct path through their networks, or whenever **TURN relay only** is selected.
|
||||
|
||||
## The project's public signalling relay
|
||||
|
||||
@@ -40,16 +40,49 @@ Both settings contain server addresses, but they are not interchangeable.
|
||||
| Setting | Required | Carries Vault contents | Purpose |
|
||||
| --- | --- | --- | --- |
|
||||
| **Signalling relay URLs** | Yes | No | Finds peers and exchanges the information needed to establish WebRTC connections. |
|
||||
| **TURN server URLs** | Only when direct WebRTC connectivity fails | Encrypted WebRTC traffic | Relays traffic between peers when NAT or firewall rules prevent a direct path. |
|
||||
| **TURN server URLs** | When direct WebRTC connectivity fails or **TURN relay only** is selected | Encrypted WebRTC traffic | Relays traffic between peers when NAT or firewall rules prevent a direct path. |
|
||||
|
||||
A TURN provider cannot read LiveSync's encrypted Vault contents, but it can observe connection metadata and traffic volume. Use a provider you trust. The project does not operate an official TURN service.
|
||||
WebRTC encrypts data between the devices, including when it passes through TURN. The TURN provider cannot read the transferred data, but it can observe network addresses and traffic volume. This transport encryption also applies when LiveSync's optional database encryption is disabled. The project does not operate an official TURN service.
|
||||
|
||||
## TURN credentials
|
||||
|
||||
In **TURN configuration**, select **Manual** to enter your own TURN server URLs,
|
||||
username, and credential, or select **Managed (Cloudflare)** to enter a **TURN Key ID** and
|
||||
**TURN Key API Token**. Cloudflare is optional; the project does not require a
|
||||
particular TURN provider or operate a credential broker. See Cloudflare's
|
||||
[credential instructions](https://developers.cloudflare.com/realtime/turn/generate-credentials/)
|
||||
for creating a TURN key and its API token.
|
||||
|
||||
The API token is saved with the P2P profile and included when sharing settings
|
||||
through an existing Setup URI or QR code. Setup URIs retain their existing
|
||||
passphrase encryption. QR codes retain their existing unencrypted format and
|
||||
'FOR YOUR EYES ONLY' display. Missing provider settings use the ordinary manual
|
||||
configuration defaults. Receiving clients need support for the selected provider
|
||||
to acquire its temporary TURN credentials.
|
||||
Markdown settings omit the connection profile group when it contains a managed
|
||||
TURN provider, including inactive profiles, and importing those omitted settings
|
||||
preserves this device's existing profiles. Diagnostic reports redact provider settings. The existing profile-URI
|
||||
encryption also covers the saved token.
|
||||
|
||||
Each device requests temporary TURN credentials when opening a new room.
|
||||
An existing room reuses its credentials while they remain valid. Cloudflare credentials have a requested lifetime
|
||||
of 24 hours and remain in memory only. Expiry is checked when LiveSync next
|
||||
reconciles the room connection. If necessary, it replaces the room and obtains
|
||||
new credentials. There is no periodic renewal: if a long-lived room cannot
|
||||
reconnect after credentials expire, disconnect and open the connection again.
|
||||
|
||||
Room replacement may interrupt replication. The next synchronisation keeps
|
||||
received Metadata and Chunks, resumes from its saved checkpoint, and compares
|
||||
revisions to fetch missing data. An unfinished network message can be sent
|
||||
again. Automatic synchronisation follows the existing peer rules; after an
|
||||
interrupted manual operation, use **Replicate now** again.
|
||||
|
||||
## Connection compatibility profiles
|
||||
|
||||
`P2P Configuration` includes a separate `Connection compatibility` section. Its defaults preserve the existing transport behaviour:
|
||||
|
||||
- **P2P message size** defaults to **Standard**. **Reduced**, **Conservative**, and **Maximum compatibility** progressively limit outgoing P2P messages when a network path appears to drop larger WebRTC messages. This is not a Vault Chunk size or an IP MTU. Smaller values add framing and processing overhead.
|
||||
- **Connection path** defaults to **Automatic**, which lets WebRTC select a viable direct or TURN-relayed path. **TURN relay only** forces the encrypted connection through TURN and is available only when the profile contains at least one valid `turn:` or `turns:` URL.
|
||||
- **Connection path** defaults to **Automatic**, which lets WebRTC select a viable direct or TURN-relayed path. **TURN relay only** forces the encrypted connection through TURN and is available when the profile contains a valid manual TURN URL or a configured TURN provider.
|
||||
|
||||
The sending device controls its outgoing message size. Select the same conservative preset on every device which may send across the constrained path. Existing devices do not receive the choice retrospectively merely because another device changed it.
|
||||
|
||||
|
||||
@@ -2,6 +2,18 @@
|
||||
|
||||
This document contains earlier published releases from the 1.0 line of the [current Self-hosted LiveSync release history](../../updates.md). Beta and release-candidate builds published before 1.0.0 are recorded in the [1.0 preview history](1.0-previews.md). Earlier release lines continue in the [0.25 history](0.25.md) and the [legacy history](legacy.md).
|
||||
|
||||
## 1.0.23
|
||||
|
||||
2nd September, 2026
|
||||
|
||||
I am sorry to make this release while several pull requests are still awaiting merge, but I believe that the safeguards provided by this work are significant, so I have decided to release it. I will merge the remaining pull requests in turn. Thank you for bearing with me while I have been less active recently.
|
||||
|
||||
### Synchronisation and storage
|
||||
|
||||
#### Fixed
|
||||
|
||||
- **Sync now** once again keeps routine progress quiet, while still opening recovery dialogues when a decision is required. Repeated OneShot Sync requests received while an earlier attempt is running are now ignored instead of starting overlapping work.
|
||||
|
||||
## 1.0.22
|
||||
|
||||
1st September, 2026
|
||||
|
||||
+20
-1
@@ -485,6 +485,25 @@ Setting key: P2P_AutoBroadcast
|
||||
|
||||
When enabled, this device notifies connected peers after a local change. The notification contains no Vault data. A receiving peer fetches the change only when it follows this device.
|
||||
|
||||
#### TURN configuration
|
||||
|
||||
Setting key: P2P_managedType
|
||||
|
||||
Select **Manual** for the existing TURN server fields, or **Managed (Cloudflare)** for a
|
||||
TURN Key ID and TURN Key API Token. The API token is persisted with the profile
|
||||
and included in Setup URI and QR code sharing. Issued temporary credentials are
|
||||
kept in memory only. Reports redact the provider settings. See
|
||||
[TURN credentials](p2p.md#turn-credentials) for sharing, expiry, and reconnect
|
||||
behaviour.
|
||||
|
||||
#### TURN Key ID and TURN Key API Token
|
||||
|
||||
Setting keys: P2P_managedId, P2P_managedToken
|
||||
|
||||
These fields appear when **Managed (Cloudflare)** is selected. Enter the TURN key's ID and
|
||||
its dedicated API token. The token field is masked. No account ID, custom
|
||||
endpoint, or renewal interval is required.
|
||||
|
||||
#### TURN Server URLs (comma-separated)
|
||||
|
||||
Setting key: P2P_turnServers
|
||||
@@ -515,7 +534,7 @@ The sender controls the size of its outgoing messages. Select the same conservat
|
||||
|
||||
Setting key: P2P_connectionPath
|
||||
|
||||
**Automatic** lets WebRTC select a viable direct or TURN-relayed path and is the default. **TURN relay only** forces `iceTransportPolicy: 'relay'` and is available only when the profile contains at least one valid `turn:` or `turns:` URL. Removing the last valid TURN URL while relay-only mode is selected restores **Automatic** and displays a Notice.
|
||||
**Automatic** lets WebRTC select a viable direct or TURN-relayed path and is the default. **TURN relay only** forces `iceTransportPolicy: 'relay'` and is available when the profile contains a valid manual TURN URL or a configured TURN credential source. Removing the manual TURN configuration while relay-only mode is selected restores **Automatic** and displays a Notice. A selected credential source which cannot supply credentials prevents the connection from opening; it does not change the connection path.
|
||||
|
||||
This choice belongs to the P2P profile and is retained in P2P connection strings and encrypted Setup URIs. Separate profiles may use the same Group ID and credentials with different compatibility choices; only the selected P2P profile is active.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user