mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-07-08 13:55:22 +00:00
Compare commits
23 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 236f9c434e | |||
| 708bd187d8 | |||
| 46081a5f47 | |||
| e5aa8d5c88 | |||
| abae4bda28 | |||
| dc9b4fd37e | |||
| cb1fb0f874 | |||
| 2cca2355fd | |||
| 5e090a3971 | |||
| fd251346e3 | |||
| 6a9918677f | |||
| 2d42b92a89 | |||
| a2b794c520 | |||
| 6f9446f447 | |||
| 3f9cd67b1c | |||
| dbcbf2c5ca | |||
| eed0fca8d3 | |||
| 05e031b90b | |||
| bec767c13f | |||
| 8046a777af | |||
| 0c58b0c513 | |||
| b66e227a02 | |||
| af6df84b5d |
+3
-1
@@ -16,7 +16,9 @@ pouchdb-browser.js
|
||||
production/
|
||||
|
||||
# Test coverage and reports
|
||||
coverage/
|
||||
coverage/
|
||||
test/bench-network/bench-results/
|
||||
src/apps/cli/testdeno/bench-results/
|
||||
|
||||
# Local environment / secrets
|
||||
.env
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# Run the Compose-packaged CLI P2P smoke benchmark.
|
||||
#
|
||||
# This workflow is intentionally non-required at first. It exercises the local
|
||||
# Compose package for CouchDB + Nostr relay + CLI runner, and uploads the
|
||||
# benchmark JSON results for inspection.
|
||||
name: cli-p2p-compose-smoke
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/workflows/cli-p2p-compose-smoke.yml'
|
||||
- 'package.json'
|
||||
- 'package-lock.json'
|
||||
- 'src/apps/cli/**'
|
||||
- 'test/bench-network/**'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
cases:
|
||||
description: 'Comma-separated benchmark cases'
|
||||
required: false
|
||||
default: 'couchdb-baseline,p2p-direct-local'
|
||||
md_files:
|
||||
description: 'Markdown file count'
|
||||
required: false
|
||||
default: '2'
|
||||
bin_files:
|
||||
description: 'Binary file count'
|
||||
required: false
|
||||
default: '1'
|
||||
couchdb_rtt_ms:
|
||||
description: 'Requested CouchDB RTT in milliseconds'
|
||||
required: false
|
||||
default: '20'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
smoke:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Show Docker versions
|
||||
run: |
|
||||
docker --version
|
||||
docker compose version
|
||||
|
||||
- name: Run Compose P2P smoke benchmark
|
||||
env:
|
||||
BENCH_CASES: ${{ inputs.cases || 'couchdb-baseline,p2p-direct-local' }}
|
||||
BENCH_MD_FILE_COUNT: ${{ inputs.md_files || '2' }}
|
||||
BENCH_MD_MIN_SIZE_BYTES: '128'
|
||||
BENCH_MD_MAX_SIZE_BYTES: '256'
|
||||
BENCH_BIN_FILE_COUNT: ${{ inputs.bin_files || '1' }}
|
||||
BENCH_BIN_SIZE_BYTES: '512'
|
||||
BENCH_COUCHDB_RTT_MS: ${{ inputs.couchdb_rtt_ms || '20' }}
|
||||
BENCH_SYNC_TIMEOUT: '180'
|
||||
BENCH_PEERS_TIMEOUT: '20'
|
||||
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: '60000'
|
||||
BENCH_LIVESYNC_TEST_TEE: '0'
|
||||
run: docker compose -f test/bench-network/compose.yml run --rm bench-runner
|
||||
|
||||
- name: Show Compose diagnostics
|
||||
if: failure()
|
||||
run: |
|
||||
docker compose -f test/bench-network/compose.yml ps
|
||||
docker compose -f test/bench-network/compose.yml logs --no-color couchdb nostr-relay
|
||||
|
||||
- name: Upload benchmark results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: cli-p2p-compose-smoke-results
|
||||
path: test/bench-network/bench-results/**
|
||||
if-no-files-found: warn
|
||||
|
||||
- name: Stop Compose services
|
||||
if: always()
|
||||
run: docker compose -f test/bench-network/compose.yml down -v --remove-orphans
|
||||
@@ -148,6 +148,20 @@ Hence, the new feature should be implemented as follows:
|
||||
- **Service Hub** (`src/modules/services/`): Central service registry using dependency injection
|
||||
- **Common Library** (`src/lib/`): Platform-independent sync logic, shared with other tools
|
||||
|
||||
### Conflict Merge Policy
|
||||
|
||||
Markdown conflict auto-merge should behave like a conservative three-way merge. The guiding rule is to merge changes when they touch non-overlapping regions, and to keep a manual conflict when the edits overlap semantically.
|
||||
|
||||
When in doubt, prefer the safer outcome: preserve data, keep the conflict visible, and ask the user rather than silently discarding content or choosing one side.
|
||||
|
||||
- If one side deletes a line and the other side leaves that same line unchanged, treat it as a safe deletion. The deleted line must not be reintroduced into the merged result.
|
||||
- If one side inserts new content in a different region while the other side deletes an unchanged old region, preserve the insertion and the deletion.
|
||||
- If one side deletes a line and the other side modifies that same line, keep the conflict for user resolution.
|
||||
- If both sides insert different content at the same position, keep both insertions in a deterministic order unless the surrounding deletion context indicates that they are competing replacements.
|
||||
- Avoid resolving conflicts by simply choosing the newest revision unless the user has explicitly selected that behaviour.
|
||||
|
||||
This policy is intentionally aligned with the conflict checkboxes and compatibility settings: automatic merge should remove avoidable prompts, but it must not silently choose between overlapping user intentions.
|
||||
|
||||
### File Structure Conventions
|
||||
|
||||
- **Platform-specific code**: Use `.platform.ts` suffix (replaced with `.obsidian.ts` in production builds via esbuild)
|
||||
|
||||
@@ -1,360 +0,0 @@
|
||||
# Architectural Decision Record: Setting Definition Repository
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
|
||||
## Release
|
||||
|
||||
Not scheduled. Intended as a design direction before refactoring the settings dialogue.
|
||||
|
||||
## Context
|
||||
|
||||
The current settings dialogue is implemented around `ObsidianLiveSyncSettingTab`.
|
||||
It is effective and feature rich, but several responsibilities are combined in the
|
||||
same layer:
|
||||
|
||||
- editing-state buffering and dirty-state tracking,
|
||||
- local-only dialogue state such as `configPassphrase`, `preset`, `syncMode`, and
|
||||
`deviceAndVaultName`,
|
||||
- persistence through `SettingService`,
|
||||
- Obsidian `Setting` component rendering,
|
||||
- pane layout and visibility rules,
|
||||
- validation and value coercion,
|
||||
- setup wizard transitions,
|
||||
- rebuild/restart side effects,
|
||||
- remote diagnostics and maintenance actions.
|
||||
|
||||
Some setting metadata already exists in `configurationNames` and related setting
|
||||
constants. This is a useful seed, but it is not a complete source of truth. The
|
||||
metadata does not currently describe storage domain, value kind, validation,
|
||||
capability requirements, migration behaviour, cross-setting dependencies, or
|
||||
whether a value should be rendered by a generic control or a custom pane.
|
||||
|
||||
The settings dialogue also contains a mix of simple controls and workflow panels.
|
||||
Examples:
|
||||
|
||||
- Simple controls: `showStatusOnEditor`, `syncOnSave`, `customChunkSize`,
|
||||
`readChunksOnline`, `useTimeouts`.
|
||||
- Derived or dialogue-only values: `syncMode`, `preset`, `configPassphrase`,
|
||||
`deviceAndVaultName`.
|
||||
- Workflow panels: remote configuration management, E2EE setup, setup wizard,
|
||||
local/remote rebuild, maintenance commands, Customisation Sync dialogue open.
|
||||
|
||||
This matters primarily for maintainability and platform independence. A setting
|
||||
should have one shared definition of its domain meaning regardless of whether it
|
||||
is displayed in Obsidian, surfaced in a CLI, used by a WebApp, or documented.
|
||||
Tests benefit from this separation, but testability is a consequence rather than
|
||||
the main design goal.
|
||||
|
||||
Real Obsidian E2E remains the right signal for Obsidian shell behaviour such as
|
||||
opening the settings tab, rendering real Obsidian components, and verifying
|
||||
user-visible workflows. Harness-based tests can then focus on deterministic
|
||||
setting semantics that no longer depend on mounting the whole Obsidian setting
|
||||
tab.
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce a platform-neutral Setting Definition Repository as the source of
|
||||
truth for setting metadata and setting semantics.
|
||||
|
||||
The repository should live in the shared domain layer, not under the Obsidian
|
||||
dialogue implementation and not under a generic model bucket. In the current
|
||||
layout this means `src/lib/src/common/settings`.
|
||||
|
||||
The repository should not be an Obsidian UI abstraction. It should describe what
|
||||
a setting is and how it behaves. Obsidian, CLI, WebApp, tests, and documentation
|
||||
can then consume the same definitions through their own renderers or adapters.
|
||||
The Obsidian settings tab should use an Obsidian renderer that maps repository
|
||||
definitions to native Obsidian `Setting` components for simple controls, while
|
||||
workflow panes remain custom.
|
||||
|
||||
The existing Obsidian setting dialogue should be migrated incrementally. Complex
|
||||
workflow panes should remain custom-rendered at first. Simple controls should
|
||||
move to repository-driven rendering first.
|
||||
|
||||
## Proposed Model
|
||||
|
||||
Each setting definition should describe a single setting key or a dialogue-only
|
||||
virtual key.
|
||||
|
||||
```ts
|
||||
type SettingStorageDomain = "persisted" | "local" | "derived" | "ephemeral";
|
||||
|
||||
type SettingValueKind = "boolean" | "text" | "password" | "number" | "select" | "textarea" | "string-list" | "custom";
|
||||
|
||||
type SettingCapability = "database-user" | "server-admin" | "filesystem" | "obsidian-shell" | "obsidian-plugin-host";
|
||||
|
||||
type SettingDefinition<TSettings, TKey extends keyof TSettings | string> = {
|
||||
key: TKey;
|
||||
storage: SettingStorageDomain;
|
||||
kind: SettingValueKind;
|
||||
defaultValue?: unknown;
|
||||
labelKey: string;
|
||||
descriptionKey?: string;
|
||||
label: string;
|
||||
description?: string;
|
||||
category: string;
|
||||
pane?: string;
|
||||
section?: string;
|
||||
level?: "ADVANCED" | "POWER_USER" | "EDGE_CASE";
|
||||
status?: "BETA" | "ALPHA" | "EXPERIMENTAL";
|
||||
obsolete?: boolean;
|
||||
internal?: boolean;
|
||||
placeholder?: string;
|
||||
options?: Record<string, string>;
|
||||
secret?: boolean;
|
||||
requiredCapabilities?: SettingCapability[];
|
||||
visible?: (context: SettingEvaluationContext<TSettings>) => boolean;
|
||||
enabled?: (context: SettingEvaluationContext<TSettings>) => boolean;
|
||||
validate?: (value: unknown, context: SettingEvaluationContext<TSettings>) => SettingValidationResult;
|
||||
coerce?: (value: unknown, context: SettingEvaluationContext<TSettings>) => unknown;
|
||||
affects?: SettingEffect[];
|
||||
commit?: SettingCommitPolicy<TKey>;
|
||||
render?: "auto" | "custom";
|
||||
};
|
||||
```
|
||||
|
||||
`SettingEvaluationContext` should carry the current editing settings, persisted
|
||||
settings, platform capabilities, and remote capability information if known. It
|
||||
should not carry an Obsidian `App`.
|
||||
|
||||
`labelKey` and `descriptionKey` should be i18n keys. During migration, the key
|
||||
may be the literal English text already used by `configurationNames` and
|
||||
`SettingInformation`. This matches the current i18n behaviour where an unknown
|
||||
key resolves to the key itself, avoids adding translation resources up front, and
|
||||
lets us later replace literal keys with stable resource keys without changing
|
||||
consumers. `label` and `description` remain compatibility aliases while existing
|
||||
code still expects resolved strings.
|
||||
|
||||
`internal` should mark settings that are not currently editable from the UI.
|
||||
Obsolete settings are internal by default. Missing UI metadata alone should not
|
||||
make a setting internal; `kind`, `render`, and explicit `internal` metadata
|
||||
should decide whether the automatic renderer can safely handle it.
|
||||
|
||||
`commit` should describe when a value is persisted, not how a button is rendered.
|
||||
Immediate settings can save on change. Explicit settings are held in the editing
|
||||
buffer until an apply action commits the configured group. This keeps apply
|
||||
buttons out of the repository while still making grouped save behaviour
|
||||
testable.
|
||||
|
||||
## Storage Domains
|
||||
|
||||
Settings should be classified by where they live:
|
||||
|
||||
- `persisted`: normal `ObsidianLiveSyncSettings` values saved through
|
||||
`SettingService`.
|
||||
- `local`: values stored outside the main settings document, for example local
|
||||
storage or device/vault identity.
|
||||
- `derived`: values computed from persisted settings, for example `syncMode`.
|
||||
- `ephemeral`: dialogue-only inputs such as `preset`.
|
||||
|
||||
This makes current special cases explicit:
|
||||
|
||||
- `configPassphrase` is local-only.
|
||||
- `deviceAndVaultName` is local/service managed.
|
||||
- `syncMode` is derived from `liveSync` and `periodicReplication`.
|
||||
- `preset` is ephemeral and expands to several persisted settings.
|
||||
|
||||
## Rendering Strategy
|
||||
|
||||
The repository should support generic rendering, but it should not force every
|
||||
pane to become schema-driven immediately.
|
||||
|
||||
Use three levels:
|
||||
|
||||
1. **Auto-rendered controls**
|
||||
Simple `boolean`, `number`, `text`, `password`, `select`, and `textarea`
|
||||
settings. These can replace many `new Setting(...).autoWire...` calls.
|
||||
|
||||
2. **Repository-defined groups with custom sections**
|
||||
A pane can declare layout, headings, and order through the repository but keep
|
||||
a custom renderer for the section body.
|
||||
|
||||
3. **Fully custom workflow panes**
|
||||
Remote configuration management, E2EE setup, setup wizard, maintenance, and
|
||||
rebuild flows should remain custom until their side effects are separately
|
||||
modelled.
|
||||
|
||||
The Obsidian setting dialogue becomes a renderer of repository definitions plus a
|
||||
host for custom workflow panes.
|
||||
|
||||
## Side Effects
|
||||
|
||||
Setting changes should distinguish value persistence from effects.
|
||||
|
||||
Examples of effects:
|
||||
|
||||
- `requires-local-rebuild`
|
||||
- `requires-remote-rebuild`
|
||||
- `requires-restart`
|
||||
- `requires-apply-settings`
|
||||
- `suspends-sync`
|
||||
- `updates-unresolved-error-ui`
|
||||
- `changes-active-remote`
|
||||
- `expands-preset`
|
||||
|
||||
The current `isNeedRebuildLocal()` and `isNeedRebuildRemote()` methods should
|
||||
eventually be replaced by repository metadata. This would make rebuild prompts
|
||||
testable without rendering the full settings tab.
|
||||
|
||||
## Capability Requirements
|
||||
|
||||
Some settings and actions require capabilities that not all users or platforms
|
||||
have.
|
||||
|
||||
Examples:
|
||||
|
||||
- CouchDB server diagnostics and automatic CouchDB repair require server-admin
|
||||
capability.
|
||||
- Normal CouchDB sync requires only database-user capability.
|
||||
- Hidden File Sync and Customisation Sync require filesystem capability.
|
||||
- Obsidian plug-in reload requires obsidian-plugin-host capability.
|
||||
- Opening settings panes and workspace views requires obsidian-shell capability.
|
||||
|
||||
Capability metadata should be used for:
|
||||
|
||||
- warning text in Obsidian settings,
|
||||
- disabling unsupported actions,
|
||||
- CLI/WebApp help output,
|
||||
- Harness tests for visibility and enabled-state rules.
|
||||
|
||||
The repository should not introduce a generic cross-platform `PluginManager`
|
||||
concept. Obsidian plug-in host behaviour should remain an Obsidian-specific
|
||||
adapter or custom workflow.
|
||||
|
||||
## Current Assumptions to Preserve
|
||||
|
||||
- Settings can be edited in a buffer before being saved.
|
||||
- Some values save immediately unless `holdValue` is set.
|
||||
- Some values require explicit Apply buttons.
|
||||
- Visibility and enabled-state often depend on other editing values.
|
||||
- Some settings are hidden in setup wizard mode.
|
||||
- Advanced, power-user, and edge-case levels remain supported.
|
||||
- The dialogue can be reloaded while preserving dirty local edits.
|
||||
- Existing `SettingService` remains responsible for encryption, persistence,
|
||||
migration, and applying settings.
|
||||
- Existing complex setup and remote configuration workflows remain custom.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### Phase 1: Repository Skeleton
|
||||
|
||||
- Create a repository module in the shared setting domain
|
||||
(`src/lib/src/common/settings`), not under the Obsidian dialogue folder.
|
||||
- Move existing `configurationNames` metadata into repository definitions without
|
||||
changing runtime behaviour.
|
||||
- Add storage domain, kind, i18n keys, internal marker, pane, section, level, and
|
||||
secret metadata for a small subset of settings.
|
||||
- Keep `getConfig()`, `getConfName()`, and existing callers working through a
|
||||
compatibility facade.
|
||||
|
||||
### Phase 2: Evaluation API
|
||||
|
||||
- Add pure functions:
|
||||
- `getSettingDefinition(key)`
|
||||
- `listSettingDefinitions(filter)`
|
||||
- `evaluateSetting(definition, context)`
|
||||
- `validateSettingValue(key, value, context)`
|
||||
- `getSettingEffects(changedKeys, context)`
|
||||
- Add unit tests for derived values, visibility, enabled-state, validation, and
|
||||
rebuild/restart effects.
|
||||
|
||||
### Phase 3: Obsidian Renderer for Simple Controls
|
||||
|
||||
- Add a small renderer that maps repository definitions to Obsidian `Setting`
|
||||
controls.
|
||||
- Migrate one low-risk pane first, likely Appearance/Logging or Advanced memory
|
||||
cache settings.
|
||||
- Keep custom panes untouched.
|
||||
- Keep `LiveSyncSetting` as a compatibility wrapper during migration.
|
||||
|
||||
### Phase 4: Derived and Local Values
|
||||
|
||||
- Model `syncMode`, `preset`, `configPassphrase`, and `deviceAndVaultName`
|
||||
explicitly.
|
||||
- Replace ad hoc save paths in `ObsidianLiveSyncSettingTab` with storage-domain
|
||||
handlers.
|
||||
- Keep user-visible behaviour unchanged.
|
||||
|
||||
### Phase 5: Effects and Capability Warnings
|
||||
|
||||
- Replace `isNeedRebuildLocal()` and `isNeedRebuildRemote()` with
|
||||
repository-driven effect calculation.
|
||||
- Model explicit commit groups for settings that must be applied together, for
|
||||
example configuration encryption passphrase settings, setting sync file, and
|
||||
database suffix changes.
|
||||
- Add capability metadata for CouchDB diagnostics, repair, Hidden File Sync, and
|
||||
Obsidian-only plug-in operations.
|
||||
- Use this to improve warnings for database-scoped CouchDB users and
|
||||
administrator-only actions.
|
||||
|
||||
### Phase 6: Documentation and Non-Obsidian Consumers
|
||||
|
||||
- Treat documentation as an authored source, not as an output that must be fully
|
||||
generated from code.
|
||||
- Optionally combine repository metadata with a documentation source such as YAML
|
||||
to generate or lint `docs/settings.md`.
|
||||
- Use the repository to verify that documented settings exist, that defaults and
|
||||
storage domains are consistent, and that internal settings are intentionally
|
||||
omitted or documented as internal.
|
||||
- Expose repository metadata to CLI/WebApp where useful.
|
||||
- Let Harness tests assert the same repository semantics used by Obsidian.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
Use Harness or unit tests for:
|
||||
|
||||
- default value coverage,
|
||||
- type/kind consistency,
|
||||
- every persisted setting has a definition or is explicitly internal,
|
||||
- visibility and enabled-state predicates,
|
||||
- derived values such as `syncMode`,
|
||||
- preset expansion,
|
||||
- rebuild/restart effect calculation,
|
||||
- capability warnings.
|
||||
|
||||
Use real Obsidian E2E for:
|
||||
|
||||
- opening the actual setting tab,
|
||||
- rendering Obsidian `Setting` components,
|
||||
- setup wizard flow,
|
||||
- remote configuration workflow,
|
||||
- actual restart prompts,
|
||||
- workflows that depend on Obsidian settings shell behaviour.
|
||||
|
||||
## Consequences
|
||||
|
||||
Positive:
|
||||
|
||||
- Setting semantics are maintained in one platform-neutral place.
|
||||
- Setting semantics become testable without mounting Obsidian UI.
|
||||
- Documentation, CLI, WebApp, and Obsidian can share setting metadata where it is
|
||||
useful.
|
||||
- Capability-sensitive settings become explicit.
|
||||
- Future settings are less likely to be implemented in only one surface.
|
||||
- The Obsidian settings dialogue can be refactored incrementally.
|
||||
|
||||
Negative:
|
||||
|
||||
- There will be a temporary compatibility layer between old setting constants and
|
||||
the repository.
|
||||
- Some panes will remain custom, so the repository will not remove all UI code.
|
||||
- Definition metadata can become stale if not enforced by tests.
|
||||
- Over-generalising workflow panes would make the repository harder to maintain.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not replace `SettingService` persistence in the first phase.
|
||||
- Do not make Obsidian plug-in host operations cross-platform.
|
||||
- Do not convert all setting panes to schema-driven UI at once.
|
||||
- Do not require real Obsidian E2E for every setting definition.
|
||||
- Do not remove custom renderers for remote setup, E2EE setup, or maintenance
|
||||
workflows.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- What should the exact `CapabilityProvider` interface look like for static
|
||||
platform capabilities and runtime-probed remote capabilities? This should be
|
||||
decided while implementing the Obsidian renderer so the interface follows a
|
||||
real consumer instead of an abstract capability model.
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"id": "obsidian-livesync",
|
||||
"name": "Self-hosted LiveSync",
|
||||
"version": "0.25.79",
|
||||
"version": "0.25.80",
|
||||
"minAppVersion": "1.7.2",
|
||||
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
|
||||
"author": "vorotamoroz",
|
||||
|
||||
Generated
+5
-5
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "obsidian-livesync",
|
||||
"version": "0.25.79",
|
||||
"version": "0.25.80",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "obsidian-livesync",
|
||||
"version": "0.25.79",
|
||||
"version": "0.25.80",
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
"src/apps/cli",
|
||||
@@ -16210,7 +16210,7 @@
|
||||
},
|
||||
"src/apps/cli": {
|
||||
"name": "self-hosted-livesync-cli",
|
||||
"version": "0.25.79-cli",
|
||||
"version": "0.25.80-cli",
|
||||
"dependencies": {
|
||||
"chokidar": "^4.0.0",
|
||||
"minimatch": "^10.2.5",
|
||||
@@ -16236,7 +16236,7 @@
|
||||
},
|
||||
"src/apps/webapp": {
|
||||
"name": "livesync-webapp",
|
||||
"version": "0.25.79-webapp",
|
||||
"version": "0.25.80-webapp",
|
||||
"dependencies": {
|
||||
"octagonal-wheels": "^0.1.47"
|
||||
},
|
||||
@@ -16251,7 +16251,7 @@
|
||||
}
|
||||
},
|
||||
"src/apps/webpeer": {
|
||||
"version": "0.25.79-webpeer",
|
||||
"version": "0.25.80-webpeer",
|
||||
"dependencies": {
|
||||
"octagonal-wheels": "^0.1.47"
|
||||
},
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "obsidian-livesync",
|
||||
"version": "0.25.79",
|
||||
"version": "0.25.80",
|
||||
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
|
||||
"main": "main.js",
|
||||
"type": "module",
|
||||
|
||||
@@ -4,12 +4,21 @@ import type { ServiceContext } from "@lib/services/base/ServiceBase";
|
||||
import { LiveSyncTrysteroReplicator } from "@lib/replication/trystero/LiveSyncTrysteroReplicator";
|
||||
import { compatGlobal } from "@lib/common/coreEnvFunctions.ts";
|
||||
import { LiveSyncError } from "@lib/common/LSError";
|
||||
import { getPeerConnectionStats } from "@lib/rpc/transports/DiagRTCPeerConnections.utils";
|
||||
import { appendFile } from "node:fs/promises";
|
||||
|
||||
type CLIP2PPeer = {
|
||||
peerId: string;
|
||||
name: string;
|
||||
};
|
||||
|
||||
type CandidateSummary = {
|
||||
id: string | "unknown";
|
||||
candidateType: string | "unknown";
|
||||
protocol: string | "unknown";
|
||||
relayProtocol: string | "unknown";
|
||||
};
|
||||
|
||||
function delay(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => compatGlobal.setTimeout(resolve, ms));
|
||||
}
|
||||
@@ -81,6 +90,74 @@ function resolvePeer(peers: CLIP2PPeer[], peerToken: string): CLIP2PPeer | undef
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function getReportValue<T extends string | number>(
|
||||
report: Record<string, unknown> | undefined,
|
||||
key: string
|
||||
): T | "unknown" {
|
||||
const value = report?.[key];
|
||||
return typeof value === "string" || typeof value === "number" ? (value as T) : "unknown";
|
||||
}
|
||||
|
||||
function summariseCandidate(reports: unknown[], candidateId: string | "unknown"): CandidateSummary | undefined {
|
||||
if (candidateId === "unknown") {
|
||||
return undefined;
|
||||
}
|
||||
const report = reports.map((r) => r as Record<string, unknown>).find((r) => r.id === candidateId);
|
||||
if (!report) {
|
||||
return undefined;
|
||||
}
|
||||
return {
|
||||
id: candidateId,
|
||||
candidateType: getReportValue<string>(report, "candidateType"),
|
||||
protocol: getReportValue<string>(report, "protocol"),
|
||||
relayProtocol: getReportValue<string>(report, "relayProtocol"),
|
||||
};
|
||||
}
|
||||
|
||||
async function writePeerConnectionStatsIfRequested(
|
||||
replicator: LiveSyncTrysteroReplicator,
|
||||
peer: CLIP2PPeer
|
||||
): Promise<void> {
|
||||
const outputPath = process.env.LIVESYNC_P2P_STATS_JSONL?.trim();
|
||||
if (!outputPath) {
|
||||
return;
|
||||
}
|
||||
|
||||
const peerConnection = replicator.rawHost?.room?.getPeers()[peer.peerId];
|
||||
const stats = peerConnection ? await getPeerConnectionStats(`cli-p2p-${peer.peerId}`, peerConnection) : undefined;
|
||||
const localCandidate = summariseCandidate(stats?.reports ?? [], stats?.localCandidateId ?? "unknown");
|
||||
const remoteCandidate = summariseCandidate(stats?.reports ?? [], stats?.remoteCandidateId ?? "unknown");
|
||||
const selectedPath =
|
||||
localCandidate && remoteCandidate
|
||||
? `${localCandidate.candidateType}<->${remoteCandidate.candidateType}`
|
||||
: "unknown";
|
||||
|
||||
const payload = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
command: "p2p-sync",
|
||||
peerId: peer.peerId,
|
||||
peerName: peer.name,
|
||||
candidatePathCollected: !!stats?.selectedPair,
|
||||
selectedPath,
|
||||
selectedPair: stats
|
||||
? {
|
||||
id: stats.selectedPairId,
|
||||
state: stats.state,
|
||||
currentRoundTripTime: stats.currentRoundTripTime,
|
||||
totalRoundTripTime: stats.totalRoundTripTime,
|
||||
requestsSent: stats.requestsSent,
|
||||
responsesReceived: stats.responsesReceived,
|
||||
packetsDiscardedOnSend: stats.packetsDiscardedOnSend,
|
||||
bytesSent: stats.bytesSent,
|
||||
bytesReceived: stats.bytesReceived,
|
||||
}
|
||||
: undefined,
|
||||
localCandidate,
|
||||
remoteCandidate,
|
||||
};
|
||||
await appendFile(outputPath, `${JSON.stringify(payload)}\n`, "utf8");
|
||||
}
|
||||
|
||||
export async function syncWithPeer(
|
||||
core: LiveSyncBaseCore<ServiceContext, never>,
|
||||
peerToken: string,
|
||||
@@ -118,6 +195,7 @@ export async function syncWithPeer(
|
||||
: LiveSyncError.fromError(err ?? "P2P sync failed while requesting remote sync");
|
||||
}
|
||||
|
||||
await writePeerConnectionStatsIfRequested(replicator, targetPeer);
|
||||
return targetPeer;
|
||||
} finally {
|
||||
await replicator.close();
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "self-hosted-livesync-cli",
|
||||
"private": true,
|
||||
"version": "0.25.79-cli",
|
||||
"version": "0.25.80-cli",
|
||||
"main": "dist/index.cjs",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts";
|
||||
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
|
||||
import { startCouchdb, stopCouchdb } from "./helpers/docker.ts";
|
||||
import { createCouchdbDatabase, startCouchdb, stopCouchdb } from "./helpers/docker.ts";
|
||||
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
|
||||
|
||||
type BenchmarkConfig = {
|
||||
caseName: string;
|
||||
couchdbBackendUri: string;
|
||||
couchdbProxyUri: string;
|
||||
couchdbUser: string;
|
||||
@@ -21,6 +22,10 @@ type BenchmarkConfig = {
|
||||
requestedRttMs: number;
|
||||
passphrase: string;
|
||||
encrypt: boolean;
|
||||
managedCouchdb: boolean;
|
||||
simulationTier: string;
|
||||
networkProfile: string;
|
||||
networkModel: string;
|
||||
};
|
||||
|
||||
function readEnvString(name: string, fallback: string): string {
|
||||
@@ -70,6 +75,7 @@ function formatBytes(value: number): string {
|
||||
|
||||
function buildConfig(): BenchmarkConfig {
|
||||
return {
|
||||
caseName: readEnvString("BENCH_CASE", "couchdb-baseline"),
|
||||
couchdbBackendUri: readEnvString("BENCH_COUCHDB_BACKEND_URI", "http://127.0.0.1:5989"),
|
||||
couchdbProxyUri: readEnvString("BENCH_COUCHDB_URI", "http://127.0.0.1:15989"),
|
||||
couchdbUser: readEnvString("BENCH_COUCHDB_USER", readEnvString("username", "admin")),
|
||||
@@ -86,6 +92,10 @@ function buildConfig(): BenchmarkConfig {
|
||||
requestedRttMs: Math.floor(readEnvNumber("BENCH_COUCHDB_RTT_MS", 50)),
|
||||
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
|
||||
encrypt: readEnvBool("BENCH_ENCRYPT", true),
|
||||
managedCouchdb: readEnvBool("BENCH_COUCHDB_MANAGED", true),
|
||||
simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"),
|
||||
networkProfile: readEnvString("BENCH_NETWORK_PROFILE", "http-latency-proxy"),
|
||||
networkModel: readEnvString("BENCH_NETWORK_MODEL", "local-http-proxy"),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -200,7 +210,17 @@ async function main(): Promise<void> {
|
||||
await initSettingsFile(settingsA);
|
||||
await initSettingsFile(settingsB);
|
||||
|
||||
await startCouchdb(config.couchdbBackendUri, config.couchdbUser, config.couchdbPassword, config.couchdbDbname);
|
||||
if (config.managedCouchdb) {
|
||||
await startCouchdb(config.couchdbBackendUri, config.couchdbUser, config.couchdbPassword, config.couchdbDbname);
|
||||
} else {
|
||||
console.log(`[INFO] using externally managed CouchDB: ${config.couchdbBackendUri}`);
|
||||
await createCouchdbDatabase(
|
||||
config.couchdbBackendUri,
|
||||
config.couchdbUser,
|
||||
config.couchdbPassword,
|
||||
config.couchdbDbname
|
||||
);
|
||||
}
|
||||
|
||||
const proxy = startCouchdbProxy({
|
||||
backendUri: config.couchdbBackendUri,
|
||||
@@ -265,10 +285,15 @@ async function main(): Promise<void> {
|
||||
}
|
||||
|
||||
const result = {
|
||||
caseName: config.caseName,
|
||||
mode: "couchdb-cli-benchmark",
|
||||
couchdbBackendUri: config.couchdbBackendUri,
|
||||
couchdbProxyUri: config.couchdbProxyUri,
|
||||
couchdbDbname: config.couchdbDbname,
|
||||
managedCouchdb: config.managedCouchdb,
|
||||
simulationTier: config.simulationTier,
|
||||
networkProfile: config.networkProfile,
|
||||
networkModel: config.networkModel,
|
||||
rttRequestedMs: config.requestedRttMs,
|
||||
proxyApplied: proxy.applied,
|
||||
proxyNote: proxy.note,
|
||||
@@ -300,7 +325,9 @@ async function main(): Promise<void> {
|
||||
);
|
||||
} finally {
|
||||
await proxy.stop();
|
||||
await stopCouchdb().catch(() => {});
|
||||
if (config.managedCouchdb) {
|
||||
await stopCouchdb().catch(() => {});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
type SweepResult = {
|
||||
name: string;
|
||||
runner: "p2p" | "couchdb";
|
||||
rttMs?: number;
|
||||
result: Record<string, unknown>;
|
||||
};
|
||||
|
||||
function readEnvString(name: string, fallback: string): string {
|
||||
const value = Deno.env.get(name)?.trim();
|
||||
return value && value.length > 0 ? value : fallback;
|
||||
}
|
||||
|
||||
function timestamp(): string {
|
||||
const d = new Date();
|
||||
const pad = (n: number) => String(n).padStart(2, "0");
|
||||
return (
|
||||
`${d.getUTCFullYear()}${pad(d.getUTCMonth() + 1)}${pad(d.getUTCDate())}-` +
|
||||
`${pad(d.getUTCHours())}${pad(d.getUTCMinutes())}${pad(d.getUTCSeconds())}`
|
||||
);
|
||||
}
|
||||
|
||||
function parseRttList(raw: string): number[] {
|
||||
const values = raw
|
||||
.split(",")
|
||||
.map((value) => Number(value.trim()))
|
||||
.filter((value) => Number.isFinite(value) && value > 0)
|
||||
.map((value) => Math.floor(value));
|
||||
if (values.length === 0) {
|
||||
throw new Error(`BENCH_SWEEP_RTT_MS must contain at least one positive number, got '${raw}'`);
|
||||
}
|
||||
return values;
|
||||
}
|
||||
|
||||
function buildBaseEnv(): Record<string, string> {
|
||||
return {
|
||||
BENCH_MD_FILE_COUNT: readEnvString("BENCH_MD_FILE_COUNT", "20"),
|
||||
BENCH_MD_MIN_SIZE_BYTES: readEnvString("BENCH_MD_MIN_SIZE_BYTES", "512"),
|
||||
BENCH_MD_MAX_SIZE_BYTES: readEnvString("BENCH_MD_MAX_SIZE_BYTES", "2048"),
|
||||
BENCH_BIN_FILE_COUNT: readEnvString("BENCH_BIN_FILE_COUNT", "5"),
|
||||
BENCH_BIN_SIZE_BYTES: readEnvString("BENCH_BIN_SIZE_BYTES", "8192"),
|
||||
BENCH_SYNC_TIMEOUT: readEnvString("BENCH_SYNC_TIMEOUT", "300"),
|
||||
BENCH_PEERS_TIMEOUT: readEnvString("BENCH_PEERS_TIMEOUT", "60"),
|
||||
BENCH_SEED: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
|
||||
LIVESYNC_TEST_TEE: readEnvString("BENCH_LIVESYNC_TEST_TEE", "0"),
|
||||
};
|
||||
}
|
||||
|
||||
async function runBenchmark(options: {
|
||||
taskName: "bench:p2p" | "bench:couchdb";
|
||||
name: string;
|
||||
outputDir: string;
|
||||
env: Record<string, string>;
|
||||
}): Promise<Record<string, unknown>> {
|
||||
const resultPath = `${options.outputDir}/${options.name}.json`;
|
||||
const env = {
|
||||
...Deno.env.toObject(),
|
||||
...options.env,
|
||||
BENCH_RESULT_JSON: resultPath,
|
||||
};
|
||||
|
||||
console.log(`[latency-sweep] running ${options.name}`);
|
||||
const child = new Deno.Command("deno", {
|
||||
args: ["task", options.taskName],
|
||||
cwd: import.meta.dirname,
|
||||
env,
|
||||
stdin: "null",
|
||||
stdout: "inherit",
|
||||
stderr: "inherit",
|
||||
}).spawn();
|
||||
const status = await child.status;
|
||||
if (status.code !== 0) {
|
||||
throw new Error(`benchmark failed: ${options.name} (exit ${status.code})`);
|
||||
}
|
||||
return JSON.parse(await Deno.readTextFile(resultPath)) as Record<string, unknown>;
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const outRoot = readEnvString("BENCH_SWEEP_ROOT", `${import.meta.dirname}/bench-results`);
|
||||
const outputDir = `${outRoot}/latency-sweep-${timestamp()}`;
|
||||
const rtts = parseRttList(readEnvString("BENCH_SWEEP_RTT_MS", "20,50,100,150,300"));
|
||||
const base = buildBaseEnv();
|
||||
|
||||
await Deno.mkdir(outputDir, { recursive: true });
|
||||
|
||||
const results: SweepResult[] = [];
|
||||
if (readEnvString("BENCH_SWEEP_INCLUDE_P2P", "true") !== "false") {
|
||||
const p2pResult = await runBenchmark({
|
||||
taskName: "bench:p2p",
|
||||
name: "p2p-direct-local",
|
||||
outputDir,
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "p2p-direct-local",
|
||||
BENCH_TURN_SERVERS: "",
|
||||
},
|
||||
});
|
||||
results.push({ name: "p2p-direct-local", runner: "p2p", result: p2pResult });
|
||||
}
|
||||
|
||||
for (const rtt of rtts) {
|
||||
const name = `couchdb-rtt-${rtt}ms`;
|
||||
const couchdbResult = await runBenchmark({
|
||||
taskName: "bench:couchdb",
|
||||
name,
|
||||
outputDir,
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: name,
|
||||
BENCH_COUCHDB_RTT_MS: String(rtt),
|
||||
},
|
||||
});
|
||||
results.push({ name, runner: "couchdb", rttMs: rtt, result: couchdbResult });
|
||||
}
|
||||
|
||||
const summary = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
outputDir,
|
||||
note:
|
||||
"This sweep models additional remote CouchDB request latency through the existing HTTP proxy. It is not a full netem model of jitter, loss, MTU, bandwidth, or VPN encapsulation.",
|
||||
rtts,
|
||||
results,
|
||||
};
|
||||
await Deno.writeTextFile(`${outputDir}/summary.json`, JSON.stringify(summary, null, 2));
|
||||
console.log(JSON.stringify(summary, null, 2));
|
||||
console.log(`[latency-sweep] result directory: ${outputDir}`);
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
main().catch((error) => {
|
||||
console.error("[Fatal Error]", error);
|
||||
Deno.exit(1);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,312 @@
|
||||
type BenchmarkCase = {
|
||||
name: string;
|
||||
runner: "p2p" | "couchdb";
|
||||
description: string;
|
||||
dataPath: string;
|
||||
trustBoundary: string;
|
||||
env: Record<string, string>;
|
||||
};
|
||||
|
||||
function readEnvString(name: string, fallback: string): string {
|
||||
const value = Deno.env.get(name)?.trim();
|
||||
return value && value.length > 0 ? value : fallback;
|
||||
}
|
||||
|
||||
function readEnvInteger(name: string, fallback: number): number {
|
||||
const value = readEnvString(name, String(fallback));
|
||||
const parsed = Number(value);
|
||||
if (!Number.isInteger(parsed) || parsed < 1) {
|
||||
throw new Error(`${name} must be a positive integer, got '${value}'`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function timestamp(): string {
|
||||
const d = new Date();
|
||||
const pad = (n: number) => String(n).padStart(2, "0");
|
||||
return (
|
||||
`${d.getUTCFullYear()}${pad(d.getUTCMonth() + 1)}${pad(d.getUTCDate())}-` +
|
||||
`${pad(d.getUTCHours())}${pad(d.getUTCMinutes())}${pad(d.getUTCSeconds())}`
|
||||
);
|
||||
}
|
||||
|
||||
function buildBaseEnv(): Record<string, string> {
|
||||
return {
|
||||
BENCH_MD_FILE_COUNT: readEnvString("BENCH_MD_FILE_COUNT", "20"),
|
||||
BENCH_MD_MIN_SIZE_BYTES: readEnvString("BENCH_MD_MIN_SIZE_BYTES", "512"),
|
||||
BENCH_MD_MAX_SIZE_BYTES: readEnvString("BENCH_MD_MAX_SIZE_BYTES", "2048"),
|
||||
BENCH_BIN_FILE_COUNT: readEnvString("BENCH_BIN_FILE_COUNT", "5"),
|
||||
BENCH_BIN_SIZE_BYTES: readEnvString("BENCH_BIN_SIZE_BYTES", "8192"),
|
||||
BENCH_SYNC_TIMEOUT: readEnvString("BENCH_SYNC_TIMEOUT", "300"),
|
||||
BENCH_PEERS_TIMEOUT: readEnvString("BENCH_PEERS_TIMEOUT", "60"),
|
||||
BENCH_SEED: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
|
||||
LIVESYNC_TEST_TEE: readEnvString("BENCH_LIVESYNC_TEST_TEE", "0"),
|
||||
};
|
||||
}
|
||||
|
||||
function buildCases(): BenchmarkCase[] {
|
||||
const base = buildBaseEnv();
|
||||
const couchdbRtt = readEnvString("BENCH_COUCHDB_RTT_MS", "20");
|
||||
const tetheringVpnRtt = readEnvString("BENCH_TETHERING_VPN_RTT_MS", "120");
|
||||
const localTurnServers = readEnvString("BENCH_LOCAL_TURN_SERVERS", "turn:127.0.0.1:3478");
|
||||
const shimCouchdbUri = readEnvString("BENCH_SHIM_COUCHDB_URI", "http://couchdb-shim:5984");
|
||||
const signallingShimRelay = readEnvString("BENCH_SIGNAL_SHIM_RELAY", "ws://p2p-signalling-shim:7777/");
|
||||
|
||||
return [
|
||||
{
|
||||
name: "couchdb-baseline",
|
||||
runner: "couchdb",
|
||||
description: "Standard self-hosted CouchDB path through a local latency proxy.",
|
||||
dataPath: "Device A -> CouchDB -> Device B",
|
||||
trustBoundary: "CouchDB operator and network path",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "couchdb-baseline",
|
||||
BENCH_COUCHDB_RTT_MS: couchdbRtt,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "p2p-direct-local",
|
||||
runner: "p2p",
|
||||
description: "Preferred direct WebRTC P2P path with Nostr signalling and TURN disabled.",
|
||||
dataPath: "Device A -> Device B",
|
||||
trustBoundary: "Nostr relay for signalling metadata; no TURN relay",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "p2p-direct-local",
|
||||
BENCH_TURN_SERVERS: "",
|
||||
BENCH_SIMULATION_TIER: "1",
|
||||
BENCH_NETWORK_PROFILE: "local-direct",
|
||||
BENCH_NETWORK_MODEL: "local-runner-webrtc",
|
||||
BENCH_P2P_CANDIDATE_PATH_VERIFICATION: "turn-disabled-but-selected-ice-pair-not-collected",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "couchdb-tethering-vpn-proxy",
|
||||
runner: "couchdb",
|
||||
description:
|
||||
"Approximate smartphone tethering/VPN remote-database path using an HTTP latency proxy. This does not model loss, jitter, MTU, or VPN encapsulation.",
|
||||
dataPath: "Device A -> VPN/network path -> CouchDB -> VPN/network path -> Device B",
|
||||
trustBoundary: "VPN/network path and CouchDB operator",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "couchdb-tethering-vpn-proxy",
|
||||
BENCH_COUCHDB_RTT_MS: tetheringVpnRtt,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "couchdb-netem-home-wifi",
|
||||
runner: "couchdb",
|
||||
description:
|
||||
"Tier 2 CouchDB path through the Compose netem TCP shim using the home-wifi profile.",
|
||||
dataPath: "Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
|
||||
trustBoundary: "CouchDB operator and constrained network shim",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "couchdb-netem-home-wifi",
|
||||
BENCH_COUCHDB_BACKEND_URI: shimCouchdbUri,
|
||||
BENCH_COUCHDB_RTT_MS: "1",
|
||||
BENCH_SIMULATION_TIER: "2",
|
||||
BENCH_NETWORK_PROFILE: "home-wifi",
|
||||
BENCH_NETWORK_MODEL: "compose-netem-tcp-shim",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "couchdb-netem-tethering-vpn",
|
||||
runner: "couchdb",
|
||||
description:
|
||||
"Tier 2 CouchDB path through the Compose netem TCP shim using a tethering-vpn profile.",
|
||||
dataPath: "Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
|
||||
trustBoundary: "CouchDB operator and constrained smartphone/VPN-like network shim",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "couchdb-netem-tethering-vpn",
|
||||
BENCH_COUCHDB_BACKEND_URI: shimCouchdbUri,
|
||||
BENCH_COUCHDB_RTT_MS: "1",
|
||||
BENCH_SIMULATION_TIER: "2",
|
||||
BENCH_NETWORK_PROFILE: "tethering-vpn",
|
||||
BENCH_NETWORK_MODEL: "compose-netem-tcp-shim",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "p2p-smartphone-vpn-direct",
|
||||
runner: "p2p",
|
||||
description:
|
||||
"Direct P2P case name for smartphone tethering/VPN measurements. In this local runner it is unshaped and should be treated as a wiring check unless executed on that network.",
|
||||
dataPath: "Device A -> Device B when WebRTC direct connectivity succeeds",
|
||||
trustBoundary: "Smartphone/VPN routing policy plus Nostr signalling metadata",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "p2p-smartphone-vpn-direct",
|
||||
BENCH_TURN_SERVERS: "",
|
||||
BENCH_SIMULATION_TIER: "unmeasured",
|
||||
BENCH_NETWORK_PROFILE: "smartphone-vpn-direct-placeholder",
|
||||
BENCH_NETWORK_MODEL: "local-runner-no-netem",
|
||||
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
|
||||
"structural-placeholder-only; selected ICE pair may be collected, but the path is not shaped",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "p2p-signalling-netem-home-wifi",
|
||||
runner: "p2p",
|
||||
description:
|
||||
"Tier 2 P2P path with only the Nostr signalling relay accessed through the home-wifi netem shim.",
|
||||
dataPath: "Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim",
|
||||
trustBoundary: "Nostr signalling metadata through constrained network shim; no TURN relay",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "p2p-signalling-netem-home-wifi",
|
||||
BENCH_RELAY: signallingShimRelay,
|
||||
BENCH_TURN_SERVERS: "",
|
||||
BENCH_SIMULATION_TIER: "2",
|
||||
BENCH_NETWORK_PROFILE: "home-wifi",
|
||||
BENCH_NETWORK_MODEL: "compose-netem-signalling-shim",
|
||||
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
|
||||
"selected ICE pair collected; only Nostr signalling path is shaped",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "p2p-signalling-netem-tethering-vpn",
|
||||
runner: "p2p",
|
||||
description:
|
||||
"Tier 2 P2P path with only the Nostr signalling relay accessed through the tethering-vpn netem shim.",
|
||||
dataPath: "Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim",
|
||||
trustBoundary: "Nostr signalling metadata through constrained smartphone/VPN-like network shim; no TURN relay",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "p2p-signalling-netem-tethering-vpn",
|
||||
BENCH_RELAY: signallingShimRelay,
|
||||
BENCH_TURN_SERVERS: "",
|
||||
BENCH_SIMULATION_TIER: "2",
|
||||
BENCH_NETWORK_PROFILE: "tethering-vpn",
|
||||
BENCH_NETWORK_MODEL: "compose-netem-signalling-shim",
|
||||
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
|
||||
"selected ICE pair collected; only Nostr signalling path is shaped",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "p2p-user-turn",
|
||||
runner: "p2p",
|
||||
description: "Optional fallback path through a local user-controlled TURN server.",
|
||||
dataPath: "Device A -> user-controlled TURN -> Device B",
|
||||
trustBoundary: "User-controlled TURN server",
|
||||
env: {
|
||||
...base,
|
||||
BENCH_CASE: "p2p-user-turn",
|
||||
BENCH_TURN_SERVERS: localTurnServers,
|
||||
BENCH_SIMULATION_TIER: "1",
|
||||
BENCH_NETWORK_PROFILE: "local-turn-fallback",
|
||||
BENCH_NETWORK_MODEL: "local-runner-webrtc-turn-configured",
|
||||
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
|
||||
"turn-configured; selected ICE pair may still be direct or relayed, so interpret the recorded candidate types",
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
async function runCase(
|
||||
testCase: BenchmarkCase,
|
||||
outputDir: string,
|
||||
repeatIndex: number,
|
||||
repeatCount: number
|
||||
): Promise<Record<string, unknown>> {
|
||||
const suffix = repeatCount > 1 ? `-r${String(repeatIndex).padStart(2, "0")}` : "";
|
||||
const resultPath = `${outputDir}/${testCase.name}${suffix}.json`;
|
||||
const taskName = testCase.runner === "p2p" ? "bench:p2p" : "bench:couchdb";
|
||||
const env = {
|
||||
...Deno.env.toObject(),
|
||||
...testCase.env,
|
||||
BENCH_RESULT_JSON: resultPath,
|
||||
BENCH_REPEAT_INDEX: String(repeatIndex),
|
||||
BENCH_REPEAT_COUNT: String(repeatCount),
|
||||
};
|
||||
|
||||
const repeatLabel = repeatCount > 1 ? ` (${repeatIndex}/${repeatCount})` : "";
|
||||
console.log(`[bench-cases] running ${testCase.name}${repeatLabel}: ${testCase.description}`);
|
||||
const command = new Deno.Command("deno", {
|
||||
args: ["task", taskName],
|
||||
cwd: import.meta.dirname,
|
||||
env,
|
||||
stdin: "null",
|
||||
stdout: "inherit",
|
||||
stderr: "inherit",
|
||||
});
|
||||
|
||||
const child = command.spawn();
|
||||
const status = await child.status;
|
||||
if (status.code !== 0) {
|
||||
throw new Error(`case failed: ${testCase.name} (exit ${status.code})`);
|
||||
}
|
||||
|
||||
const result = JSON.parse(await Deno.readTextFile(resultPath)) as Record<string, unknown>;
|
||||
return {
|
||||
...testCase,
|
||||
repeatIndex,
|
||||
repeatCount,
|
||||
resultPath,
|
||||
result,
|
||||
};
|
||||
}
|
||||
|
||||
function selectCases(allCases: BenchmarkCase[]): BenchmarkCase[] {
|
||||
const requested = readEnvString("BENCH_CASES", "couchdb-baseline,p2p-direct-local");
|
||||
const names = requested
|
||||
.split(",")
|
||||
.map((v) => v.trim())
|
||||
.filter((v) => v.length > 0);
|
||||
const byName = new Map(allCases.map((c) => [c.name, c]));
|
||||
return names.map((name) => {
|
||||
const found = byName.get(name);
|
||||
if (!found) {
|
||||
throw new Error(`Unknown BENCH_CASES entry '${name}'. Available: ${allCases.map((c) => c.name).join(", ")}`);
|
||||
}
|
||||
return found;
|
||||
});
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const outRoot = readEnvString("BENCH_CASES_ROOT", `${import.meta.dirname}/bench-results`);
|
||||
const outputDir = `${outRoot}/cases-${timestamp()}`;
|
||||
await Deno.mkdir(outputDir, { recursive: true });
|
||||
|
||||
const allCases = buildCases();
|
||||
const cases = selectCases(allCases);
|
||||
const repeatCount = readEnvInteger("BENCH_REPEAT_COUNT", 1);
|
||||
await Deno.writeTextFile(
|
||||
`${outputDir}/case-manifest.json`,
|
||||
JSON.stringify(
|
||||
{
|
||||
generatedAt: new Date().toISOString(),
|
||||
repeatCount,
|
||||
selectedCases: cases,
|
||||
availableCases: allCases,
|
||||
},
|
||||
null,
|
||||
2
|
||||
)
|
||||
);
|
||||
|
||||
const results: Record<string, unknown>[] = [];
|
||||
for (const testCase of cases) {
|
||||
for (let repeatIndex = 1; repeatIndex <= repeatCount; repeatIndex++) {
|
||||
results.push(await runCase(testCase, outputDir, repeatIndex, repeatCount));
|
||||
}
|
||||
}
|
||||
|
||||
const summary = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
outputDir,
|
||||
repeatCount,
|
||||
results,
|
||||
};
|
||||
await Deno.writeTextFile(`${outputDir}/summary.json`, JSON.stringify(summary, null, 2));
|
||||
console.log(JSON.stringify(summary, null, 2));
|
||||
console.log(`[bench-cases] result directory: ${outputDir}`);
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
main().catch((error) => {
|
||||
console.error("[Fatal Error]", error);
|
||||
Deno.exit(1);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,435 @@
|
||||
import { join } from "@std/path";
|
||||
import { startCliInBackground } from "./helpers/backgroundCli.ts";
|
||||
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
|
||||
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
|
||||
import { discoverPeer } from "./helpers/p2p.ts";
|
||||
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
|
||||
|
||||
type Role = "host" | "client";
|
||||
|
||||
type NetemSummary = {
|
||||
enabled: boolean;
|
||||
profile: string;
|
||||
interface: string;
|
||||
delayMs: number;
|
||||
jitterMs: number;
|
||||
lossPercent: number;
|
||||
bandwidthMbit: number;
|
||||
mtu: number;
|
||||
tcQdisc?: string;
|
||||
ipAddr?: string;
|
||||
ipRoute?: string;
|
||||
};
|
||||
|
||||
type HostReady = {
|
||||
generatedAt: string;
|
||||
totalFiles: number;
|
||||
totalBytes: number;
|
||||
mdFileCount: number;
|
||||
binFileCount: number;
|
||||
mirrorElapsedMs: number;
|
||||
netem: NetemSummary;
|
||||
};
|
||||
|
||||
type P2PConnectionStats = {
|
||||
candidatePathCollected: boolean;
|
||||
selectedPath: string;
|
||||
localCandidate?: { candidateType: string; protocol: string; relayProtocol: string };
|
||||
remoteCandidate?: { candidateType: string; protocol: string; relayProtocol: string };
|
||||
};
|
||||
|
||||
function errorToRecord(error: unknown): Record<string, unknown> {
|
||||
if (error instanceof Error) {
|
||||
return {
|
||||
name: error.name,
|
||||
message: error.message,
|
||||
stack: error.stack,
|
||||
};
|
||||
}
|
||||
return {
|
||||
name: "UnknownError",
|
||||
message: String(error),
|
||||
};
|
||||
}
|
||||
|
||||
function readEnvString(name: string, fallback: string): string {
|
||||
const value = Deno.env.get(name)?.trim();
|
||||
return value && value.length > 0 ? value : fallback;
|
||||
}
|
||||
|
||||
function readEnvNumber(name: string, fallback: number): number {
|
||||
const raw = Deno.env.get(name);
|
||||
if (raw === undefined || raw.trim() === "") {
|
||||
return fallback;
|
||||
}
|
||||
const parsed = Number(raw);
|
||||
if (!Number.isFinite(parsed) || parsed < 0) {
|
||||
throw new Error(`${name} must be a non-negative number, got '${raw}'`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function nowMs(): number {
|
||||
return performance.now();
|
||||
}
|
||||
|
||||
async function commandOutput(command: string, args: string[]): Promise<string> {
|
||||
const output = await new Deno.Command(command, {
|
||||
args,
|
||||
stdin: "null",
|
||||
stdout: "piped",
|
||||
stderr: "piped",
|
||||
}).output();
|
||||
const stdout = new TextDecoder().decode(output.stdout);
|
||||
const stderr = new TextDecoder().decode(output.stderr);
|
||||
if (!output.success) {
|
||||
throw new Error(`${command} ${args.join(" ")} failed\nstdout: ${stdout}\nstderr: ${stderr}`);
|
||||
}
|
||||
return stdout.trim();
|
||||
}
|
||||
|
||||
async function commandOk(command: string, args: string[]): Promise<void> {
|
||||
await commandOutput(command, args);
|
||||
}
|
||||
|
||||
async function applyNetemIfRequested(): Promise<NetemSummary> {
|
||||
const enabled = readEnvString("BENCH_NETEM_ENABLED", "0") === "1";
|
||||
const profile = readEnvString("NETEM_PROFILE", "home-wifi");
|
||||
const iface = readEnvString("NETEM_INTERFACE", "eth0");
|
||||
const delayMs = readEnvNumber("NETEM_DELAY_MS", 20);
|
||||
const jitterMs = readEnvNumber("NETEM_JITTER_MS", 5);
|
||||
const lossPercent = readEnvNumber("NETEM_LOSS_PERCENT", 0.1);
|
||||
const bandwidthMbit = readEnvNumber("NETEM_BANDWIDTH_MBIT", 100);
|
||||
const mtu = readEnvNumber("NETEM_MTU", 1500);
|
||||
|
||||
const summary: NetemSummary = {
|
||||
enabled,
|
||||
profile,
|
||||
interface: iface,
|
||||
delayMs,
|
||||
jitterMs,
|
||||
lossPercent,
|
||||
bandwidthMbit,
|
||||
mtu,
|
||||
};
|
||||
|
||||
if (!enabled) {
|
||||
return summary;
|
||||
}
|
||||
|
||||
await commandOk("ip", ["link", "set", "dev", iface, "mtu", String(mtu)]);
|
||||
await new Deno.Command("tc", { args: ["qdisc", "del", "dev", iface, "root"] }).output();
|
||||
await commandOk("tc", [
|
||||
"qdisc",
|
||||
"add",
|
||||
"dev",
|
||||
iface,
|
||||
"root",
|
||||
"netem",
|
||||
"delay",
|
||||
`${delayMs}ms`,
|
||||
`${jitterMs}ms`,
|
||||
"loss",
|
||||
`${lossPercent}%`,
|
||||
"rate",
|
||||
`${bandwidthMbit}mbit`,
|
||||
]);
|
||||
summary.tcQdisc = await commandOutput("tc", ["qdisc", "show", "dev", iface]);
|
||||
summary.ipAddr = await commandOutput("ip", ["addr", "show", iface]);
|
||||
summary.ipRoute = await commandOutput("ip", ["route"]);
|
||||
return summary;
|
||||
}
|
||||
|
||||
async function waitForFile(path: string, timeoutMs: number): Promise<void> {
|
||||
const started = Date.now();
|
||||
while (Date.now() - started < timeoutMs) {
|
||||
try {
|
||||
const stat = await Deno.stat(path);
|
||||
if (stat.isFile) {
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
// wait
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for ${path}`);
|
||||
}
|
||||
|
||||
async function readJsonFile<T>(path: string): Promise<T> {
|
||||
return JSON.parse(await Deno.readTextFile(path)) as T;
|
||||
}
|
||||
|
||||
function pickSampleFiles(entries: DatasetEntry[]): DatasetEntry[] {
|
||||
const unique = new Map<string, DatasetEntry>();
|
||||
for (const entry of [entries.find((e) => e.kind === "md"), entries.find((e) => e.kind === "bin"), entries.at(-1)]) {
|
||||
if (entry) {
|
||||
unique.set(entry.relativePath, entry);
|
||||
}
|
||||
}
|
||||
return [...unique.values()];
|
||||
}
|
||||
|
||||
async function readLatestP2PConnectionStats(path: string): Promise<P2PConnectionStats | undefined> {
|
||||
try {
|
||||
const lines = (await Deno.readTextFile(path))
|
||||
.split(/\r?\n/)
|
||||
.map((line) => line.trim())
|
||||
.filter((line) => line.length > 0);
|
||||
return lines.length === 0 ? undefined : (JSON.parse(lines.at(-1)!) as P2PConnectionStats);
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
function buildCommonConfig() {
|
||||
const runId = readEnvString("BENCH_SPLIT_RUN_ID", readEnvString("BENCH_ROOM_ID", "bench-split-run"));
|
||||
const baseWorkRoot = readEnvString("BENCH_SPLIT_WORK_ROOT", "/p2p-work");
|
||||
return {
|
||||
runId,
|
||||
workRoot: join(baseWorkRoot, runId),
|
||||
resultRoot: readEnvString("BENCH_SPLIT_RESULT_ROOT", "/workspace/src/apps/cli/testdeno/bench-results"),
|
||||
relay: readEnvString("BENCH_RELAY", "ws://nostr-relay:7777/"),
|
||||
appId: readEnvString("BENCH_APP_ID", "self-hosted-livesync-cli-benchmark"),
|
||||
roomId: readEnvString("BENCH_ROOM_ID", "bench-split-room"),
|
||||
passphrase: readEnvString("BENCH_PASSPHRASE", "bench-split-passphrase"),
|
||||
turnServers: readEnvString("BENCH_TURN_SERVERS", ""),
|
||||
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
|
||||
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
|
||||
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 20)),
|
||||
mdMinSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 512)),
|
||||
mdMaxSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 2048)),
|
||||
binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 5)),
|
||||
binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 8192)),
|
||||
peersTimeoutSeconds: readEnvNumber("BENCH_PEERS_TIMEOUT", 60),
|
||||
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 300),
|
||||
nodeTimeoutMs: readEnvNumber("BENCH_SPLIT_NODE_TIMEOUT_MS", 360_000),
|
||||
profile: readEnvString("BENCH_NETWORK_PROFILE", readEnvString("NETEM_PROFILE", "split-compose")),
|
||||
};
|
||||
}
|
||||
|
||||
async function prepareP2PSettings(
|
||||
settingsPath: string,
|
||||
peerName: string,
|
||||
config: ReturnType<typeof buildCommonConfig>
|
||||
) {
|
||||
await initSettingsFile(settingsPath);
|
||||
await applyP2pSettings(
|
||||
settingsPath,
|
||||
config.roomId,
|
||||
config.passphrase,
|
||||
config.appId,
|
||||
config.relay,
|
||||
"~.*",
|
||||
config.turnServers
|
||||
);
|
||||
await applyP2pTestTweaks(settingsPath, peerName, config.passphrase);
|
||||
}
|
||||
|
||||
async function runHost(): Promise<void> {
|
||||
const config = buildCommonConfig();
|
||||
const netem = await applyNetemIfRequested();
|
||||
await Deno.mkdir(config.workRoot, { recursive: true });
|
||||
await Deno.mkdir(config.resultRoot, { recursive: true });
|
||||
|
||||
const hostVault = join(config.workRoot, "vault-host");
|
||||
const hostSettings = join(config.workRoot, "settings-host.json");
|
||||
await Deno.mkdir(hostVault, { recursive: true });
|
||||
await prepareP2PSettings(hostSettings, "p2p-split-host", config);
|
||||
|
||||
const seedFiles = await createDeterministicDataset({
|
||||
rootDir: hostVault,
|
||||
datasetDirName: config.datasetDirName,
|
||||
seed: config.datasetSeed,
|
||||
mdCount: config.mdFileCount,
|
||||
mdMinSizeBytes: config.mdMinSizeBytes,
|
||||
mdMaxSizeBytes: config.mdMaxSizeBytes,
|
||||
binCount: config.binFileCount,
|
||||
binSizeBytes: config.binSizeBytes,
|
||||
});
|
||||
await Deno.writeTextFile(
|
||||
join(config.workRoot, "sample-files.json"),
|
||||
JSON.stringify(pickSampleFiles(seedFiles.entries), null, 2)
|
||||
);
|
||||
|
||||
const mirrorStart = nowMs();
|
||||
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
|
||||
const mirrorElapsedMs = Number((nowMs() - mirrorStart).toFixed(1));
|
||||
const hostReady: HostReady = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
totalFiles: seedFiles.totalFiles,
|
||||
totalBytes: seedFiles.totalBytes,
|
||||
mdFileCount: seedFiles.mdCount,
|
||||
binFileCount: seedFiles.binCount,
|
||||
mirrorElapsedMs,
|
||||
netem,
|
||||
};
|
||||
await Deno.writeTextFile(join(config.workRoot, "host-ready.json"), JSON.stringify(hostReady, null, 2));
|
||||
|
||||
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
|
||||
try {
|
||||
await host.waitUntilContains("P2P host is running", 20_000);
|
||||
await Deno.writeTextFile(
|
||||
join(config.workRoot, "p2p-host-ready.json"),
|
||||
JSON.stringify({ generatedAt: new Date().toISOString() })
|
||||
);
|
||||
await waitForFile(join(config.workRoot, "client-done.json"), config.nodeTimeoutMs);
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
}
|
||||
|
||||
async function runClient(): Promise<void> {
|
||||
const config = buildCommonConfig();
|
||||
const netem = await applyNetemIfRequested();
|
||||
await Deno.mkdir(config.resultRoot, { recursive: true });
|
||||
await waitForFile(join(config.workRoot, "host-ready.json"), config.nodeTimeoutMs);
|
||||
await waitForFile(join(config.workRoot, "p2p-host-ready.json"), config.nodeTimeoutMs);
|
||||
|
||||
const clientVault = join(config.workRoot, "vault-client");
|
||||
const clientSettings = join(config.workRoot, "settings-client.json");
|
||||
const statsPath = join(config.workRoot, "p2p-connection-stats.jsonl");
|
||||
await Deno.mkdir(clientVault, { recursive: true });
|
||||
await prepareP2PSettings(clientSettings, "p2p-split-client", config);
|
||||
|
||||
const hostReady = await readJsonFile<HostReady>(join(config.workRoot, "host-ready.json"));
|
||||
const timestamp = new Date().toISOString().replace(/[-:]/g, "").slice(0, 15);
|
||||
const outputDir = join(config.resultRoot, `p2p-split-${config.profile}-${timestamp}`);
|
||||
await Deno.mkdir(outputDir, { recursive: true });
|
||||
|
||||
const previousStatsPath = Deno.env.get("LIVESYNC_P2P_STATS_JSONL");
|
||||
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", statsPath);
|
||||
let stage = "peer-discovery";
|
||||
let peerDiscoveryCommandElapsedMs: number | undefined;
|
||||
let syncElapsedMs: number | undefined;
|
||||
try {
|
||||
const peerDiscoveryCommandStart = nowMs();
|
||||
const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
|
||||
peerDiscoveryCommandElapsedMs = Number((nowMs() - peerDiscoveryCommandStart).toFixed(1));
|
||||
|
||||
stage = "p2p-sync";
|
||||
const syncStart = nowMs();
|
||||
await runCliOrFail(
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"p2p-sync",
|
||||
peer.id,
|
||||
String(config.syncTimeoutSeconds)
|
||||
);
|
||||
syncElapsedMs = Number((nowMs() - syncStart).toFixed(1));
|
||||
|
||||
stage = "sample-verification";
|
||||
const samples = await readJsonFile<DatasetEntry[]>(join(config.workRoot, "sample-files.json"));
|
||||
for (const sample of samples) {
|
||||
const pulledPath = join(config.workRoot, `pulled-${sample.relativePath.replaceAll("/", "_")}`);
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "pull", sample.relativePath, pulledPath);
|
||||
await assertFilesEqual(
|
||||
sample.absolutePath,
|
||||
pulledPath,
|
||||
`sample file mismatch after split sync: ${sample.relativePath}`
|
||||
);
|
||||
}
|
||||
|
||||
const p2pConnectionStats = await readLatestP2PConnectionStats(statsPath);
|
||||
const result = {
|
||||
ok: true,
|
||||
generatedAt: new Date().toISOString(),
|
||||
caseName: "p2p-split-compose",
|
||||
mode: "p2p-split-compose-benchmark",
|
||||
runId: config.runId,
|
||||
simulationTier: Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "2" : "1",
|
||||
networkProfile: config.profile,
|
||||
networkModel:
|
||||
Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "split-compose-egress-netem" : "split-compose-no-netem",
|
||||
relay: config.relay,
|
||||
turnServers: config.turnServers,
|
||||
turnEnabled: config.turnServers.trim().length > 0,
|
||||
p2pCandidatePathVerified: p2pConnectionStats?.candidatePathCollected === true,
|
||||
p2pConnectionStats,
|
||||
hostNetem: hostReady.netem,
|
||||
clientNetem: netem,
|
||||
totalFiles: hostReady.totalFiles,
|
||||
totalBytes: hostReady.totalBytes,
|
||||
mdFileCount: hostReady.mdFileCount,
|
||||
binFileCount: hostReady.binFileCount,
|
||||
mirrorElapsedMs: hostReady.mirrorElapsedMs,
|
||||
peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds,
|
||||
peerDiscoveryCommandElapsedMs,
|
||||
syncElapsedMs,
|
||||
throughputBytesPerSec: Number((hostReady.totalBytes / (syncElapsedMs / 1000)).toFixed(2)),
|
||||
throughputMiBPerSec: Number((hostReady.totalBytes / (syncElapsedMs / 1000) / 1024 / 1024).toFixed(4)),
|
||||
};
|
||||
await Deno.writeTextFile(join(outputDir, "summary.json"), JSON.stringify(result, null, 2));
|
||||
await Deno.writeTextFile(
|
||||
join(config.workRoot, "client-done.json"),
|
||||
JSON.stringify({ generatedAt: new Date().toISOString(), outputDir, ok: true })
|
||||
);
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} catch (error) {
|
||||
const p2pConnectionStats = await readLatestP2PConnectionStats(statsPath);
|
||||
const result = {
|
||||
ok: false,
|
||||
generatedAt: new Date().toISOString(),
|
||||
caseName: "p2p-split-compose",
|
||||
mode: "p2p-split-compose-benchmark",
|
||||
runId: config.runId,
|
||||
simulationTier: Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "2" : "1",
|
||||
networkProfile: config.profile,
|
||||
networkModel:
|
||||
Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "split-compose-egress-netem" : "split-compose-no-netem",
|
||||
relay: config.relay,
|
||||
turnServers: config.turnServers,
|
||||
turnEnabled: config.turnServers.trim().length > 0,
|
||||
p2pCandidatePathVerified: p2pConnectionStats?.candidatePathCollected === true,
|
||||
p2pConnectionStats,
|
||||
hostNetem: hostReady.netem,
|
||||
clientNetem: netem,
|
||||
totalFiles: hostReady.totalFiles,
|
||||
totalBytes: hostReady.totalBytes,
|
||||
mdFileCount: hostReady.mdFileCount,
|
||||
binFileCount: hostReady.binFileCount,
|
||||
mirrorElapsedMs: hostReady.mirrorElapsedMs,
|
||||
peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds,
|
||||
peerDiscoveryCommandElapsedMs,
|
||||
syncElapsedMs,
|
||||
failure: {
|
||||
stage,
|
||||
...errorToRecord(error),
|
||||
},
|
||||
};
|
||||
await Deno.writeTextFile(join(outputDir, "summary.json"), JSON.stringify(result, null, 2));
|
||||
await Deno.writeTextFile(
|
||||
join(config.workRoot, "client-done.json"),
|
||||
JSON.stringify({ generatedAt: new Date().toISOString(), outputDir, ok: false })
|
||||
);
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
throw error;
|
||||
} finally {
|
||||
if (previousStatsPath === undefined) {
|
||||
Deno.env.delete("LIVESYNC_P2P_STATS_JSONL");
|
||||
} else {
|
||||
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", previousStatsPath);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const role = readEnvString("BENCH_P2P_SPLIT_ROLE", "") as Role;
|
||||
if (role === "host") {
|
||||
await runHost();
|
||||
return;
|
||||
}
|
||||
if (role === "client") {
|
||||
await runClient();
|
||||
return;
|
||||
}
|
||||
throw new Error("BENCH_P2P_SPLIT_ROLE must be 'host' or 'client'");
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
main().catch((error) => {
|
||||
console.error("[Fatal Error]", error);
|
||||
Deno.exit(1);
|
||||
});
|
||||
}
|
||||
@@ -1,15 +1,23 @@
|
||||
import { TempDir } from "./helpers/temp.ts";
|
||||
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
|
||||
import { startCliInBackground } from "./helpers/backgroundCli.ts";
|
||||
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
|
||||
import {
|
||||
discoverPeer,
|
||||
maybeStartCoturn,
|
||||
maybeStartLocalRelay,
|
||||
stopCoturnIfStarted,
|
||||
stopLocalRelayIfStarted,
|
||||
} from "./helpers/p2p.ts";
|
||||
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
|
||||
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
|
||||
|
||||
type BenchmarkConfig = {
|
||||
caseName: string;
|
||||
relay: string;
|
||||
appId: string;
|
||||
roomId: string;
|
||||
passphrase: string;
|
||||
turnServers: string;
|
||||
datasetDirName: string;
|
||||
datasetSeed: string;
|
||||
mdFileCount: number;
|
||||
@@ -19,6 +27,42 @@ type BenchmarkConfig = {
|
||||
binSizeBytes: number;
|
||||
peersTimeoutSeconds: number;
|
||||
syncTimeoutSeconds: number;
|
||||
simulationTier: string;
|
||||
networkProfile: string;
|
||||
networkModel: string;
|
||||
candidatePathVerification: string;
|
||||
};
|
||||
|
||||
type P2PConnectionStats = {
|
||||
generatedAt: string;
|
||||
command: string;
|
||||
peerId: string;
|
||||
peerName: string;
|
||||
candidatePathCollected: boolean;
|
||||
selectedPath: string;
|
||||
selectedPair?: {
|
||||
id: string;
|
||||
state: string;
|
||||
currentRoundTripTime: number | "unknown";
|
||||
totalRoundTripTime: number | "unknown";
|
||||
requestsSent: number | "unknown";
|
||||
responsesReceived: number | "unknown";
|
||||
packetsDiscardedOnSend: number | "unknown";
|
||||
bytesSent: number | "unknown";
|
||||
bytesReceived: number | "unknown";
|
||||
};
|
||||
localCandidate?: {
|
||||
id: string;
|
||||
candidateType: string;
|
||||
protocol: string;
|
||||
relayProtocol: string;
|
||||
};
|
||||
remoteCandidate?: {
|
||||
id: string;
|
||||
candidateType: string;
|
||||
protocol: string;
|
||||
relayProtocol: string;
|
||||
};
|
||||
};
|
||||
|
||||
function readEnvString(name: string, fallback: string): string {
|
||||
@@ -61,10 +105,12 @@ function formatBytes(value: number): string {
|
||||
|
||||
function buildConfig(): BenchmarkConfig {
|
||||
return {
|
||||
caseName: readEnvString("BENCH_CASE", "p2p-direct-local"),
|
||||
relay: readEnvString("BENCH_RELAY", "ws://localhost:4000/"),
|
||||
appId: readEnvString("BENCH_APP_ID", "self-hosted-livesync-cli-benchmark"),
|
||||
roomId: readEnvString("BENCH_ROOM_ID", `bench-room-${Date.now()}`),
|
||||
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
|
||||
turnServers: readEnvString("BENCH_TURN_SERVERS", ""),
|
||||
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
|
||||
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
|
||||
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)),
|
||||
@@ -74,6 +120,10 @@ function buildConfig(): BenchmarkConfig {
|
||||
binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024)),
|
||||
peersTimeoutSeconds: readEnvNumber("BENCH_PEERS_TIMEOUT", 20),
|
||||
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240),
|
||||
simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"),
|
||||
networkProfile: readEnvString("BENCH_NETWORK_PROFILE", "local-direct"),
|
||||
networkModel: readEnvString("BENCH_NETWORK_MODEL", "local-runner-webrtc"),
|
||||
candidatePathVerification: readEnvString("BENCH_P2P_CANDIDATE_PATH_VERIFICATION", "not-collected"),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -102,115 +152,182 @@ function pickSampleFiles(entries: DatasetEntry[]): DatasetEntry[] {
|
||||
return [...unique.values()];
|
||||
}
|
||||
|
||||
async function readLatestP2PConnectionStats(statsPath: string): Promise<P2PConnectionStats | undefined> {
|
||||
try {
|
||||
const text = await Deno.readTextFile(statsPath);
|
||||
const lines = text
|
||||
.split(/\r?\n/)
|
||||
.map((line) => line.trim())
|
||||
.filter((line) => line.length > 0);
|
||||
if (lines.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
return JSON.parse(lines[lines.length - 1]) as P2PConnectionStats;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const config = buildConfig();
|
||||
const resultPath = readOptionalResultPath();
|
||||
|
||||
const relayStarted = await maybeStartLocalRelay(config.relay);
|
||||
const coturnStarted = await maybeStartCoturn(config.turnServers);
|
||||
await using workDir = await TempDir.create("livesync-cli-p2p-bench");
|
||||
|
||||
const hostVault = workDir.join("vault-host");
|
||||
const clientVault = workDir.join("vault-client");
|
||||
const hostSettings = workDir.join("settings-host.json");
|
||||
const clientSettings = workDir.join("settings-client.json");
|
||||
const p2pStatsPath = workDir.join("p2p-connection-stats.jsonl");
|
||||
const previousStatsPath = Deno.env.get("LIVESYNC_P2P_STATS_JSONL");
|
||||
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", p2pStatsPath);
|
||||
|
||||
await Promise.all([
|
||||
Deno.mkdir(hostVault, { recursive: true }),
|
||||
Deno.mkdir(clientVault, { recursive: true }),
|
||||
initSettingsFile(hostSettings),
|
||||
initSettingsFile(clientSettings),
|
||||
]);
|
||||
|
||||
await Promise.all([
|
||||
applyP2pSettings(hostSettings, config.roomId, config.passphrase, config.appId, config.relay, "~.*"),
|
||||
applyP2pSettings(clientSettings, config.roomId, config.passphrase, config.appId, config.relay, "~.*"),
|
||||
]);
|
||||
|
||||
await Promise.all([
|
||||
applyP2pTestTweaks(hostSettings, "p2p-bench-host", config.passphrase),
|
||||
applyP2pTestTweaks(clientSettings, "p2p-bench-client", config.passphrase),
|
||||
]);
|
||||
|
||||
const seedFiles = await createDeterministicDataset({
|
||||
rootDir: hostVault,
|
||||
datasetDirName: config.datasetDirName,
|
||||
seed: config.datasetSeed,
|
||||
mdCount: config.mdFileCount,
|
||||
mdMinSizeBytes: config.mdMinSizeBytes,
|
||||
mdMaxSizeBytes: config.mdMaxSizeBytes,
|
||||
binCount: config.binFileCount,
|
||||
binSizeBytes: config.binSizeBytes,
|
||||
});
|
||||
|
||||
const mirrorStart = nowMs();
|
||||
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
|
||||
const mirrorElapsed = nowMs() - mirrorStart;
|
||||
|
||||
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
|
||||
try {
|
||||
const hostReadyStart = nowMs();
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const hostReadyElapsed = nowMs() - hostReadyStart;
|
||||
await Promise.all([
|
||||
Deno.mkdir(hostVault, { recursive: true }),
|
||||
Deno.mkdir(clientVault, { recursive: true }),
|
||||
initSettingsFile(hostSettings),
|
||||
initSettingsFile(clientSettings),
|
||||
]);
|
||||
|
||||
const peerDiscoveryStart = nowMs();
|
||||
const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
|
||||
const peerDiscoveryElapsed = nowMs() - peerDiscoveryStart;
|
||||
await Promise.all([
|
||||
applyP2pSettings(
|
||||
hostSettings,
|
||||
config.roomId,
|
||||
config.passphrase,
|
||||
config.appId,
|
||||
config.relay,
|
||||
"~.*",
|
||||
config.turnServers
|
||||
),
|
||||
applyP2pSettings(
|
||||
clientSettings,
|
||||
config.roomId,
|
||||
config.passphrase,
|
||||
config.appId,
|
||||
config.relay,
|
||||
"~.*",
|
||||
config.turnServers
|
||||
),
|
||||
]);
|
||||
|
||||
const syncStart = nowMs();
|
||||
await runCliOrFail(
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"p2p-sync",
|
||||
peer.id,
|
||||
String(config.syncTimeoutSeconds)
|
||||
);
|
||||
const syncElapsed = nowMs() - syncStart;
|
||||
await Promise.all([
|
||||
applyP2pTestTweaks(hostSettings, "p2p-bench-host", config.passphrase),
|
||||
applyP2pTestTweaks(clientSettings, "p2p-bench-client", config.passphrase),
|
||||
]);
|
||||
|
||||
const sampleFiles = pickSampleFiles(seedFiles.entries);
|
||||
for (const sample of sampleFiles) {
|
||||
const pulledPath = workDir.join(`pulled-${sample.relativePath.replaceAll("/", "_")}`);
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "pull", sample.relativePath, pulledPath);
|
||||
await assertFilesEqual(
|
||||
sample.absolutePath,
|
||||
pulledPath,
|
||||
`sample file mismatch after sync: ${sample.relativePath}`
|
||||
);
|
||||
}
|
||||
|
||||
const result = {
|
||||
mode: "p2p-cli-benchmark",
|
||||
relay: config.relay,
|
||||
appId: config.appId,
|
||||
roomId: config.roomId,
|
||||
datasetSeed: config.datasetSeed,
|
||||
const seedFiles = await createDeterministicDataset({
|
||||
rootDir: hostVault,
|
||||
datasetDirName: config.datasetDirName,
|
||||
peerId: peer.id,
|
||||
peerName: peer.name,
|
||||
totalFiles: seedFiles.totalFiles,
|
||||
totalBytes: seedFiles.totalBytes,
|
||||
mdFileCount: seedFiles.mdCount,
|
||||
binFileCount: seedFiles.binCount,
|
||||
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
|
||||
hostReadyElapsedMs: Number(hostReadyElapsed.toFixed(1)),
|
||||
peerDiscoveryElapsedMs: Number(peerDiscoveryElapsed.toFixed(1)),
|
||||
syncElapsedMs: Number(syncElapsed.toFixed(1)),
|
||||
throughputBytesPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000)).toFixed(2)),
|
||||
throughputMiBPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000) / 1024 / 1024).toFixed(4)),
|
||||
};
|
||||
seed: config.datasetSeed,
|
||||
mdCount: config.mdFileCount,
|
||||
mdMinSizeBytes: config.mdMinSizeBytes,
|
||||
mdMaxSizeBytes: config.mdMaxSizeBytes,
|
||||
binCount: config.binFileCount,
|
||||
binSizeBytes: config.binSizeBytes,
|
||||
});
|
||||
|
||||
if (resultPath) {
|
||||
await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
|
||||
const mirrorStart = nowMs();
|
||||
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
|
||||
const mirrorElapsed = nowMs() - mirrorStart;
|
||||
|
||||
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
|
||||
try {
|
||||
const hostReadyStart = nowMs();
|
||||
await host.waitUntilContains("P2P host is running", 20000);
|
||||
const hostReadyElapsed = nowMs() - hostReadyStart;
|
||||
|
||||
const peerDiscoveryCommandStart = nowMs();
|
||||
const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
|
||||
const peerDiscoveryCommandElapsed = nowMs() - peerDiscoveryCommandStart;
|
||||
|
||||
const syncStart = nowMs();
|
||||
await runCliOrFail(
|
||||
clientVault,
|
||||
"--settings",
|
||||
clientSettings,
|
||||
"p2p-sync",
|
||||
peer.id,
|
||||
String(config.syncTimeoutSeconds)
|
||||
);
|
||||
const syncElapsed = nowMs() - syncStart;
|
||||
|
||||
const sampleFiles = pickSampleFiles(seedFiles.entries);
|
||||
for (const sample of sampleFiles) {
|
||||
const pulledPath = workDir.join(`pulled-${sample.relativePath.replaceAll("/", "_")}`);
|
||||
await runCliOrFail(clientVault, "--settings", clientSettings, "pull", sample.relativePath, pulledPath);
|
||||
await assertFilesEqual(
|
||||
sample.absolutePath,
|
||||
pulledPath,
|
||||
`sample file mismatch after sync: ${sample.relativePath}`
|
||||
);
|
||||
}
|
||||
|
||||
const p2pConnectionStats = await readLatestP2PConnectionStats(p2pStatsPath);
|
||||
const result = {
|
||||
caseName: config.caseName,
|
||||
mode: "p2p-cli-benchmark",
|
||||
relay: config.relay,
|
||||
turnServers: config.turnServers,
|
||||
turnEnabled: config.turnServers.trim().length > 0,
|
||||
simulationTier: config.simulationTier,
|
||||
networkProfile: config.networkProfile,
|
||||
networkModel: config.networkModel,
|
||||
p2pCandidatePathVerified: p2pConnectionStats?.candidatePathCollected === true,
|
||||
p2pCandidatePathVerification: p2pConnectionStats?.candidatePathCollected
|
||||
? "selected ICE candidate pair collected from RTCPeerConnection.getStats"
|
||||
: config.candidatePathVerification,
|
||||
p2pCandidatePathNote: p2pConnectionStats?.candidatePathCollected
|
||||
? "The selected ICE candidate pair was collected by the CLI benchmark. Interpret the path from the candidate types; do not infer TURN use from configuration alone."
|
||||
: config.turnServers.trim().length > 0
|
||||
? "TURN is configured, so the selected WebRTC path may be direct, server-reflexive, or relayed. The selected ICE candidate pair was not exported by this run."
|
||||
: "TURN is disabled, so a TURN-relayed path is not expected. The selected ICE candidate pair was not exported by this run.",
|
||||
p2pConnectionStats,
|
||||
appId: config.appId,
|
||||
roomId: config.roomId,
|
||||
datasetSeed: config.datasetSeed,
|
||||
datasetDirName: config.datasetDirName,
|
||||
peerId: peer.id,
|
||||
peerName: peer.name,
|
||||
totalFiles: seedFiles.totalFiles,
|
||||
totalBytes: seedFiles.totalBytes,
|
||||
mdFileCount: seedFiles.mdCount,
|
||||
binFileCount: seedFiles.binCount,
|
||||
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
|
||||
hostReadyElapsedMs: Number(hostReadyElapsed.toFixed(1)),
|
||||
peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds,
|
||||
peerDiscoveryCommandElapsedMs: Number(peerDiscoveryCommandElapsed.toFixed(1)),
|
||||
peerDiscoveryNote:
|
||||
"p2p-peers waits for the requested timeout before printing discovered peers, so this is command duration, not first-peer latency.",
|
||||
syncElapsedMs: Number(syncElapsed.toFixed(1)),
|
||||
throughputBytesPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000)).toFixed(2)),
|
||||
throughputMiBPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000) / 1024 / 1024).toFixed(4)),
|
||||
};
|
||||
|
||||
if (resultPath) {
|
||||
await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
|
||||
}
|
||||
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
console.error(
|
||||
`[Benchmark] mirrored ${seedFiles.totalFiles} files (${formatBytes(
|
||||
seedFiles.totalBytes
|
||||
)}) in ${formatMs(mirrorElapsed)}, ` +
|
||||
`synced in ${formatMs(syncElapsed)} ` +
|
||||
`(${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
|
||||
);
|
||||
} finally {
|
||||
await host.stop();
|
||||
}
|
||||
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
console.error(
|
||||
`[Benchmark] mirrored ${seedFiles.totalFiles} files (${formatBytes(seedFiles.totalBytes)}) in ${formatMs(mirrorElapsed)}, ` +
|
||||
`synced in ${formatMs(syncElapsed)} ` +
|
||||
`(${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
|
||||
);
|
||||
} finally {
|
||||
await host.stop();
|
||||
if (previousStatsPath === undefined) {
|
||||
Deno.env.delete("LIVESYNC_P2P_STATS_JSONL");
|
||||
} else {
|
||||
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", previousStatsPath);
|
||||
}
|
||||
await stopCoturnIfStarted(coturnStarted);
|
||||
await stopLocalRelayIfStarted(relayStarted);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,6 +17,9 @@
|
||||
"test:p2p-upload-download": "deno test --env-file=.test.env -A --no-check test-p2p-upload-download-repro.ts",
|
||||
"bench:p2p": "deno run --env-file=.test.env -A --no-check bench-p2p.ts",
|
||||
"bench:couchdb": "deno run --env-file=.test.env -A --no-check bench-couchdb.ts",
|
||||
"bench:cases": "deno run --env-file=.test.env -A --no-check bench-network-cases.ts",
|
||||
"bench:latency-sweep": "deno run --env-file=.test.env -A --no-check bench-latency-sweep.ts",
|
||||
"bench:p2p-split-node": "deno run --env-file=.test.env -A --no-check bench-p2p-split-node.ts",
|
||||
"bench:item1": "bash ./bench-run-item1.sh",
|
||||
"bench:item1:full": "BENCH_MD_FILE_COUNT=1500 BENCH_MD_MIN_SIZE_BYTES=1024 BENCH_MD_MAX_SIZE_BYTES=20480 BENCH_BIN_FILE_COUNT=500 BENCH_BIN_SIZE_BYTES=102400 BENCH_COUCHDB_RTT_MS=50 bash ./bench-run-item1.sh",
|
||||
"test:e2e-couchdb": "deno test --env-file=.test.env -A --no-check test-e2e-two-vaults-couchdb.ts",
|
||||
|
||||
@@ -9,7 +9,7 @@ function sleep(ms: number): Promise<void> {
|
||||
}
|
||||
|
||||
async function connectWithTimeout(hostname: string, port: number, timeoutMs: number): Promise<void> {
|
||||
let timer: number | undefined;
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
try {
|
||||
const connPromise = Deno.connect({ hostname, port });
|
||||
const timeoutPromise = new Promise<never>((_, reject) => {
|
||||
|
||||
@@ -76,8 +76,10 @@ export async function discoverPeer(
|
||||
}
|
||||
|
||||
export async function maybeStartLocalRelay(relay: string): Promise<boolean> {
|
||||
if (!isLocalP2pRelay(relay)) return false;
|
||||
await startP2pRelay();
|
||||
const shouldStart = isLocalP2pRelay(relay);
|
||||
if (shouldStart) {
|
||||
await startP2pRelay();
|
||||
}
|
||||
const endpoint = parseRelayEndpoint(relay);
|
||||
await waitForPort(endpoint.hostname, endpoint.port, {
|
||||
timeoutMs: Number(Deno.env.get("LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS") ?? "15000"),
|
||||
@@ -86,8 +88,10 @@ export async function maybeStartLocalRelay(relay: string): Promise<boolean> {
|
||||
});
|
||||
// Docker proxy accepts TCP connections instantly before the container's internal process is fully ready.
|
||||
// Wait an additional few seconds to ensure strfry is actually accepting WebSockets.
|
||||
await sleep(3000);
|
||||
return true;
|
||||
if (shouldStart) {
|
||||
await sleep(3000);
|
||||
}
|
||||
return shouldStart;
|
||||
}
|
||||
|
||||
export async function stopLocalRelayIfStarted(started: boolean): Promise<void> {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "livesync-webapp",
|
||||
"private": true,
|
||||
"version": "0.25.79-webapp",
|
||||
"version": "0.25.80-webapp",
|
||||
"type": "module",
|
||||
"description": "Browser-based Self-hosted LiveSync using FileSystem API",
|
||||
"scripts": {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "webpeer",
|
||||
"private": true,
|
||||
"version": "0.25.79-webpeer",
|
||||
"version": "0.25.80-webpeer",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
@@ -51,6 +51,7 @@ import { EVENT_SETTING_SAVED, eventHub } from "@/common/events.ts";
|
||||
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
|
||||
import type { LiveSyncCore } from "@/main.ts";
|
||||
import { tryGetFilePath } from "@lib/common/utils.doc.ts";
|
||||
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
|
||||
type SyncDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce";
|
||||
|
||||
declare global {
|
||||
@@ -1832,43 +1833,35 @@ ${messageFetch}${messageOverwrite}${messageMerge}
|
||||
}
|
||||
|
||||
async configureHiddenFileSync(mode: keyof OPTIONAL_SYNC_FEATURES) {
|
||||
if (
|
||||
mode != "FETCH" &&
|
||||
mode != "OVERWRITE" &&
|
||||
mode != "MERGE" &&
|
||||
mode != "DISABLE" &&
|
||||
mode != "DISABLE_HIDDEN"
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (mode == "DISABLE" || mode == "DISABLE_HIDDEN") {
|
||||
// await this.core.$allSuspendExtraSync();
|
||||
await this.core.services.setting.applyPartial(
|
||||
{
|
||||
syncInternalFiles: false,
|
||||
},
|
||||
true
|
||||
);
|
||||
// this.core.settings.syncInternalFiles = false;
|
||||
// await this.core.saveSettings();
|
||||
return;
|
||||
}
|
||||
this._log("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
|
||||
if (mode == "FETCH") {
|
||||
await this.initialiseInternalFileSync("pullForce", true);
|
||||
} else if (mode == "OVERWRITE") {
|
||||
await this.initialiseInternalFileSync("pushForce", true);
|
||||
} else if (mode == "MERGE") {
|
||||
await this.initialiseInternalFileSync("safe", true);
|
||||
}
|
||||
await this.core.services.setting.applyPartial(
|
||||
{
|
||||
useAdvancedMode: true,
|
||||
syncInternalFiles: true,
|
||||
const result = await configureHiddenFileSyncMode(mode, {
|
||||
disable: async () => {
|
||||
// await this.core.$allSuspendExtraSync();
|
||||
await this.core.services.setting.applyPartial(
|
||||
{
|
||||
syncInternalFiles: false,
|
||||
},
|
||||
true
|
||||
);
|
||||
// this.core.settings.syncInternalFiles = false;
|
||||
// await this.core.saveSettings();
|
||||
},
|
||||
true
|
||||
);
|
||||
enable: async () => {
|
||||
this._log("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
|
||||
await this.core.services.setting.applyPartial(
|
||||
{
|
||||
useAdvancedMode: true,
|
||||
syncInternalFiles: true,
|
||||
},
|
||||
true
|
||||
);
|
||||
},
|
||||
initialise: async (direction) => {
|
||||
await this.initialiseInternalFileSync(direction, true);
|
||||
},
|
||||
});
|
||||
if (result == "ignored" || result == "disabled") {
|
||||
return;
|
||||
}
|
||||
// this.plugin.settings.useAdvancedMode = true;
|
||||
// this.plugin.settings.syncInternalFiles = true;
|
||||
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
type HiddenFileSyncMode = "FETCH" | "OVERWRITE" | "MERGE" | "DISABLE" | "DISABLE_HIDDEN";
|
||||
type HiddenFileSyncDirection = "pullForce" | "pushForce" | "safe";
|
||||
|
||||
type ConfigureHiddenFileSyncHandlers = {
|
||||
disable: () => Promise<void>;
|
||||
enable: () => Promise<void>;
|
||||
initialise: (direction: HiddenFileSyncDirection) => Promise<void>;
|
||||
};
|
||||
|
||||
export type ConfigureHiddenFileSyncResult = "ignored" | "disabled" | "enabled";
|
||||
|
||||
function getInitialiseDirection(mode: keyof OPTIONAL_SYNC_FEATURES): HiddenFileSyncDirection | false {
|
||||
if (mode == "FETCH") return "pullForce";
|
||||
if (mode == "OVERWRITE") return "pushForce";
|
||||
if (mode == "MERGE") return "safe";
|
||||
return false;
|
||||
}
|
||||
|
||||
function isDisableMode(mode: keyof OPTIONAL_SYNC_FEATURES): boolean {
|
||||
return mode == "DISABLE" || mode == "DISABLE_HIDDEN";
|
||||
}
|
||||
|
||||
function isHiddenFileSyncMode(mode: keyof OPTIONAL_SYNC_FEATURES): mode is HiddenFileSyncMode {
|
||||
return mode == "FETCH" || mode == "OVERWRITE" || mode == "MERGE" || isDisableMode(mode);
|
||||
}
|
||||
|
||||
export async function configureHiddenFileSyncMode(
|
||||
mode: keyof OPTIONAL_SYNC_FEATURES,
|
||||
handlers: ConfigureHiddenFileSyncHandlers
|
||||
): Promise<ConfigureHiddenFileSyncResult> {
|
||||
if (!isHiddenFileSyncMode(mode)) {
|
||||
return "ignored";
|
||||
}
|
||||
if (isDisableMode(mode)) {
|
||||
await handlers.disable();
|
||||
return "disabled";
|
||||
}
|
||||
const direction = getInitialiseDirection(mode);
|
||||
if (direction === false) {
|
||||
return "ignored";
|
||||
}
|
||||
await handlers.enable();
|
||||
await handlers.initialise(direction);
|
||||
return "enabled";
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
|
||||
|
||||
describe("configureHiddenFileSyncMode", () => {
|
||||
it.each([
|
||||
["FETCH", "pullForce"],
|
||||
["OVERWRITE", "pushForce"],
|
||||
["MERGE", "safe"],
|
||||
] as const)("enables hidden file sync before initialising %s", async (mode, direction) => {
|
||||
const calls: string[] = [];
|
||||
|
||||
const result = await configureHiddenFileSyncMode(mode, {
|
||||
disable: vi.fn(async () => {
|
||||
calls.push("disable");
|
||||
}),
|
||||
enable: vi.fn(async () => {
|
||||
calls.push("enable");
|
||||
}),
|
||||
initialise: vi.fn(async (actualDirection) => {
|
||||
calls.push(`init:${actualDirection}`);
|
||||
}),
|
||||
});
|
||||
|
||||
expect(result).toBe("enabled");
|
||||
expect(calls).toEqual(["enable", `init:${direction}`]);
|
||||
});
|
||||
|
||||
it.each(["DISABLE", "DISABLE_HIDDEN"] as const)("disables hidden file sync immediately for %s", async (mode) => {
|
||||
const calls: string[] = [];
|
||||
|
||||
const result = await configureHiddenFileSyncMode(mode, {
|
||||
disable: vi.fn(async () => {
|
||||
calls.push("disable");
|
||||
}),
|
||||
enable: vi.fn(async () => {
|
||||
calls.push("enable");
|
||||
}),
|
||||
initialise: vi.fn(async (direction) => {
|
||||
calls.push(`init:${direction}`);
|
||||
}),
|
||||
});
|
||||
|
||||
expect(result).toBe("disabled");
|
||||
expect(calls).toEqual(["disable"]);
|
||||
});
|
||||
});
|
||||
+1
-1
Submodule src/lib updated: c7c5fe21be...a0efb7274e
@@ -8,13 +8,7 @@ import {
|
||||
type ValueComponent,
|
||||
} from "@/deps.ts";
|
||||
import { unique } from "octagonal-wheels/collection";
|
||||
import {
|
||||
LEVEL_ADVANCED,
|
||||
LEVEL_POWER_USER,
|
||||
statusDisplay,
|
||||
type ConfigurationItem,
|
||||
type SettingDefinition,
|
||||
} from "@lib/common/types.ts";
|
||||
import { LEVEL_ADVANCED, LEVEL_POWER_USER, statusDisplay, type ConfigurationItem } from "@lib/common/types.ts";
|
||||
import { type ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import {
|
||||
type AllSettingItemKey,
|
||||
@@ -24,21 +18,9 @@ import {
|
||||
type AllNumericItemKey,
|
||||
type AllBooleanItemKey,
|
||||
} from "./settingConstants.ts";
|
||||
import { $msg, $t } from "@lib/common/i18n.ts";
|
||||
import { $msg } from "@lib/common/i18n.ts";
|
||||
import { wrapMemo, type AutoWireOption, type OnUpdateResult } from "./SettingPane.ts";
|
||||
|
||||
function configurationFromDefinition(definition: SettingDefinition<AllSettingItemKey>): ConfigurationItem {
|
||||
return {
|
||||
name: $t(definition.labelKey),
|
||||
desc: definition.descriptionKey ? $t(definition.descriptionKey) : undefined,
|
||||
placeHolder: definition.placeholder,
|
||||
status: definition.status,
|
||||
obsolete: definition.obsolete,
|
||||
level: definition.level,
|
||||
isHidden: definition.secret,
|
||||
};
|
||||
}
|
||||
|
||||
export class LiveSyncSetting extends Setting {
|
||||
autoWiredComponent?: TextComponent | ToggleComponent | DropdownComponent | ButtonComponent | TextAreaComponent;
|
||||
applyButtonComponent?: ButtonComponent;
|
||||
@@ -74,7 +56,7 @@ export class LiveSyncSetting extends Setting {
|
||||
return this;
|
||||
}
|
||||
autoWireSetting(key: AllSettingItemKey, opt?: AutoWireOption) {
|
||||
const conf = opt?.settingDefinition ? configurationFromDefinition(opt.settingDefinition) : getConfig(key);
|
||||
const conf = getConfig(key);
|
||||
if (!conf) {
|
||||
// throw new Error($msg("liveSyncSetting.errorNoSuchSettingItem", { key }));
|
||||
return;
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
import {
|
||||
getExplicitSettingCommitGroup,
|
||||
getSettingDefinition,
|
||||
type AllBooleanItemKey,
|
||||
type AllNumericItemKey,
|
||||
type AllSettingItemKey,
|
||||
type AllStringItemKey,
|
||||
} from "./settingConstants.ts";
|
||||
import type { LiveSyncSetting } from "./LiveSyncSetting.ts";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import type { AutoWireOption } from "./SettingPane.ts";
|
||||
|
||||
export type ObsidianSettingRenderOption<TOption extends string = string> = AutoWireOption & {
|
||||
options?: Record<TOption, string>;
|
||||
clampMin?: number;
|
||||
clampMax?: number;
|
||||
acceptZero?: boolean;
|
||||
renderInternal?: boolean;
|
||||
};
|
||||
|
||||
export function addObsidianApplyButton(setting: LiveSyncSetting, group: string, text?: string): LiveSyncSetting {
|
||||
const commitGroup = getExplicitSettingCommitGroup(group);
|
||||
if (!commitGroup) {
|
||||
throw new Error(`No explicit setting commit group found for '${group}'`);
|
||||
}
|
||||
return setting.addApplyButton(commitGroup.applyKeys, text);
|
||||
}
|
||||
|
||||
export function renderObsidianApplyButton(containerEl: HTMLElement, group: string, text?: string): LiveSyncSetting {
|
||||
return addObsidianApplyButton(new Setting(containerEl), group, text);
|
||||
}
|
||||
|
||||
export function renderObsidianSetting<TOption extends string = string>(
|
||||
containerEl: HTMLElement,
|
||||
key: AllSettingItemKey,
|
||||
opt: ObsidianSettingRenderOption<TOption> = {}
|
||||
): LiveSyncSetting {
|
||||
const definition = getSettingDefinition(key);
|
||||
if (!definition) {
|
||||
throw new Error(`No setting definition found for '${key}'`);
|
||||
}
|
||||
if (definition.internal && !opt.renderInternal) {
|
||||
throw new Error(`Setting '${key}' is internal and cannot be rendered automatically`);
|
||||
}
|
||||
if (definition.render === "custom" || definition.kind === "custom") {
|
||||
throw new Error(`Setting '${key}' requires a custom renderer`);
|
||||
}
|
||||
|
||||
const isExplicitCommit = definition.commit?.mode === "explicit";
|
||||
const holdValue = opt.holdValue ?? isExplicitCommit;
|
||||
const setting = new Setting(containerEl);
|
||||
const wireOption = {
|
||||
...opt,
|
||||
holdValue,
|
||||
settingDefinition: definition,
|
||||
};
|
||||
|
||||
if (opt.options) {
|
||||
return setting.autoWireDropDown(key as AllStringItemKey, { ...wireOption, options: opt.options });
|
||||
}
|
||||
|
||||
switch (definition.kind) {
|
||||
case "boolean":
|
||||
return setting.autoWireToggle(key as AllBooleanItemKey, wireOption);
|
||||
case "number":
|
||||
return setting.autoWireNumeric(key as AllNumericItemKey, wireOption);
|
||||
case "password":
|
||||
return setting.autoWireText(key as AllStringItemKey, { ...wireOption, isPassword: true });
|
||||
case "textarea":
|
||||
return setting.autoWireTextArea(key as AllStringItemKey, wireOption);
|
||||
case "select":
|
||||
throw new Error(`Setting '${key}' requires select options`);
|
||||
case "text":
|
||||
return setting.autoWireText(key as AllStringItemKey, wireOption);
|
||||
}
|
||||
}
|
||||
@@ -1,37 +1,41 @@
|
||||
import { ChunkAlgorithmNames } from "@lib/common/types.ts";
|
||||
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
|
||||
export function paneAdvanced(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
|
||||
void addPanel(paneEl, "Memory cache").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "hashCacheMaxCount", { clampMin: 10 });
|
||||
// renderObsidianSetting(paneEl, "hashCacheMaxAmount", { clampMin: 1 });
|
||||
new Setting(paneEl).autoWireNumeric("hashCacheMaxCount", { clampMin: 10 });
|
||||
// new Setting(paneEl).autoWireNumeric("hashCacheMaxAmount", { clampMin: 1 });
|
||||
});
|
||||
void addPanel(paneEl, "Local Database Tweak").then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
|
||||
const items = ChunkAlgorithmNames;
|
||||
renderObsidianSetting(paneEl, "chunkSplitterVersion", {
|
||||
new Setting(paneEl).autoWireDropDown("chunkSplitterVersion", {
|
||||
options: items,
|
||||
});
|
||||
renderObsidianSetting(paneEl, "customChunkSize", { clampMin: 0, acceptZero: true });
|
||||
new Setting(paneEl).autoWireNumeric("customChunkSize", { clampMin: 0, acceptZero: true });
|
||||
});
|
||||
|
||||
void addPanel(paneEl, "Transfer Tweak").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "readChunksOnline", { onUpdate: this.onlyOnCouchDB }).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "useOnlyLocalChunk", { onUpdate: this.onlyOnCouchDB }).setClass("wizardHidden");
|
||||
new Setting(paneEl)
|
||||
.setClass("wizardHidden")
|
||||
.autoWireToggle("readChunksOnline", { onUpdate: this.onlyOnCouchDB });
|
||||
new Setting(paneEl)
|
||||
.setClass("wizardHidden")
|
||||
.autoWireToggle("useOnlyLocalChunk", { onUpdate: this.onlyOnCouchDB });
|
||||
|
||||
renderObsidianSetting(paneEl, "concurrencyOfReadChunksOnline", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("concurrencyOfReadChunksOnline", {
|
||||
clampMin: 10,
|
||||
onUpdate: this.onlyOnCouchDB,
|
||||
}).setClass("wizardHidden");
|
||||
});
|
||||
|
||||
renderObsidianSetting(paneEl, "minimumIntervalOfReadChunksOnline", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("minimumIntervalOfReadChunksOnline", {
|
||||
clampMin: 10,
|
||||
onUpdate: this.onlyOnCouchDB,
|
||||
}).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "autoAcceptCompatibleTweak").setClass("wizardHidden");
|
||||
});
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("autoAcceptCompatibleTweak");
|
||||
// new Setting(paneEl)
|
||||
// .setClass("wizardHidden")
|
||||
// .autoWireToggle("sendChunksBulk", { onUpdate: onlyOnCouchDB })
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import { EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG, eventHub } from "@/common/events.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
import { enableOnly, visibleOnly } from "./SettingPane.ts";
|
||||
export function paneCustomisationSync(
|
||||
@@ -36,27 +35,27 @@ export function paneCustomisationSync(
|
||||
visibleOnly(() => this.isConfiguredAs("usePluginSync", true))
|
||||
);
|
||||
|
||||
renderObsidianSetting(paneEl, "deviceAndVaultName", {
|
||||
new Setting(paneEl).autoWireText("deviceAndVaultName", {
|
||||
placeHolder: "desktop",
|
||||
onUpdate: enableOnlyOnPluginSyncIsNotEnabled,
|
||||
});
|
||||
|
||||
renderObsidianSetting(paneEl, "usePluginSyncV2");
|
||||
new Setting(paneEl).autoWireToggle("usePluginSyncV2");
|
||||
|
||||
renderObsidianSetting(paneEl, "usePluginSync", {
|
||||
new Setting(paneEl).autoWireToggle("usePluginSync", {
|
||||
onUpdate: enableOnly(() => !this.isConfiguredAs("deviceAndVaultName", "")),
|
||||
});
|
||||
|
||||
renderObsidianSetting(paneEl, "autoSweepPlugins", {
|
||||
new Setting(paneEl).autoWireToggle("autoSweepPlugins", {
|
||||
onUpdate: visibleOnlyOnPluginSyncEnabled,
|
||||
});
|
||||
|
||||
renderObsidianSetting(paneEl, "autoSweepPluginsPeriodic", {
|
||||
new Setting(paneEl).autoWireToggle("autoSweepPluginsPeriodic", {
|
||||
onUpdate: visibleOnly(
|
||||
() => this.isConfiguredAs("usePluginSync", true) && this.isConfiguredAs("autoSweepPlugins", true)
|
||||
),
|
||||
});
|
||||
renderObsidianSetting(paneEl, "notifyPluginOrSettingUpdated", {
|
||||
new Setting(paneEl).autoWireToggle("notifyPluginOrSettingUpdated", {
|
||||
onUpdate: visibleOnlyOnPluginSyncEnabled,
|
||||
});
|
||||
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { $msg, $t } from "@lib/common/i18n.ts";
|
||||
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "@lib/common/rosetta.ts";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
import { visibleOnly } from "./SettingPane.ts";
|
||||
@@ -17,20 +16,20 @@ export function paneGeneral(
|
||||
// ["", $msg("obsidianLiveSyncSettingTab.defaultLanguage")],
|
||||
...SUPPORTED_I18N_LANGS.map((e) => [e, $t(`lang-${e}`)]),
|
||||
]) as Record<I18N_LANGS, string>;
|
||||
renderObsidianSetting(paneEl, "displayLanguage", {
|
||||
new Setting(paneEl).autoWireDropDown("displayLanguage", {
|
||||
options: languages,
|
||||
});
|
||||
this.addOnSaved("displayLanguage", () => this.display());
|
||||
renderObsidianSetting(paneEl, "showStatusOnEditor");
|
||||
new Setting(paneEl).autoWireToggle("showStatusOnEditor");
|
||||
this.addOnSaved("showStatusOnEditor", () => {
|
||||
eventHub.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
|
||||
});
|
||||
renderObsidianSetting(paneEl, "showOnlyIconsOnEditor", {
|
||||
new Setting(paneEl).autoWireToggle("showOnlyIconsOnEditor", {
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("showStatusOnEditor", true)),
|
||||
});
|
||||
renderObsidianSetting(paneEl, "showStatusOnStatusbar");
|
||||
renderObsidianSetting(paneEl, "hideFileWarningNotice");
|
||||
renderObsidianSetting(paneEl, "networkWarningStyle", {
|
||||
new Setting(paneEl).autoWireToggle("showStatusOnStatusbar");
|
||||
new Setting(paneEl).autoWireToggle("hideFileWarningNotice");
|
||||
new Setting(paneEl).autoWireDropDown("networkWarningStyle", {
|
||||
options: {
|
||||
[NetworkWarningStyles.BANNER]: "Show full banner",
|
||||
[NetworkWarningStyles.ICON]: "Show icon only",
|
||||
@@ -44,9 +43,9 @@ export function paneGeneral(
|
||||
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleLogging")).then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
|
||||
renderObsidianSetting(paneEl, "lessInformationInLog");
|
||||
new Setting(paneEl).autoWireToggle("lessInformationInLog");
|
||||
|
||||
renderObsidianSetting(paneEl, "showVerboseLog", {
|
||||
new Setting(paneEl).autoWireToggle("showVerboseLog", {
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("lessInformationInLog", false)),
|
||||
});
|
||||
});
|
||||
|
||||
@@ -25,7 +25,6 @@ import { ICHeader, ICXHeader, PSCHeader } from "@/common/types.ts";
|
||||
import { HiddenFileSync } from "@/features/HiddenFileSync/CmdHiddenFileSync.ts";
|
||||
import { EVENT_REQUEST_SHOW_HISTORY } from "@/common/obsidianEvents.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
import { isNotFoundError } from "@lib/common/utils.doc.ts";
|
||||
export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
|
||||
@@ -88,14 +87,14 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
|
||||
eventHub.emitEvent(EVENT_REQUEST_CHECK_REMOTE_SIZE);
|
||||
})
|
||||
);
|
||||
renderObsidianSetting(paneEl, "writeLogToTheFile");
|
||||
new Setting(paneEl).autoWireToggle("writeLogToTheFile");
|
||||
});
|
||||
|
||||
void addPanel(paneEl, "Scram Switches").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "suspendFileWatching");
|
||||
new Setting(paneEl).autoWireToggle("suspendFileWatching");
|
||||
this.addOnSaved("suspendFileWatching", () => this.services.appLifecycle.askRestart());
|
||||
|
||||
renderObsidianSetting(paneEl, "suspendParseReplicationResult");
|
||||
new Setting(paneEl).autoWireToggle("suspendParseReplicationResult");
|
||||
this.addOnSaved("suspendParseReplicationResult", () => this.services.appLifecycle.askRestart());
|
||||
});
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ import {
|
||||
} from "@lib/common/types.ts";
|
||||
import { Logger } from "@lib/common/logger.ts";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import { addObsidianApplyButton, renderObsidianApplyButton, renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
import { visibleOnly } from "./SettingPane.ts";
|
||||
@@ -17,17 +16,17 @@ import { migrateDatabases } from "./settingUtils.ts";
|
||||
|
||||
export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
|
||||
void addPanel(paneEl, "Compatibility (Metadata)").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "deleteMetadataOfDeletedFiles").setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("deleteMetadataOfDeletedFiles");
|
||||
|
||||
renderObsidianSetting(paneEl, "automaticallyDeleteMetadataOfDeletedFiles", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("automaticallyDeleteMetadataOfDeletedFiles", {
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("deleteMetadataOfDeletedFiles", true)),
|
||||
}).setClass("wizardHidden");
|
||||
});
|
||||
});
|
||||
|
||||
void addPanel(paneEl, "Compatibility (Conflict Behaviour)").then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "disableMarkdownAutoMerge").setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "writeDocumentsIfConflicted").setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("disableMarkdownAutoMerge");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("writeDocumentsIfConflicted");
|
||||
});
|
||||
|
||||
void addPanel(paneEl, "Compatibility (Database structure)").then((paneEl) => {
|
||||
@@ -114,18 +113,18 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
|
||||
});
|
||||
}
|
||||
}
|
||||
renderObsidianSetting(paneEl, "handleFilenameCaseSensitive", { holdValue: true }).setClass("wizardHidden");
|
||||
new Setting(paneEl).autoWireToggle("handleFilenameCaseSensitive", { holdValue: true }).setClass("wizardHidden");
|
||||
});
|
||||
|
||||
void addPanel(paneEl, "Compatibility (Internal API Usage)").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "watchInternalFileChanges", { invert: true });
|
||||
new Setting(paneEl).autoWireToggle("watchInternalFileChanges", { invert: true });
|
||||
});
|
||||
void addPanel(paneEl, "Compatibility (Remote Database)").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "E2EEAlgorithm", {
|
||||
new Setting(paneEl).autoWireDropDown("E2EEAlgorithm", {
|
||||
options: E2EEAlgorithmNames,
|
||||
});
|
||||
});
|
||||
renderObsidianSetting(paneEl, "useDynamicIterationCount", {
|
||||
new Setting(paneEl).autoWireToggle("useDynamicIterationCount", {
|
||||
holdValue: true,
|
||||
onUpdate: visibleOnly(
|
||||
() =>
|
||||
@@ -135,15 +134,16 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
|
||||
});
|
||||
|
||||
void addPanel(paneEl, "Edge case addressing (Database)").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "additionalSuffixOfDatabaseName");
|
||||
renderObsidianApplyButton(paneEl, "database-suffix");
|
||||
new Setting(paneEl)
|
||||
.autoWireText("additionalSuffixOfDatabaseName", { holdValue: true })
|
||||
.addApplyButton(["additionalSuffixOfDatabaseName"]);
|
||||
|
||||
this.addOnSaved("additionalSuffixOfDatabaseName", async (key) => {
|
||||
Logger("Suffix has been changed. Reopening database...", LOG_LEVEL_NOTICE);
|
||||
await this.services.databaseEvents.initialiseDatabase();
|
||||
});
|
||||
|
||||
renderObsidianSetting(paneEl, "hashAlg", {
|
||||
new Setting(paneEl).autoWireDropDown("hashAlg", {
|
||||
options: {
|
||||
"": "Old Algorithm",
|
||||
xxhash32: "xxhash32 (Fast but less collision resistance)",
|
||||
@@ -157,15 +157,15 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
|
||||
});
|
||||
});
|
||||
void addPanel(paneEl, "Edge case addressing (Behaviour)").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "doNotSuspendOnFetching");
|
||||
renderObsidianSetting(paneEl, "doNotDeleteFolder").setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "processSizeMismatchedFiles");
|
||||
new Setting(paneEl).autoWireToggle("doNotSuspendOnFetching");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("doNotDeleteFolder");
|
||||
new Setting(paneEl).autoWireToggle("processSizeMismatchedFiles");
|
||||
});
|
||||
|
||||
void addPanel(paneEl, "Edge case addressing (Processing)").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "disableWorkerForGeneratingChunks");
|
||||
new Setting(paneEl).autoWireToggle("disableWorkerForGeneratingChunks");
|
||||
|
||||
renderObsidianSetting(paneEl, "processSmallFilesInUIThread", {
|
||||
new Setting(paneEl).autoWireToggle("processSmallFilesInUIThread", {
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("disableWorkerForGeneratingChunks", false)),
|
||||
});
|
||||
});
|
||||
@@ -173,11 +173,11 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
|
||||
// new Setting(paneEl).autoWireToggle("useRequestAPI");
|
||||
// });
|
||||
void addPanel(paneEl, "Compatibility (Trouble addressed)").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "disableCheckingConfigMismatch");
|
||||
new Setting(paneEl).autoWireToggle("disableCheckingConfigMismatch");
|
||||
});
|
||||
void addPanel(paneEl, "Remediation").then((paneEl) => {
|
||||
let dateEl: HTMLSpanElement;
|
||||
const remediationSetting = new Setting(paneEl)
|
||||
new Setting(paneEl)
|
||||
.addText((text) => {
|
||||
const updateDateText = () => {
|
||||
if (this.editingSettings.maxMTimeForReflectEvents == 0) {
|
||||
@@ -212,8 +212,8 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
|
||||
updateDateText();
|
||||
return text;
|
||||
})
|
||||
.setAuto("maxMTimeForReflectEvents");
|
||||
addObsidianApplyButton(remediationSetting, "remediation-reflect-events");
|
||||
.setAuto("maxMTimeForReflectEvents")
|
||||
.addApplyButton(["maxMTimeForReflectEvents"]);
|
||||
|
||||
this.addOnSaved("maxMTimeForReflectEvents", async (key) => {
|
||||
const buttons = ["Restart Now", "Later"] as const;
|
||||
@@ -240,6 +240,6 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
|
||||
// .setClass("wizardHidden");
|
||||
// new Setting(paneEl).autoWireNumeric("maxAgeInEden", { onUpdate: onlyUsingEden }).setClass("wizardHidden");
|
||||
|
||||
renderObsidianSetting(paneEl, "enableCompression").setClass("wizardHidden");
|
||||
new Setting(paneEl).autoWireToggle("enableCompression").setClass("wizardHidden");
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { type ConfigPassphraseStore } from "@lib/common/types.ts";
|
||||
import { renderObsidianApplyButton, renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
|
||||
@@ -21,14 +21,14 @@ export function panePowerUsers(
|
||||
this.onlyOnCouchDB
|
||||
).addClass("wizardHidden");
|
||||
|
||||
renderObsidianSetting(paneEl, "batch_size", { clampMin: 2, onUpdate: this.onlyOnCouchDB }).setClass(
|
||||
"wizardHidden"
|
||||
);
|
||||
renderObsidianSetting(paneEl, "batches_limit", {
|
||||
new Setting(paneEl)
|
||||
.setClass("wizardHidden")
|
||||
.autoWireNumeric("batch_size", { clampMin: 2, onUpdate: this.onlyOnCouchDB });
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("batches_limit", {
|
||||
clampMin: 2,
|
||||
onUpdate: this.onlyOnCouchDB,
|
||||
}).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "useTimeouts", { onUpdate: this.onlyOnCouchDB }).setClass("wizardHidden");
|
||||
});
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("useTimeouts", { onUpdate: this.onlyOnCouchDB });
|
||||
});
|
||||
void addPanel(paneEl, "Configuration Encryption").then((paneEl) => {
|
||||
const passphrase_options: Record<ConfigPassphraseStore, string> = {
|
||||
@@ -37,18 +37,23 @@ export function panePowerUsers(
|
||||
ASK_AT_LAUNCH: "Ask an passphrase at every launch",
|
||||
};
|
||||
|
||||
renderObsidianSetting(paneEl, "configPassphraseStore", {
|
||||
options: passphrase_options,
|
||||
}).setClass("wizardHidden");
|
||||
new Setting(paneEl)
|
||||
.setName("Encrypting sensitive configuration items")
|
||||
.autoWireDropDown("configPassphraseStore", {
|
||||
options: passphrase_options,
|
||||
holdValue: true,
|
||||
})
|
||||
.setClass("wizardHidden");
|
||||
|
||||
renderObsidianSetting(paneEl, "configPassphrase")
|
||||
new Setting(paneEl)
|
||||
.autoWireText("configPassphrase", { isPassword: true, holdValue: true })
|
||||
.setClass("wizardHidden")
|
||||
.addOnUpdate(() => ({
|
||||
disabled: !this.isConfiguredAs("configPassphraseStore", "LOCALSTORAGE"),
|
||||
}));
|
||||
renderObsidianApplyButton(paneEl, "configuration-encryption").setClass("wizardHidden");
|
||||
new Setting(paneEl).addApplyButton(["configPassphrase", "configPassphraseStore"]).setClass("wizardHidden");
|
||||
});
|
||||
void addPanel(paneEl, "Developer").then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "enableDebugTools").setClass("wizardHidden");
|
||||
new Setting(paneEl).autoWireToggle("enableDebugTools").setClass("wizardHidden");
|
||||
});
|
||||
}
|
||||
|
||||
@@ -10,7 +10,6 @@ import {
|
||||
import { Menu, type ButtonComponent } from "@/deps.ts";
|
||||
import { $msg } from "@lib/common/i18n.ts";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
// import { visibleOnly } from "./SettingPane.ts";
|
||||
@@ -675,7 +674,7 @@ export function paneRemoteConfig(
|
||||
|
||||
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleNotification"), () => {}).then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "notifyThresholdOfRemoteStorageSize", {}).setClass("wizardHidden");
|
||||
new Setting(paneEl).autoWireNumeric("notifyThresholdOfRemoteStorageSize", {}).setClass("wizardHidden");
|
||||
});
|
||||
|
||||
// new Setting(paneEl).setClass("wizardOnly").addButton((button) =>
|
||||
|
||||
@@ -4,7 +4,6 @@ import MultipleRegExpControl from "./MultipleRegExpControl.svelte";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import { mount } from "svelte";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
import { visibleOnly } from "./SettingPane.ts";
|
||||
export function paneSelector(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
|
||||
@@ -47,12 +46,12 @@ export function paneSelector(this: ObsidianLiveSyncSettingTab, paneEl: HTMLEleme
|
||||
},
|
||||
},
|
||||
});
|
||||
renderObsidianSetting(paneEl, "syncMaxSizeInMB", { clampMin: 0 }).setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("syncMaxSizeInMB", { clampMin: 0 });
|
||||
|
||||
renderObsidianSetting(paneEl, "useIgnoreFiles").setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "ignoreFiles", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("useIgnoreFiles");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireTextArea("ignoreFiles", {
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("useIgnoreFiles", true)),
|
||||
}).setClass("wizardHidden");
|
||||
});
|
||||
});
|
||||
void addPanel(paneEl, "Hidden Files", undefined, undefined, LEVEL_ADVANCED).then((paneEl) => {
|
||||
const targetPatternSetting = new Setting(paneEl)
|
||||
|
||||
@@ -15,7 +15,6 @@ import { DEFAULT_SETTINGS } from "@lib/common/types.ts";
|
||||
import { request } from "@/deps.ts";
|
||||
import { SetupManager, UserMode } from "@/modules/features/SetupManager.ts";
|
||||
import { LiveSyncError } from "@lib/common/LSError.ts";
|
||||
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
export function paneSetup(
|
||||
this: ObsidianLiveSyncSettingTab,
|
||||
paneEl: HTMLElement,
|
||||
@@ -108,9 +107,10 @@ export function paneSetup(
|
||||
});
|
||||
|
||||
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleExtraFeatures")).then((paneEl) => {
|
||||
renderObsidianSetting(paneEl, "useAdvancedMode");
|
||||
renderObsidianSetting(paneEl, "usePowerUserMode");
|
||||
renderObsidianSetting(paneEl, "useEdgeCaseMode");
|
||||
new Setting(paneEl).autoWireToggle("useAdvancedMode");
|
||||
|
||||
new Setting(paneEl).autoWireToggle("usePowerUserMode");
|
||||
new Setting(paneEl).autoWireToggle("useEdgeCaseMode");
|
||||
|
||||
this.addOnSaved("useAdvancedMode", () => this.display());
|
||||
this.addOnSaved("usePowerUserMode", () => this.display());
|
||||
|
||||
@@ -2,7 +2,6 @@ import { type ObsidianLiveSyncSettings, LOG_LEVEL_NOTICE, REMOTE_COUCHDB, LEVEL_
|
||||
import { Logger } from "@lib/common/logger.ts";
|
||||
import { $msg } from "@lib/common/i18n.ts";
|
||||
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
|
||||
import { renderObsidianApplyButton, renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
|
||||
import { EVENT_REQUEST_COPY_SETUP_URI, eventHub } from "@/common/events.ts";
|
||||
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
|
||||
import type { PageFunctions } from "./SettingPane.ts";
|
||||
@@ -32,16 +31,18 @@ export function paneSyncSettings(
|
||||
DISABLE: $msg("obsidianLiveSyncSettingTab.optionDisableAllAutomatic"),
|
||||
};
|
||||
|
||||
renderObsidianSetting(paneEl, "preset", {
|
||||
options: options,
|
||||
holdValue: true,
|
||||
}).addButton((button) => {
|
||||
button.setButtonText($msg("obsidianLiveSyncSettingTab.btnApply"));
|
||||
button.onClick(async () => {
|
||||
// await this.saveSettings(["preset"]);
|
||||
await this.saveAllDirtySettings();
|
||||
new Setting(paneEl)
|
||||
.autoWireDropDown("preset", {
|
||||
options: options,
|
||||
holdValue: true,
|
||||
})
|
||||
.addButton((button) => {
|
||||
button.setButtonText($msg("obsidianLiveSyncSettingTab.btnApply"));
|
||||
button.onClick(async () => {
|
||||
// await this.saveSettings(["preset"]);
|
||||
await this.saveAllDirtySettings();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
this.addOnSaved("preset", async (currentPreset) => {
|
||||
if (currentPreset == "") {
|
||||
@@ -135,7 +136,7 @@ export function paneSyncSettings(
|
||||
const onlyOnNonLiveSync = visibleOnly(() => !this.isConfiguredAs("syncMode", "LIVESYNC"));
|
||||
const onlyOnPeriodic = visibleOnly(() => this.isConfiguredAs("syncMode", "PERIODIC"));
|
||||
|
||||
const optionsSyncMode: Record<string, string> =
|
||||
const optionsSyncMode =
|
||||
this.editingSettings.remoteType == REMOTE_COUCHDB
|
||||
? {
|
||||
ONEVENTS: $msg("obsidianLiveSyncSettingTab.optionOnEvents"),
|
||||
@@ -147,9 +148,12 @@ export function paneSyncSettings(
|
||||
PERIODIC: $msg("obsidianLiveSyncSettingTab.optionPeriodicAndEvents"),
|
||||
};
|
||||
|
||||
renderObsidianSetting(paneEl, "syncMode", {
|
||||
options: optionsSyncMode,
|
||||
}).setClass("wizardHidden");
|
||||
new Setting(paneEl)
|
||||
.autoWireDropDown("syncMode", {
|
||||
//@ts-ignore
|
||||
options: optionsSyncMode,
|
||||
})
|
||||
.setClass("wizardHidden");
|
||||
this.addOnSaved("syncMode", async (value) => {
|
||||
this.editingSettings.liveSync = false;
|
||||
this.editingSettings.periodicReplication = false;
|
||||
@@ -163,28 +167,32 @@ export function paneSyncSettings(
|
||||
await this.services.control.applySettings();
|
||||
});
|
||||
|
||||
renderObsidianSetting(paneEl, "periodicReplicationInterval", {
|
||||
clampMax: 5000,
|
||||
onUpdate: onlyOnPeriodic,
|
||||
}).setClass("wizardHidden");
|
||||
new Setting(paneEl)
|
||||
.autoWireNumeric("periodicReplicationInterval", {
|
||||
clampMax: 5000,
|
||||
onUpdate: onlyOnPeriodic,
|
||||
})
|
||||
.setClass("wizardHidden");
|
||||
|
||||
renderObsidianSetting(paneEl, "syncMinimumInterval", {
|
||||
new Setting(paneEl).autoWireNumeric("syncMinimumInterval", {
|
||||
onUpdate: onlyOnNonLiveSync,
|
||||
});
|
||||
renderObsidianSetting(paneEl, "syncOnSave", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "syncOnEditorSave", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "syncOnFileOpen", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "syncOnStart", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "syncAfterMerge", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncOnSave", { onUpdate: onlyOnNonLiveSync });
|
||||
new Setting(paneEl)
|
||||
.setClass("wizardHidden")
|
||||
.autoWireToggle("syncOnEditorSave", { onUpdate: onlyOnNonLiveSync });
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncOnFileOpen", { onUpdate: onlyOnNonLiveSync });
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncOnStart", { onUpdate: onlyOnNonLiveSync });
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncAfterMerge", { onUpdate: onlyOnNonLiveSync });
|
||||
// Desktop app only, and only for the sync modes that keep a background replication channel
|
||||
// (LiveSync and Periodic). Ignored on mobile, where suspending preserves battery. The
|
||||
// visibility predicate mirrors the runtime guard in ModuleObsidianEvents.
|
||||
if (!this.services.API.isMobile()) {
|
||||
renderObsidianSetting(paneEl, "keepReplicationActiveInBackground", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("keepReplicationActiveInBackground", {
|
||||
onUpdate: visibleOnly(
|
||||
() => this.isConfiguredAs("syncMode", "LIVESYNC") || this.isConfiguredAs("syncMode", "PERIODIC")
|
||||
),
|
||||
}).setClass("wizardHidden");
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
@@ -195,15 +203,15 @@ export function paneSyncSettings(
|
||||
visibleOnly(() => !this.isConfiguredAs("syncMode", "LIVESYNC"))
|
||||
).then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "batchSave").setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "batchSaveMinimumDelay", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("batchSave");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("batchSaveMinimumDelay", {
|
||||
acceptZero: true,
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
|
||||
}).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "batchSaveMaximumDelay", {
|
||||
});
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("batchSaveMaximumDelay", {
|
||||
acceptZero: true,
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
|
||||
}).setClass("wizardHidden");
|
||||
});
|
||||
});
|
||||
|
||||
void addPanel(
|
||||
@@ -214,9 +222,9 @@ export function paneSyncSettings(
|
||||
LEVEL_ADVANCED
|
||||
).then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "trashInsteadDelete").setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("trashInsteadDelete");
|
||||
|
||||
renderObsidianSetting(paneEl, "doNotDeleteFolder").setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("doNotDeleteFolder");
|
||||
});
|
||||
void addPanel(
|
||||
paneEl,
|
||||
@@ -227,11 +235,11 @@ export function paneSyncSettings(
|
||||
).then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
|
||||
renderObsidianSetting(paneEl, "resolveConflictsByNewerFile").setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("resolveConflictsByNewerFile");
|
||||
|
||||
renderObsidianSetting(paneEl, "checkConflictOnlyOnOpen").setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("checkConflictOnlyOnOpen");
|
||||
|
||||
renderObsidianSetting(paneEl, "showMergeDialogOnlyOnActive").setClass("wizardHidden");
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("showMergeDialogOnlyOnActive");
|
||||
});
|
||||
|
||||
void addPanel(
|
||||
@@ -242,12 +250,11 @@ export function paneSyncSettings(
|
||||
LEVEL_ADVANCED
|
||||
).then((paneEl) => {
|
||||
paneEl.addClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "settingSyncFile");
|
||||
renderObsidianApplyButton(paneEl, "setting-sync-file");
|
||||
new Setting(paneEl).autoWireText("settingSyncFile", { holdValue: true }).addApplyButton(["settingSyncFile"]);
|
||||
|
||||
renderObsidianSetting(paneEl, "writeCredentialsForSettingSync");
|
||||
new Setting(paneEl).autoWireToggle("writeCredentialsForSettingSync");
|
||||
|
||||
renderObsidianSetting(paneEl, "notifyAllSettingSyncFile");
|
||||
new Setting(paneEl).autoWireToggle("notifyAllSettingSyncFile");
|
||||
});
|
||||
|
||||
void addPanel(
|
||||
@@ -306,14 +313,14 @@ export function paneSyncSettings(
|
||||
});
|
||||
}
|
||||
|
||||
renderObsidianSetting(paneEl, "suppressNotifyHiddenFilesChange", {}).setClass("wizardHidden");
|
||||
renderObsidianSetting(paneEl, "syncInternalFilesBeforeReplication", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("suppressNotifyHiddenFilesChange", {});
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncInternalFilesBeforeReplication", {
|
||||
onUpdate: visibleOnly(() => this.isConfiguredAs("watchInternalFileChanges", true)),
|
||||
}).setClass("wizardHidden");
|
||||
});
|
||||
|
||||
renderObsidianSetting(paneEl, "syncInternalFilesInterval", {
|
||||
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("syncInternalFilesInterval", {
|
||||
clampMin: 10,
|
||||
acceptZero: true,
|
||||
}).setClass("wizardHidden");
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,11 +1,5 @@
|
||||
import { $msg } from "@lib/common/i18n";
|
||||
import {
|
||||
LEVEL_ADVANCED,
|
||||
LEVEL_EDGE_CASE,
|
||||
LEVEL_POWER_USER,
|
||||
type ConfigLevel,
|
||||
type SettingDefinition,
|
||||
} from "@lib/common/types";
|
||||
import { LEVEL_ADVANCED, LEVEL_EDGE_CASE, LEVEL_POWER_USER, type ConfigLevel } from "@lib/common/types";
|
||||
import type { AllSettingItemKey, AllSettings } from "./settingConstants";
|
||||
|
||||
export const combineOnUpdate = (func1: OnUpdateFunc, func2: OnUpdateFunc): OnUpdateFunc => {
|
||||
@@ -83,7 +77,6 @@ export type AutoWireOption = {
|
||||
invert?: boolean;
|
||||
onUpdate?: OnUpdateFunc;
|
||||
obsolete?: boolean;
|
||||
settingDefinition?: SettingDefinition<AllSettingItemKey>;
|
||||
};
|
||||
|
||||
export function findAttrFromParent(el: HTMLElement, attr: string): string {
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
bench-results/
|
||||
@@ -0,0 +1,8 @@
|
||||
FROM alpine:3.22
|
||||
|
||||
RUN apk add --no-cache iproute2
|
||||
|
||||
COPY test/bench-network/netem-smoke.sh /usr/local/bin/livesync-netem-smoke
|
||||
RUN chmod +x /usr/local/bin/livesync-netem-smoke
|
||||
|
||||
CMD ["livesync-netem-smoke"]
|
||||
@@ -0,0 +1,35 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
|
||||
FROM node:24-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates curl unzip python3 make g++ iproute2 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
ENV DENO_INSTALL=/usr/local
|
||||
RUN curl -fsSL https://deno.land/install.sh | sh
|
||||
|
||||
WORKDIR /workspace
|
||||
|
||||
COPY package.json package-lock.json ./
|
||||
COPY src/apps/cli/package.json ./src/apps/cli/package.json
|
||||
COPY src/apps/webapp/package.json ./src/apps/webapp/package.json
|
||||
COPY src/apps/webpeer/package.json ./src/apps/webpeer/package.json
|
||||
RUN npm ci
|
||||
|
||||
COPY . .
|
||||
RUN npm run build -w self-hosted-livesync-cli
|
||||
|
||||
WORKDIR /workspace/src/apps/cli/testdeno
|
||||
|
||||
RUN deno cache --lock=deno.lock \
|
||||
bench-network-cases.ts \
|
||||
bench-latency-sweep.ts \
|
||||
bench-p2p-split-node.ts \
|
||||
bench-p2p.ts \
|
||||
bench-couchdb.ts
|
||||
|
||||
COPY test/bench-network/run-bench.sh /usr/local/bin/run-livesync-bench
|
||||
RUN chmod +x /usr/local/bin/run-livesync-bench
|
||||
|
||||
CMD ["run-livesync-bench"]
|
||||
@@ -0,0 +1,10 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
|
||||
FROM alpine:3.22
|
||||
|
||||
RUN apk add --no-cache iproute2 socat
|
||||
|
||||
COPY test/bench-network/netem-tcp-shim.sh /usr/local/bin/livesync-netem-tcp-shim
|
||||
RUN chmod +x /usr/local/bin/livesync-netem-tcp-shim
|
||||
|
||||
CMD ["livesync-netem-tcp-shim"]
|
||||
@@ -0,0 +1,263 @@
|
||||
# Network benchmark package
|
||||
|
||||
This directory packages the CLI benchmark cases with Docker Compose. It is
|
||||
intended for reproducible local benchmark runs where CouchDB, the Nostr
|
||||
signalling relay, optional TURN, and the benchmark runner are fixed by the
|
||||
Compose file.
|
||||
|
||||
## Quick smoke run
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
docker compose -f test/bench-network/compose.yml run --rm bench-runner
|
||||
```
|
||||
|
||||
By default this runs:
|
||||
|
||||
- `couchdb-baseline`
|
||||
- `p2p-direct-local`
|
||||
|
||||
The dataset is intentionally small by default. Results are written to
|
||||
`test/bench-network/bench-results/`.
|
||||
|
||||
## GitHub Actions smoke run
|
||||
|
||||
`.github/workflows/cli-p2p-compose-smoke.yml` provides a manual
|
||||
`workflow_dispatch` smoke run for the same Compose package. It is intentionally
|
||||
not a required check yet, because WebRTC peer discovery can still be slow or
|
||||
environment-sensitive on GitHub-hosted runners. Keep the dataset small and use
|
||||
the uploaded JSON artefact to inspect whether failures are caused by peer
|
||||
discovery, synchronisation, CouchDB startup, or Docker networking.
|
||||
|
||||
## Select cases
|
||||
|
||||
```bash
|
||||
BENCH_CASES=couchdb-baseline,p2p-direct-local,p2p-user-turn \
|
||||
docker compose -f test/bench-network/compose.yml --profile turn run --rm bench-runner
|
||||
```
|
||||
|
||||
Available local cases:
|
||||
|
||||
- `couchdb-baseline`
|
||||
- `p2p-direct-local`
|
||||
- `couchdb-tethering-vpn-proxy`
|
||||
- `couchdb-netem-home-wifi`
|
||||
- `couchdb-netem-tethering-vpn`
|
||||
- `p2p-smartphone-vpn-direct`
|
||||
- `p2p-user-turn`
|
||||
|
||||
Set `BENCH_REPEAT_COUNT` to run each selected case more than once. Repeated
|
||||
results are written with suffixes such as `-r01`, `-r02`, and `-r03`, and the
|
||||
summary records the repeat index for each run.
|
||||
|
||||
`p2p-smartphone-vpn-direct` is a structural case name. When it is run inside
|
||||
this Compose package it is not a real smartphone tethering/VPN measurement; it
|
||||
uses the local Compose network. Use it only for wiring checks unless the runner
|
||||
is executed in an actual tethered/VPN environment.
|
||||
|
||||
## Comparison model
|
||||
|
||||
The primary local comparison is between a remote-database path and a direct P2P
|
||||
path:
|
||||
|
||||
| Case | Data path | What is measured | What is not measured |
|
||||
| ------------------ | ------------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- |
|
||||
| `couchdb-baseline` | Device A -> CouchDB -> Device B | Two one-shot CLI synchronisation commands through a local HTTP latency proxy | Real WAN jitter, packet loss, bandwidth limits, VPN encapsulation, and server contention |
|
||||
| `p2p-direct-local` | Device A -> Device B after Nostr signalling | One CLI P2P synchronisation command over WebRTC DataChannel with TURN disabled | Public relay operation, mobile carrier behaviour, TURN relay throughput, and first-peer discovery latency |
|
||||
|
||||
Use the CouchDB result as the remote-store baseline and the P2P result as the
|
||||
direct-transfer comparison. The Nostr relay is used for signalling in the P2P
|
||||
case, but synchronised note content is transferred over the WebRTC DataChannel.
|
||||
The P2P result JSON records the selected WebRTC ICE candidate pair when the CLI
|
||||
can collect it from `RTCPeerConnection.getStats()`. Interpret P2P paths from
|
||||
the recorded candidate types rather than from TURN configuration alone. Do not
|
||||
report P2P runs as Tier 2 constrained-network measurements until host and
|
||||
client are captured under an equivalent shaped topology.
|
||||
|
||||
## Dataset and latency controls
|
||||
|
||||
```bash
|
||||
BENCH_MD_FILE_COUNT=100 \
|
||||
BENCH_MD_MIN_SIZE_BYTES=512 \
|
||||
BENCH_MD_MAX_SIZE_BYTES=2048 \
|
||||
BENCH_BIN_FILE_COUNT=25 \
|
||||
BENCH_BIN_SIZE_BYTES=8192 \
|
||||
BENCH_COUCHDB_RTT_MS=20 \
|
||||
BENCH_PEERS_TIMEOUT=60 \
|
||||
docker compose -f test/bench-network/compose.yml run --rm bench-runner
|
||||
```
|
||||
|
||||
The current CouchDB latency model is the existing HTTP proxy inside
|
||||
`bench-couchdb.ts`. It models a remote database path with additional request
|
||||
latency, but it does not model packet loss, jitter, MTU, bandwidth limits,
|
||||
bufferbloat, or VPN encapsulation.
|
||||
|
||||
For P2P runs, `BENCH_PEERS_TIMEOUT` is passed to `p2p-peers`. That command waits
|
||||
for the requested observation window before printing discovered peers, so the
|
||||
reported peer discovery command time should not be read as first-peer latency.
|
||||
|
||||
## Latency sweep
|
||||
|
||||
To run P2P once and CouchDB at several requested RTT values:
|
||||
|
||||
```bash
|
||||
BENCH_COMMAND=latency-sweep \
|
||||
BENCH_SWEEP_RTT_MS=20,50,100,150,300 \
|
||||
BENCH_MD_FILE_COUNT=100 \
|
||||
BENCH_MD_MIN_SIZE_BYTES=512 \
|
||||
BENCH_MD_MAX_SIZE_BYTES=2048 \
|
||||
BENCH_BIN_FILE_COUNT=25 \
|
||||
BENCH_BIN_SIZE_BYTES=8192 \
|
||||
BENCH_SYNC_TIMEOUT=300 \
|
||||
BENCH_PEERS_TIMEOUT=60 \
|
||||
docker compose -f test/bench-network/compose.yml run --rm bench-runner
|
||||
```
|
||||
|
||||
This sweep is useful for finding where the remote CouchDB path falls behind the
|
||||
local direct P2P path in the current HTTP-proxy latency model. It should not be
|
||||
presented as a full smartphone/VPN model.
|
||||
|
||||
## Network emulation smoke
|
||||
|
||||
The optional `netem` profile checks whether a Linux runner can apply traffic
|
||||
shaping inside a Compose-managed container. This is a fixture smoke test for a
|
||||
second-tier simulation design; it does not produce synchronisation performance
|
||||
results by itself.
|
||||
|
||||
```bash
|
||||
docker compose -f test/bench-network/compose.yml --profile netem run --rm netem-smoke
|
||||
```
|
||||
|
||||
The smoke writes `tc qdisc`, route, and interface details under
|
||||
`test/bench-network/bench-results/`. Profile parameters can be overridden:
|
||||
|
||||
```bash
|
||||
NETEM_PROFILE=tethering-vpn \
|
||||
NETEM_DELAY_MS=140 \
|
||||
NETEM_JITTER_MS=50 \
|
||||
NETEM_LOSS_PERCENT=1.0 \
|
||||
NETEM_BANDWIDTH_MBIT=10 \
|
||||
NETEM_MTU=1380 \
|
||||
docker compose -f test/bench-network/compose.yml --profile netem run --rm netem-smoke
|
||||
```
|
||||
|
||||
## Split-container P2P emulation
|
||||
|
||||
The optional `p2p-split` profile runs the P2P host and client in separate
|
||||
Compose services. Each service can apply `tc netem` to its own egress interface
|
||||
and the client result records the selected WebRTC ICE candidate pair.
|
||||
|
||||
```bash
|
||||
BENCH_MD_FILE_COUNT=2 \
|
||||
BENCH_BIN_FILE_COUNT=1 \
|
||||
BENCH_PEERS_TIMEOUT=10 \
|
||||
BENCH_SPLIT_RUN_ID="$(date -u +%Y%m%d%H%M%S)" \
|
||||
docker compose -f test/bench-network/compose.yml --profile p2p-split up \
|
||||
--abort-on-container-exit --exit-code-from p2p-split-client \
|
||||
p2p-split-host p2p-split-client
|
||||
```
|
||||
|
||||
By default this uses the `home-wifi` profile (`20 ms` delay, `5 ms` jitter,
|
||||
`0.1%` loss, `100 Mbit`, and `1500` MTU) on both P2P containers. Override the
|
||||
same `NETEM_*` variables used by the TCP shim to model a stricter profile.
|
||||
|
||||
```bash
|
||||
BENCH_MD_FILE_COUNT=100 \
|
||||
BENCH_MD_MIN_SIZE_BYTES=512 \
|
||||
BENCH_MD_MAX_SIZE_BYTES=2048 \
|
||||
BENCH_BIN_FILE_COUNT=25 \
|
||||
BENCH_BIN_SIZE_BYTES=8192 \
|
||||
BENCH_PEERS_TIMEOUT=60 \
|
||||
BENCH_SYNC_TIMEOUT=420 \
|
||||
BENCH_SPLIT_RUN_ID="$(date -u +%Y%m%d%H%M%S)" \
|
||||
BENCH_NETWORK_PROFILE=tethering-vpn \
|
||||
NETEM_PROFILE=tethering-vpn \
|
||||
NETEM_DELAY_MS=140 \
|
||||
NETEM_JITTER_MS=50 \
|
||||
NETEM_LOSS_PERCENT=1.0 \
|
||||
NETEM_BANDWIDTH_MBIT=10 \
|
||||
NETEM_MTU=1380 \
|
||||
docker compose -f test/bench-network/compose.yml --profile p2p-split up \
|
||||
--abort-on-container-exit --exit-code-from p2p-split-client \
|
||||
p2p-split-host p2p-split-client
|
||||
```
|
||||
|
||||
This is a Linux-only manual benchmark fixture, not a required pull-request CI
|
||||
job. It shapes each P2P container's egress path, including signalling traffic,
|
||||
and should be reported separately from the CouchDB TCP-shim measurements. The
|
||||
result JSON includes `ok: true` for completed runs; failed runs still write a
|
||||
summary with `ok: false` and a `failure` object before returning a non-zero
|
||||
exit code.
|
||||
|
||||
Remove the shared work volume between repeated manual runs when you do not use
|
||||
a unique `BENCH_SPLIT_RUN_ID`:
|
||||
|
||||
```bash
|
||||
docker compose -f test/bench-network/compose.yml --profile p2p-split down --volumes
|
||||
```
|
||||
|
||||
## P2P Signalling-Only Emulation
|
||||
|
||||
The optional `signalling-shim` profile shapes only the Nostr signalling relay
|
||||
path. The P2P host and client run in the benchmark runner as usual, and the
|
||||
configured relay URL points at a TCP netem shim in front of `nostr-relay`.
|
||||
This is the preferred fixture when evaluating the hypothesis that P2P avoids a
|
||||
constrained remote database data path while still depending on a signalling
|
||||
server for rendezvous.
|
||||
|
||||
```bash
|
||||
BENCH_CASES=p2p-signalling-netem-home-wifi \
|
||||
docker compose -f test/bench-network/compose.yml --profile signalling-shim run --rm \
|
||||
bench-runner-signalling-shim
|
||||
```
|
||||
|
||||
For a stricter signalling path:
|
||||
|
||||
```bash
|
||||
NETEM_PROFILE=tethering-vpn \
|
||||
NETEM_DELAY_MS=140 \
|
||||
NETEM_JITTER_MS=50 \
|
||||
NETEM_LOSS_PERCENT=1.0 \
|
||||
NETEM_BANDWIDTH_MBIT=10 \
|
||||
NETEM_MTU=1380 \
|
||||
BENCH_CASES=p2p-signalling-netem-tethering-vpn \
|
||||
docker compose -f test/bench-network/compose.yml --profile signalling-shim run --rm \
|
||||
bench-runner-signalling-shim
|
||||
```
|
||||
|
||||
Use this separately from `p2p-split`. The `p2p-split` profile shapes each peer's
|
||||
egress path, so it constrains both signalling and the selected WebRTC data
|
||||
path. The `signalling-shim` profile constrains only relay access, which keeps
|
||||
it focused on peer-to-signalling-server reachability rather than peer-to-peer
|
||||
note-data transfer.
|
||||
|
||||
## Shimmed CouchDB benchmark
|
||||
|
||||
The optional `shim` profile runs a CouchDB benchmark through a TCP forwarding
|
||||
container that applies `tc netem`. This is a manual Tier 2 synchronisation
|
||||
measurement path; it is intentionally separate from required pull-request CI.
|
||||
|
||||
```bash
|
||||
docker compose -f test/bench-network/compose.yml --profile shim run --rm bench-runner-shim
|
||||
```
|
||||
|
||||
The default profile is `home-wifi`. A smartphone/VPN-like profile can be
|
||||
requested by overriding both the shim parameters and the benchmark case:
|
||||
|
||||
```bash
|
||||
NETEM_PROFILE=tethering-vpn \
|
||||
NETEM_DELAY_MS=140 \
|
||||
NETEM_JITTER_MS=50 \
|
||||
NETEM_LOSS_PERCENT=1.0 \
|
||||
NETEM_BANDWIDTH_MBIT=10 \
|
||||
NETEM_MTU=1380 \
|
||||
BENCH_CASES=couchdb-netem-tethering-vpn \
|
||||
docker compose -f test/bench-network/compose.yml --profile shim run --rm bench-runner-shim
|
||||
```
|
||||
|
||||
The benchmark result records `simulationTier`, `networkProfile`, and
|
||||
`networkModel`. The shim also writes its applied `tc qdisc`, route, and
|
||||
interface state under `test/bench-network/bench-results/`.
|
||||
This shim currently measures the CouchDB path only. It does not shape or verify
|
||||
the WebRTC P2P data path.
|
||||
@@ -0,0 +1,329 @@
|
||||
services:
|
||||
couchdb:
|
||||
image: couchdb:3.5.0
|
||||
environment:
|
||||
COUCHDB_USER: ${BENCH_COUCHDB_USER:-admin}
|
||||
COUCHDB_PASSWORD: ${BENCH_COUCHDB_PASSWORD:-testpassword}
|
||||
COUCHDB_SINGLE_NODE: "true"
|
||||
healthcheck:
|
||||
test:
|
||||
[
|
||||
"CMD-SHELL",
|
||||
"curl -fsS -u ${BENCH_COUCHDB_USER:-admin}:${BENCH_COUCHDB_PASSWORD:-testpassword} http://127.0.0.1:5984/_up >/dev/null",
|
||||
]
|
||||
interval: 2s
|
||||
timeout: 5s
|
||||
retries: 30
|
||||
|
||||
nostr-relay:
|
||||
image: ghcr.io/hoytech/strfry:latest
|
||||
entrypoint: sh
|
||||
command:
|
||||
- -lc
|
||||
- |
|
||||
cat > /tmp/strfry.conf <<'EOF'
|
||||
db = "./strfry-db/"
|
||||
|
||||
relay {
|
||||
bind = "0.0.0.0"
|
||||
port = 7777
|
||||
nofiles = 65536
|
||||
|
||||
info {
|
||||
name = "livesync bench relay"
|
||||
description = "local relay for livesync compose benchmarks"
|
||||
}
|
||||
|
||||
maxWebsocketPayloadSize = 131072
|
||||
autoPingSeconds = 55
|
||||
|
||||
writePolicy {
|
||||
plugin = ""
|
||||
}
|
||||
}
|
||||
EOF
|
||||
exec /app/strfry --config /tmp/strfry.conf relay
|
||||
tmpfs:
|
||||
- /app/strfry-db:rw,size=256m
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "nc -z 127.0.0.1 7777"]
|
||||
interval: 2s
|
||||
timeout: 5s
|
||||
retries: 30
|
||||
|
||||
coturn:
|
||||
image: coturn/coturn:latest
|
||||
command:
|
||||
- --log-file=stdout
|
||||
- --listening-port=3478
|
||||
- --user=${BENCH_TURN_USERNAME:-testuser}:${BENCH_TURN_CREDENTIAL:-testpass}
|
||||
- --realm=${BENCH_TURN_REALM:-livesync.test}
|
||||
profiles:
|
||||
- turn
|
||||
|
||||
bench-runner:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.runner
|
||||
depends_on:
|
||||
couchdb:
|
||||
condition: service_healthy
|
||||
nostr-relay:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
BENCH_COMMAND: ${BENCH_COMMAND:-cases}
|
||||
BENCH_CASES: ${BENCH_CASES:-couchdb-baseline,p2p-direct-local}
|
||||
BENCH_REPEAT_COUNT: ${BENCH_REPEAT_COUNT:-1}
|
||||
BENCH_CASES_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_SWEEP_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_SWEEP_RTT_MS: ${BENCH_SWEEP_RTT_MS:-20,50,100,150,300}
|
||||
BENCH_SWEEP_INCLUDE_P2P: ${BENCH_SWEEP_INCLUDE_P2P:-true}
|
||||
BENCH_COUCHDB_MANAGED: "false"
|
||||
BENCH_COUCHDB_BACKEND_URI: http://couchdb:5984
|
||||
BENCH_COUCHDB_URI: http://127.0.0.1:15989
|
||||
BENCH_COUCHDB_USER: ${BENCH_COUCHDB_USER:-admin}
|
||||
BENCH_COUCHDB_PASSWORD: ${BENCH_COUCHDB_PASSWORD:-testpassword}
|
||||
BENCH_RELAY: ws://nostr-relay:7777/
|
||||
BENCH_LOCAL_TURN_SERVERS: turn:coturn:3478
|
||||
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
|
||||
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
|
||||
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
|
||||
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
|
||||
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
|
||||
BENCH_COUCHDB_RTT_MS: ${BENCH_COUCHDB_RTT_MS:-20}
|
||||
BENCH_TETHERING_VPN_RTT_MS: ${BENCH_TETHERING_VPN_RTT_MS:-120}
|
||||
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
|
||||
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
|
||||
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
|
||||
BENCH_LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
|
||||
volumes:
|
||||
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
|
||||
|
||||
couchdb-shim:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.shim
|
||||
profiles:
|
||||
- shim
|
||||
depends_on:
|
||||
couchdb:
|
||||
condition: service_healthy
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
environment:
|
||||
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
|
||||
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
|
||||
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
|
||||
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
|
||||
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
|
||||
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
|
||||
NETEM_MTU: ${NETEM_MTU:-1500}
|
||||
NETEM_RESULT_ROOT: /bench-results
|
||||
SHIM_LISTEN_PORT: 5984
|
||||
SHIM_TARGET_HOST: couchdb
|
||||
SHIM_TARGET_PORT: 5984
|
||||
volumes:
|
||||
- ./bench-results:/bench-results
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "nc -z 127.0.0.1 5984"]
|
||||
interval: 2s
|
||||
timeout: 5s
|
||||
retries: 30
|
||||
|
||||
bench-runner-shim:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.runner
|
||||
profiles:
|
||||
- shim
|
||||
depends_on:
|
||||
couchdb-shim:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
BENCH_COMMAND: ${BENCH_COMMAND:-cases}
|
||||
BENCH_CASES: ${BENCH_CASES:-couchdb-netem-home-wifi}
|
||||
BENCH_REPEAT_COUNT: ${BENCH_REPEAT_COUNT:-1}
|
||||
BENCH_CASES_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_SWEEP_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_COUCHDB_MANAGED: "false"
|
||||
BENCH_COUCHDB_BACKEND_URI: http://couchdb-shim:5984
|
||||
BENCH_SHIM_COUCHDB_URI: http://couchdb-shim:5984
|
||||
BENCH_COUCHDB_URI: http://127.0.0.1:15989
|
||||
BENCH_COUCHDB_USER: ${BENCH_COUCHDB_USER:-admin}
|
||||
BENCH_COUCHDB_PASSWORD: ${BENCH_COUCHDB_PASSWORD:-testpassword}
|
||||
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
|
||||
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
|
||||
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
|
||||
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
|
||||
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
|
||||
BENCH_COUCHDB_RTT_MS: ${BENCH_COUCHDB_RTT_MS:-1}
|
||||
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
|
||||
BENCH_LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
|
||||
volumes:
|
||||
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
|
||||
|
||||
p2p-signalling-shim:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.shim
|
||||
profiles:
|
||||
- signalling-shim
|
||||
depends_on:
|
||||
nostr-relay:
|
||||
condition: service_healthy
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
environment:
|
||||
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
|
||||
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
|
||||
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
|
||||
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
|
||||
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
|
||||
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
|
||||
NETEM_MTU: ${NETEM_MTU:-1500}
|
||||
NETEM_RESULT_ROOT: /bench-results
|
||||
SHIM_LISTEN_PORT: 7777
|
||||
SHIM_TARGET_HOST: nostr-relay
|
||||
SHIM_TARGET_PORT: 7777
|
||||
volumes:
|
||||
- ./bench-results:/bench-results
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "nc -z 127.0.0.1 7777"]
|
||||
interval: 2s
|
||||
timeout: 5s
|
||||
retries: 30
|
||||
|
||||
bench-runner-signalling-shim:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.runner
|
||||
profiles:
|
||||
- signalling-shim
|
||||
depends_on:
|
||||
p2p-signalling-shim:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
BENCH_COMMAND: ${BENCH_COMMAND:-cases}
|
||||
BENCH_CASES: ${BENCH_CASES:-p2p-signalling-netem-home-wifi}
|
||||
BENCH_REPEAT_COUNT: ${BENCH_REPEAT_COUNT:-1}
|
||||
BENCH_CASES_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_SIGNAL_SHIM_RELAY: ws://p2p-signalling-shim:7777/
|
||||
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
|
||||
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
|
||||
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
|
||||
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
|
||||
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
|
||||
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
|
||||
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
|
||||
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
|
||||
BENCH_LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
|
||||
volumes:
|
||||
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
|
||||
|
||||
p2p-split-host:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.runner
|
||||
profiles:
|
||||
- p2p-split
|
||||
depends_on:
|
||||
nostr-relay:
|
||||
condition: service_healthy
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
environment:
|
||||
BENCH_COMMAND: p2p-split-node
|
||||
BENCH_P2P_SPLIT_ROLE: host
|
||||
BENCH_SPLIT_RUN_ID: ${BENCH_SPLIT_RUN_ID:-bench-split-run}
|
||||
BENCH_SPLIT_WORK_ROOT: /p2p-work
|
||||
BENCH_SPLIT_RESULT_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_RELAY: ws://nostr-relay:7777/
|
||||
BENCH_APP_ID: ${BENCH_APP_ID:-self-hosted-livesync-cli-benchmark}
|
||||
BENCH_ROOM_ID: ${BENCH_ROOM_ID:-bench-split-room}
|
||||
BENCH_PASSPHRASE: ${BENCH_PASSPHRASE:-bench-split-passphrase}
|
||||
BENCH_TURN_SERVERS: ${BENCH_TURN_SERVERS:-}
|
||||
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
|
||||
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
|
||||
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
|
||||
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
|
||||
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
|
||||
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
|
||||
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
|
||||
BENCH_NETEM_ENABLED: ${BENCH_NETEM_ENABLED:-1}
|
||||
BENCH_NETWORK_PROFILE: ${BENCH_NETWORK_PROFILE:-home-wifi}
|
||||
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
|
||||
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
|
||||
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
|
||||
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
|
||||
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
|
||||
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
|
||||
NETEM_MTU: ${NETEM_MTU:-1500}
|
||||
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
|
||||
LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
|
||||
volumes:
|
||||
- p2p-split-work:/p2p-work
|
||||
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
|
||||
|
||||
p2p-split-client:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.runner
|
||||
profiles:
|
||||
- p2p-split
|
||||
depends_on:
|
||||
nostr-relay:
|
||||
condition: service_healthy
|
||||
p2p-split-host:
|
||||
condition: service_started
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
environment:
|
||||
BENCH_COMMAND: p2p-split-node
|
||||
BENCH_P2P_SPLIT_ROLE: client
|
||||
BENCH_SPLIT_RUN_ID: ${BENCH_SPLIT_RUN_ID:-bench-split-run}
|
||||
BENCH_SPLIT_WORK_ROOT: /p2p-work
|
||||
BENCH_SPLIT_RESULT_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_RELAY: ws://nostr-relay:7777/
|
||||
BENCH_APP_ID: ${BENCH_APP_ID:-self-hosted-livesync-cli-benchmark}
|
||||
BENCH_ROOM_ID: ${BENCH_ROOM_ID:-bench-split-room}
|
||||
BENCH_PASSPHRASE: ${BENCH_PASSPHRASE:-bench-split-passphrase}
|
||||
BENCH_TURN_SERVERS: ${BENCH_TURN_SERVERS:-}
|
||||
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
|
||||
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
|
||||
BENCH_NETEM_ENABLED: ${BENCH_NETEM_ENABLED:-1}
|
||||
BENCH_NETWORK_PROFILE: ${BENCH_NETWORK_PROFILE:-home-wifi}
|
||||
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
|
||||
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
|
||||
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
|
||||
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
|
||||
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
|
||||
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
|
||||
NETEM_MTU: ${NETEM_MTU:-1500}
|
||||
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
|
||||
LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
|
||||
volumes:
|
||||
- p2p-split-work:/p2p-work
|
||||
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
|
||||
|
||||
netem-smoke:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: test/bench-network/Dockerfile.netem
|
||||
profiles:
|
||||
- netem
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
environment:
|
||||
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
|
||||
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
|
||||
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
|
||||
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
|
||||
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
|
||||
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
|
||||
NETEM_MTU: ${NETEM_MTU:-1500}
|
||||
NETEM_RESULT_ROOT: /bench-results
|
||||
volumes:
|
||||
- ./bench-results:/bench-results
|
||||
|
||||
volumes:
|
||||
p2p-split-work:
|
||||
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env sh
|
||||
set -eu
|
||||
|
||||
profile="${NETEM_PROFILE:-home-wifi}"
|
||||
iface="${NETEM_INTERFACE:-eth0}"
|
||||
delay_ms="${NETEM_DELAY_MS:-20}"
|
||||
jitter_ms="${NETEM_JITTER_MS:-5}"
|
||||
loss_percent="${NETEM_LOSS_PERCENT:-0.1}"
|
||||
bandwidth_mbit="${NETEM_BANDWIDTH_MBIT:-100}"
|
||||
mtu="${NETEM_MTU:-1500}"
|
||||
out_root="${NETEM_RESULT_ROOT:-/bench-results}"
|
||||
timestamp="$(date -u +%Y%m%d-%H%M%S)"
|
||||
out_dir="${out_root}/netem-smoke-${timestamp}"
|
||||
out_file="${out_dir}/summary.json"
|
||||
|
||||
mkdir -p "$out_dir"
|
||||
|
||||
if ! ip link show "$iface" >/dev/null 2>&1; then
|
||||
echo "Network interface '$iface' was not found" >&2
|
||||
ip addr >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
ip link set dev "$iface" mtu "$mtu"
|
||||
tc qdisc del dev "$iface" root >/dev/null 2>&1 || true
|
||||
tc qdisc add dev "$iface" root netem \
|
||||
delay "${delay_ms}ms" "${jitter_ms}ms" \
|
||||
loss "${loss_percent}%" \
|
||||
rate "${bandwidth_mbit}mbit"
|
||||
|
||||
json_lines() {
|
||||
awk '
|
||||
{
|
||||
gsub(/\\/, "\\\\");
|
||||
gsub(/"/, "\\\"");
|
||||
printf "%s \"%s\"", (NR == 1 ? "" : ",\n"), $0;
|
||||
}
|
||||
'
|
||||
}
|
||||
|
||||
ip_addr="$(ip addr show "$iface" | json_lines)"
|
||||
ip_route="$(ip route | json_lines)"
|
||||
tc_qdisc="$(tc qdisc show dev "$iface" | json_lines)"
|
||||
|
||||
cat > "$out_file" <<EOF
|
||||
{
|
||||
"simulationTier": 2,
|
||||
"mode": "netem-smoke",
|
||||
"profile": "$profile",
|
||||
"interface": "$iface",
|
||||
"netem": {
|
||||
"delayMs": $delay_ms,
|
||||
"jitterMs": $jitter_ms,
|
||||
"lossPercent": $loss_percent,
|
||||
"bandwidthMbit": $bandwidth_mbit,
|
||||
"mtu": $mtu
|
||||
},
|
||||
"ipAddr": [
|
||||
$ip_addr
|
||||
],
|
||||
"ipRoute": [
|
||||
$ip_route
|
||||
],
|
||||
"tcQdisc": [
|
||||
$tc_qdisc
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cat "$out_file"
|
||||
echo "[netem-smoke] result file: $out_file"
|
||||
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env sh
|
||||
set -eu
|
||||
|
||||
profile="${NETEM_PROFILE:-home-wifi}"
|
||||
iface="${NETEM_INTERFACE:-eth0}"
|
||||
delay_ms="${NETEM_DELAY_MS:-20}"
|
||||
jitter_ms="${NETEM_JITTER_MS:-5}"
|
||||
loss_percent="${NETEM_LOSS_PERCENT:-0.1}"
|
||||
bandwidth_mbit="${NETEM_BANDWIDTH_MBIT:-100}"
|
||||
mtu="${NETEM_MTU:-1500}"
|
||||
listen_port="${SHIM_LISTEN_PORT:-5984}"
|
||||
target_host="${SHIM_TARGET_HOST:-couchdb}"
|
||||
target_port="${SHIM_TARGET_PORT:-5984}"
|
||||
out_root="${NETEM_RESULT_ROOT:-/bench-results}"
|
||||
timestamp="$(date -u +%Y%m%d-%H%M%S)"
|
||||
out_dir="${out_root}/netem-shim-${profile}-${timestamp}"
|
||||
out_file="${out_dir}/summary.json"
|
||||
|
||||
json_lines() {
|
||||
awk '
|
||||
{
|
||||
gsub(/\\/, "\\\\");
|
||||
gsub(/"/, "\\\"");
|
||||
printf "%s \"%s\"", (NR == 1 ? "" : ",\n"), $0;
|
||||
}
|
||||
'
|
||||
}
|
||||
|
||||
mkdir -p "$out_dir"
|
||||
|
||||
if ! ip link show "$iface" >/dev/null 2>&1; then
|
||||
echo "Network interface '$iface' was not found" >&2
|
||||
ip addr >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
ip link set dev "$iface" mtu "$mtu"
|
||||
tc qdisc del dev "$iface" root >/dev/null 2>&1 || true
|
||||
tc qdisc add dev "$iface" root netem \
|
||||
delay "${delay_ms}ms" "${jitter_ms}ms" \
|
||||
loss "${loss_percent}%" \
|
||||
rate "${bandwidth_mbit}mbit"
|
||||
|
||||
ip_addr="$(ip addr show "$iface" | json_lines)"
|
||||
ip_route="$(ip route | json_lines)"
|
||||
tc_qdisc="$(tc qdisc show dev "$iface" | json_lines)"
|
||||
|
||||
cat > "$out_file" <<EOF
|
||||
{
|
||||
"simulationTier": 2,
|
||||
"mode": "netem-tcp-shim",
|
||||
"profile": "$profile",
|
||||
"interface": "$iface",
|
||||
"listenPort": $listen_port,
|
||||
"targetHost": "$target_host",
|
||||
"targetPort": $target_port,
|
||||
"netem": {
|
||||
"delayMs": $delay_ms,
|
||||
"jitterMs": $jitter_ms,
|
||||
"lossPercent": $loss_percent,
|
||||
"bandwidthMbit": $bandwidth_mbit,
|
||||
"mtu": $mtu
|
||||
},
|
||||
"ipAddr": [
|
||||
$ip_addr
|
||||
],
|
||||
"ipRoute": [
|
||||
$ip_route
|
||||
],
|
||||
"tcQdisc": [
|
||||
$tc_qdisc
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cat "$out_file"
|
||||
echo "[netem-shim] forwarding 0.0.0.0:${listen_port} to ${target_host}:${target_port}"
|
||||
echo "[netem-shim] result file: $out_file"
|
||||
|
||||
exec socat "TCP-LISTEN:${listen_port},fork,reuseaddr" "TCP:${target_host}:${target_port}"
|
||||
@@ -0,0 +1,19 @@
|
||||
#!/usr/bin/env sh
|
||||
set -eu
|
||||
|
||||
case "${BENCH_COMMAND:-cases}" in
|
||||
cases)
|
||||
exec deno task bench:cases
|
||||
;;
|
||||
latency-sweep)
|
||||
exec deno task bench:latency-sweep
|
||||
;;
|
||||
p2p-split-node)
|
||||
exec deno task bench:p2p-split-node
|
||||
;;
|
||||
*)
|
||||
echo "Unknown BENCH_COMMAND: ${BENCH_COMMAND}" >&2
|
||||
echo "Expected one of: cases, latency-sweep, p2p-split-node" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
+16
-36
@@ -3,6 +3,22 @@ Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
|
||||
|
||||
The head note of 0.25 is now in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Because 0.25 got a lot of updates, thankfully, compatibility is kept and we do not need breaking changes! In other words, when get enough stabled. The next version will be v1.0.0. Even though it my hope.
|
||||
|
||||
## 0.25.80
|
||||
|
||||
7th July, 2026
|
||||
|
||||
### Fixed
|
||||
|
||||
- Improved Markdown conflict auto-merge so that non-overlapping edits are merged while overlapping delete-and-edit cases remain visible for manual resolution (#993).
|
||||
- Behaviour change:
|
||||
- When one side deletes an unchanged line and the other side edits a different region, the deleted line is no longer reintroduced into the merged result.
|
||||
- When one side deletes a line and the other side modifies that same line, the conflict is preserved instead of silently choosing one side.
|
||||
- Fixed an issue where applying a newer database entry to storage could incorrectly preserve an older local file as a conflict (#994).
|
||||
- Behaviour change:
|
||||
- Local storage is preserved as a conflict when it may contain unsynchronised changes that are not represented in the revision history. A newer incoming text entry is applied without creating a conflict only when it clearly extends the existing local text.
|
||||
- Fixed an issue where choosing Disable and then Overwrite in Hidden File Sync could silently skip hidden files, because the overwrite setup ran while hidden file synchronisation was still disabled (#989, PR #992).
|
||||
- Hidden File Sync is now re-enabled before the Fetch, Overwrite, or Merge initialisation runs, instead of after it completes. If that initialisation fails, the setting may remain enabled.
|
||||
|
||||
## 0.25.79
|
||||
|
||||
29th June, 2026
|
||||
@@ -96,41 +112,5 @@ Also, this update is a very large one, even if we had a lot of time, and we had
|
||||
- Some dependencies have been updated.
|
||||
- Now we check the compatibility with iOS 15 in the CI tests to ensure the plugin continues to work on older iOS versions even after we upgrade some dependencies.
|
||||
|
||||
## 0.25.74
|
||||
|
||||
8th June, 2026
|
||||
|
||||
### Fixed
|
||||
|
||||
- Fixed an issue where disabling hidden file synchronisation did not take effect, allowing non-target hidden files to continue to be processed and synchronised by replication or boot-sequence scan (#941).
|
||||
- Prevented the automatic merging of conflicted revisions when one of the revisions has been deleted, which was causing deleted files to reappear (#911).
|
||||
- The startup sequence now saves the state more effectively (Thank you so much for @bmcyver)!
|
||||
|
||||
## Only CLI
|
||||
|
||||
8th June, 2026
|
||||
|
||||
I should also consider the version numbering for the CLI...
|
||||
|
||||
### Improved
|
||||
|
||||
- Added new remote database management commands: `remote-status`, `unlock-remote`, `lock-remote`, and `mark-resolved`.
|
||||
- --vault option is now available for daemon and mirror commands! (Thank you so much for @starskyzheng)!
|
||||
- Decoupled the database directory path from the actual vault directory path using the `--vault` (or `-V`) option.
|
||||
|
||||
### Fixed (preventive)
|
||||
|
||||
- Validated that the specified vault path exists and is indeed a directory before starting the CLI.
|
||||
- Integrated path resolution and validations for one-off commands (such as `'push'`, `'pull'`, `'cat'`, `'rm'`, `'info'`, and `'resolve'`) against the decoupled vault path instead of the database path.
|
||||
|
||||
## 0.25.73
|
||||
|
||||
4th June, 2026
|
||||
|
||||
### Fixed
|
||||
|
||||
- Adjust CouchDB's database name checking to its specification (#926).
|
||||
- `Reset Syncronisation on This Device` for minio and P2P is now working properly.
|
||||
|
||||
Full notes are in
|
||||
[updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).
|
||||
|
||||
@@ -4,6 +4,64 @@ Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
|
||||
The head note of 0.25 is now in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Because 0.25 got a lot of updates, thankfully, compatibility is kept and we do not need breaking changes! In other words, when get enough stabled. The next version will be v1.0.0. Even though it my hope.
|
||||
|
||||
|
||||
## 0.25.80
|
||||
|
||||
7th July, 2026
|
||||
|
||||
### Fixed
|
||||
|
||||
- Improved Markdown conflict auto-merge so that non-overlapping edits are merged while overlapping delete-and-edit cases remain visible for manual resolution (#993).
|
||||
- Behaviour change:
|
||||
- When one side deletes an unchanged line and the other side edits a different region, the deleted line is no longer reintroduced into the merged result.
|
||||
- When one side deletes a line and the other side modifies that same line, the conflict is preserved instead of silently choosing one side.
|
||||
- Fixed an issue where applying a newer database entry to storage could incorrectly preserve an older local file as a conflict (#994).
|
||||
- Behaviour change:
|
||||
- Local storage is preserved as a conflict when it may contain unsynchronised changes that are not represented in the revision history. A newer incoming text entry is applied without creating a conflict only when it clearly extends the existing local text.
|
||||
- Fixed an issue where choosing Disable and then Overwrite in Hidden File Sync could silently skip hidden files, because the overwrite setup ran while hidden file synchronisation was still disabled (#989, PR #992).
|
||||
- Hidden File Sync is now re-enabled before the Fetch, Overwrite, or Merge initialisation runs, instead of after it completes. If that initialisation fails, the setting may remain enabled.
|
||||
|
||||
## 0.25.79
|
||||
|
||||
29th June, 2026
|
||||
|
||||
### Fixed
|
||||
|
||||
- Fast Fetch now retries transient stream interruptions and resumes from the latest persisted checkpoint, instead of starting over after ordinary network or platform interruptions (#977, PR #978; commonlib PR #59). Thank you so much for @apple-ouyang for the fix!
|
||||
- Simple Fetch now remembers the selected setup choices while an interrupted Fetch All operation is still pending, so users are not asked the same questions again on retry (#977, PR #978). Thank you so much for @apple-ouyang for the fix!
|
||||
- No longer hidden storage events, such as `.git` paths, reach the normal target-file filter when internal file synchronisation is disabled. This avoids noisy non-target logs before those files are skipped (commonlib PR #60). Thank you so much for @apple-ouyang for the fix!
|
||||
- Fixed an issue where a file deleted from storage could be resurrected by the offline scanner because the database tombstone was not written when the storage file was already gone (commonlib PR #56). Thank you so much for @cosmic-fire-eng for the fix!
|
||||
|
||||
### Improved
|
||||
|
||||
- Local database maintenance commands now ask before applying the required chunk settings, and can apply those prerequisites before continuing (#980, PR #981). Thank you so much for @apple-ouyang for the improvement!
|
||||
- Improved CouchDB replication event handling by using the new `StreamInbox` helper from `octagonal-wheels` (commonlib PR #62).
|
||||
|
||||
### Documentation
|
||||
|
||||
- Added `nginx` to the setup documentation table of contents (PR #976). Thank you so much for @kiraventom for the improvement!
|
||||
|
||||
### Miscellaneous
|
||||
|
||||
- Updated `octagonal-wheels` to `0.1.47` across the plug-in and workspace packages to use the newly published helper modules.
|
||||
|
||||
## 0.25.78
|
||||
|
||||
23rd June, 2026
|
||||
|
||||
### Fixed
|
||||
- No longer fast synchronisation (a.k.a. Fast Fetch) causes a rewind and re-fetch of the entire database when some errors occur during the process (#972, PR #973). Thank you so much for @apple-ouyang for the fix!
|
||||
|
||||
### Improved
|
||||
|
||||
- Overhauled the Object Storage (e.g., MinIO and S3) replication engine ('Journal Replicator 2nd Edition').
|
||||
- It now leverages the standard Web Streams API for a resilient, backpressure-aware architecture, reducing memory footprints/temporary storage usage on large vaults.
|
||||
- Decoupled the physical storage logic to make it easier to add new storage backends in the future.
|
||||
- Stricter compliance with CouchDB's replication protocol (proper `_revisions` transfers with `new_edits: false`) when using Object Storage.
|
||||
|
||||
### Testing
|
||||
- Added comprehensive unit tests for the new `JournalSyncCore` engine, covering streams, backpressure, and `new_edits: false` validation.
|
||||
- Improved integration test workflows in the CI pipeline to run MinIO tests automatically using standard environment variables.
|
||||
|
||||
## 0.25.77
|
||||
|
||||
19th June, 2026
|
||||
|
||||
Reference in New Issue
Block a user