public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-07-17 01:56:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

Normalise tracked text files and formatter scope

Browse Source
This commit is contained in:
vorotamoroz
2026-07-17 01:05:29 +00:00
parent e114f66fb2
commit 3a1a8948d2
13 changed files with 474 additions and 730 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
+30 -30
View File
@@ -1,33 +1,33 @@
# Git history # Git history
.git/ .git/
.gitignore .gitignore
# Dependencies — re-installed inside Docker # Dependencies — re-installed inside Docker
node_modules/ node_modules/
src/apps/cli/node_modules/ src/apps/cli/node_modules/
# Pre-built CLI output — rebuilt inside Docker # Pre-built CLI output — rebuilt inside Docker
src/apps/cli/dist/ src/apps/cli/dist/
# Obsidian plugin build outputs # Obsidian plugin build outputs
main.js main.js
main_org.js main_org.js
pouchdb-browser.js pouchdb-browser.js
production/ production/
# Test coverage and reports # Test coverage and reports
coverage/ coverage/
test/bench-network/bench-results/ test/bench-network/bench-results/
src/apps/cli/testdeno/bench-results/ src/apps/cli/testdeno/bench-results/
# Local environment / secrets # Local environment / secrets
.env .env
*.env *.env
.test.env .test.env
# local config files # local config files
*.local *.local
# OS artefacts # OS artefacts
.DS_Store .DS_Store
Thumbs.db Thumbs.db
.gitattributes
+5 -1
View File
@@ -13,6 +13,11 @@
*.mjs text eol=lf *.mjs text eol=lf
*.css text eol=lf *.css text eol=lf
# Extensionless text files
.gitignore text eol=lf
.dockerignore text eol=lf
Dockerfile text eol=lf
# Binary files — no line ending conversion # Binary files — no line ending conversion
*.png binary *.png binary
*.jpg binary *.jpg binary
@@ -21,4 +26,3 @@
*.ico binary *.ico binary
*.woff2 binary *.woff2 binary
*.woff binary *.woff binary
*.sh text eol=lf
.prettierignore
+2 -1
View File
@@ -1,4 +1,5 @@
pouchdb-browser.js pouchdb-browser.js
main_org.js main_org.js
main.js main.js
_types/** _types/**
src/lib/**
docs/setup_own_server_cn.md
+152 -152
View File
@@ -1,152 +1,152 @@
# 在你自己的服务器上设置 CouchDB # 在你自己的服务器上设置 CouchDB
## 目录 ## 目录
- [配置 CouchDB](#配置-CouchDB) - [配置 CouchDB](#配置-CouchDB)
- [运行 CouchDB](#运行-CouchDB) - [运行 CouchDB](#运行-CouchDB)
- [Docker CLI](#docker-cli) - [Docker CLI](#docker-cli)
- [Docker Compose](#docker-compose) - [Docker Compose](#docker-compose)
- [创建数据库](#创建数据库) - [创建数据库](#创建数据库)
- [从移动设备访问](#从移动设备访问) - [从移动设备访问](#从移动设备访问)
- [移动设备测试](#移动设备测试) - [移动设备测试](#移动设备测试)
- [设置你的域名](#设置你的域名) - [设置你的域名](#设置你的域名)
--- ---
> 注：提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。推荐直接使用该 docker-compose 配置进行搭建。（若使用，请查阅链接中的文档，而不是这个文档） > 注：提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。推荐直接使用该 docker-compose 配置进行搭建。（若使用，请查阅链接中的文档，而不是这个文档）
## 配置 CouchDB ## 配置 CouchDB
设置 CouchDB 的最简单方法是使用 [CouchDB docker image]((https://hub.docker.com/_/couchdb)). 设置 CouchDB 的最简单方法是使用 [CouchDB docker image]((https://hub.docker.com/_/couchdb)).
需要修改一些 `local.ini` 中的配置，以让它可以用于 Self-hosted LiveSync，如下： 需要修改一些 `local.ini` 中的配置，以让它可以用于 Self-hosted LiveSync，如下：
``` ```
[couchdb] [couchdb]
single_node=true single_node=true
max_document_size = 50000000 max_document_size = 50000000
[chttpd] [chttpd]
require_valid_user = true require_valid_user = true
max_http_request_size = 4294967296 max_http_request_size = 4294967296
[chttpd_auth] [chttpd_auth]
require_valid_user = true require_valid_user = true
authentication_redirect = /_utils/session.html authentication_redirect = /_utils/session.html
[httpd] [httpd]
WWW-Authenticate = Basic realm="couchdb" WWW-Authenticate = Basic realm="couchdb"
enable_cors = true enable_cors = true
[cors] [cors]
origins = app://obsidian.md,capacitor://localhost,http://localhost origins = app://obsidian.md,capacitor://localhost,http://localhost
credentials = true credentials = true
headers = accept, authorization, content-type, origin, referer headers = accept, authorization, content-type, origin, referer
methods = GET, PUT, POST, HEAD, DELETE methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600 max_age = 3600
``` ```
## 运行 CouchDB ## 运行 CouchDB
### Docker CLI ### Docker CLI
你可以通过指定 `local.ini` 配置运行 CouchDB: 你可以通过指定 `local.ini` 配置运行 CouchDB:
``` ```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb $ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
``` ```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径* *记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
后台运行: 后台运行:
``` ```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb $ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
``` ```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径* *记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
### Docker Compose ### Docker Compose
创建一个文件夹, 将你的 `local.ini` 放在文件夹内, 然后在文件夹内创建 `docker-compose.yml`. 请确保对 `local.ini` 有读写权限并且确保在容器运行后能创建 `data` 文件夹. 文件夹结构大概如下: 创建一个文件夹, 将你的 `local.ini` 放在文件夹内, 然后在文件夹内创建 `docker-compose.yml`. 请确保对 `local.ini` 有读写权限并且确保在容器运行后能创建 `data` 文件夹. 文件夹结构大概如下:
``` ```
obsidian-livesync obsidian-livesync
├── docker-compose.yml ├── docker-compose.yml
└── local.ini └── local.ini
``` ```
可以参照以下内容编辑 `docker-compose.yml`: 可以参照以下内容编辑 `docker-compose.yml`:
```yaml ```yaml
services: services:
couchdb: couchdb:
image: couchdb image: couchdb
container_name: obsidian-livesync container_name: obsidian-livesync
user: 1000:1000 user: 1000:1000
environment: environment:
- COUCHDB_USER=admin - COUCHDB_USER=admin
- COUCHDB_PASSWORD=password - COUCHDB_PASSWORD=password
volumes: volumes:
- ./data:/opt/couchdb/data - ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini - ./local.ini:/opt/couchdb/etc/local.ini
ports: ports:
- 5984:5984 - 5984:5984
restart: unless-stopped restart: unless-stopped
``` ```
最后, 创建并启动容器: 最后, 创建并启动容器:
``` ```
# -d will launch detached so the container runs in background # -d will launch detached so the container runs in background
docker-compose up -d docker-compose up -d
``` ```
## 创建数据库 ## 创建数据库
CouchDB 部署成功后, 需要手动创建一个数据库, 方便插件连接并同步. CouchDB 部署成功后, 需要手动创建一个数据库, 方便插件连接并同步.
1. 访问 `http://localhost:5984/_utils`, 输入帐号密码后进入管理页面 1. 访问 `http://localhost:5984/_utils`, 输入帐号密码后进入管理页面
2. 点击 Create Database, 然后根据个人喜好创建数据库 2. 点击 Create Database, 然后根据个人喜好创建数据库
## 从移动设备访问 ## 从移动设备访问
如果你想要从移动设备访问 Self-hosted LiveSync，你需要一个合法的 SSL 证书。 如果你想要从移动设备访问 Self-hosted LiveSync，你需要一个合法的 SSL 证书。
### 移动设备测试 ### 移动设备测试
测试时，[localhost.run](http://localhost.run/) 这一类的反向隧道服务很实用。（非必须，只是用于终端设备不方便 ssh 的时候的备选方案） 测试时，[localhost.run](http://localhost.run/) 这一类的反向隧道服务很实用。（非必须，只是用于终端设备不方便 ssh 的时候的备选方案）
``` ```
$ ssh -R 80:localhost:5984 nokey@localhost.run $ ssh -R 80:localhost:5984 nokey@localhost.run
Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts. Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts.
=============================================================================== ===============================================================================
Welcome to localhost.run! Welcome to localhost.run!
Follow your favourite reverse tunnel at [https://twitter.com/localhost_run]. Follow your favourite reverse tunnel at [https://twitter.com/localhost_run].
**You need a SSH key to access this service.** **You need a SSH key to access this service.**
If you get a permission denied follow Gitlab's most excellent howto: If you get a permission denied follow Gitlab's most excellent howto:
https://docs.gitlab.com/ee/ssh/ https://docs.gitlab.com/ee/ssh/
*Only rsa and ed25519 keys are supported* *Only rsa and ed25519 keys are supported*
To set up and manage custom domains go to https://admin.localhost.run/ To set up and manage custom domains go to https://admin.localhost.run/
More details on custom domains (and how to enable subdomains of your custom More details on custom domains (and how to enable subdomains of your custom
domain) at https://localhost.run/docs/custom-domains domain) at https://localhost.run/docs/custom-domains
To explore using localhost.run visit the documentation site: To explore using localhost.run visit the documentation site:
https://localhost.run/docs/ https://localhost.run/docs/
=============================================================================== ===============================================================================
** your connection id is xxxxxxxxxxxxxxxxxxxxxxxxxxxx, please mention it if you send me a message about an issue. ** ** your connection id is xxxxxxxxxxxxxxxxxxxxxxxxxxxx, please mention it if you send me a message about an issue. **
xxxxxxxx.localhost.run tunneled with tls termination, https://xxxxxxxx.localhost.run xxxxxxxx.localhost.run tunneled with tls termination, https://xxxxxxxx.localhost.run
Connection to localhost.run closed by remote host. Connection to localhost.run closed by remote host.
Connection to localhost.run closed. Connection to localhost.run closed.
``` ```
https://xxxxxxxx.localhost.run 即为临时服务器地址。 https://xxxxxxxx.localhost.run 即为临时服务器地址。
### 设置你的域名 ### 设置你的域名
设置一个指向你服务器的 A 记录，并根据需要设置反向代理。 设置一个指向你服务器的 A 记录，并根据需要设置反向代理。
Note: 不推荐将 CouchDB 挂载到根目录 Note: 不推荐将 CouchDB 挂载到根目录
可以使用 Caddy 很方便的给服务器加上 SSL 功能 可以使用 Caddy 很方便的给服务器加上 SSL 功能
提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。 提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。
注意检查服务器日志，当心恶意访问。 注意检查服务器日志，当心恶意访问。
src/apps/_test/storageAdapterContract.ts
+12 -3
View File
@@ -87,7 +87,10 @@ export const storageAdapterContractCases: readonly StorageAdapterContractCase[]
name: "keeps operations inside the configured root", name: "keeps operations inside the configured root",
async run(adapter) { async run(adapter) {
await assertRejects(() => adapter.exists("../outside"), "parent traversal should be rejected"); await assertRejects(() => adapter.exists("../outside"), "parent traversal should be rejected");
await assertRejects(() => adapter.write("nested/../outside", "content"), "nested traversal should be rejected"); await assertRejects(
() => adapter.write("nested/../outside", "content"),
"nested traversal should be rejected"
);
await assertRejects(() => adapter.read("/absolute"), "absolute paths should be rejected"); await assertRejects(() => adapter.read("/absolute"), "absolute paths should be rejected");
await assertRejects(() => adapter.read("C:\\absolute"), "drive-qualified paths should be rejected"); await assertRejects(() => adapter.read("C:\\absolute"), "drive-qualified paths should be rejected");
await assertRejects(() => adapter.read("nested\\outside"), "backslash-separated paths should be rejected"); await assertRejects(() => adapter.read("nested\\outside"), "backslash-separated paths should be rejected");
@@ -101,8 +104,14 @@ export const storageAdapterContractCases: readonly StorageAdapterContractCase[]
assertEqual(await adapter.exists(""), true, "the configured root should exist"); assertEqual(await adapter.exists(""), true, "the configured root should exist");
assertEqual((await adapter.stat(""))?.type, "folder", "the configured root should be a folder"); assertEqual((await adapter.stat(""))?.type, "folder", "the configured root should be a folder");
assertEqual(await adapter.list(""), { files: [], folders: [] }, "the configured root should be listable"); assertEqual(await adapter.list(""), { files: [], folders: [] }, "the configured root should be listable");
await assertRejects(() => adapter.write("", "content"), "writing over the configured root should be rejected"); await assertRejects(
await assertRejects(() => adapter.append("", "content"), "appending to the configured root should be rejected"); () => adapter.write("", "content"),
"writing over the configured root should be rejected"
);
await assertRejects(
() => adapter.append("", "content"),
"appending to the configured root should be rejected"
);
}, },
}, },
]; ];
src/apps/cli/.gitignore
+8 -8
View File
@@ -1,9 +1,9 @@
.livesync .livesync
test/* test/*
!test/*.sh !test/*.sh
test/test-init.local.sh test/test-init.local.sh
node_modules node_modules
.*.json .*.json
*.env *.env
!.test.env !.test.env
bench-results bench-results
src/apps/cli/Dockerfile
+111 -111
View File
@@ -1,111 +1,111 @@
# syntax=docker/dockerfile:1 # syntax=docker/dockerfile:1
# #
# Self-hosted LiveSync CLI — Docker image # Self-hosted LiveSync CLI — Docker image
# #
# Build (from the repository root): # Build (from the repository root):
# docker build -f src/apps/cli/Dockerfile -t livesync-cli . # docker build -f src/apps/cli/Dockerfile -t livesync-cli .
# #
# Run: # Run:
# docker run --rm -v /path/to/your/vault:/data livesync-cli sync # docker run --rm -v /path/to/your/vault:/data livesync-cli sync
# docker run --rm -v /path/to/your/vault:/data livesync-cli ls # docker run --rm -v /path/to/your/vault:/data livesync-cli ls
# docker run --rm -v /path/to/your/vault:/data livesync-cli init-settings # docker run --rm -v /path/to/your/vault:/data livesync-cli init-settings
# docker run --rm -v /path/to/your/vault:/data livesync-cli --help # docker run --rm -v /path/to/your/vault:/data livesync-cli --help
# #
# The first positional argument (database-path) is automatically set to /data. # The first positional argument (database-path) is automatically set to /data.
# Mount your vault at /data, or override with: -e LIVESYNC_DB_PATH=/other/path # Mount your vault at /data, or override with: -e LIVESYNC_DB_PATH=/other/path
# #
# P2P (WebRTC) networking — important notes # P2P (WebRTC) networking — important notes
# ----------------------------------------- # -----------------------------------------
# The P2P replicator (p2p-host / p2p-sync / p2p-peers) uses WebRTC, which # The P2P replicator (p2p-host / p2p-sync / p2p-peers) uses WebRTC, which
# generates ICE candidates of three kinds: # generates ICE candidates of three kinds:
# #
# host — the container's bridge IP (172.17.x.x). Unreachable from outside # host — the container's bridge IP (172.17.x.x). Unreachable from outside
# the Docker bridge, so LAN peers cannot connect via this candidate. # the Docker bridge, so LAN peers cannot connect via this candidate.
# srflx — the host's public IP, obtained via STUN reflection. Works fine # srflx — the host's public IP, obtained via STUN reflection. Works fine
# over the internet even with the default bridge network. # over the internet even with the default bridge network.
# relay — traffic relayed through a TURN server. Always reachable regardless # relay — traffic relayed through a TURN server. Always reachable regardless
# of network mode. # of network mode.
# #
# Recommended network modes per use-case: # Recommended network modes per use-case:
# #
# LAN P2P (Linux only) # LAN P2P (Linux only)
# docker run --network host ... # docker run --network host ...
# This exposes the real host IP as the 'host' candidate so LAN peers can # This exposes the real host IP as the 'host' candidate so LAN peers can
# connect directly. --network host is not available on Docker Desktop for # connect directly. --network host is not available on Docker Desktop for
# macOS or Windows. # macOS or Windows.
# #
# LAN P2P (macOS / Windows Docker Desktop) # LAN P2P (macOS / Windows Docker Desktop)
# Configure a TURN server in settings (P2P_turnServers / P2P_turnUsername / # Configure a TURN server in settings (P2P_turnServers / P2P_turnUsername /
# P2P_turnCredential). All data is then relayed through the TURN server, # P2P_turnCredential). All data is then relayed through the TURN server,
# bypassing the bridge-network limitation. # bypassing the bridge-network limitation.
# #
# Internet P2P # Internet P2P
# Default bridge network is sufficient; the srflx candidate carries the # Default bridge network is sufficient; the srflx candidate carries the
# host's public IP and peers can connect normally. # host's public IP and peers can connect normally.
# #
# CouchDB sync only (no P2P) # CouchDB sync only (no P2P)
# Default bridge network. No special configuration required. # Default bridge network. No special configuration required.
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Stage 1 — builder # Stage 1 — builder
# Full Node.js environment to compile native modules and bundle the CLI. # Full Node.js environment to compile native modules and bundle the CLI.
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
FROM node:22-slim AS builder FROM node:22-slim AS builder
# Build tools required by native Node.js addons (mainly leveldown) # Build tools required by native Node.js addons (mainly leveldown)
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 make g++ \ && apt-get install -y --no-install-recommends python3 make g++ \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /build WORKDIR /build
# Install workspace dependencies first (layer-cache friendly) # Install workspace dependencies first (layer-cache friendly)
COPY package.json package-lock.json ./ COPY package.json package-lock.json ./
RUN npm install RUN npm install
# Copy the full source tree and build the CLI bundle # Copy the full source tree and build the CLI bundle
COPY . . COPY . .
RUN cd src/apps/cli && npm run build RUN cd src/apps/cli && npm run build
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Stage 2 — runtime-deps # Stage 2 — runtime-deps
# Install only the external (unbundled) packages that the CLI requires at # Install only the external (unbundled) packages that the CLI requires at
# runtime. Native addons are compiled here against the same base image that # runtime. Native addons are compiled here against the same base image that
# the final runtime stage uses. # the final runtime stage uses.
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
FROM node:22-slim AS runtime-deps FROM node:22-slim AS runtime-deps
# Build tools required to compile native addons # Build tools required to compile native addons
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 make g++ \ && apt-get install -y --no-install-recommends python3 make g++ \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /deps WORKDIR /deps
# package.json lists only the packages that the CLI requires # package.json lists only the packages that the CLI requires
COPY src/apps/cli/package.json ./package.json COPY src/apps/cli/package.json ./package.json
RUN npm install --omit=dev RUN npm install --omit=dev
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Stage 3 — runtime # Stage 3 — runtime
# Minimal image: pre-compiled native modules + CLI bundle only. # Minimal image: pre-compiled native modules + CLI bundle only.
# No build tools are included, keeping the image small. # No build tools are included, keeping the image small.
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
FROM node:22-slim FROM node:22-slim
WORKDIR /app WORKDIR /app
# Copy pre-compiled external node_modules from runtime-deps stage # Copy pre-compiled external node_modules from runtime-deps stage
COPY --from=runtime-deps /deps/node_modules ./node_modules COPY --from=runtime-deps /deps/node_modules ./node_modules
# Copy the built CLI bundle from builder stage # Copy the built CLI bundle from builder stage
COPY --from=builder /build/src/apps/cli/dist ./dist COPY --from=builder /build/src/apps/cli/dist ./dist
# Install entrypoint wrapper # Install entrypoint wrapper
COPY src/apps/cli/docker-entrypoint.sh /usr/local/bin/livesync-cli COPY src/apps/cli/docker-entrypoint.sh /usr/local/bin/livesync-cli
RUN chmod +x /usr/local/bin/livesync-cli RUN chmod +x /usr/local/bin/livesync-cli
# Mount your vault / local database directory here # Mount your vault / local database directory here
VOLUME ["/data"] VOLUME ["/data"]
ENTRYPOINT ["livesync-cli"] ENTRYPOINT ["livesync-cli"]
src/apps/cli/testdeno/bench-couchdb.ts
+46 -124
View File
@@ -1,17 +1,8 @@
import { TempDir } from "./helpers/temp.ts"; import { TempDir } from "./helpers/temp.ts";
import { import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts";
applyRemoteSyncSettings,
initSettingsFile,
} from "./helpers/settings.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts"; import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
import { import { createCouchdbDatabase, startCouchdb, stopCouchdb } from "./helpers/docker.ts";
createCouchdbDatabase, import { createDeterministicDataset } from "./helpers/dataset.ts";
startCouchdb,
stopCouchdb,
} from "./helpers/docker.ts";
import {
createDeterministicDataset,
} from "./helpers/dataset.ts";
import { import {
type BenchmarkVerificationMode, type BenchmarkVerificationMode,
parseBenchmarkVerificationMode, parseBenchmarkVerificationMode,
@@ -81,10 +72,7 @@ function readEnvStringArray(name: string, fallback: string[]): string[] {
try { try {
const parsed = JSON.parse(raw); const parsed = JSON.parse(raw);
if ( if (Array.isArray(parsed) && parsed.every((item) => typeof item === "string")) {
Array.isArray(parsed) &&
parsed.every((item) => typeof item === "string")
) {
return parsed; return parsed;
} }
} catch { } catch {
@@ -119,60 +107,34 @@ function formatBytes(value: number): string {
function buildConfig(): BenchmarkConfig { function buildConfig(): BenchmarkConfig {
return { return {
caseName: readEnvString("BENCH_CASE", "couchdb-baseline"), caseName: readEnvString("BENCH_CASE", "couchdb-baseline"),
couchdbBackendUri: readEnvString( couchdbBackendUri: readEnvString("BENCH_COUCHDB_BACKEND_URI", "http://127.0.0.1:5989"),
"BENCH_COUCHDB_BACKEND_URI", couchdbProxyUri: readEnvString("BENCH_COUCHDB_URI", "http://127.0.0.1:15989"),
"http://127.0.0.1:5989", couchdbUser: readEnvString("BENCH_COUCHDB_USER", readEnvString("username", "admin")),
), couchdbPassword: readEnvString("BENCH_COUCHDB_PASSWORD", readEnvString("password", "password")),
couchdbProxyUri: readEnvString( couchdbDbname: readEnvString("BENCH_COUCHDB_DBNAME", `bench-couchdb-${Date.now()}`),
"BENCH_COUCHDB_URI",
"http://127.0.0.1:15989",
),
couchdbUser: readEnvString(
"BENCH_COUCHDB_USER",
readEnvString("username", "admin"),
),
couchdbPassword: readEnvString(
"BENCH_COUCHDB_PASSWORD",
readEnvString("password", "password"),
),
couchdbDbname: readEnvString(
"BENCH_COUCHDB_DBNAME",
`bench-couchdb-${Date.now()}`,
),
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"), datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"), datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)), mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)),
mdMinSizeBytes: Math.floor( mdMinSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 1024)),
readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 1024), mdMaxSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 20 * 1024)),
),
mdMaxSizeBytes: Math.floor(
readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 20 * 1024),
),
binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 500)), binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 500)),
binSizeBytes: Math.floor( binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024)),
readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024),
),
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240), syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240),
requestedRttMs: Math.floor(readEnvNumber("BENCH_COUCHDB_RTT_MS", 50)), requestedRttMs: Math.floor(readEnvNumber("BENCH_COUCHDB_RTT_MS", 50)),
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`), passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
encrypt: readEnvBool("BENCH_ENCRYPT", true), encrypt: readEnvBool("BENCH_ENCRYPT", true),
managedCouchdb: readEnvBool("BENCH_COUCHDB_MANAGED", true), managedCouchdb: readEnvBool("BENCH_COUCHDB_MANAGED", true),
simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"), simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"),
networkProfile: readEnvString( networkProfile: readEnvString("BENCH_NETWORK_PROFILE", "http-latency-proxy"),
"BENCH_NETWORK_PROFILE",
"http-latency-proxy",
),
networkModel: readEnvString("BENCH_NETWORK_MODEL", "local-http-proxy"), networkModel: readEnvString("BENCH_NETWORK_MODEL", "local-http-proxy"),
measurementScope: readEnvString( measurementScope: readEnvString(
"BENCH_MEASUREMENT_SCOPE", "BENCH_MEASUREMENT_SCOPE",
"Two one-shot synchronisation phases through a CouchDB-compatible remote-store path.", "Two one-shot synchronisation phases through a CouchDB-compatible remote-store path."
), ),
limitations: readEnvStringArray("BENCH_LIMITATIONS_JSON", [ limitations: readEnvStringArray("BENCH_LIMITATIONS_JSON", [
"This benchmark result is scoped to the configured dataset, remote store, and network model.", "This benchmark result is scoped to the configured dataset, remote store, and network model.",
]), ]),
verificationMode: parseBenchmarkVerificationMode( verificationMode: parseBenchmarkVerificationMode(Deno.env.get("BENCH_VERIFY_MODE")),
Deno.env.get("BENCH_VERIFY_MODE"),
),
repeatIndex: Math.floor(readEnvNumber("BENCH_REPEAT_INDEX", 1)), repeatIndex: Math.floor(readEnvNumber("BENCH_REPEAT_INDEX", 1)),
repeatCount: Math.floor(readEnvNumber("BENCH_REPEAT_COUNT", 1)), repeatCount: Math.floor(readEnvNumber("BENCH_REPEAT_COUNT", 1)),
}; };
@@ -193,20 +155,17 @@ export type CouchdbProxyHandle = {
directionalDelayMs: number; directionalDelayMs: number;
}; };
export function startCouchdbProxy( export function startCouchdbProxy(options: {
options: { backendUri: string;
backendUri: string; proxyUri: string;
proxyUri: string; requestedRttMs: number;
requestedRttMs: number; delay?: (milliseconds: number) => Promise<void>;
delay?: (milliseconds: number) => Promise<void>; }): CouchdbProxyHandle {
},
): CouchdbProxyHandle {
const backend = new URL(options.backendUri); const backend = new URL(options.backendUri);
const proxy = new URL(options.proxyUri); const proxy = new URL(options.proxyUri);
const halfDelayMs = options.requestedRttMs / 2; const halfDelayMs = options.requestedRttMs / 2;
const delay = options.delay ?? const delay =
((milliseconds: number) => options.delay ?? ((milliseconds: number) => new Promise<void>((resolve) => setTimeout(resolve, milliseconds)));
new Promise<void>((resolve) => setTimeout(resolve, milliseconds)));
const controller = new AbortController(); const controller = new AbortController();
const listener = Deno.serve( const listener = Deno.serve(
@@ -256,14 +215,13 @@ export function startCouchdbProxy(
statusText: upstream.statusText, statusText: upstream.statusText,
headers: responseHeaders, headers: responseHeaders,
}); });
}, }
); );
return { return {
applied: true, applied: true,
directionalDelayMs: halfDelayMs, directionalDelayMs: halfDelayMs,
note: note: `local reverse proxy on ${proxy.origin} with ${halfDelayMs}ms request-path and ${halfDelayMs}ms response-path delay`,
`local reverse proxy on ${proxy.origin} with ${halfDelayMs}ms request-path and ${halfDelayMs}ms response-path delay`,
stop: async () => { stop: async () => {
controller.abort(); controller.abort();
await listener.finished.catch(() => {}); await listener.finished.catch(() => {});
@@ -287,21 +245,14 @@ async function main(): Promise<void> {
await initSettingsFile(settingsB); await initSettingsFile(settingsB);
if (config.managedCouchdb) { if (config.managedCouchdb) {
await startCouchdb( await startCouchdb(config.couchdbBackendUri, config.couchdbUser, config.couchdbPassword, config.couchdbDbname);
config.couchdbBackendUri,
config.couchdbUser,
config.couchdbPassword,
config.couchdbDbname,
);
} else { } else {
console.log( console.log(`[INFO] using externally managed CouchDB: ${config.couchdbBackendUri}`);
`[INFO] using externally managed CouchDB: ${config.couchdbBackendUri}`,
);
await createCouchdbDatabase( await createCouchdbDatabase(
config.couchdbBackendUri, config.couchdbBackendUri,
config.couchdbUser, config.couchdbUser,
config.couchdbPassword, config.couchdbPassword,
config.couchdbDbname, config.couchdbDbname
); );
} }
@@ -356,28 +307,15 @@ async function main(): Promise<void> {
await runCliOrFail(vaultB, "--settings", settingsB, "sync"); await runCliOrFail(vaultB, "--settings", settingsB, "sync");
const syncBElapsed = nowMs() - syncBStart; const syncBElapsed = nowMs() - syncBStart;
const verification = await verifyBenchmarkDataset( const verification = await verifyBenchmarkDataset(seedFiles.entries, config.verificationMode, async (entry) => {
seedFiles.entries, const pulledPath = workDir.join(`pulled-${entry.relativePath.split("/").join("_")}`);
config.verificationMode, await runCliOrFail(vaultB, "--settings", settingsB, "pull", entry.relativePath, pulledPath);
async (entry) => { await assertFilesEqual(
const pulledPath = workDir.join( entry.absolutePath,
`pulled-${entry.relativePath.split("/").join("_")}`, pulledPath,
); `file mismatch after CouchDB sync: ${entry.relativePath}`
await runCliOrFail( );
vaultB, });
"--settings",
settingsB,
"pull",
entry.relativePath,
pulledPath,
);
await assertFilesEqual(
entry.absolutePath,
pulledPath,
`file mismatch after CouchDB sync: ${entry.relativePath}`,
);
},
);
const result = { const result = {
caseName: config.caseName, caseName: config.caseName,
@@ -409,40 +347,24 @@ async function main(): Promise<void> {
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)), mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
syncAElapsedMs: Number(syncAElapsed.toFixed(1)), syncAElapsedMs: Number(syncAElapsed.toFixed(1)),
syncBElapsedMs: Number(syncBElapsed.toFixed(1)), syncBElapsedMs: Number(syncBElapsed.toFixed(1)),
totalSyncElapsedMs: Number( totalSyncElapsedMs: Number((syncAElapsed + syncBElapsed).toFixed(1)),
(syncAElapsed + syncBElapsed).toFixed(1), throughputBytesPerSec: Number((seedFiles.totalBytes / ((syncAElapsed + syncBElapsed) / 1000)).toFixed(2)),
),
throughputBytesPerSec: Number(
(seedFiles.totalBytes / ((syncAElapsed + syncBElapsed) / 1000))
.toFixed(
2,
),
),
throughputMiBPerSec: Number( throughputMiBPerSec: Number(
(seedFiles.totalBytes / ((syncAElapsed + syncBElapsed) / 1000) / (seedFiles.totalBytes / ((syncAElapsed + syncBElapsed) / 1000) / 1024 / 1024).toFixed(4)
1024 /
1024).toFixed(4),
), ),
}; };
if (resultPath) { if (resultPath) {
await Deno.writeTextFile( await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
resultPath,
JSON.stringify(result, null, 2),
);
} }
console.log(JSON.stringify(result, null, 2)); console.log(JSON.stringify(result, null, 2));
console.error( console.error(
`[Benchmark] couchdb mirrored ${seedFiles.totalFiles} files (${ `[Benchmark] couchdb mirrored ${seedFiles.totalFiles} files (${formatBytes(
formatBytes(seedFiles.totalBytes) seedFiles.totalBytes
}) in ${ )}) in ${formatMs(mirrorElapsed)}, synced in ${formatMs(
formatMs( syncAElapsed + syncBElapsed
mirrorElapsed, )} (${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
)
}, synced in ${
formatMs(syncAElapsed + syncBElapsed)
} (${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`,
); );
} finally { } finally {
await proxy.stop(); await proxy.stop();
src/apps/cli/testdeno/bench-latency-sweep.ts
+2 -5
View File
@@ -65,9 +65,7 @@ async function runBenchmark(options: {
repeatIndex: number; repeatIndex: number;
repeatCount: number; repeatCount: number;
}): Promise<Record<string, unknown>> { }): Promise<Record<string, unknown>> {
const suffix = options.repeatCount > 1 const suffix = options.repeatCount > 1 ? `-r${String(options.repeatIndex).padStart(2, "0")}` : "";
? `-r${String(options.repeatIndex).padStart(2, "0")}`
: "";
const resultPath = `${options.outputDir}/${options.name}${suffix}.json`; const resultPath = `${options.outputDir}/${options.name}${suffix}.json`;
const env = { const env = {
...Deno.env.toObject(), ...Deno.env.toObject(),
@@ -156,8 +154,7 @@ async function main(): Promise<void> {
const summary = { const summary = {
generatedAt: new Date().toISOString(), generatedAt: new Date().toISOString(),
outputDir, outputDir,
note: note: "This sweep applies half of each requested CouchDB RTT before forwarding requests and half before returning responses. It is not a full netem model of jitter, loss, MTU, bandwidth, or VPN encapsulation.",
"This sweep applies half of each requested CouchDB RTT before forwarding requests and half before returning responses. It is not a full netem model of jitter, loss, MTU, bandwidth, or VPN encapsulation.",
rtts, rtts,
repeatCount, repeatCount,
results, results,
src/apps/cli/testdeno/bench-network-cases.ts
+35 -91
View File
@@ -27,26 +27,16 @@ function timestamp(): string {
const d = new Date(); const d = new Date();
const pad = (n: number) => String(n).padStart(2, "0"); const pad = (n: number) => String(n).padStart(2, "0");
return ( return (
`${d.getUTCFullYear()}${pad(d.getUTCMonth() + 1)}${ `${d.getUTCFullYear()}${pad(d.getUTCMonth() + 1)}${pad(d.getUTCDate())}-` +
pad(d.getUTCDate()) `${pad(d.getUTCHours())}${pad(d.getUTCMinutes())}${pad(d.getUTCSeconds())}`
}-` +
`${pad(d.getUTCHours())}${pad(d.getUTCMinutes())}${
pad(d.getUTCSeconds())
}`
); );
} }
function buildBaseEnv(): Record<string, string> { function buildBaseEnv(): Record<string, string> {
return { return {
BENCH_MD_FILE_COUNT: readEnvString("BENCH_MD_FILE_COUNT", "20"), BENCH_MD_FILE_COUNT: readEnvString("BENCH_MD_FILE_COUNT", "20"),
BENCH_MD_MIN_SIZE_BYTES: readEnvString( BENCH_MD_MIN_SIZE_BYTES: readEnvString("BENCH_MD_MIN_SIZE_BYTES", "512"),
"BENCH_MD_MIN_SIZE_BYTES", BENCH_MD_MAX_SIZE_BYTES: readEnvString("BENCH_MD_MAX_SIZE_BYTES", "2048"),
"512",
),
BENCH_MD_MAX_SIZE_BYTES: readEnvString(
"BENCH_MD_MAX_SIZE_BYTES",
"2048",
),
BENCH_BIN_FILE_COUNT: readEnvString("BENCH_BIN_FILE_COUNT", "5"), BENCH_BIN_FILE_COUNT: readEnvString("BENCH_BIN_FILE_COUNT", "5"),
BENCH_BIN_SIZE_BYTES: readEnvString("BENCH_BIN_SIZE_BYTES", "8192"), BENCH_BIN_SIZE_BYTES: readEnvString("BENCH_BIN_SIZE_BYTES", "8192"),
BENCH_SYNC_TIMEOUT: readEnvString("BENCH_SYNC_TIMEOUT", "300"), BENCH_SYNC_TIMEOUT: readEnvString("BENCH_SYNC_TIMEOUT", "300"),
@@ -59,7 +49,7 @@ function buildBaseEnv(): Record<string, string> {
function withScopeEnv( function withScopeEnv(
env: Record<string, string>, env: Record<string, string>,
options: Pick<BenchmarkCase, "measurementScope" | "limitations">, options: Pick<BenchmarkCase, "measurementScope" | "limitations">
): Record<string, string> { ): Record<string, string> {
return { return {
...env, ...env,
@@ -79,25 +69,15 @@ export function buildCases(): BenchmarkCase[] {
const base = buildBaseEnv(); const base = buildBaseEnv();
const couchdbRtt = readEnvString("BENCH_COUCHDB_RTT_MS", "20"); const couchdbRtt = readEnvString("BENCH_COUCHDB_RTT_MS", "20");
const tetheringVpnRtt = readEnvString("BENCH_TETHERING_VPN_RTT_MS", "120"); const tetheringVpnRtt = readEnvString("BENCH_TETHERING_VPN_RTT_MS", "120");
const localTurnServers = readEnvString( const localTurnServers = readEnvString("BENCH_LOCAL_TURN_SERVERS", "turn:127.0.0.1:3478");
"BENCH_LOCAL_TURN_SERVERS", const shimCouchdbUri = readEnvString("BENCH_SHIM_COUCHDB_URI", "http://couchdb-shim:5984");
"turn:127.0.0.1:3478", const signallingShimRelay = readEnvString("BENCH_SIGNAL_SHIM_RELAY", "ws://p2p-signalling-shim:7777/");
);
const shimCouchdbUri = readEnvString(
"BENCH_SHIM_COUCHDB_URI",
"http://couchdb-shim:5984",
);
const signallingShimRelay = readEnvString(
"BENCH_SIGNAL_SHIM_RELAY",
"ws://p2p-signalling-shim:7777/",
);
return [ return [
defineCase({ defineCase({
name: "couchdb-baseline", name: "couchdb-baseline",
runner: "couchdb", runner: "couchdb",
description: description: "Standard self-hosted CouchDB path through a local latency proxy.",
"Standard self-hosted CouchDB path through a local latency proxy.",
dataPath: "Device A -> CouchDB -> Device B", dataPath: "Device A -> CouchDB -> Device B",
trustBoundary: "CouchDB operator and network path", trustBoundary: "CouchDB operator and network path",
measurementScope: measurementScope:
@@ -115,8 +95,7 @@ export function buildCases(): BenchmarkCase[] {
defineCase({ defineCase({
name: "p2p-direct-local", name: "p2p-direct-local",
runner: "p2p", runner: "p2p",
description: description: "Preferred direct WebRTC P2P path with Nostr signalling and TURN disabled.",
"Preferred direct WebRTC P2P path with Nostr signalling and TURN disabled.",
dataPath: "Device A -> Device B", dataPath: "Device A -> Device B",
trustBoundary: "Nostr relay for signalling metadata; no TURN relay", trustBoundary: "Nostr relay for signalling metadata; no TURN relay",
measurementScope: measurementScope:
@@ -133,8 +112,7 @@ export function buildCases(): BenchmarkCase[] {
BENCH_SIMULATION_TIER: "1", BENCH_SIMULATION_TIER: "1",
BENCH_NETWORK_PROFILE: "local-direct", BENCH_NETWORK_PROFILE: "local-direct",
BENCH_NETWORK_MODEL: "local-runner-webrtc", BENCH_NETWORK_MODEL: "local-runner-webrtc",
BENCH_P2P_CANDIDATE_PATH_VERIFICATION: BENCH_P2P_CANDIDATE_PATH_VERIFICATION: "turn-disabled-but-selected-ice-pair-not-collected",
"turn-disabled-but-selected-ice-pair-not-collected",
}, },
}), }),
defineCase({ defineCase({
@@ -142,8 +120,7 @@ export function buildCases(): BenchmarkCase[] {
runner: "couchdb", runner: "couchdb",
description: description:
"Approximate smartphone tethering/VPN remote-database path using an HTTP latency proxy. This does not model loss, jitter, MTU, or VPN encapsulation.", "Approximate smartphone tethering/VPN remote-database path using an HTTP latency proxy. This does not model loss, jitter, MTU, or VPN encapsulation.",
dataPath: dataPath: "Device A -> VPN/network path -> CouchDB -> VPN/network path -> Device B",
"Device A -> VPN/network path -> CouchDB -> VPN/network path -> Device B",
trustBoundary: "VPN/network path and CouchDB operator", trustBoundary: "VPN/network path and CouchDB operator",
measurementScope: measurementScope:
"Two one-shot CouchDB synchronisation phases with additional requested RTT through the local HTTP proxy.", "Two one-shot CouchDB synchronisation phases with additional requested RTT through the local HTTP proxy.",
@@ -160,10 +137,8 @@ export function buildCases(): BenchmarkCase[] {
defineCase({ defineCase({
name: "couchdb-netem-home-wifi", name: "couchdb-netem-home-wifi",
runner: "couchdb", runner: "couchdb",
description: description: "Tier 2 CouchDB path through the Compose netem TCP shim using the home-wifi profile.",
"Tier 2 CouchDB path through the Compose netem TCP shim using the home-wifi profile.", dataPath: "Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
dataPath:
"Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
trustBoundary: "CouchDB operator and constrained network shim", trustBoundary: "CouchDB operator and constrained network shim",
measurementScope: measurementScope:
"Tier 2 CouchDB synchronisation through a Compose TCP shim that applies the home-wifi netem profile.", "Tier 2 CouchDB synchronisation through a Compose TCP shim that applies the home-wifi netem profile.",
@@ -184,12 +159,9 @@ export function buildCases(): BenchmarkCase[] {
defineCase({ defineCase({
name: "couchdb-netem-tethering-vpn", name: "couchdb-netem-tethering-vpn",
runner: "couchdb", runner: "couchdb",
description: description: "Tier 2 CouchDB path through the Compose netem TCP shim using a tethering-vpn profile.",
"Tier 2 CouchDB path through the Compose netem TCP shim using a tethering-vpn profile.", dataPath: "Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
dataPath: trustBoundary: "CouchDB operator and constrained smartphone/VPN-like network shim",
"Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
trustBoundary:
"CouchDB operator and constrained smartphone/VPN-like network shim",
measurementScope: measurementScope:
"Tier 2 CouchDB synchronisation through a Compose TCP shim that applies the tethering-vpn netem profile.", "Tier 2 CouchDB synchronisation through a Compose TCP shim that applies the tethering-vpn netem profile.",
limitations: [ limitations: [
@@ -211,10 +183,8 @@ export function buildCases(): BenchmarkCase[] {
runner: "p2p", runner: "p2p",
description: description:
"Direct P2P case name for smartphone tethering/VPN measurements. In this local runner it is unshaped and should be treated as a wiring check unless executed on that network.", "Direct P2P case name for smartphone tethering/VPN measurements. In this local runner it is unshaped and should be treated as a wiring check unless executed on that network.",
dataPath: dataPath: "Device A -> Device B when WebRTC direct connectivity succeeds",
"Device A -> Device B when WebRTC direct connectivity succeeds", trustBoundary: "Smartphone/VPN routing policy plus Nostr signalling metadata",
trustBoundary:
"Smartphone/VPN routing policy plus Nostr signalling metadata",
measurementScope: measurementScope:
"Structural placeholder for direct P2P measurements on a real smartphone tethering/VPN path.", "Structural placeholder for direct P2P measurements on a real smartphone tethering/VPN path.",
limitations: [ limitations: [
@@ -237,10 +207,8 @@ export function buildCases(): BenchmarkCase[] {
runner: "p2p", runner: "p2p",
description: description:
"Tier 2 P2P path with only the Nostr signalling relay accessed through the home-wifi netem shim.", "Tier 2 P2P path with only the Nostr signalling relay accessed through the home-wifi netem shim.",
dataPath: dataPath: "Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim",
"Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim", trustBoundary: "Nostr signalling metadata through constrained network shim; no TURN relay",
trustBoundary:
"Nostr signalling metadata through constrained network shim; no TURN relay",
measurementScope: measurementScope:
"One fresh CLI p2p-sync command where only Nostr signalling access is shaped by the home-wifi netem profile; the selected WebRTC note-data path is unshaped.", "One fresh CLI p2p-sync command where only Nostr signalling access is shaped by the home-wifi netem profile; the selected WebRTC note-data path is unshaped.",
limitations: [ limitations: [
@@ -265,8 +233,7 @@ export function buildCases(): BenchmarkCase[] {
runner: "p2p", runner: "p2p",
description: description:
"Tier 2 P2P path with only the Nostr signalling relay accessed through the tethering-vpn netem shim.", "Tier 2 P2P path with only the Nostr signalling relay accessed through the tethering-vpn netem shim.",
dataPath: dataPath: "Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim",
"Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim",
trustBoundary: trustBoundary:
"Nostr signalling metadata through constrained smartphone/VPN-like network shim; no TURN relay", "Nostr signalling metadata through constrained smartphone/VPN-like network shim; no TURN relay",
measurementScope: measurementScope:
@@ -291,8 +258,7 @@ export function buildCases(): BenchmarkCase[] {
defineCase({ defineCase({
name: "p2p-user-turn", name: "p2p-user-turn",
runner: "p2p", runner: "p2p",
description: description: "Optional fallback path through a local user-controlled TURN server.",
"Optional fallback path through a local user-controlled TURN server.",
dataPath: "Device A -> user-controlled TURN -> Device B", dataPath: "Device A -> user-controlled TURN -> Device B",
trustBoundary: "User-controlled TURN server", trustBoundary: "User-controlled TURN server",
measurementScope: measurementScope:
@@ -319,11 +285,9 @@ async function runCase(
testCase: BenchmarkCase, testCase: BenchmarkCase,
outputDir: string, outputDir: string,
repeatIndex: number, repeatIndex: number,
repeatCount: number, repeatCount: number
): Promise<Record<string, unknown>> { ): Promise<Record<string, unknown>> {
const suffix = repeatCount > 1 const suffix = repeatCount > 1 ? `-r${String(repeatIndex).padStart(2, "0")}` : "";
? `-r${String(repeatIndex).padStart(2, "0")}`
: "";
const resultPath = `${outputDir}/${testCase.name}${suffix}.json`; const resultPath = `${outputDir}/${testCase.name}${suffix}.json`;
const taskName = testCase.runner === "p2p" ? "bench:p2p" : "bench:couchdb"; const taskName = testCase.runner === "p2p" ? "bench:p2p" : "bench:couchdb";
const env = { const env = {
@@ -334,12 +298,8 @@ async function runCase(
BENCH_REPEAT_COUNT: String(repeatCount), BENCH_REPEAT_COUNT: String(repeatCount),
}; };
const repeatLabel = repeatCount > 1 const repeatLabel = repeatCount > 1 ? ` (${repeatIndex}/${repeatCount})` : "";
? ` (${repeatIndex}/${repeatCount})` console.log(`[bench-cases] running ${testCase.name}${repeatLabel}: ${testCase.description}`);
: "";
console.log(
`[bench-cases] running ${testCase.name}${repeatLabel}: ${testCase.description}`,
);
const command = new Deno.Command("deno", { const command = new Deno.Command("deno", {
args: ["task", taskName], args: ["task", taskName],
cwd: import.meta.dirname, cwd: import.meta.dirname,
@@ -355,10 +315,7 @@ async function runCase(
throw new Error(`case failed: ${testCase.name} (exit ${status.code})`); throw new Error(`case failed: ${testCase.name} (exit ${status.code})`);
} }
const result = JSON.parse(await Deno.readTextFile(resultPath)) as Record< const result = JSON.parse(await Deno.readTextFile(resultPath)) as Record<string, unknown>;
string,
unknown
>;
return { return {
...testCase, ...testCase,
repeatIndex, repeatIndex,
@@ -369,10 +326,7 @@ async function runCase(
} }
function selectCases(allCases: BenchmarkCase[]): BenchmarkCase[] { function selectCases(allCases: BenchmarkCase[]): BenchmarkCase[] {
const requested = readEnvString( const requested = readEnvString("BENCH_CASES", "couchdb-baseline,p2p-direct-local");
"BENCH_CASES",
"couchdb-baseline,p2p-direct-local",
);
const names = requested const names = requested
.split(",") .split(",")
.map((v) => v.trim()) .map((v) => v.trim())
@@ -382,9 +336,7 @@ function selectCases(allCases: BenchmarkCase[]): BenchmarkCase[] {
const found = byName.get(name); const found = byName.get(name);
if (!found) { if (!found) {
throw new Error( throw new Error(
`Unknown BENCH_CASES entry '${name}'. Available: ${ `Unknown BENCH_CASES entry '${name}'. Available: ${allCases.map((c) => c.name).join(", ")}`
allCases.map((c) => c.name).join(", ")
}`,
); );
} }
return found; return found;
@@ -392,10 +344,7 @@ function selectCases(allCases: BenchmarkCase[]): BenchmarkCase[] {
} }
async function main(): Promise<void> { async function main(): Promise<void> {
const outRoot = readEnvString( const outRoot = readEnvString("BENCH_CASES_ROOT", `${import.meta.dirname}/bench-results`);
"BENCH_CASES_ROOT",
`${import.meta.dirname}/bench-results`,
);
const outputDir = `${outRoot}/cases-${timestamp()}`; const outputDir = `${outRoot}/cases-${timestamp()}`;
await Deno.mkdir(outputDir, { recursive: true }); await Deno.mkdir(outputDir, { recursive: true });
@@ -412,16 +361,14 @@ async function main(): Promise<void> {
availableCases: allCases, availableCases: allCases,
}, },
null, null,
2, 2
), )
); );
const results: Record<string, unknown>[] = []; const results: Record<string, unknown>[] = [];
for (const testCase of cases) { for (const testCase of cases) {
for (let repeatIndex = 1; repeatIndex <= repeatCount; repeatIndex++) { for (let repeatIndex = 1; repeatIndex <= repeatCount; repeatIndex++) {
results.push( results.push(await runCase(testCase, outputDir, repeatIndex, repeatCount));
await runCase(testCase, outputDir, repeatIndex, repeatCount),
);
} }
} }
@@ -431,10 +378,7 @@ async function main(): Promise<void> {
repeatCount, repeatCount,
results, results,
}; };
await Deno.writeTextFile( await Deno.writeTextFile(`${outputDir}/summary.json`, JSON.stringify(summary, null, 2));
`${outputDir}/summary.json`,
JSON.stringify(summary, null, 2),
);
console.log(JSON.stringify(summary, null, 2)); console.log(JSON.stringify(summary, null, 2));
console.log(`[bench-cases] result directory: ${outputDir}`); console.log(`[bench-cases] result directory: ${outputDir}`);
} }
src/apps/cli/testdeno/bench-p2p.ts
+39 -105
View File
@@ -1,9 +1,5 @@
import { TempDir } from "./helpers/temp.ts"; import { TempDir } from "./helpers/temp.ts";
import { import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
applyP2pSettings,
applyP2pTestTweaks,
initSettingsFile,
} from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts"; import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { import {
discoverPeer, discoverPeer,
@@ -13,9 +9,7 @@ import {
stopLocalRelayIfStarted, stopLocalRelayIfStarted,
} from "./helpers/p2p.ts"; } from "./helpers/p2p.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts"; import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
import { import { createDeterministicDataset } from "./helpers/dataset.ts";
createDeterministicDataset,
} from "./helpers/dataset.ts";
import { import {
type BenchmarkVerificationMode, type BenchmarkVerificationMode,
parseBenchmarkVerificationMode, parseBenchmarkVerificationMode,
@@ -107,10 +101,7 @@ function readEnvStringArray(name: string, fallback: string[]): string[] {
try { try {
const parsed = JSON.parse(raw); const parsed = JSON.parse(raw);
if ( if (Array.isArray(parsed) && parsed.every((item) => typeof item === "string")) {
Array.isArray(parsed) &&
parsed.every((item) => typeof item === "string")
) {
return parsed; return parsed;
} }
} catch { } catch {
@@ -147,48 +138,31 @@ function buildConfig(): BenchmarkConfig {
return { return {
caseName: readEnvString("BENCH_CASE", "p2p-direct-local"), caseName: readEnvString("BENCH_CASE", "p2p-direct-local"),
relay: readEnvString("BENCH_RELAY", "ws://localhost:4000/"), relay: readEnvString("BENCH_RELAY", "ws://localhost:4000/"),
appId: readEnvString( appId: readEnvString("BENCH_APP_ID", "self-hosted-livesync-cli-benchmark"),
"BENCH_APP_ID",
"self-hosted-livesync-cli-benchmark",
),
roomId: readEnvString("BENCH_ROOM_ID", `bench-room-${Date.now()}`), roomId: readEnvString("BENCH_ROOM_ID", `bench-room-${Date.now()}`),
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`), passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
turnServers: readEnvString("BENCH_TURN_SERVERS", ""), turnServers: readEnvString("BENCH_TURN_SERVERS", ""),
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"), datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"), datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)), mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)),
mdMinSizeBytes: Math.floor( mdMinSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 1024)),
readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 1024), mdMaxSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 20 * 1024)),
),
mdMaxSizeBytes: Math.floor(
readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 20 * 1024),
),
binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 500)), binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 500)),
binSizeBytes: Math.floor( binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024)),
readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024),
),
peersTimeoutSeconds: readEnvNumber("BENCH_PEERS_TIMEOUT", 20), peersTimeoutSeconds: readEnvNumber("BENCH_PEERS_TIMEOUT", 20),
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240), syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240),
simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"), simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"),
networkProfile: readEnvString("BENCH_NETWORK_PROFILE", "local-direct"), networkProfile: readEnvString("BENCH_NETWORK_PROFILE", "local-direct"),
networkModel: readEnvString( networkModel: readEnvString("BENCH_NETWORK_MODEL", "local-runner-webrtc"),
"BENCH_NETWORK_MODEL", candidatePathVerification: readEnvString("BENCH_P2P_CANDIDATE_PATH_VERIFICATION", "not-collected"),
"local-runner-webrtc",
),
candidatePathVerification: readEnvString(
"BENCH_P2P_CANDIDATE_PATH_VERIFICATION",
"not-collected",
),
measurementScope: readEnvString( measurementScope: readEnvString(
"BENCH_MEASUREMENT_SCOPE", "BENCH_MEASUREMENT_SCOPE",
"One fresh CLI p2p-sync command, including process start-up and WebRTC connection establishment; the earlier peer-list observation command is excluded.", "One fresh CLI p2p-sync command, including process start-up and WebRTC connection establishment; the earlier peer-list observation command is excluded."
), ),
limitations: readEnvStringArray("BENCH_LIMITATIONS_JSON", [ limitations: readEnvStringArray("BENCH_LIMITATIONS_JSON", [
"This benchmark result is scoped to the configured dataset, network model, and selected ICE path.", "This benchmark result is scoped to the configured dataset, network model, and selected ICE path.",
]), ]),
verificationMode: parseBenchmarkVerificationMode( verificationMode: parseBenchmarkVerificationMode(Deno.env.get("BENCH_VERIFY_MODE")),
Deno.env.get("BENCH_VERIFY_MODE"),
),
repeatIndex: Math.floor(readEnvNumber("BENCH_REPEAT_INDEX", 1)), repeatIndex: Math.floor(readEnvNumber("BENCH_REPEAT_INDEX", 1)),
repeatCount: Math.floor(readEnvNumber("BENCH_REPEAT_COUNT", 1)), repeatCount: Math.floor(readEnvNumber("BENCH_REPEAT_COUNT", 1)),
}; };
@@ -202,9 +176,7 @@ function readOptionalResultPath(): string | undefined {
return raw; return raw;
} }
async function readLatestP2PConnectionStats( async function readLatestP2PConnectionStats(statsPath: string): Promise<P2PConnectionStats | undefined> {
statsPath: string,
): Promise<P2PConnectionStats | undefined> {
try { try {
const text = await Deno.readTextFile(statsPath); const text = await Deno.readTextFile(statsPath);
const lines = text const lines = text
@@ -252,7 +224,7 @@ async function main(): Promise<void> {
config.appId, config.appId,
config.relay, config.relay,
"~.*", "~.*",
config.turnServers, config.turnServers
), ),
applyP2pSettings( applyP2pSettings(
clientSettings, clientSettings,
@@ -261,21 +233,13 @@ async function main(): Promise<void> {
config.appId, config.appId,
config.relay, config.relay,
"~.*", "~.*",
config.turnServers, config.turnServers
), ),
]); ]);
await Promise.all([ await Promise.all([
applyP2pTestTweaks( applyP2pTestTweaks(hostSettings, "p2p-bench-host", config.passphrase),
hostSettings, applyP2pTestTweaks(clientSettings, "p2p-bench-client", config.passphrase),
"p2p-bench-host",
config.passphrase,
),
applyP2pTestTweaks(
clientSettings,
"p2p-bench-client",
config.passphrase,
),
]); ]);
const seedFiles = await createDeterministicDataset({ const seedFiles = await createDeterministicDataset({
@@ -293,25 +257,15 @@ async function main(): Promise<void> {
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror"); await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
const mirrorElapsed = nowMs() - mirrorStart; const mirrorElapsed = nowMs() - mirrorStart;
const host = startCliInBackground( const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
hostVault,
"--settings",
hostSettings,
"p2p-host",
);
try { try {
const hostReadyStart = nowMs(); const hostReadyStart = nowMs();
await host.waitUntilContains("P2P host is running", 20000); await host.waitUntilContains("P2P host is running", 20000);
const hostReadyElapsed = nowMs() - hostReadyStart; const hostReadyElapsed = nowMs() - hostReadyStart;
const peerDiscoveryCommandStart = nowMs(); const peerDiscoveryCommandStart = nowMs();
const peer = await discoverPeer( const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
clientVault, const peerDiscoveryCommandElapsed = nowMs() - peerDiscoveryCommandStart;
clientSettings,
config.peersTimeoutSeconds,
);
const peerDiscoveryCommandElapsed = nowMs() -
peerDiscoveryCommandStart;
const syncStart = nowMs(); const syncStart = nowMs();
await runCliOrFail( await runCliOrFail(
@@ -320,7 +274,7 @@ async function main(): Promise<void> {
clientSettings, clientSettings,
"p2p-sync", "p2p-sync",
peer.id, peer.id,
String(config.syncTimeoutSeconds), String(config.syncTimeoutSeconds)
); );
const syncElapsed = nowMs() - syncStart; const syncElapsed = nowMs() - syncStart;
@@ -328,28 +282,24 @@ async function main(): Promise<void> {
seedFiles.entries, seedFiles.entries,
config.verificationMode, config.verificationMode,
async (entry) => { async (entry) => {
const pulledPath = workDir.join( const pulledPath = workDir.join(`pulled-${entry.relativePath.replaceAll("/", "_")}`);
`pulled-${entry.relativePath.replaceAll("/", "_")}`,
);
await runCliOrFail( await runCliOrFail(
clientVault, clientVault,
"--settings", "--settings",
clientSettings, clientSettings,
"pull", "pull",
entry.relativePath, entry.relativePath,
pulledPath, pulledPath
); );
await assertFilesEqual( await assertFilesEqual(
entry.absolutePath, entry.absolutePath,
pulledPath, pulledPath,
`file mismatch after P2P sync: ${entry.relativePath}`, `file mismatch after P2P sync: ${entry.relativePath}`
); );
}, }
); );
const p2pConnectionStats = await readLatestP2PConnectionStats( const p2pConnectionStats = await readLatestP2PConnectionStats(p2pStatsPath);
p2pStatsPath,
);
const result = { const result = {
caseName: config.caseName, caseName: config.caseName,
mode: "p2p-cli-benchmark", mode: "p2p-cli-benchmark",
@@ -363,17 +313,15 @@ async function main(): Promise<void> {
limitations: config.limitations, limitations: config.limitations,
repeatIndex: config.repeatIndex, repeatIndex: config.repeatIndex,
repeatCount: config.repeatCount, repeatCount: config.repeatCount,
p2pCandidatePathVerified: p2pCandidatePathVerified: p2pConnectionStats?.candidatePathCollected === true,
p2pConnectionStats?.candidatePathCollected === true, p2pCandidatePathVerification: p2pConnectionStats?.candidatePathCollected
p2pCandidatePathVerification: ? "selected ICE candidate pair collected from RTCPeerConnection.getStats"
p2pConnectionStats?.candidatePathCollected : config.candidatePathVerification,
? "selected ICE candidate pair collected from RTCPeerConnection.getStats"
: config.candidatePathVerification,
p2pCandidatePathNote: p2pConnectionStats?.candidatePathCollected p2pCandidatePathNote: p2pConnectionStats?.candidatePathCollected
? "The selected ICE candidate pair was collected by the CLI benchmark. Interpret the path from the candidate types; do not infer TURN use from configuration alone." ? "The selected ICE candidate pair was collected by the CLI benchmark. Interpret the path from the candidate types; do not infer TURN use from configuration alone."
: config.turnServers.trim().length > 0 : config.turnServers.trim().length > 0
? "TURN is configured, so the selected WebRTC path may be direct, server-reflexive, or relayed. The selected ICE candidate pair was not exported by this run." ? "TURN is configured, so the selected WebRTC path may be direct, server-reflexive, or relayed. The selected ICE candidate pair was not exported by this run."
: "TURN is disabled, so a TURN-relayed path is not expected. The selected ICE candidate pair was not exported by this run.", : "TURN is disabled, so a TURN-relayed path is not expected. The selected ICE candidate pair was not exported by this run.",
p2pConnectionStats, p2pConnectionStats,
appId: config.appId, appId: config.appId,
roomId: config.roomId, roomId: config.roomId,
@@ -389,39 +337,25 @@ async function main(): Promise<void> {
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)), mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
hostReadyElapsedMs: Number(hostReadyElapsed.toFixed(1)), hostReadyElapsedMs: Number(hostReadyElapsed.toFixed(1)),
peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds, peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds,
peerDiscoveryCommandElapsedMs: Number( peerDiscoveryCommandElapsedMs: Number(peerDiscoveryCommandElapsed.toFixed(1)),
peerDiscoveryCommandElapsed.toFixed(1),
),
peerDiscoveryNote: peerDiscoveryNote:
"p2p-peers waits for the requested timeout before printing discovered peers, so this is command duration, not first-peer latency.", "p2p-peers waits for the requested timeout before printing discovered peers, so this is command duration, not first-peer latency.",
syncElapsedMs: Number(syncElapsed.toFixed(1)), syncElapsedMs: Number(syncElapsed.toFixed(1)),
throughputBytesPerSec: Number( throughputBytesPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000)).toFixed(2)),
(seedFiles.totalBytes / (syncElapsed / 1000)).toFixed(2), throughputMiBPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000) / 1024 / 1024).toFixed(4)),
),
throughputMiBPerSec: Number(
(seedFiles.totalBytes / (syncElapsed / 1000) / 1024 / 1024)
.toFixed(
4,
),
),
}; };
if (resultPath) { if (resultPath) {
await Deno.writeTextFile( await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
resultPath,
JSON.stringify(result, null, 2),
);
} }
console.log(JSON.stringify(result, null, 2)); console.log(JSON.stringify(result, null, 2));
console.error( console.error(
`[Benchmark] mirrored ${seedFiles.totalFiles} files (${ `[Benchmark] mirrored ${seedFiles.totalFiles} files (${formatBytes(
formatBytes( seedFiles.totalBytes
seedFiles.totalBytes, )}) in ${formatMs(mirrorElapsed)}, ` +
)
}) in ${formatMs(mirrorElapsed)}, ` +
`synced in ${formatMs(syncElapsed)} ` + `synced in ${formatMs(syncElapsed)} ` +
`(${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`, `(${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
); );
} finally { } finally {
await host.stop(); await host.stop();
src/apps/cli/testdeno/helpers/benchmarkVerification.ts
+6 -15
View File
@@ -10,9 +10,7 @@ export type BenchmarkVerificationResult = {
}; };
function toHex(bytes: ArrayBuffer): string { function toHex(bytes: ArrayBuffer): string {
return [...new Uint8Array(bytes)] return [...new Uint8Array(bytes)].map((value) => value.toString(16).padStart(2, "0")).join("");
.map((value) => value.toString(16).padStart(2, "0"))
.join("");
} }
async function sha256(bytes: Uint8Array): Promise<string> { async function sha256(bytes: Uint8Array): Promise<string> {
@@ -23,7 +21,7 @@ async function sha256(bytes: Uint8Array): Promise<string> {
export function parseBenchmarkVerificationMode( export function parseBenchmarkVerificationMode(
raw: string | undefined, raw: string | undefined,
fallback: BenchmarkVerificationMode = "sample", fallback: BenchmarkVerificationMode = "sample"
): BenchmarkVerificationMode { ): BenchmarkVerificationMode {
const value = raw?.trim().toLowerCase(); const value = raw?.trim().toLowerCase();
if (!value) return fallback; if (!value) return fallback;
@@ -31,10 +29,7 @@ export function parseBenchmarkVerificationMode(
throw new Error(`BENCH_VERIFY_MODE must be 'all' or 'sample', got '${raw}'`); throw new Error(`BENCH_VERIFY_MODE must be 'all' or 'sample', got '${raw}'`);
} }
export function selectVerificationEntries( export function selectVerificationEntries(entries: DatasetEntry[], mode: BenchmarkVerificationMode): DatasetEntry[] {
entries: DatasetEntry[],
mode: BenchmarkVerificationMode,
): DatasetEntry[] {
if (mode === "all" || entries.length === 0) return [...entries]; if (mode === "all" || entries.length === 0) return [...entries];
const md = entries.find((entry) => entry.kind === "md"); const md = entries.find((entry) => entry.kind === "md");
@@ -48,15 +43,11 @@ export function selectVerificationEntries(
return [...selected.values()]; return [...selected.values()];
} }
export async function computeDatasetDigestSha256( export async function computeDatasetDigestSha256(entries: DatasetEntry[]): Promise<string> {
entries: DatasetEntry[],
): Promise<string> {
const manifest: string[] = []; const manifest: string[] = [];
for (const entry of entries) { for (const entry of entries) {
const contentDigest = await sha256(await Deno.readFile(entry.absolutePath)); const contentDigest = await sha256(await Deno.readFile(entry.absolutePath));
manifest.push( manifest.push(`${entry.kind}\t${entry.relativePath}\t${entry.size}\t${contentDigest}`);
`${entry.kind}\t${entry.relativePath}\t${entry.size}\t${contentDigest}`,
);
} }
return await sha256(new TextEncoder().encode(manifest.join("\n"))); return await sha256(new TextEncoder().encode(manifest.join("\n")));
} }
@@ -64,7 +55,7 @@ export async function computeDatasetDigestSha256(
export async function verifyBenchmarkDataset( export async function verifyBenchmarkDataset(
entries: DatasetEntry[], entries: DatasetEntry[],
mode: BenchmarkVerificationMode, mode: BenchmarkVerificationMode,
verifyEntry: (entry: DatasetEntry) => Promise<void>, verifyEntry: (entry: DatasetEntry) => Promise<void>
): Promise<BenchmarkVerificationResult> { ): Promise<BenchmarkVerificationResult> {
const selected = selectVerificationEntries(entries, mode); const selected = selectVerificationEntries(entries, mode);
for (const entry of selected) { for (const entry of selected) {
src/apps/cli/testdeno/test-benchmark-contract.ts
+26 -84
View File
@@ -1,10 +1,7 @@
import { assert, assertEquals, assertStringIncludes } from "@std/assert"; import { assert, assertEquals, assertStringIncludes } from "@std/assert";
import { type BenchmarkCase, buildCases } from "./bench-network-cases.ts"; import { type BenchmarkCase, buildCases } from "./bench-network-cases.ts";
import { startCouchdbProxy } from "./bench-couchdb.ts"; import { startCouchdbProxy } from "./bench-couchdb.ts";
import { import { parseBenchmarkVerificationMode, selectVerificationEntries } from "./helpers/benchmarkVerification.ts";
parseBenchmarkVerificationMode,
selectVerificationEntries,
} from "./helpers/benchmarkVerification.ts";
import type { DatasetEntry } from "./helpers/dataset.ts"; import type { DatasetEntry } from "./helpers/dataset.ts";
function getFreePort(): number { function getFreePort(): number {
@@ -24,20 +21,10 @@ function getCase(cases: BenchmarkCase[], name: string): BenchmarkCase {
function parsedLimitations(testCase: BenchmarkCase): string[] { function parsedLimitations(testCase: BenchmarkCase): string[] {
const raw = testCase.env.BENCH_LIMITATIONS_JSON; const raw = testCase.env.BENCH_LIMITATIONS_JSON;
assert( assert(raw, `${testCase.name} must pass BENCH_LIMITATIONS_JSON to benchmark result output`);
raw,
`${testCase.name} must pass BENCH_LIMITATIONS_JSON to benchmark result output`,
);
const parsed = JSON.parse(raw); const parsed = JSON.parse(raw);
assert( assert(Array.isArray(parsed), `${testCase.name} limitations must be an array`);
Array.isArray(parsed), assert(parsed.every((item) => typeof item === "string" && item.trim().length > 0));
`${testCase.name} limitations must be an array`,
);
assert(
parsed.every((item) =>
typeof item === "string" && item.trim().length > 0
),
);
return parsed; return parsed;
} }
@@ -46,36 +33,14 @@ Deno.test("benchmark cases record scope and limitations for paper use", () => {
assert(cases.length > 0); assert(cases.length > 0);
for (const testCase of cases) { for (const testCase of cases) {
assert( assert(testCase.description.trim().length > 0, `${testCase.name} must describe the case`);
testCase.description.trim().length > 0, assert(testCase.dataPath.trim().length > 0, `${testCase.name} must describe the data path`);
`${testCase.name} must describe the case`, assert(testCase.trustBoundary.trim().length > 0, `${testCase.name} must describe the trust boundary`);
); assert(testCase.measurementScope.trim().length > 0, `${testCase.name} must describe the measurement scope`);
assert( assert(testCase.limitations.length > 0, `${testCase.name} must list limitations`);
testCase.dataPath.trim().length > 0, assertEquals(testCase.env.BENCH_MEASUREMENT_SCOPE, testCase.measurementScope);
`${testCase.name} must describe the data path`,
);
assert(
testCase.trustBoundary.trim().length > 0,
`${testCase.name} must describe the trust boundary`,
);
assert(
testCase.measurementScope.trim().length > 0,
`${testCase.name} must describe the measurement scope`,
);
assert(
testCase.limitations.length > 0,
`${testCase.name} must list limitations`,
);
assertEquals(
testCase.env.BENCH_MEASUREMENT_SCOPE,
testCase.measurementScope,
);
assertEquals(parsedLimitations(testCase), testCase.limitations); assertEquals(parsedLimitations(testCase), testCase.limitations);
assertEquals( assertEquals(testCase.env.BENCH_VERIFY_MODE, "all", `${testCase.name} must verify the complete dataset`);
testCase.env.BENCH_VERIFY_MODE,
"all",
`${testCase.name} must verify the complete dataset`,
);
} }
}); });
@@ -89,7 +54,7 @@ Deno.test("CouchDB latency proxy applies half the requested RTT in each directio
port: backendPort, port: backendPort,
onListen() {}, onListen() {},
}, },
() => new Response("ok"), () => new Response("ok")
); );
const proxy = startCouchdbProxy({ const proxy = startCouchdbProxy({
backendUri: `http://127.0.0.1:${backendPort}`, backendUri: `http://127.0.0.1:${backendPort}`,
@@ -143,34 +108,22 @@ Deno.test("benchmark verification mode selects either all files or a labelled sa
Deno.test("P2P signalling-shim cases do not claim to shape the note-data path", () => { Deno.test("P2P signalling-shim cases do not claim to shape the note-data path", () => {
const cases = buildCases(); const cases = buildCases();
for ( for (const name of ["p2p-signalling-netem-home-wifi", "p2p-signalling-netem-tethering-vpn"]) {
const name of [
"p2p-signalling-netem-home-wifi",
"p2p-signalling-netem-tethering-vpn",
]
) {
const testCase = getCase(cases, name); const testCase = getCase(cases, name);
assertEquals(testCase.runner, "p2p"); assertEquals(testCase.runner, "p2p");
assertEquals(testCase.env.BENCH_TURN_SERVERS, ""); assertEquals(testCase.env.BENCH_TURN_SERVERS, "");
assertEquals(testCase.env.BENCH_SIMULATION_TIER, "2"); assertEquals(testCase.env.BENCH_SIMULATION_TIER, "2");
assertEquals( assertEquals(testCase.env.BENCH_NETWORK_MODEL, "compose-netem-signalling-shim");
testCase.env.BENCH_NETWORK_MODEL,
"compose-netem-signalling-shim",
);
assertStringIncludes(testCase.dataPath, "WebRTC DataChannel"); assertStringIncludes(testCase.dataPath, "WebRTC DataChannel");
assertStringIncludes(testCase.dataPath, "Nostr signalling"); assertStringIncludes(testCase.dataPath, "Nostr signalling");
assertStringIncludes(testCase.measurementScope, "fresh CLI p2p-sync"); assertStringIncludes(testCase.measurementScope, "fresh CLI p2p-sync");
assert( assert(
testCase.limitations.some((limitation) => testCase.limitations.some((limitation) => limitation.includes("connection establishment")),
limitation.includes("connection establishment") `${name} must state that connection establishment is timed`
),
`${name} must state that connection establishment is timed`,
); );
assert( assert(
testCase.limitations.some((limitation) => testCase.limitations.some((limitation) => limitation.includes("does not shape the selected WebRTC")),
limitation.includes("does not shape the selected WebRTC") `${name} must avoid claiming that the P2P note-data path was shaped`
),
`${name} must avoid claiming that the P2P note-data path was shaped`,
); );
} }
}); });
@@ -182,42 +135,31 @@ Deno.test("placeholder and TURN cases are clearly non-evidence for broad P2P per
assertEquals(smartphone.env.BENCH_SIMULATION_TIER, "unmeasured"); assertEquals(smartphone.env.BENCH_SIMULATION_TIER, "unmeasured");
assertEquals(smartphone.env.BENCH_NETWORK_MODEL, "local-runner-no-netem"); assertEquals(smartphone.env.BENCH_NETWORK_MODEL, "local-runner-no-netem");
assert( assert(
smartphone.limitations.some((limitation) => smartphone.limitations.some((limitation) => limitation.includes("must not be reported as smartphone")),
limitation.includes("must not be reported as smartphone") "smartphone/VPN placeholder must not be usable as field evidence by accident"
),
"smartphone/VPN placeholder must not be usable as field evidence by accident",
); );
const turn = getCase(cases, "p2p-user-turn"); const turn = getCase(cases, "p2p-user-turn");
assertStringIncludes(turn.env.BENCH_TURN_SERVERS, "turn:"); assertStringIncludes(turn.env.BENCH_TURN_SERVERS, "turn:");
assert( assert(
turn.limitations.some((limitation) => turn.limitations.some((limitation) =>
limitation.includes( limitation.includes("does not prove that the selected ICE path was relayed")
"does not prove that the selected ICE path was relayed",
)
), ),
"TURN case must require selected ICE candidate interpretation", "TURN case must require selected ICE candidate interpretation"
); );
}); });
Deno.test("CouchDB netem cases are marked as remote-store baselines", () => { Deno.test("CouchDB netem cases are marked as remote-store baselines", () => {
const cases = buildCases(); const cases = buildCases();
for ( for (const name of ["couchdb-netem-home-wifi", "couchdb-netem-tethering-vpn"]) {
const name of ["couchdb-netem-home-wifi", "couchdb-netem-tethering-vpn"]
) {
const testCase = getCase(cases, name); const testCase = getCase(cases, name);
assertEquals(testCase.runner, "couchdb"); assertEquals(testCase.runner, "couchdb");
assertEquals(testCase.env.BENCH_SIMULATION_TIER, "2"); assertEquals(testCase.env.BENCH_SIMULATION_TIER, "2");
assertEquals( assertEquals(testCase.env.BENCH_NETWORK_MODEL, "compose-netem-tcp-shim");
testCase.env.BENCH_NETWORK_MODEL,
"compose-netem-tcp-shim",
);
assertStringIncludes(testCase.measurementScope, "CouchDB"); assertStringIncludes(testCase.measurementScope, "CouchDB");
assert( assert(
testCase.limitations.some((limitation) => testCase.limitations.some((limitation) => limitation.includes("not the WebRTC P2P data path")),
limitation.includes("not the WebRTC P2P data path") `${name} must remain scoped to the CouchDB remote-store path`
),
`${name} must remain scoped to the CouchDB remote-store path`,
); );
} }
}); });