mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-07-23 13:02:58 +00:00
Re-evaluate optional features for 1.0
This commit is contained in:
@@ -3,7 +3,7 @@
|
||||
FROM node:24-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates curl unzip python3 make g++ iproute2 \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates curl unzip python3 make g++ iproute2 libjpeg-turbo-progs time \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
ENV DENO_INSTALL=/usr/local
|
||||
@@ -28,6 +28,7 @@ RUN deno cache --lock=deno.lock \
|
||||
bench-p2p-split-node.ts \
|
||||
bench-p2p.ts \
|
||||
bench-couchdb.ts \
|
||||
bench-compression.ts \
|
||||
test-p2p-sync.ts \
|
||||
test-p2p-replicator-replacement.ts \
|
||||
test-p2p-relay-disconnect.ts
|
||||
|
||||
@@ -127,6 +127,49 @@ This sweep is useful for finding where the remote CouchDB path falls behind the
|
||||
local direct P2P path in the current HTTP-proxy latency model. It should not be
|
||||
presented as a full smartphone/VPN model.
|
||||
|
||||
## Data Compression benchmark
|
||||
|
||||
The [Data Compression specification](../../docs/specs_data_compression.md) records the current storage contract, 1.0 decision, measured result, and follow-up optimisation candidates. This section is the benchmark runbook.
|
||||
|
||||
The compression benchmark compares the exact CLI and Commonlib CouchDB path in
|
||||
four configurations: E2EE off or on, each with Data Compression off or on. It
|
||||
uses the normal CLI `mirror` and `sync` commands, so the recorded CouchDB
|
||||
documents have passed through the current Rabin–Karp chunk splitter, optional
|
||||
fflate compression, and E2EE V2 rather than a synthetic document transform.
|
||||
|
||||
```bash
|
||||
BENCH_COMMAND=compression \
|
||||
BENCH_COMPRESSION_REPEAT_COUNT=3 \
|
||||
BENCH_COUCHDB_RTT_MS=1 \
|
||||
docker compose -f test/bench-network/compose.yml run --build --rm bench-runner
|
||||
```
|
||||
|
||||
The fixture contains current repository Markdown, PNG, JSON, and TypeScript
|
||||
files; two deterministic JPEGs generated with `cjpeg`; a gzip-compressed
|
||||
Markdown file; and deterministic high-entropy binary data. Every run verifies
|
||||
all files after the second client has synchronised them. The JSON result under
|
||||
`test/bench-network/bench-results/` records:
|
||||
|
||||
- source bytes and stored chunk bytes by file kind;
|
||||
- raw CouchDB external, active, and file sizes;
|
||||
- request and response body bytes observed by the local HTTP proxy, including
|
||||
the combined initial sync and full materialisation download;
|
||||
- upload and download wall time, user and system CPU time, and maximum resident
|
||||
memory for each CLI process; and
|
||||
- percentage changes caused by enabling compression with E2EE both off and on.
|
||||
|
||||
Use at least three repeats when making a default-setting decision. A `1 ms`
|
||||
requested RTT keeps the local run focused on transform and storage costs. Run a
|
||||
separate representative RTT when evaluating whether reduced request bodies
|
||||
outweigh compression CPU on the intended network. HTTP byte counters cover
|
||||
decoded bodies and exclude headers, while process timings include CLI start-up.
|
||||
Full materialisation starts one CLI process per file and can repeat lazy chunk
|
||||
fetches, so treat that phase as a CLI workflow measurement rather than a raw
|
||||
download lower bound. Stored chunk size and upload request size are the more
|
||||
direct transform comparisons.
|
||||
The generated JPEGs are deterministic image-like fixtures, not a photographic
|
||||
corpus, so broader media conclusions require a separately reviewed corpus.
|
||||
|
||||
## Network emulation smoke
|
||||
|
||||
The optional `netem` profile checks whether a Linux runner can apply traffic
|
||||
|
||||
@@ -75,6 +75,8 @@ services:
|
||||
BENCH_CASES: ${BENCH_CASES:-couchdb-baseline,p2p-direct-local}
|
||||
BENCH_REPEAT_COUNT: ${BENCH_REPEAT_COUNT:-1}
|
||||
BENCH_CASES_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_COMPRESSION_RESULT_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_COMPRESSION_REPEAT_COUNT: ${BENCH_COMPRESSION_REPEAT_COUNT:-1}
|
||||
BENCH_SWEEP_ROOT: /workspace/src/apps/cli/testdeno/bench-results
|
||||
BENCH_SWEEP_RTT_MS: ${BENCH_SWEEP_RTT_MS:-20,50,100,150,300}
|
||||
BENCH_SWEEP_INCLUDE_P2P: ${BENCH_SWEEP_INCLUDE_P2P:-true}
|
||||
@@ -97,6 +99,7 @@ services:
|
||||
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
|
||||
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
|
||||
BENCH_LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
|
||||
LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
|
||||
LIVESYNC_CLI_DEBUG: ${LIVESYNC_CLI_DEBUG:-0}
|
||||
LIVESYNC_CLI_VERBOSE: ${LIVESYNC_CLI_VERBOSE:-0}
|
||||
CLI_E2E_TASK: ${CLI_E2E_TASK:-test:p2p:ci}
|
||||
|
||||
@@ -8,12 +8,15 @@ case "${BENCH_COMMAND:-cases}" in
|
||||
latency-sweep)
|
||||
exec deno task bench:latency-sweep
|
||||
;;
|
||||
compression)
|
||||
exec deno task bench:compression
|
||||
;;
|
||||
p2p-split-node)
|
||||
exec deno task bench:p2p-split-node
|
||||
;;
|
||||
*)
|
||||
echo "Unknown BENCH_COMMAND: ${BENCH_COMMAND}" >&2
|
||||
echo "Expected one of: cases, latency-sweep, p2p-split-node" >&2
|
||||
echo "Expected one of: cases, latency-sweep, compression, p2p-split-node" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
|
||||
Reference in New Issue
Block a user