Compare commits

..

8 Commits

Author SHA1 Message Date
vorotamoroz e916683b8d (test): customisation sync 2 2026-06-26 12:16:39 +00:00
vorotamoroz 7028a0857e (test): customisation sync 2026-06-26 12:02:29 +00:00
vorotamoroz 150f0700b0 (test): hidden file sync roundtrip and misc 2026-06-26 11:39:02 +00:00
vorotamoroz 36590ee762 (test): improve e2e CouchDB 2026-06-26 10:55:06 +00:00
vorotamoroz 4a33236c8f (test): improve e2e session helper 2026-06-26 10:46:45 +00:00
vorotamoroz be23fa51a1 (test): the E2E test on the real-Obsidian 2026-06-26 10:33:51 +00:00
vorotamoroz 8e24739b96 Tag tidying 2026-06-24 05:55:05 +01:00
vorotamoroz 2d5b5da904 Merge pull request #974 from vrtmrz/0_25_78
Releasing v0.25.78
2026-06-23 20:19:31 +09:00
25 changed files with 4244 additions and 636 deletions
+3 -2
View File
@@ -54,6 +54,7 @@ jobs:
id: meta id: meta
run: | run: |
VERSION=$(jq -r '.version' manifest.json) VERSION=$(jq -r '.version' manifest.json)
MAJOR_MINOR=$(echo "${VERSION}" | cut -d. -f1,2)
SHORT_SHA=$(git rev-parse --short HEAD) SHORT_SHA=$(git rev-parse --short HEAD)
IMAGE="ghcr.io/${{ github.repository_owner }}/livesync-cli" IMAGE="ghcr.io/${{ github.repository_owner }}/livesync-cli"
@@ -61,10 +62,10 @@ jobs:
TAGS="" TAGS=""
if [[ "${{ github.ref }}" == refs/tags/* ]]; then if [[ "${{ github.ref }}" == refs/tags/* ]]; then
# Stable release builds # Stable release builds
TAGS="${IMAGE}:${VERSION}-cli,${IMAGE}:latest,${IMAGE}:${VERSION}-sha-${SHORT_SHA}-cli" TAGS="${IMAGE}:${VERSION}-cli,${IMAGE}:${MAJOR_MINOR}-cli,${IMAGE}:latest,${IMAGE}:${VERSION}-sha-${SHORT_SHA}-cli"
elif [[ "${{ github.ref }}" == refs/heads/main ]]; then elif [[ "${{ github.ref }}" == refs/heads/main ]]; then
# Bleeding-edge / nightly builds # Bleeding-edge / nightly builds
TAGS="${IMAGE}:edge,${IMAGE}:${VERSION}-dev-sha-${SHORT_SHA}-cli" TAGS="${IMAGE}:edge"
else else
# Other branches / manual run fallback # Other branches / manual run fallback
TAGS="${IMAGE}:${VERSION}-dev-sha-${SHORT_SHA}-cli" TAGS="${IMAGE}:${VERSION}-dev-sha-${SHORT_SHA}-cli"
+231
View File
@@ -0,0 +1,231 @@
# Architectural Decision Record: Real Obsidian End-to-End Test Runner
## Status
Proposed / Spike Implemented
## Release
Not yet. Planned after the serviceFeature refactoring branch is reviewed.
## Context
The current end-to-end tests run through Vitest browser mode and a mocked Obsidian environment in `test/harness`. This has been useful for exercising synchronisation flows without launching Obsidian, but it is no longer a reliable final signal for plug-in behaviour.
The main issues are:
- The harness reimplements a large part of the Obsidian API surface, including vault files, workspace events, settings, and lifecycle behaviour. This mock can drift from real Obsidian behaviour without failing.
- The tests run inside a browser-style environment, while the desktop plug-in runs inside Obsidian's Electron environment with its own application lifecycle, storage paths, command registry, and event ordering.
- Several high-value regressions are about integration boundaries: boot-up sequence timing, real vault file reflection, Obsidian command registration, settings persistence, restart prompts, and file watcher behaviour. These are precisely the areas where a mock harness gives weak confidence.
- Maintaining the harness competes with maintaining the plug-in. Adding behaviour to the plug-in often requires teaching the mock another Obsidian detail before the actual regression can be tested.
The current harness should therefore stop being treated as the primary E2E layer.
## Decision
Introduce a new E2E layer that launches real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in installed into those vaults.
The long-term test pyramid should be:
1. Unit tests for deterministic operations and serviceFeature boundaries.
2. Integration tests for CouchDB, Object Storage, P2P services, database operations, and replication protocols.
3. Real Obsidian E2E tests for boot-up sequence, vault reflection, command registration, settings dialogues, restart scheduling, and user-visible workflows.
The existing `test/harness` should be demoted to a transitional compatibility layer. It may remain temporarily while the real Obsidian runner reaches parity for critical flows, but new high-level E2E coverage should target the real runner.
## Non-Goals
- Do not replace unit or integration tests with slow UI tests.
- Do not keep extending the Obsidian mock to cover new Obsidian APIs unless a short-term compatibility bridge is required.
- Do not require real Obsidian E2E for every pull request initially. The first CI integration should be opt-in or nightly until stability is proven.
- Do not test every setting dialogue through UI clicks if the behaviour is already covered by unit or integration tests. Use UI automation only for workflows whose risk is in real Obsidian integration.
## Proposed Architecture
### Runner
Create a dedicated runner under `test/e2e-obsidian/`.
The runner should:
- Create one or more temporary vault directories.
- Build the plug-in once with `npm run build` or a narrower production build command.
- Install `main.js`, `manifest.json`, and `styles.css` when present into `.obsidian/plugins/obsidian-livesync/`.
- Prepare `.obsidian/community-plugins.json` and `.obsidian/plugins/obsidian-livesync/data.json` as needed.
- Launch Obsidian against the temporary vault.
- Wait until the plug-in reports readiness through a deterministic probe.
- Drive assertions through a narrow control channel rather than fragile visual selectors wherever possible.
- Dispose of Obsidian and temporary vaults after each scenario.
### Obsidian Launch
The preferred desktop target is the installed Obsidian application. The launch mechanism should be platform-specific but hidden behind a small adapter:
- Linux: launch the Obsidian executable with a vault path or Obsidian URI, depending on what is most reliable. If an AppImage is used and FUSE is not available, extract it with `--appimage-extract` and launch the extracted `squashfs-root/obsidian` binary.
- macOS: launch the app bundle through `open` or the executable inside the bundle.
- Windows: launch the installed executable or the registered application protocol.
The first implementation can support Linux only if that is the local and CI target. Cross-platform support can be added after the runner contract is stable.
In headless Linux environments, launch through `xvfb-run`, pass Electron flags such as `--no-sandbox` and `--disable-gpu`, and isolate `HOME`, `XDG_CONFIG_HOME`, and `--user-data-dir` per temporary vault.
### Control Channel
The runner needs a stable way to observe readiness and issue test commands. Prefer a test-only plug-in bridge compiled only in test builds or enabled only by an environment variable.
Possible bridge options:
- The official Obsidian CLI, using the installed `obsidian-cli` helper to open vaults, reload the plug-in, run `eval`, and call developer commands.
- A local HTTP/WebSocket bridge bound to `127.0.0.1` with a random port and token.
- A file-based bridge in the vault, where Obsidian writes status files and consumes command files.
- A DevTools protocol bridge if Obsidian exposes a stable debugging port in the test environment.
The first implementation uses Obsidian's CLI for orchestration and readiness checks. The CLI handles vault opening through `obsidian://open?path=...`, enables community plug-ins through `app.plugins.setEnable(true)`, reloads Self-hosted LiveSync through `plugin:reload id=obsidian-livesync`, and verifies that `app.plugins.plugins['obsidian-livesync']` is loaded.
This keeps E2E-only behaviour out of the production plug-in bundle. The runner should not require Self-hosted LiveSync to write marker files or expose a test server merely to prove that Obsidian loaded it.
The DevTools protocol remains useful for diagnostics. Obsidian's CLI exposes developer commands such as `dev:cdp`, `dev:errors`, and `dev:console`, so the runner should prefer the CLI path first and fall back to direct DevTools attachment only if the CLI cannot provide the required signal.
### Test Data and Services
Keep the existing Docker scripts for CouchDB, MinIO, and P2P services. The real Obsidian runner should reuse these service fixtures instead of creating another service orchestration stack.
Each test should use unique database names, bucket prefixes, vault names, and P2P room IDs. This prevents tests from depending on cleanup and makes interrupted runs less harmful.
## Migration Plan
### Phase 0: Discovery
- Confirm how Obsidian can be launched reliably on the local development environment.
- Confirm whether Obsidian accepts a vault path directly, requires an Obsidian URI, or needs a pre-existing vault registry.
- Identify where Obsidian stores per-user state in the test environment and decide how to isolate it.
- Decide whether the first bridge is file-based or HTTP/WebSocket.
Initial discovery on Linux ARM64 found that:
- `Obsidian-1.12.7-arm64.AppImage` requires `libfuse.so.2` for direct AppImage execution.
- Extracting the AppImage with `--appimage-extract` works without FUSE.
- Launching the extracted `squashfs-root/obsidian` binary under `xvfb-run` with isolated user data stays alive for the smoke timeout.
- No missing shared libraries were reported by `ldd` for the extracted binary in the tested environment.
- Obsidian's CLI is disabled unless the global `obsidian.json` contains `cli: true`.
- Passing only `.obsidian/community-plugins.json` is not enough to load community plug-ins on Obsidian 1.12. The runner also has to enable the global community plug-in switch through `app.plugins.setEnable(true)`.
- The reliable launch sequence is: start Obsidian, send `obsidian://open?path=...` through `obsidian-cli`, wait until the vault-side CLI exposes the plug-in catalogue, enable community plug-ins, reload Self-hosted LiveSync, and verify plug-in readiness through `obsidian-cli eval`.
### Phase 1: Smoke Runner
- Add `test/e2e-obsidian/runner` utilities for temporary vault creation, plug-in installation, launch, readiness wait, and cleanup.
- Add one smoke test:
- launch Obsidian with an empty vault,
- load Self-hosted LiveSync,
- wait for the boot-up sequence to become ready,
- read the plug-in version or status through the control channel,
- close Obsidian cleanly.
- Add an npm script such as `test:e2e:obsidian`.
Current implementation status:
- Added `test/e2e-obsidian/runner` helpers for Obsidian discovery, CLI discovery, temporary vault creation, plug-in installation, process launch, CLI execution, and readiness polling.
- Added `test:e2e:obsidian:discover`, `test:e2e:obsidian:cli-help`, `test:e2e:obsidian:smoke`, `test:e2e:obsidian:vault-reflection`, `test:e2e:obsidian:couchdb-upload`, `test:e2e:obsidian:startup-scan`, `test:e2e:obsidian:two-vault-sync`, `test:e2e:obsidian:hidden-file-snippet-sync`, `test:e2e:obsidian:customisation-sync`, `test:e2e:obsidian:setting-markdown-export`, and `test:e2e:obsidian:install-appimage`.
- Added `startObsidianLiveSyncSession()` so future workflows can reuse the launch, vault open, community plug-in enablement, plug-in reload, and readiness sequence without duplicating smoke runner code.
- Added CouchDB runner utilities that reuse `.test.env`/process environment values, create unique temporary databases, query uploaded documents directly, and clean up the database unless `E2E_OBSIDIAN_KEEP_COUCHDB=true` is set.
- Added a manual AppImage installer that downloads Obsidian `1.12.7` for `arm64` or `x86_64`, stores it under `_testdata/obsidian`, and extracts it for FUSE-free execution.
- Confirmed the smoke runner on Linux ARM64 with the extracted Obsidian `1.12.7` AppImage, `xvfb-run`, and the built Self-hosted LiveSync bundle.
- Confirmed the runner can enable the Obsidian CLI through isolated `obsidian.json` state, open the temporary vault through `obsidian-cli`, enable community plug-ins through `app.plugins.setEnable(true)`, reload Self-hosted LiveSync, and verify readiness through `obsidian-cli eval`.
- Removed the first test-only ready-marker bridge from the plug-in bundle. The current runner observes readiness from outside the plug-in through Obsidian's own CLI, so normal user vaults do not receive E2E marker files.
Current verification:
- `npm run tsc-check` passes.
- `npm run build` passes with existing Svelte warnings.
- `npm run test:e2e:obsidian:discover` finds `_testdata/obsidian/squashfs-root/obsidian` when the extracted AppImage is present.
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS=1000 npm run test:e2e:obsidian:smoke` passes locally.
- `npm run test:e2e:obsidian:vault-reflection` creates a note through Obsidian's vault API, verifies the reflected file on disk, and reads it back through Obsidian.
- `npm run test:e2e:obsidian:couchdb-upload` configures a unique CouchDB database, creates a note through Obsidian, commits it into the local database, runs one-shot synchronisation, and verifies that CouchDB contains the metadata document and all referenced chunk documents.
- `npm run test:e2e:obsidian:startup-scan` verifies that a file written while Obsidian is stopped is picked up during the next real Obsidian boot and uploaded to CouchDB after one-shot synchronisation.
- `npm run test:e2e:obsidian:two-vault-sync` verifies two-vault note synchronisation: creation, update, deletion, Markdown conflict automatic merging with the merged result propagated by a second synchronisation, and per-device target-filter differences.
- `npm run test:e2e:obsidian:hidden-file-snippet-sync` verifies hidden file synchronisation as a two-vault round-trip: creation, deletion, automatic JSON conflict merging with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target-pattern differences.
- `npm run test:e2e:obsidian:customisation-sync` verifies a two-vault Customisation Sync workflow: scan a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronise them through CouchDB, apply them on the second vault, assert the resulting `.obsidian` files, propagate a snippet update, and verify deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
- `npm run test:e2e:obsidian:setting-markdown-export` verifies that setting Markdown export creates a vault file and omits credentials when credential export is disabled.
- `npm run test:e2e:obsidian:install-appimage` reuses the existing AppImage and extracted binary when they are already present.
Known limits:
- The smoke runner currently proves only one-vault launch and plug-in load readiness. Broader workflows are covered by separate real Obsidian scripts, including CouchDB upload, startup scan, two-vault note synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export.
- Cross-platform support is still discovery-level. The working path has been validated on Linux ARM64.
- CI wiring is not yet implemented. CI should use `OBSIDIAN_BINARY` or a cached `_testdata/obsidian/squashfs-root` rather than downloading the AppImage on every run.
### Phase 2: First Real Workflow
- Add a one-vault local workflow:
- configure a temporary CouchDB database,
- create a note in the real vault,
- wait for metadata and chunks to be stored,
- restart Obsidian,
- verify that the plug-in loads and the note remains consistent.
This validates real boot-up, settings persistence, vault file access, database writes, and restart-sensitive state.
Current implementation status:
- Added a pre-CouchDB workflow that creates a note through Obsidian's vault API, confirms the note is reflected as a real vault file, and reads the same note back through Obsidian. This covers the vault reflection part of the Phase 2 path before remote database setup is introduced.
- Added a first CouchDB-backed upload workflow, modelled after the CLI Deno tests: reuse the standard CouchDB environment variables, create a unique remote database, apply CouchDB settings through the plug-in's setting service, commit the note through the real Obsidian vault path, run one-shot synchronisation, and assert that remote metadata and chunks exist.
- Added Obsidian-specific workflows for boot-time vault scanning, two-vault note synchronisation, hidden `.obsidian/snippets` file round-tripping, hidden JSON conflict resolution, Customisation Sync application for snippets, config JSON files, and plug-in fixtures, per-device target-filter differences, and setting Markdown export. These scenarios assert against CouchDB documents, vault files, or real Obsidian UI outcomes instead of internal service state.
### Phase 3: Two-Vault Synchronisation
- Launch two Obsidian instances with two temporary vaults.
- Configure both against the same temporary remote database.
- Create, modify, rename, and delete notes in one vault.
- Verify reflection in the other vault.
- Cover encrypted and non-encrypted configurations separately.
### Phase 4: Harness Retirement
- Mark `test/harness` as deprecated in documentation.
- Stop adding new tests to `test/suite` unless they are explicitly transitional.
- Move critical existing scenarios from `test/suite` to real Obsidian E2E or lower-level integration tests.
- Remove the harness only after the new runner covers the critical boot-up and synchronisation workflows.
## CI Strategy
Start with local-only execution. After the smoke runner is stable:
- Run the smoke test in CI on Linux.
- Keep full two-vault synchronisation scenarios as nightly or manually triggered jobs until runtime and flakiness are understood.
- Do not download the Obsidian AppImage on every CI run. Use a pre-installed Obsidian binary, a CI cache for `_testdata/obsidian/squashfs-root`, or a manually triggered preparation job.
- Capture Obsidian logs, plug-in logs, vault snapshots, and service logs on failure.
- Fail fast on launch failures, readiness timeouts, and cleanup failures with clear diagnostics.
## Risks and Mitigations
- **Obsidian licensing and installation**: CI may need a cached installer or a pre-installed binary. Keep the runner capable of using `OBSIDIAN_BINARY`.
- **Flakiness from UI timing**: Prefer a control channel and service-level probes over visual selectors.
- **Multiple instances**: Obsidian may not support multiple independent instances cleanly on all platforms. Start with one-instance smoke tests, then validate two-instance behaviour on Linux before expanding scope.
- **State leakage**: Isolate vault directories, Obsidian user data, remote database names, and bridge tokens per test.
- **Security of E2E controls**: Keep readiness and control outside the production plug-in bundle. Prefer Obsidian CLI probes over E2E-only plug-in code.
- **Runtime cost**: Keep the default PR gate small. Move slow synchronisation matrices to scheduled jobs.
## Open Questions
- Which launch mechanism is most reliable for Obsidian on Linux in this repository's CI environment?
- Can two Obsidian instances run with isolated user data at the same time?
- Do future scenarios need a richer control channel than Obsidian CLI, or can CLI `eval` and developer commands cover the required workflows?
- Should any future E2E-only plug-in code live in a separate test build, or should the production bundle remain free of E2E controls?
- Which existing `test/suite` scenarios are critical enough to port before deprecating the harness?
## Initial Implementation Checklist
1. Add an Obsidian launch discovery script that prints the detected executable, version, and launch mode.
2. Add temporary vault and plug-in installation helpers.
3. Add CLI-based plug-in readiness polling.
4. Add `test:e2e:obsidian:smoke` for one-vault plug-in load.
5. Document required local environment variables, especially `OBSIDIAN_BINARY`.
6. Port one CouchDB-backed workflow after the smoke test is stable.
7. Mark `test/harness` as transitional and block new broad E2E work from targeting it.
## Consequences
- Real Obsidian E2E becomes the source of truth for plug-in lifecycle and vault integration.
- Unit and integration tests remain the primary fast feedback loops.
- The old browser harness can be deleted once the new runner covers the critical workflows.
- The project will gain slower but higher-confidence tests for the behaviours most likely to differ between mocks and Obsidian itself.
+11
View File
@@ -38,6 +38,17 @@
"test:unit:coverage": "vitest run --config vitest.config.unit.ts --coverage", "test:unit:coverage": "vitest run --config vitest.config.unit.ts --coverage",
"test:install-playwright": "npx playwright install chromium", "test:install-playwright": "npx playwright install chromium",
"test:install-dependencies": "npm run test:install-playwright", "test:install-dependencies": "npm run test:install-playwright",
"test:e2e:obsidian:install-appimage": "tsx test/e2e-obsidian/scripts/install-appimage.ts",
"test:e2e:obsidian:discover": "tsx test/e2e-obsidian/scripts/discover.ts",
"test:e2e:obsidian:cli-help": "tsx test/e2e-obsidian/scripts/cli-help.ts",
"test:e2e:obsidian:smoke": "tsx test/e2e-obsidian/scripts/smoke.ts",
"test:e2e:obsidian:vault-reflection": "tsx test/e2e-obsidian/scripts/vault-reflection.ts",
"test:e2e:obsidian:couchdb-upload": "tsx test/e2e-obsidian/scripts/couchdb-upload.ts",
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
"test:e2e:obsidian:setting-markdown-export": "tsx test/e2e-obsidian/scripts/setting-markdown-export.ts",
"test:coverage": "vitest run --coverage", "test:coverage": "vitest run --coverage",
"test:docker-couchdb:up": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-start.sh", "test:docker-couchdb:up": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-start.sh",
"test:docker-couchdb:init": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-init.sh", "test:docker-couchdb:init": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-init.sh",
+98
View File
@@ -0,0 +1,98 @@
# Real Obsidian E2E Runner
This directory contains the experimental real Obsidian end-to-end runner.
The current smoke runner verifies only the launch path:
1. create a temporary vault,
2. install the built Self-hosted LiveSync plug-in artifacts,
3. launch real Obsidian,
4. open the temporary vault through `obsidian-cli`,
5. enable Obsidian community plug-ins for the temporary app profile,
6. reload Self-hosted LiveSync through `obsidian-cli`,
7. verify through `obsidian-cli eval` that the plug-in is loaded,
8. optionally drive a real vault or CouchDB workflow through Obsidian's own API,
9. terminate Obsidian and remove the temporary vault.
The runner does not require Self-hosted LiveSync to expose an E2E-only bridge. Readiness is checked from outside the plug-in through Obsidian's own CLI.
Obsidian 1.12 stores the global community plug-in switch outside `.obsidian/community-plugins.json`. The smoke runner enables it through `app.plugins.setEnable(true)` after the vault window is available.
Future workflows should use `startObsidianLiveSyncSession()` from `runner/session.ts` rather than repeating the launch and plug-in readiness sequence.
## Local Setup
Set `OBSIDIAN_BINARY` when Obsidian is not installed in a standard location.
For an AppImage on Linux without FUSE, use the helper script:
```bash
npm run test:e2e:obsidian:install-appimage
```
The script downloads Obsidian `1.12.7` for the current architecture, stores it in `_testdata/obsidian`, and extracts it to `_testdata/obsidian/squashfs-root`. The runner checks `_testdata/obsidian/squashfs-root/obsidian` before the AppImage path.
Do not download the AppImage on every CI run. Prefer one of these approaches:
- set `OBSIDIAN_BINARY` to a pre-installed Obsidian executable,
- restore `_testdata/obsidian/squashfs-root` from a CI cache, or
- run `test:e2e:obsidian:install-appimage` only in a manually triggered preparation job.
## Commands
```bash
npm run test:e2e:obsidian:install-appimage
npm run test:e2e:obsidian:discover
npm run test:e2e:obsidian:cli-help -- vaults verbose
npm run test:e2e:obsidian:smoke
npm run test:e2e:obsidian:vault-reflection
npm run test:e2e:obsidian:couchdb-upload
npm run test:e2e:obsidian:startup-scan
npm run test:e2e:obsidian:two-vault-sync
npm run test:e2e:obsidian:hidden-file-snippet-sync
npm run test:e2e:obsidian:customisation-sync
npm run test:e2e:obsidian:setting-markdown-export
```
`test:e2e:obsidian:couchdb-upload` reuses the CouchDB variables from `.test.env` or the process environment. It expects a reachable CouchDB service, creates a unique database, configures Self-hosted LiveSync through `obsidian-cli eval`, creates a note in real Obsidian, commits the note into the local database, runs one-shot synchronisation, and verifies that the remote database contains both the metadata document and its chunk documents.
`test:e2e:obsidian:startup-scan` configures a temporary CouchDB database, stops Obsidian, writes a note directly into the vault, restarts Obsidian, and verifies from CouchDB that the boot-time scan picked up the offline file.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, deletion, Markdown conflict automatic merging with the merged result propagated by a second synchronisation, and per-device target filters where one vault ignores a note that the other vault synchronises.
`test:e2e:obsidian:hidden-file-snippet-sync` runs a two-vault hidden file round-trip. It verifies creation and deletion of a real `.obsidian/snippets/*.css` file, automatic JSON conflict merging for a hidden file with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target patterns where one vault ignores a hidden file that the other vault synchronises.
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`.
Start the local CouchDB fixture first when one is not already running:
```bash
npm run test:docker-couchdb:start
```
Useful environment variables:
- `OBSIDIAN_BINARY`: explicit Obsidian executable path.
- `E2E_OBSIDIAN_VERSION`: Obsidian AppImage version for `test:e2e:obsidian:install-appimage`; default is `1.12.7`.
- `E2E_OBSIDIAN_APPIMAGE_ARCH`: AppImage architecture override, such as `arm64` or `x86_64`.
- `E2E_OBSIDIAN_APPIMAGE_URL`: explicit AppImage URL override.
- `E2E_OBSIDIAN_DOWNLOAD_DIR`: AppImage download and extraction directory; default is `_testdata/obsidian`.
- `E2E_OBSIDIAN_FORCE_DOWNLOAD=true`: re-download the AppImage even when it exists.
- `E2E_OBSIDIAN_SKIP_EXTRACT=true`: download the AppImage without extracting it.
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS`: smoke timeout in milliseconds.
- `E2E_OBSIDIAN_READY_TIMEOUT_MS`: plug-in readiness timeout in milliseconds.
- `E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS`: timeout for waiting until the vault-side Obsidian CLI exposes the plug-in catalogue.
- `E2E_OBSIDIAN_CLI_TIMEOUT_MS`: timeout for each `obsidian-cli` invocation.
- `E2E_OBSIDIAN_FILE_TIMEOUT_MS`: timeout for waiting until a note created through Obsidian's vault API is reflected to disk.
- `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready.
- `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database.
- `E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS`: timeout for waiting until CouchDB contains uploaded E2E documents.
- `E2E_OBSIDIAN_KEEP_COUCHDB=true`: keep the temporary CouchDB database for inspection.
- `E2E_OBSIDIAN_STARTUP_GRACE_MS`: early process-exit detection window in milliseconds.
- `E2E_OBSIDIAN_KEEP_VAULT=true`: keep the temporary vault for inspection.
- `E2E_OBSIDIAN_USE_XVFB=false`: disable automatic `xvfb-run` on headless Linux.
- `E2E_OBSIDIAN_ARGS`: override the default Obsidian launch arguments.
On headless Linux, the runner automatically uses `/usr/bin/xvfb-run` when no `DISPLAY` or `WAYLAND_DISPLAY` is present.
+102
View File
@@ -0,0 +1,102 @@
import { spawn } from "node:child_process";
export type ObsidianCliResult = {
code: number | null;
signal: NodeJS.Signals | null;
stdout: string;
stderr: string;
};
function parseEvalJson(stdout: string): unknown {
const marker = "=> ";
const markerIndex = stdout.indexOf(marker);
const text = markerIndex >= 0 ? stdout.slice(markerIndex + marker.length) : stdout;
return JSON.parse(text.trim());
}
export async function runObsidianCli(
cliBinary: string,
args: string[],
env: NodeJS.ProcessEnv = process.env,
timeoutMs = Number(process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ?? 10000)
): Promise<ObsidianCliResult> {
return await new Promise((resolve, reject) => {
const child = spawn(cliBinary, args, {
stdio: ["ignore", "pipe", "pipe"],
env,
});
let stdout = "";
let stderr = "";
const timeout = setTimeout(() => {
child.kill("SIGKILL");
reject(new Error(`Obsidian CLI timed out: ${cliBinary} ${args.join(" ")}`));
}, timeoutMs);
child.stdout?.on("data", (chunk: Buffer) => {
stdout += chunk.toString();
});
child.stderr?.on("data", (chunk: Buffer) => {
stderr += chunk.toString();
});
child.on("error", (error) => {
clearTimeout(timeout);
reject(error);
});
child.on("exit", (code, signal) => {
clearTimeout(timeout);
resolve({ code, signal, stdout, stderr });
});
});
}
export async function openVaultWithObsidianCli(
cliBinary: string,
vaultPath: string,
env: NodeJS.ProcessEnv = process.env
): Promise<void> {
const result = await runObsidianCli(cliBinary, [`obsidian://open?path=${encodeURIComponent(vaultPath)}`], env);
if (result.code !== 0) {
throw new Error(
[
`Failed to open Obsidian vault through CLI. code=${result.code}, signal=${result.signal}`,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
export async function evalObsidianJson<T>(
cliBinary: string,
code: string,
env: NodeJS.ProcessEnv = process.env
): Promise<T> {
const result = await runObsidianCli(cliBinary, ["eval", `code=${code}`], env);
if (result.code !== 0) {
throw new Error(
[
`Failed to evaluate Obsidian JavaScript through CLI. code=${result.code}, signal=${result.signal}`,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
try {
return parseEvalJson(result.stdout) as T;
} catch (error) {
throw new Error(
[
`Failed to parse Obsidian CLI eval JSON. code=${result.code}, signal=${result.signal}`,
error instanceof Error ? `parse error: ${error.message}` : undefined,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
+174
View File
@@ -0,0 +1,174 @@
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
export type CouchDbConfig = {
uri: string;
username: string;
password: string;
dbPrefix: string;
};
export type CouchDbDocument = {
_id: string;
_rev?: string;
type?: string;
path?: string;
children?: string[];
[key: string]: unknown;
};
export type CouchDbAllDocsResponse = {
rows: Array<{
id: string;
key: string;
value: { rev: string; deleted?: boolean };
doc?: CouchDbDocument;
}>;
};
function parseEnvFile(content: string): Record<string, string> {
const entries = content
.split(/\r?\n/u)
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("#"))
.map((line) => {
const equalsAt = line.indexOf("=");
if (equalsAt < 0) {
return undefined;
}
const key = line.slice(0, equalsAt).trim();
const rawValue = line.slice(equalsAt + 1).trim();
const value = rawValue.replace(/^['"]|['"]$/gu, "");
return [key, value] as const;
})
.filter((entry): entry is readonly [string, string] => entry !== undefined);
return Object.fromEntries(entries);
}
function getEnvValue(values: Record<string, string | undefined>, ...keys: string[]): string {
for (const key of keys) {
const value = values[key]?.trim();
if (value) {
return value;
}
}
throw new Error(`Required CouchDB environment value is missing: ${keys.join(" or ")}`);
}
function authHeader(config: Pick<CouchDbConfig, "username" | "password">): string {
return `Basic ${Buffer.from(`${config.username}:${config.password}`).toString("base64")}`;
}
function databaseUrl(config: Pick<CouchDbConfig, "uri">, dbName: string, suffix = ""): string {
return `${config.uri.replace(/\/+$/u, "")}/${encodeURIComponent(dbName)}${suffix}`;
}
async function couchDbRequest(
config: Pick<CouchDbConfig, "uri" | "username" | "password">,
path: string,
init: RequestInit = {}
): Promise<Response> {
const response = await fetch(`${config.uri.replace(/\/+$/u, "")}${path}`, {
...init,
headers: {
authorization: authHeader(config),
...init.headers,
},
});
return response;
}
export async function loadCouchDbConfig(envFile = ".test.env"): Promise<CouchDbConfig> {
let fileValues: Record<string, string> = {};
try {
fileValues = parseEnvFile(await readFile(resolve(envFile), "utf-8"));
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
throw error;
}
}
const values = { ...fileValues, ...process.env };
return {
uri: getEnvValue(values, "COUCHDB_URI", "hostname").replace(/\/+$/u, ""),
username: getEnvValue(values, "COUCHDB_USER", "username"),
password: getEnvValue(values, "COUCHDB_PASSWORD", "password"),
dbPrefix: getEnvValue(values, "COUCHDB_DBNAME", "dbname"),
};
}
export function makeUniqueDatabaseName(prefix: string, label: string): string {
const safePrefix = prefix
.toLowerCase()
.replace(/[^a-z0-9_$()+/-]+/gu, "-")
.replace(/^-+/u, "")
.slice(0, 80);
const random = Math.random().toString(36).slice(2, 8);
return `${safePrefix || "livesync-e2e"}-${label}-${Date.now()}-${random}`;
}
export async function assertCouchDbReachable(config: CouchDbConfig): Promise<void> {
const response = await couchDbRequest(config, "/_up");
if (!response.ok) {
throw new Error(`CouchDB is not reachable at ${config.uri}. HTTP ${response.status}: ${await response.text()}`);
}
}
export async function createCouchDbDatabase(config: CouchDbConfig, dbName: string): Promise<void> {
const response = await fetch(databaseUrl(config, dbName), {
method: "PUT",
headers: { authorization: authHeader(config) },
});
if (!response.ok && response.status !== 412) {
throw new Error(
`Failed to create CouchDB database ${dbName}. HTTP ${response.status}: ${await response.text()}`
);
}
}
export async function deleteCouchDbDatabase(config: CouchDbConfig, dbName: string): Promise<void> {
const response = await fetch(databaseUrl(config, dbName), {
method: "DELETE",
headers: { authorization: authHeader(config) },
});
if (!response.ok && response.status !== 404) {
throw new Error(
`Failed to delete CouchDB database ${dbName}. HTTP ${response.status}: ${await response.text()}`
);
}
}
export async function fetchAllCouchDbDocs(config: CouchDbConfig, dbName: string): Promise<CouchDbAllDocsResponse> {
const response = await fetch(databaseUrl(config, dbName, "/_all_docs?include_docs=true"), {
headers: { authorization: authHeader(config) },
});
if (!response.ok) {
throw new Error(
`Failed to read CouchDB documents from ${dbName}. HTTP ${response.status}: ${await response.text()}`
);
}
return (await response.json()) as CouchDbAllDocsResponse;
}
export async function waitForCouchDbDocs(
config: CouchDbConfig,
dbName: string,
predicate: (docs: CouchDbDocument[]) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ?? 15000)
): Promise<CouchDbDocument[]> {
const deadline = Date.now() + timeoutMs;
let lastDocs: CouchDbDocument[] = [];
while (Date.now() < deadline) {
const response = await fetchAllCouchDbDocs(config, dbName);
lastDocs = response.rows.flatMap((row) => (row.doc ? [row.doc] : []));
if (predicate(lastDocs)) {
return lastDocs;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(
`Timed out waiting for CouchDB documents in ${dbName}. Last document IDs: ${lastDocs
.map((doc) => doc._id)
.join(", ")}`
);
}
+149
View File
@@ -0,0 +1,149 @@
import { accessSync, constants, existsSync } from "node:fs";
import { resolve } from "node:path";
import { platform } from "node:process";
export type ObsidianDiscoveryResult = {
binary?: string;
source?: string;
checked: string[];
};
const defaultCandidatesByPlatform: Record<NodeJS.Platform, string[]> = {
aix: [],
android: [],
darwin: [
"/Applications/Obsidian.app/Contents/MacOS/Obsidian",
"/Applications/Obsidian.app/Contents/MacOS/obsidian",
],
freebsd: [],
haiku: [],
linux: [
"_testdata/obsidian/squashfs-root/obsidian",
"_testdata/obsidian/squashfs-root/AppRun",
"_testdata/obsidian/Obsidian-1.12.7-arm64.AppImage",
"_testdata/obsidian/Obsidian-1.12.7-x86_64.AppImage",
"/usr/bin/obsidian",
"/usr/local/bin/obsidian",
"/snap/bin/obsidian",
"/opt/Obsidian/obsidian",
"/opt/obsidian/obsidian",
"/app/bin/obsidian",
],
openbsd: [],
sunos: [],
win32: ["C:\\Program Files\\Obsidian\\Obsidian.exe", "C:\\Program Files (x86)\\Obsidian\\Obsidian.exe"],
cygwin: [],
netbsd: [],
};
const defaultCliCandidatesByPlatform: Record<NodeJS.Platform, string[]> = {
aix: [],
android: [],
darwin: [
"/Applications/Obsidian.app/Contents/MacOS/obsidian-cli",
"/Applications/Obsidian.app/Contents/Resources/obsidian-cli",
],
freebsd: [],
haiku: [],
linux: [
"_testdata/obsidian/squashfs-root/obsidian-cli",
"/usr/bin/obsidian-cli",
"/usr/local/bin/obsidian-cli",
"/snap/bin/obsidian-cli",
"/opt/Obsidian/obsidian-cli",
"/opt/obsidian/obsidian-cli",
],
openbsd: [],
sunos: [],
win32: ["C:\\Program Files\\Obsidian\\obsidian-cli.exe", "C:\\Program Files (x86)\\Obsidian\\obsidian-cli.exe"],
cygwin: [],
netbsd: [],
};
function isUsableFile(path: string): boolean {
const resolvedPath = resolve(path);
if (!existsSync(resolvedPath)) {
return false;
}
if (platform === "win32") {
return true;
}
try {
accessSync(resolvedPath, constants.X_OK);
return true;
} catch {
return false;
}
}
export function discoverObsidianBinary(env: NodeJS.ProcessEnv = process.env): ObsidianDiscoveryResult {
const checked: string[] = [];
const envBinary = env.OBSIDIAN_BINARY?.trim();
if (envBinary) {
checked.push(envBinary);
if (isUsableFile(envBinary)) {
return {
binary: resolve(envBinary),
source: "OBSIDIAN_BINARY",
checked,
};
}
}
const candidates = defaultCandidatesByPlatform[platform] ?? [];
for (const candidate of candidates) {
checked.push(candidate);
if (isUsableFile(candidate)) {
return {
binary: resolve(candidate),
source: "default-path",
checked,
};
}
}
return { checked };
}
export function requireObsidianBinary(env: NodeJS.ProcessEnv = process.env): string {
const result = discoverObsidianBinary(env);
if (!result.binary) {
throw new Error(
[
"Could not find an Obsidian executable.",
"Set OBSIDIAN_BINARY to the installed Obsidian executable path.",
`Checked paths: ${result.checked.length > 0 ? result.checked.join(", ") : "(none)"}`,
].join("\n")
);
}
return result.binary;
}
export function discoverObsidianCli(env: NodeJS.ProcessEnv = process.env): ObsidianDiscoveryResult {
const checked: string[] = [];
const envBinary = env.OBSIDIAN_CLI?.trim();
if (envBinary) {
checked.push(envBinary);
if (isUsableFile(envBinary)) {
return {
binary: resolve(envBinary),
source: "OBSIDIAN_CLI",
checked,
};
}
}
const candidates = defaultCliCandidatesByPlatform[platform] ?? [];
for (const candidate of candidates) {
checked.push(candidate);
if (isUsableFile(candidate)) {
return {
binary: resolve(candidate),
source: "default-path",
checked,
};
}
}
return { checked };
}
+119
View File
@@ -0,0 +1,119 @@
import { spawn, type ChildProcess } from "node:child_process";
import { once } from "node:events";
import { existsSync } from "node:fs";
import { dirname } from "node:path";
import { platform } from "node:process";
export type ObsidianProcess = {
process: ChildProcess;
stop: () => Promise<void>;
};
export type LaunchObsidianOptions = {
binary: string;
vaultPath: string;
homePath?: string;
xdgConfigPath?: string;
userDataPath?: string;
startupGraceMs?: number;
};
function splitArgs(args: string): string[] {
return args.split(" ").filter((arg) => arg.length > 0);
}
function launchArgs(options: LaunchObsidianOptions): string[] {
const explicitArgs = process.env.E2E_OBSIDIAN_ARGS;
if (explicitArgs) {
return splitArgs(explicitArgs);
}
return [
"--no-sandbox",
"--disable-gpu",
"--disable-software-rasterizer",
...(process.env.E2E_OBSIDIAN_USE_USER_DATA_DIR === "true" && options.userDataPath
? [`--user-data-dir=${options.userDataPath}`]
: []),
];
}
function shouldUseXvfb(): boolean {
if (process.env.E2E_OBSIDIAN_USE_XVFB === "false") {
return false;
}
if (process.env.DISPLAY || process.env.WAYLAND_DISPLAY) {
return false;
}
return platform === "linux" && existsSync("/usr/bin/xvfb-run");
}
export async function launchObsidian(options: LaunchObsidianOptions): Promise<ObsidianProcess> {
const startupGraceMs = options.startupGraceMs ?? 1000;
const args = launchArgs(options);
const useXvfb = shouldUseXvfb();
const command = useXvfb ? "/usr/bin/xvfb-run" : options.binary;
const commandArgs = useXvfb ? ["-a", options.binary, ...args] : args;
const child = spawn(command, commandArgs, {
cwd: dirname(options.binary),
detached: true,
stdio: ["ignore", "pipe", "pipe"],
env: {
...process.env,
...(options.homePath ? { HOME: options.homePath } : {}),
...(options.xdgConfigPath ? { XDG_CONFIG_HOME: options.xdgConfigPath } : {}),
OBSIDIAN_DISABLE_GPU: process.env.OBSIDIAN_DISABLE_GPU ?? "1",
},
});
let stderr = "";
let stdout = "";
child.stderr?.on("data", (chunk: Buffer) => {
stderr += chunk.toString();
});
child.stdout?.on("data", (chunk: Buffer) => {
stdout += chunk.toString();
});
const exitPromise = once(child, "exit").then(([code, signal]) => ({ code, signal }));
const timer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), startupGraceMs);
});
const firstResult = await Promise.race([exitPromise, timer]);
if (firstResult !== "timeout") {
throw new Error(
[
`Obsidian exited before the smoke timeout. code=${firstResult.code}, signal=${firstResult.signal}`,
stdout ? `stdout:\n${stdout}` : undefined,
stderr ? `stderr:\n${stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
return {
process: child,
stop: async () => {
if (child.exitCode !== null || child.signalCode !== null) {
return;
}
if (child.pid) {
process.kill(-child.pid, "SIGTERM");
} else {
child.kill("SIGTERM");
}
const stopTimer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), 5000);
});
const stopResult = await Promise.race([exitPromise, stopTimer]);
if (stopResult === "timeout") {
if (child.pid) {
process.kill(-child.pid, "SIGKILL");
} else {
child.kill("SIGKILL");
}
await exitPromise;
}
},
};
}
@@ -0,0 +1,182 @@
import { evalObsidianJson } from "./cli.ts";
import type { CouchDbConfig } from "./couchdb.ts";
export type ConfiguredSettings = {
isConfigured: boolean;
liveSync: boolean;
syncOnStart: boolean;
syncOnSave: boolean;
couchDB_URI: string;
couchDB_DBNAME: string;
};
export type CoreReadiness = {
databaseReady: boolean;
appReady: boolean;
};
export type LocalDatabaseEntry = {
id: string;
rev: string;
path: string;
type: string;
children: string[];
};
export function assertEqual(actual: unknown, expected: unknown, message: string): void {
if (actual !== expected) {
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
}
}
export async function configureCouchDb(
cliBinary: string,
env: NodeJS.ProcessEnv,
settings: Pick<CouchDbConfig, "uri" | "username" | "password"> & { dbName: string },
overrides: Record<string, unknown> = {}
): Promise<ConfiguredSettings> {
return await evalObsidianJson<ConfiguredSettings>(
cliBinary,
[
"(async()=>{",
"const plugin=app.plugins.plugins['obsidian-livesync'];",
"const core=plugin.core;",
"const nextSettings={",
`couchDB_URI:${JSON.stringify(settings.uri)},`,
`couchDB_USER:${JSON.stringify(settings.username)},`,
`couchDB_PASSWORD:${JSON.stringify(settings.password)},`,
`couchDB_DBNAME:${JSON.stringify(settings.dbName)},`,
"remoteType:'',",
"liveSync:false,",
"syncOnStart:false,",
"syncOnSave:false,",
"usePluginSync:false,",
"usePluginSyncV2:true,",
"useEden:false,",
"customChunkSize:1,",
"sendChunksBulkMaxSize:1,",
"chunkSplitterVersion:'v3-rabin-karp',",
"readChunksOnline:false,",
"disableCheckingConfigMismatch:true,",
"isConfigured:true,",
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
"};",
"await core.services.setting.applyExternalSettings(nextSettings,true);",
"await core.services.control.applySettings();",
"const current=core.services.setting.currentSettings();",
"return JSON.stringify({",
"isConfigured:current.isConfigured,",
"liveSync:current.liveSync,",
"syncOnStart:current.syncOnStart,",
"syncOnSave:current.syncOnSave,",
"couchDB_URI:current.couchDB_URI,",
"couchDB_DBNAME:current.couchDB_DBNAME,",
"});",
"})()",
].join(""),
env
);
}
export async function waitForLiveSyncCoreReady(
cliBinary: string,
env: NodeJS.ProcessEnv,
timeoutMs = Number(process.env.E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS ?? 20000)
): Promise<CoreReadiness> {
const deadline = Date.now() + timeoutMs;
let lastReadiness: CoreReadiness | undefined;
while (Date.now() < deadline) {
lastReadiness = await evalObsidianJson<CoreReadiness>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"return JSON.stringify({",
"databaseReady:core.services.database.isDatabaseReady(),",
"appReady:core.services.appLifecycle.isReady(),",
"});",
"})()",
].join(""),
env
);
if (lastReadiness.databaseReady && lastReadiness.appReady) {
return lastReadiness;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Self-hosted LiveSync core readiness: ${JSON.stringify(lastReadiness)}`);
}
export async function prepareRemote(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const settings=core.services.setting.currentSettings();",
"const replicator=core.services.replicator.getActiveReplicator();",
"await replicator.tryCreateRemoteDatabase(settings);",
"await replicator.markRemoteResolved(settings);",
"const status=await replicator.getRemoteStatus(settings);",
"return JSON.stringify({status});",
"})()",
].join(""),
env
);
}
export async function pushLocalChanges(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"await core.services.fileProcessing.commitPendingFileEvents();",
"const result=await core.services.replication.replicate(true);",
"return JSON.stringify({result:!!result});",
"})()",
].join(""),
env
);
}
export async function waitForLocalDatabaseEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
options: { hidden?: boolean; timeoutMs?: number } = {}
): Promise<LocalDatabaseEntry> {
const timeoutMs = options.timeoutMs ?? Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000);
return await evalObsidianJson<LocalDatabaseEntry>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const hidden=${JSON.stringify(options.hidden === true)};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const deadline=Date.now()+timeoutMs;",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"while(Date.now()<deadline){",
"await core.services.fileProcessing.commitPendingFileEvents();",
"const dbPath=hidden?`i:${path}`:path;",
"entry=await core.localDatabase.getDBEntry(dbPath,undefined,false,true).catch(()=>false);",
"if(!entry||!entry._id){",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"entry=rows.map((row)=>row.doc).find((doc)=>doc&&(",
"doc._id===dbPath||doc._id===path||doc.path===dbPath||doc.path===path||",
"(typeof doc.path==='string'&&doc.path.endsWith(path))||",
"(typeof doc._id==='string'&&doc._id.endsWith(path))",
"))||false;",
"}",
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
"await sleep(250);",
"}",
"if(!entry||!entry._id) throw new Error(`Timed out waiting for local database entry: ${path}`);",
"return JSON.stringify({id:entry._id,rev:entry._rev,path:entry.path,type:entry.type,children:entry.children||[]});",
"})()",
].join(""),
env
);
}
@@ -0,0 +1,39 @@
import { copyFile, mkdir, writeFile } from "node:fs/promises";
import { existsSync } from "node:fs";
import { join, resolve } from "node:path";
export type PluginInstallResult = {
pluginDir: string;
copied: string[];
};
const pluginId = "obsidian-livesync";
export async function installBuiltPlugin(vaultPath: string, rootDir = process.cwd()): Promise<PluginInstallResult> {
const pluginDir = join(vaultPath, ".obsidian", "plugins", pluginId);
const copied: string[] = [];
await mkdir(pluginDir, { recursive: true });
const requiredArtifacts = ["main.js", "manifest.json"];
for (const artifact of requiredArtifacts) {
const source = resolve(rootDir, artifact);
if (!existsSync(source)) {
throw new Error(`Required plug-in artifact is missing: ${source}`);
}
await copyFile(source, join(pluginDir, artifact));
copied.push(artifact);
}
const optionalArtifacts = ["styles.css"];
for (const artifact of optionalArtifacts) {
const source = resolve(rootDir, artifact);
if (!existsSync(source)) {
continue;
}
await copyFile(source, join(pluginDir, artifact));
copied.push(artifact);
}
await writeFile(join(vaultPath, ".obsidian", "community-plugins.json"), JSON.stringify([pluginId], null, 4));
return { pluginDir, copied };
}
+41
View File
@@ -0,0 +1,41 @@
import { evalObsidianJson } from "./cli.ts";
export type PluginReadiness = {
status: "ready";
pluginId: string;
pluginVersion: string;
vaultName: string;
};
export async function waitForPluginReady(
cliBinary: string,
env: NodeJS.ProcessEnv,
timeoutMs = Number(process.env.E2E_OBSIDIAN_READY_TIMEOUT_MS ?? 20000)
): Promise<PluginReadiness> {
const deadline = Date.now() + timeoutMs;
let lastOutput = "";
while (Date.now() < deadline) {
try {
const readiness = await evalObsidianJson<PluginReadiness>(
cliBinary,
[
"(async()=>JSON.stringify({",
"status:!!app.plugins.plugins['obsidian-livesync']?'ready':'pending',",
"pluginId:'obsidian-livesync',",
"pluginVersion:app.plugins.manifests['obsidian-livesync']?.version,",
"vaultName:app.vault.getName()",
"}))()",
].join(""),
env
);
if (readiness.status === "ready") {
return readiness;
}
} catch (error) {
lastOutput = error instanceof Error ? error.message : String(error);
// Keep polling until Obsidian exposes the vault-side CLI and plug-in state.
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Self-hosted LiveSync readiness through Obsidian CLI.\n${lastOutput}`);
}
+94
View File
@@ -0,0 +1,94 @@
import { openVaultWithObsidianCli, runObsidianCli } from "./cli.ts";
import { launchObsidian, type ObsidianProcess } from "./launch.ts";
import { installBuiltPlugin, type PluginInstallResult } from "./pluginInstaller.ts";
import { waitForPluginReady, type PluginReadiness } from "./readiness.ts";
import type { TemporaryVault } from "./vault.ts";
export type ObsidianLiveSyncSession = {
app: ObsidianProcess;
cliEnv: NodeJS.ProcessEnv;
install: PluginInstallResult;
readiness: PluginReadiness;
};
export type StartObsidianLiveSyncSessionOptions = {
binary: string;
cliBinary: string;
vault: TemporaryVault;
startupGraceMs?: number;
};
async function waitForPluginCatalogue(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS ?? 15000);
let lastOutput = "";
while (Date.now() < deadline) {
const result = await runObsidianCli(cliBinary, ["plugins", "filter=community"], env);
lastOutput = [result.stdout, result.stderr].filter(Boolean).join("\n");
if (result.stdout.includes("obsidian-livesync")) {
return;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Obsidian plug-in catalogue through CLI.\n${lastOutput}`);
}
async function enableCommunityPlugins(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
const result = await runObsidianCli(cliBinary, ["eval", "code=(async()=>app.plugins.setEnable(true))()"], env);
if (result.code !== 0 || result.stdout.includes("Error:")) {
throw new Error(
[
`Failed to enable Obsidian community plug-ins through CLI. code=${result.code}, signal=${result.signal}`,
result.stdout ? `stdout:\n${result.stdout}` : undefined,
result.stderr ? `stderr:\n${result.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
async function reloadLiveSyncPlugin(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
const reload = await runObsidianCli(cliBinary, ["plugin:reload", "id=obsidian-livesync"], env);
if (reload.code !== 0 || !reload.stdout.includes("Reloaded: obsidian-livesync")) {
throw new Error(
[
`Failed to reload Self-hosted LiveSync through Obsidian CLI. code=${reload.code}, signal=${reload.signal}`,
reload.stdout ? `stdout:\n${reload.stdout}` : undefined,
reload.stderr ? `stderr:\n${reload.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
export async function startObsidianLiveSyncSession(
options: StartObsidianLiveSyncSessionOptions
): Promise<ObsidianLiveSyncSession> {
const install = await installBuiltPlugin(options.vault.path);
const app = await launchObsidian({
binary: options.binary,
vaultPath: options.vault.path,
homePath: options.vault.homePath,
xdgConfigPath: options.vault.xdgConfigPath,
userDataPath: options.vault.userDataPath,
startupGraceMs: options.startupGraceMs,
});
const cliEnv = {
...process.env,
HOME: options.vault.homePath,
XDG_CONFIG_HOME: options.vault.xdgConfigPath,
};
try {
await openVaultWithObsidianCli(options.cliBinary, options.vault.path, cliEnv);
await waitForPluginCatalogue(options.cliBinary, cliEnv);
await enableCommunityPlugins(options.cliBinary, cliEnv);
await reloadLiveSyncPlugin(options.cliBinary, cliEnv);
const readiness = await waitForPluginReady(options.cliBinary, cliEnv);
return { app, cliEnv, install, readiness };
} catch (error) {
await app.stop();
throw error;
}
}
+77
View File
@@ -0,0 +1,77 @@
import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { tmpdir } from "node:os";
export type TemporaryVault = {
path: string;
name: string;
homePath: string;
xdgConfigPath: string;
userDataPath: string;
dispose: () => Promise<void>;
};
export async function createTemporaryVault(prefix = "obsidian-livesync-e2e-"): Promise<TemporaryVault> {
const vaultPath = await mkdtemp(join(tmpdir(), prefix));
const statePath = await mkdtemp(join(tmpdir(), `${prefix}state-`));
const name = vaultPath.split(/[\\/]/).pop() ?? "obsidian-livesync-e2e";
await mkdir(join(vaultPath, ".obsidian"), { recursive: true });
const homePath = join(statePath, "home");
const xdgConfigPath = join(statePath, "xdg-config");
const userDataPath = join(statePath, "user-data");
await mkdir(homePath, { recursive: true });
await mkdir(xdgConfigPath, { recursive: true });
await mkdir(userDataPath, { recursive: true });
await writeFile(
join(vaultPath, ".obsidian", "app.json"),
JSON.stringify({ legacyEditor: false, safeMode: false }, null, 4)
);
await writeObsidianVaultRegistry(vaultPath, name, homePath, xdgConfigPath, userDataPath);
return {
path: vaultPath,
name,
homePath,
xdgConfigPath,
userDataPath,
dispose: async () => {
if (process.env.E2E_OBSIDIAN_KEEP_VAULT === "true") {
console.log(`Keeping temporary vault: ${vaultPath}`);
console.log(`Keeping temporary Obsidian state: ${statePath}`);
return;
}
await Promise.all([
rm(vaultPath, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 }),
rm(statePath, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 }),
]);
},
};
}
async function writeObsidianVaultRegistry(
vaultPath: string,
vaultName: string,
homePath: string,
xdgConfigPath: string,
userDataPath: string
): Promise<void> {
const vaultId = `livesync-e2e-${Date.now()}`;
const registry = {
cli: true,
vaults: {
[vaultId]: {
path: vaultPath,
ts: Date.now(),
open: true,
name: vaultName,
},
},
};
const registryText = JSON.stringify(registry, null, 4);
for (const configRoot of [join(homePath, ".config"), xdgConfigPath]) {
const obsidianConfigDir = join(configRoot, "obsidian");
await mkdir(obsidianConfigDir, { recursive: true });
await writeFile(join(obsidianConfigDir, "obsidian.json"), registryText);
}
await writeFile(join(userDataPath, "obsidian.json"), registryText);
}
+51
View File
@@ -0,0 +1,51 @@
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { launchObsidian } from "../runner/launch.ts";
import { runObsidianCli } from "../runner/cli.ts";
import { createTemporaryVault } from "../runner/vault.ts";
import { installBuiltPlugin } from "../runner/pluginInstaller.ts";
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let app;
try {
await installBuiltPlugin(vault.path);
app = await launchObsidian({
binary,
vaultPath: vault.path,
homePath: vault.homePath,
xdgConfigPath: vault.xdgConfigPath,
userDataPath: vault.userDataPath,
});
const cliEnv = {
...process.env,
HOME: vault.homePath,
XDG_CONFIG_HOME: vault.xdgConfigPath,
};
await runObsidianCli(cli.binary, [`obsidian://open?path=${encodeURIComponent(vault.path)}`], cliEnv);
await new Promise((resolve) => setTimeout(resolve, 3000));
if (process.env.E2E_OBSIDIAN_RELOAD_PLUGIN === "true") {
await runObsidianCli(cli.binary, ["eval", "code=(async()=>app.plugins.setEnable(true))()"], cliEnv);
await runObsidianCli(cli.binary, ["plugin:reload", "id=obsidian-livesync"], cliEnv);
}
const cliArgs = process.argv.slice(2);
const result = await runObsidianCli(cli.binary, cliArgs.length > 0 ? cliArgs : ["--help"], cliEnv);
console.log(result.stdout);
console.error(result.stderr);
process.exitCode = result.code ?? 1;
} finally {
if (app) {
await app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+140
View File
@@ -0,0 +1,140 @@
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/couchdb-upload.md";
const noteContent = [
"# CouchDB upload from real Obsidian",
"",
"This note is created through Obsidian and uploaded by Self-hosted LiveSync.",
"The content is intentionally long enough to require chunk metadata in the local database.",
"0123456789 abcdefghijklmnopqrstuvwxyz 0123456789 abcdefghijklmnopqrstuvwxyz",
"0123456789 abcdefghijklmnopqrstuvwxyz 0123456789 abcdefghijklmnopqrstuvwxyz",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.ProcessEnv): Promise<LocalDatabaseEntry> {
return await evalObsidianJson<LocalDatabaseEntry>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
`const content=${JSON.stringify(noteContent)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"await app.vault.create(path,content);",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"for(let i=0;i<40;i++){",
"await core.services.fileProcessing.commitPendingFileEvents();",
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true).catch(()=>false);",
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
"await sleep(250);",
"}",
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
"})()",
].join(""),
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "obsidian-upload");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureCouchDb(cli.binary, session.cliEnv, {
uri: couchDb.uri,
username: couchDb.username,
password: couchDb.password,
dbName,
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
assertEqual(configured.couchDB_URI, couchDb.uri, "Configured CouchDB URI did not match.");
assertEqual(configured.couchDB_DBNAME, dbName, "Configured CouchDB database name did not match.");
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during this one-shot workflow.");
assertEqual(configured.syncOnStart, false, "Sync on start should remain disabled during this workflow.");
assertEqual(configured.syncOnSave, false, "Sync on save should remain disabled during this workflow.");
await prepareRemote(cli.binary, session.cliEnv);
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
await pushLocalChanges(cli.binary, session.cliEnv);
const remoteDocs = await waitForCouchDbDocs(couchDb, dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(localEntry.id) && localEntry.children.every((childId) => ids.has(childId));
});
const remoteMetadata = remoteDocs.find((doc) => doc._id === localEntry.id);
assertEqual(
remoteMetadata?.path,
localEntry.path,
"Remote metadata path did not match the local database entry."
);
console.log(
`Uploaded metadata ${localEntry.id} and ${localEntry.children.length} chunk(s) to CouchDB database ${dbName}`
);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,601 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
type CouchDbConfig,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const snippetPath = ".obsidian/snippets/livesync-customisation-e2e.css";
const snippetContent = [
"body {",
" --livesync-customisation-e2e-colour: #3d6f54;",
"}",
"",
".livesync-customisation-e2e {",
" color: var(--livesync-customisation-e2e-colour);",
"}",
"",
].join("\n");
const snippetUpdatedContent = [
"body {",
" --livesync-customisation-e2e-colour: #73548f;",
"}",
"",
".livesync-customisation-e2e {",
" background-color: var(--livesync-customisation-e2e-colour);",
"}",
"",
].join("\n");
const configPath = ".obsidian/livesync-customisation-e2e.json";
const configContent = JSON.stringify({ source: "customisation-sync", enabled: true }, null, 4) + "\n";
const pluginDir = ".obsidian/plugins/livesync-e2e-sample";
const pluginManifestPath = `${pluginDir}/manifest.json`;
const pluginMainPath = `${pluginDir}/main.js`;
const pluginStylesPath = `${pluginDir}/styles.css`;
const pluginManifestContent =
JSON.stringify(
{
id: "livesync-e2e-sample",
name: "LiveSync E2E Sample",
version: "0.0.1",
minAppVersion: "1.0.0",
description: "A sample plug-in fixture for real Obsidian E2E.",
author: "Self-hosted LiveSync",
isDesktopOnly: false,
},
null,
4
) + "\n";
const pluginMainContent = [
"module.exports = class LiveSyncE2ESamplePlugin extends Plugin {",
" async onload() {",
" this.register(() => undefined);",
" }",
"};",
"",
].join("\n");
const pluginStylesContent = ".livesync-e2e-sample { color: #73548f; }\n";
const sourceDeviceName = "customisation-sync-a";
const targetDeviceName = "customisation-sync-b";
type RunnerContext = {
binary: string;
cliBinary: string;
couchDb: CouchDbConfig;
dbName: string;
};
type CustomisationEntry = {
id: string;
path: string;
children: string[];
};
type CustomisationScanResult = {
enabled: boolean;
useV2: boolean;
device: string;
configDir: string;
files: string[];
};
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
await rm(join(vaultPath, path), { force: true });
}
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
return await readFile(join(vaultPath, path), "utf-8");
}
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
try {
await readFile(join(vaultPath, path));
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
async function waitForPathContent(
vaultPath: string,
path: string,
predicate: (content: string) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const deadline = Date.now() + timeoutMs;
let lastContent = "";
while (Date.now() < deadline) {
if (await pathExists(vaultPath, path)) {
lastContent = await readVaultFile(vaultPath, path);
if (predicate(lastContent)) {
return lastContent;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
deviceName: string
): Promise<ObsidianLiveSyncSession> {
const session = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await configureCouchDb(
context.cliBinary,
session.cliEnv,
{
uri: context.couchDb.uri,
username: context.couchDb.username,
password: context.couchDb.password,
dbName: context.dbName,
},
{
deviceAndVaultName: deviceName,
usePluginSync: true,
usePluginSyncV2: true,
autoSweepPlugins: false,
autoSweepPluginsPeriodic: false,
syncInternalFiles: false,
}
);
await evalObsidianJson<unknown>(
context.cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
`core.services.setting.setDeviceAndVaultName(${JSON.stringify(deviceName)});`,
"await core.services.setting.saveSettingData();",
"return JSON.stringify({device:core.services.setting.getDeviceAndVaultName()});",
"})()",
].join(""),
session.cliEnv
);
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await prepareRemote(context.cliBinary, session.cliEnv);
return session;
}
async function scanCustomisations(cliBinary: string, env: NodeJS.ProcessEnv): Promise<CustomisationScanResult> {
return await evalObsidianJson<CustomisationScanResult>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const before=await addOn.scanInternalFiles();",
"await addOn.scanAllConfigFiles(false);",
"return JSON.stringify({",
"ok:true,",
"enabled:core.settings.usePluginSync,",
"useV2:core.settings.usePluginSyncV2,",
"device:core.services.setting.getDeviceAndVaultName(),",
"configDir:addOn.configDir,",
"files:before,",
"});",
"})()",
].join(""),
env
);
}
async function storeCustomisationFile(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const term=core.services.setting.getDeviceAndVaultName();",
"const stat=await core.storageAccess.statHidden(path);",
"const category=addOn.getFileCategory(path);",
"const result=await addOn.storeCustomizationFiles(path,term);",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>doc.path);",
"if(!result){",
" throw new Error(`Could not store Customisation Sync file: path=${path}; term=${term}; category=${category}; stat=${JSON.stringify(stat)}; result=${JSON.stringify(result)}; entries=${JSON.stringify(entries)}`);",
"}",
"return JSON.stringify({ok:true,path,term,category,entries});",
"})()",
].join(""),
env
);
}
async function deleteCustomisationSyncEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&!doc.deleted&&!doc._deleted)||false;",
"if(!entry) throw new Error(`Could not find customisation sync entry to delete: ${category}/${name}`);",
"if(!(await addOn.deleteConfigOnDatabase(entry.path))){",
" throw new Error(`Could not delete Customisation Sync entry: ${entry.path}`);",
"}",
"return JSON.stringify({ok:true,path:entry.path});",
"})()",
].join(""),
env
);
}
async function waitForCustomisationEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
): Promise<CustomisationEntry> {
const entries = await waitForCustomisationEntries(cliBinary, env, category, name, 1, term, timeoutMs);
return entries[0];
}
async function waitForCustomisationEntries(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
count: number,
term?: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
): Promise<CustomisationEntry[]> {
return await evalObsidianJson<CustomisationEntry[]>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const count=${JSON.stringify(count)};`,
`const term=${JSON.stringify(term ?? "")};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const deadline=Date.now()+timeoutMs;",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entries=[];",
"while(Date.now()<deadline){",
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
" entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&Array.isArray(doc.children)&&doc.children.length>0);",
" if(entries.length>=count) break;",
" await sleep(250);",
"}",
"if(entries.length<count){",
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
" const found=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>({id:doc._id,path:doc.path,children:doc.children?.length??0}));",
" throw new Error(`Timed out waiting for customisation sync entries: ${category}/${name}; expected=${count}; entries=${JSON.stringify(found)}`);",
"}",
"return JSON.stringify(entries.map((entry)=>({id:entry._id,path:entry.path,children:entry.children||[]})));",
"})()",
].join(""),
env
);
}
async function waitForCustomisationEntryAbsent(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const deadline=Date.now()+timeoutMs;",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"while(Date.now()<deadline){",
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
" entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&!doc.deleted&&!doc._deleted)||false;",
" if(!entry) return JSON.stringify({ok:true});",
" await sleep(250);",
"}",
"throw new Error(`Timed out waiting for customisation sync entry deletion: ${category}/${name}; entry=${JSON.stringify(entry)}`);",
"})()",
].join(""),
env
);
}
async function applyRemoteCustomisationEntry(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
name: string,
term?: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`)))||false;",
"if(!entry) throw new Error(`Could not find remote customisation entry: ${category}/${name}`);",
"const display=addOn.createPluginDataFromV2(entry.path);",
"if(!display) throw new Error(`Could not create Customisation Sync display entry: ${entry.path}`);",
"const file=await addOn.createPluginDataExFileV2(entry.path);",
"if(!file) throw new Error(`Could not load Customisation Sync file entry: ${entry.path}`);",
"await display.setFile(file);",
"if(!(await addOn.applyDataV2(display))){",
" throw new Error(`Could not apply Customisation Sync entry: ${entry.path}`);",
"}",
"return JSON.stringify({ok:true,path:entry.path});",
"})()",
].join(""),
env
);
}
async function applyRemoteCustomisationGroup(
cliBinary: string,
env: NodeJS.ProcessEnv,
category: "PLUGIN_MAIN",
name: string,
term?: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const category=${JSON.stringify(category)};`,
`const name=${JSON.stringify(name)};`,
`const term=${JSON.stringify(term ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('ConfigSync');",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`)));",
"if(entries.length===0) throw new Error(`Could not find remote customisation entries: ${category}/${name}`);",
"const display=addOn.createPluginDataFromV2(entries[0].path);",
"if(!display) throw new Error(`Could not create Customisation Sync display entry: ${entries[0].path}`);",
"for(const entry of entries){",
" const file=await addOn.createPluginDataExFileV2(entry.path);",
" if(!file) throw new Error(`Could not load Customisation Sync file entry: ${entry.path}`);",
" await display.setFile(file);",
"}",
"if(!(await addOn.applyDataV2(display))){",
" throw new Error(`Could not apply Customisation Sync group: ${category}/${name}`);",
"}",
"return JSON.stringify({ok:true,count:entries.length});",
"})()",
].join(""),
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "customisation-sync");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
const snippetPathParts = snippetPath.split("/");
const snippetName = snippetPathParts[snippetPathParts.length - 1] ?? snippetPath;
const configName = configPath.split("/").pop() ?? configPath;
const pluginName = pluginDir.split("/").pop() ?? pluginDir;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
await writeVaultFile(vaultA.path, configPath, configContent);
await writeVaultFile(vaultA.path, pluginManifestPath, pluginManifestContent);
await writeVaultFile(vaultA.path, pluginMainPath, pluginMainContent);
await writeVaultFile(vaultA.path, pluginStylesPath, pluginStylesContent);
let session = await startConfiguredSession(context, vaultA, sourceDeviceName);
const scanResult = await scanCustomisations(context.cliBinary, session.cliEnv);
console.log(`Customisation scan files: ${scanResult.files.join(", ") || "(none)"}`);
await storeCustomisationFile(context.cliBinary, session.cliEnv, snippetPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, configPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginManifestPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginMainPath);
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginStylesPath);
const entry = await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName);
const configEntry = await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "CONFIG", configName);
const pluginEntries = await waitForCustomisationEntries(
context.cliBinary,
session.cliEnv,
"PLUGIN_MAIN",
pluginName,
3
);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
const entries = [entry, configEntry, ...pluginEntries];
return entries.every(
(target) => ids.has(target.id) && target.children.every((childId) => ids.has(childId))
);
});
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
assertEqual(
await pathExists(vaultB.path, snippetPath),
false,
"Customisation Sync snippet was reflected before explicit application."
);
await applyRemoteCustomisationEntry(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
const applied = await waitForPathContent(vaultB.path, snippetPath, (content) => content === snippetContent);
await applyRemoteCustomisationEntry(context.cliBinary, session.cliEnv, "CONFIG", configName, sourceDeviceName);
const appliedConfig = await waitForPathContent(vaultB.path, configPath, (content) => content === configContent);
await applyRemoteCustomisationGroup(
context.cliBinary,
session.cliEnv,
"PLUGIN_MAIN",
pluginName,
sourceDeviceName
);
const appliedPluginManifest = await waitForPathContent(
vaultB.path,
pluginManifestPath,
(content) => content === pluginManifestContent
);
const appliedPluginMain = await waitForPathContent(
vaultB.path,
pluginMainPath,
(content) => content === pluginMainContent
);
const appliedPluginStyles = await waitForPathContent(
vaultB.path,
pluginStylesPath,
(content) => content === pluginStylesContent
);
await session.app.stop();
assertEqual(applied, snippetContent, "Customisation Sync snippet content did not match after application.");
assertEqual(appliedConfig, configContent, "Customisation Sync config content did not match after application.");
assertEqual(
appliedPluginManifest,
pluginManifestContent,
"Customisation Sync plug-in manifest did not match after application."
);
assertEqual(appliedPluginMain, pluginMainContent, "Customisation Sync plug-in main file did not match.");
assertEqual(appliedPluginStyles, pluginStylesContent, "Customisation Sync plug-in stylesheet did not match.");
await writeVaultFile(vaultA.path, snippetPath, snippetUpdatedContent);
session = await startConfiguredSession(context, vaultA, sourceDeviceName);
await storeCustomisationFile(context.cliBinary, session.cliEnv, snippetPath);
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await applyRemoteCustomisationEntry(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
const updated = await waitForPathContent(
vaultB.path,
snippetPath,
(content) => content === snippetUpdatedContent
);
await session.app.stop();
assertEqual(updated, snippetUpdatedContent, "Updated Customisation Sync snippet did not apply.");
await removeVaultFile(vaultA.path, snippetPath);
session = await startConfiguredSession(context, vaultA, sourceDeviceName);
await deleteCustomisationSyncEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
await waitForCustomisationEntryAbsent(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCustomisationEntryAbsent(
context.cliBinary,
session.cliEnv,
"SNIPPET",
snippetName,
sourceDeviceName
);
await session.app.stop();
console.log(
`Customisation Sync applied snippet, config, and plug-in fixtures, then propagated snippet update and sync-data deletion.`
);
} finally {
await vaultA.dispose();
await vaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+13
View File
@@ -0,0 +1,13 @@
import { discoverObsidianBinary } from "../runner/environment.ts";
const result = discoverObsidianBinary();
if (result.binary) {
console.log(`Obsidian executable: ${result.binary}`);
console.log(`Source: ${result.source}`);
process.exit(0);
}
console.error("Obsidian executable was not found.");
console.error("Set OBSIDIAN_BINARY to the installed Obsidian executable path.");
console.error(`Checked paths: ${result.checked.length > 0 ? result.checked.join(", ") : "(none)"}`);
process.exit(1);
@@ -0,0 +1,541 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
type CouchDbConfig,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const snippetPath = ".obsidian/snippets/livesync-e2e.css";
const snippetContent = [
"body {",
" --livesync-e2e-snippet-colour: #245a70;",
"}",
"",
".livesync-e2e-snippet {",
" color: var(--livesync-e2e-snippet-colour);",
"}",
"",
].join("\n");
const mergeJsonPath = ".obsidian/livesync-e2e-merge.json";
const manualMergeJsonPath = ".obsidian/livesync-e2e-manual-merge.json";
const targetPath = ".obsidian/livesync-targeted/only-a.json";
type RunnerContext = {
binary: string;
cliBinary: string;
couchDb: CouchDbConfig;
dbName: string;
};
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
await rm(join(vaultPath, path), { force: true });
}
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
return await readFile(join(vaultPath, path), "utf-8");
}
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
try {
await readFile(join(vaultPath, path));
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
async function waitForPathContent(
vaultPath: string,
path: string,
predicate: (content: string) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const deadline = Date.now() + timeoutMs;
let lastContent = "";
while (Date.now() < deadline) {
if (await pathExists(vaultPath, path)) {
lastContent = await readVaultFile(vaultPath, path);
if (predicate(lastContent)) {
return lastContent;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
}
async function waitForPathDeleted(
vaultPath: string,
path: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (!(await pathExists(vaultPath, path))) {
return;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
}
function hasJsonValues(content: string, values: Record<string, unknown>): boolean {
try {
const parsed = JSON.parse(content) as Record<string, unknown>;
return Object.entries(values).every(([key, value]) => parsed[key] === value);
} catch {
return false;
}
}
async function scanHiddenStorage(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('HiddenFileSync');",
"await addOn.scanAllStorageChanges(true);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function scanHiddenDatabase(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('HiddenFileSync');",
"await addOn.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('HiddenFileSync');",
"await addOn.resolveConflictOnInternalFiles();",
"await addOn.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function autoMergeHiddenJsonConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('HiddenFileSync');",
"let doc=false;",
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
" if(entry.path===prefixedPath){ doc=entry; break; }",
"}",
"if(!doc) throw new Error(`Could not find hidden conflict candidate: ${path}`);",
"if(!doc._conflicts?.length) throw new Error(`Hidden file has no conflicts: ${path}`);",
"const conflicts=doc._conflicts.sort((a,b)=>Number(a.split('-')[0])-Number(b.split('-')[0]));",
"const conflictedRev=conflicts[0];",
"const conflictedRevNo=Number(conflictedRev.split('-')[0]);",
"const revFrom=await core.localDatabase.getRaw(doc._id,{revs_info:true});",
"const commonBase=(revFrom._revs_info||[])",
" .filter((rev)=>rev.status==='available'&&Number(rev.rev.split('-')[0])<conflictedRevNo)",
" .map((rev)=>rev.rev)[0]||'';",
"const result=await core.localDatabase.managers.conflictManager.mergeObject(",
" doc.path, commonBase, doc._rev, conflictedRev",
");",
"if(!result){",
" throw new Error(`Hidden JSON conflict was not auto-mergeable: ${path}; base=${commonBase}; current=${doc._rev}; conflict=${conflictedRev}`);",
"}",
"await addOn.ensureDir(path);",
"const stat=await addOn.writeFile(path,result);",
"if(!stat) throw new Error(`Could not write merged hidden file: ${path}`);",
"await addOn.storeInternalFileToDatabase({path,mtime:stat.mtime,ctime:stat.ctime,size:stat.size},true);",
"await core.localDatabase.removeRevision(doc._id,conflictedRev);",
"await addOn.extractInternalFileFromDatabase(path);",
"await addOn.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true,merged:JSON.parse(result)});",
"})()",
].join(""),
env
);
}
async function openHiddenJsonResolveModal(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('HiddenFileSync');",
"let doc=false;",
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
" if(entry.path===prefixedPath){ doc=entry; break; }",
"}",
"if(!doc?._conflicts?.length) throw new Error(`Could not find hidden JSON conflict: ${path}`);",
"const conflicts=doc._conflicts.sort((a,b)=>Number(a.split('-')[0])-Number(b.split('-')[0]));",
"const docA=await core.localDatabase.getDBEntry(prefixedPath,{rev:doc._rev});",
"const docB=await core.localDatabase.getDBEntry(prefixedPath,{rev:conflicts[0]});",
"if(docA===false||docB===false) throw new Error(`Could not load conflicted hidden JSON entries: ${path}`);",
"void addOn.showJSONMergeDialogAndMerge(docA,docB);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function clickJsonResolveOption(cliBinary: string, env: NodeJS.ProcessEnv, mode: "AB" | "BA"): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const mode=${JSON.stringify(mode)};`,
"const deadline=Date.now()+10000;",
"while(Date.now()<deadline){",
" const input=[...document.querySelectorAll('input[name=\"disp\"]')].find((candidate)=>candidate.value===mode);",
" const apply=[...document.querySelectorAll('button')].find((button)=>button.textContent?.trim()==='Apply');",
" if(input&&apply){",
" input.click();",
" input.dispatchEvent(new Event('change',{bubbles:true}));",
" await new Promise((resolve)=>setTimeout(resolve,100));",
" apply.click();",
" return JSON.stringify({ok:true});",
" }",
" await new Promise((resolve)=>setTimeout(resolve,250));",
"}",
"const buttons=[...document.querySelectorAll('button')].map((button)=>button.textContent?.trim()).filter(Boolean);",
"const inputs=[...document.querySelectorAll('input[name=\"disp\"]')].map((input)=>input.value);",
"throw new Error(`Timed out waiting for JSON resolve modal; buttons=${JSON.stringify(buttons)}; inputs=${JSON.stringify(inputs)}`);",
"})()",
].join(""),
env
);
}
async function storeHiddenFileAsConflict(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
baseRev: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const baseRev=${JSON.stringify(baseRev)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const addOn=core.getAddOn('HiddenFileSync');",
"const fileInfo=await addOn.loadFileWithInfo(path);",
"if(fileInfo.deleted) throw new Error(`Hidden file was unexpectedly deleted: ${path}`);",
"const baseData=await addOn.__loadBaseSaveData(path,true);",
"if(baseData===false) throw new Error(`Could not load base save data: ${path}`);",
"const saveData={",
" ...baseData,",
" data:fileInfo.body,",
" mtime:fileInfo.stat.mtime,",
" ctime:fileInfo.stat.ctime,",
" size:fileInfo.stat.size,",
" children:[],",
" deleted:false,",
" type:baseData.datatype,",
"};",
"const result=await core.localDatabase.putDBEntry(saveData,false,baseRev);",
"if(!result?.ok) throw new Error(`Could not store conflicted hidden file: ${path}`);",
"return JSON.stringify({ok:true,rev:result.rev});",
"})()",
].join(""),
env
);
}
async function createHiddenJsonConflict(
context: RunnerContext,
session: ObsidianLiveSyncSession,
vault: TemporaryVault,
path: string,
base: string,
left: string,
right: string
): Promise<void> {
await writeVaultFile(vault.path, path, base);
await scanHiddenStorage(context.cliBinary, session.cliEnv);
const baseEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
await writeVaultFile(vault.path, path, left);
await scanHiddenStorage(context.cliBinary, session.cliEnv);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
await writeVaultFile(vault.path, path, right);
await storeHiddenFileAsConflict(context.cliBinary, session.cliEnv, path, baseEntry.rev);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
overrides: Record<string, unknown> = {}
): Promise<ObsidianLiveSyncSession> {
const session = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await configureCouchDb(
context.cliBinary,
session.cliEnv,
{
uri: context.couchDb.uri,
username: context.couchDb.username,
password: context.couchDb.password,
dbName: context.dbName,
},
{
syncInternalFiles: true,
syncInternalFilesBeforeReplication: true,
watchInternalFileChanges: false,
syncInternalFilesTargetPatterns: "",
...overrides,
}
);
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await prepareRemote(context.cliBinary, session.cliEnv);
return session;
}
async function uploadHiddenFile(
context: RunnerContext,
session: ObsidianLiveSyncSession,
path: string
): Promise<LocalDatabaseEntry> {
await scanHiddenStorage(context.cliBinary, session.cliEnv);
const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
});
return entry;
}
async function pullAndApplyHiddenFiles(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
await pushLocalChanges(context.cliBinary, session.cliEnv);
await resolveHiddenConflicts(context.cliBinary, session.cliEnv);
await scanHiddenDatabase(context.cliBinary, session.cliEnv);
}
async function runCreateRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
let session = await startConfiguredSession(context, vaultA);
const entry = await uploadHiddenFile(context, session, snippetPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await pullAndApplyHiddenFiles(context, session);
const received = await waitForPathContent(vaultB.path, snippetPath, (content) => content === snippetContent);
await session.app.stop();
assertEqual(received, snippetContent, "Hidden snippet content did not round-trip to the second vault.");
console.log(`Hidden create round-trip copied ${entry.id} to the second vault.`);
}
async function runDeleteRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
await removeVaultFile(vaultA.path, snippetPath);
let session = await startConfiguredSession(context, vaultA);
await scanHiddenStorage(context.cliBinary, session.cliEnv);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await pullAndApplyHiddenFiles(context, session);
await waitForPathDeleted(vaultB.path, snippetPath);
await session.app.stop();
console.log("Hidden delete round-trip removed the snippet from the second vault.");
}
async function runJsonConflictRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const base = JSON.stringify({ base: true, fromA: false, fromB: false }, null, 4) + "\n";
const left = JSON.stringify({ base: true, fromA: true, fromB: false }, null, 4) + "\n";
const right = JSON.stringify({ base: true, fromA: false, fromB: true }, null, 4) + "\n";
let session = await startConfiguredSession(context, vaultB);
await createHiddenJsonConflict(context, session, vaultB, mergeJsonPath, base, left, right);
await autoMergeHiddenJsonConflict(context.cliBinary, session.cliEnv, mergeJsonPath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
const mergedOnB = await waitForPathContent(vaultB.path, mergeJsonPath, (content) =>
hasJsonValues(content, { fromA: true, fromB: true })
);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await pullAndApplyHiddenFiles(context, session);
const mergedOnA = await waitForPathContent(vaultA.path, mergeJsonPath, (content) =>
hasJsonValues(content, { fromA: true, fromB: true })
);
await session.app.stop();
assertEqual(mergedOnA, mergedOnB, "Merged hidden JSON content was not consistent across both vaults.");
console.log("Hidden JSON conflict was automatically merged and round-tripped.");
}
async function runJsonManualConflictResolution(context: RunnerContext, vault: TemporaryVault): Promise<void> {
const base = JSON.stringify({ shared: "base" }, null, 4) + "\n";
const left = JSON.stringify({ shared: "left", fromA: true }, null, 4) + "\n";
const right = JSON.stringify({ shared: "right", fromB: true }, null, 4) + "\n";
const session = await startConfiguredSession(context, vault);
await createHiddenJsonConflict(context, session, vault, manualMergeJsonPath, base, left, right);
await openHiddenJsonResolveModal(context.cliBinary, session.cliEnv, manualMergeJsonPath);
await clickJsonResolveOption(context.cliBinary, session.cliEnv, "AB");
const merged = await waitForPathContent(vault.path, manualMergeJsonPath, (content) =>
hasJsonValues(content, { shared: "right", fromA: true, fromB: true })
);
await session.app.stop();
const parsed = JSON.parse(merged);
assertEqual(parsed.shared, "right", "Manual JSON conflict resolution did not apply the selected merged result.");
assertEqual(parsed.fromA, true, "Manual JSON conflict resolution lost the first-side value.");
assertEqual(parsed.fromB, true, "Manual JSON conflict resolution lost the second-side value.");
console.log("Hidden JSON conflict modal applied the selected merged result.");
}
async function runTargetMismatch(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const targetContent = JSON.stringify({ onlyA: true, targetMismatch: true }, null, 4) + "\n";
await writeVaultFile(vaultA.path, targetPath, targetContent);
let session = await startConfiguredSession(context, vaultA);
await uploadHiddenFile(context, session, targetPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "snippets",
});
await pullAndApplyHiddenFiles(context, session);
assertEqual(
await pathExists(vaultB.path, targetPath),
false,
"Hidden file was applied on a device where it was not a target file."
);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "",
});
await pullAndApplyHiddenFiles(context, session);
const received = await waitForPathContent(vaultB.path, targetPath, (content) => content === targetContent);
await session.app.stop();
assertEqual(received, targetContent, "Hidden file was not applied after it became a target file.");
console.log("Hidden target mismatch respected per-device target patterns, then applied after enabling the target.");
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "hidden-roundtrip");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
await runCreateRoundTrip(context, vaultA, vaultB);
await runDeleteRoundTrip(context, vaultA, vaultB);
await runJsonConflictRoundTrip(context, vaultA, vaultB);
await runJsonManualConflictResolution(context, vaultB);
await runTargetMismatch(context, vaultA, vaultB);
} finally {
await vaultA.dispose();
await vaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,121 @@
import { createWriteStream, existsSync } from "node:fs";
import { chmod, mkdir } from "node:fs/promises";
import { get } from "node:https";
import { arch } from "node:process";
import { basename, join, resolve } from "node:path";
import { spawn } from "node:child_process";
const defaultVersion = "1.12.7";
function appImageArch(): string {
const requestedArch = process.env.E2E_OBSIDIAN_APPIMAGE_ARCH?.trim();
if (requestedArch) {
return requestedArch;
}
if (arch === "arm64") {
return "arm64";
}
if (arch === "x64") {
return "x86_64";
}
throw new Error(`Unsupported architecture for Obsidian AppImage: ${arch}`);
}
function appImageUrl(version: string, imageArch: string): string {
return `https://github.com/obsidianmd/obsidian-releases/releases/download/v${version}/Obsidian-${version}-${imageArch}.AppImage`;
}
function download(url: string, destination: string, redirectsLeft = 5): Promise<void> {
return new Promise((resolveDownload, reject) => {
const request = get(url, (response) => {
const statusCode = response.statusCode ?? 0;
const location = response.headers.location;
if (statusCode >= 300 && statusCode < 400 && location) {
response.resume();
if (redirectsLeft <= 0) {
reject(new Error(`Too many redirects while downloading ${url}`));
return;
}
download(new URL(location, url).toString(), destination, redirectsLeft - 1)
.then(resolveDownload)
.catch(reject);
return;
}
if (statusCode !== 200) {
response.resume();
reject(new Error(`Failed to download ${url}: HTTP ${statusCode}`));
return;
}
const file = createWriteStream(destination, { mode: 0o755 });
response.pipe(file);
file.on("finish", () => {
file.close((error) => {
if (error) {
reject(error);
} else {
resolveDownload();
}
});
});
file.on("error", reject);
});
request.on("error", reject);
});
}
function extractAppImage(appImagePath: string, cwd: string): Promise<void> {
return new Promise((resolveExtract, reject) => {
const child = spawn(appImagePath, ["--appimage-extract"], {
cwd,
stdio: "inherit",
});
child.on("error", reject);
child.on("exit", (code, signal) => {
if (code === 0) {
resolveExtract();
return;
}
reject(new Error(`AppImage extraction failed. code=${code}, signal=${signal}`));
});
});
}
async function main(): Promise<void> {
const version = process.env.E2E_OBSIDIAN_VERSION?.trim() || defaultVersion;
const imageArch = appImageArch();
const targetDir = resolve(process.env.E2E_OBSIDIAN_DOWNLOAD_DIR?.trim() || "_testdata/obsidian");
const url = process.env.E2E_OBSIDIAN_APPIMAGE_URL?.trim() || appImageUrl(version, imageArch);
const appImagePath = join(targetDir, basename(new URL(url).pathname));
const extractedBinary = join(targetDir, "squashfs-root", "obsidian");
const forceDownload = process.env.E2E_OBSIDIAN_FORCE_DOWNLOAD === "true";
const skipExtract = process.env.E2E_OBSIDIAN_SKIP_EXTRACT === "true";
await mkdir(targetDir, { recursive: true });
if (!existsSync(appImagePath) || forceDownload) {
console.log(`Downloading Obsidian AppImage: ${url}`);
console.log(`Destination: ${appImagePath}`);
await download(url, appImagePath);
await chmod(appImagePath, 0o755);
} else {
console.log(`Using existing Obsidian AppImage: ${appImagePath}`);
}
if (!skipExtract) {
if (existsSync(extractedBinary)) {
console.log(`Using existing extracted Obsidian binary: ${extractedBinary}`);
} else {
console.log(`Extracting Obsidian AppImage in ${targetDir}`);
await extractAppImage(appImagePath, targetDir);
console.log(`Extracted Obsidian binary: ${extractedBinary}`);
}
}
console.log(`Set OBSIDIAN_BINARY=${extractedBinary} to use the extracted binary explicitly.`);
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,106 @@
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { assertEqual, waitForLiveSyncCoreReady } from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const settingPath = "LiveSync/settings-export.md";
async function waitForFileContaining(
vaultPath: string,
path: string,
predicates: ((content: string) => boolean)[],
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const fullPath = join(vaultPath, path);
const deadline = Date.now() + timeoutMs;
let lastContent = "";
let lastError: unknown;
while (Date.now() < deadline) {
try {
lastContent = await readFile(fullPath, "utf-8");
if (predicates.every((predicate) => predicate(lastContent))) {
return lastContent;
}
} catch (error) {
lastError = error;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for setting Markdown: ${fullPath}\nLast error: ${String(lastError)}`);
}
async function configureSettingMarkdown(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"await core.services.setting.applyExternalSettings({",
`settingSyncFile:${JSON.stringify(settingPath)},`,
"writeCredentialsForSettingSync:false,",
"couchDB_USER:'e2e-user',",
"couchDB_PASSWORD:'e2e-password',",
"passphrase:'e2e-passphrase',",
"showVerboseLog:true,",
"},true);",
"await core.services.setting.saveSettingData();",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
await configureSettingMarkdown(cli.binary, session.cliEnv);
const content = await waitForFileContaining(vault.path, settingPath, [
(value) => value.includes("````yaml:livesync-setting"),
(value) => value.includes(`settingSyncFile: ${settingPath}`),
(value) => value.includes("showVerboseLog: true"),
]);
assertEqual(
content.includes("couchDB_PASSWORD: e2e-password"),
false,
"Credential leaked into setting Markdown."
);
assertEqual(content.includes("passphrase: e2e-passphrase"), false, "Passphrase leaked into setting Markdown.");
console.log(`Generated setting Markdown without credentials: ${settingPath}`);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+41
View File
@@ -0,0 +1,41 @@
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
console.log(`Installed plug-in artifacts: ${session.install.copied.join(", ")}`);
const { readiness } = session;
console.log(
`Obsidian plug-in ready: ${readiness.pluginId}@${readiness.pluginVersion} in ${readiness.vaultName}`
);
await new Promise((resolve) => setTimeout(resolve, Number(process.env.E2E_OBSIDIAN_SMOKE_TIMEOUT_MS ?? 1000)));
console.log("Obsidian stayed alive after the plug-in readiness check.");
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+116
View File
@@ -0,0 +1,116 @@
import { mkdir, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/startup-scan.md";
const noteContent = [
"# Startup scan",
"",
"This note was written while Obsidian was stopped.",
"The test verifies that the next real Obsidian boot scans it into the local database.",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "startup-scan");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureCouchDb(cli.binary, session.cliEnv, {
uri: couchDb.uri,
username: couchDb.username,
password: couchDb.password,
dbName,
});
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not configured.");
await prepareRemote(cli.binary, session.cliEnv);
await session.app.stop();
session = undefined;
await writeVaultFile(vault.path, notePath, noteContent);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const localEntry = await waitForLocalDatabaseEntry(cli.binary, session.cliEnv, notePath);
await pushLocalChanges(cli.binary, session.cliEnv);
const remoteDocs = await waitForCouchDbDocs(couchDb, dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(localEntry.id) && localEntry.children.every((childId) => ids.has(childId));
});
const remoteMetadata = remoteDocs.find((doc) => doc._id === localEntry.id);
assertEqual(remoteMetadata?.path, localEntry.path, "Startup-scanned remote metadata path did not match.");
console.log(`Startup scan uploaded metadata ${localEntry.id} and ${localEntry.children.length} chunk(s).`);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+438
View File
@@ -0,0 +1,438 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
type CouchDbConfig,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const createPath = "E2E/two-vault/create.md";
const updatePath = "E2E/two-vault/update.md";
const deletePath = "E2E/two-vault/delete.md";
const conflictPath = "E2E/two-vault/conflict.md";
const targetMismatchPath = "E2E/two-vault/target-mismatch.md";
type RunnerContext = {
binary: string;
cliBinary: string;
couchDb: CouchDbConfig;
dbName: string;
};
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, content, "utf-8");
}
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
await rm(join(vaultPath, path), { force: true });
}
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
return await readFile(join(vaultPath, path), "utf-8");
}
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
try {
await readFile(join(vaultPath, path));
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
async function waitForPathContent(
vaultPath: string,
path: string,
predicate: (content: string) => boolean,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<string> {
const deadline = Date.now() + timeoutMs;
let lastContent = "";
while (Date.now() < deadline) {
if (await pathExists(vaultPath, path)) {
lastContent = await readVaultFile(vaultPath, path);
if (predicate(lastContent)) {
return lastContent;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
}
async function waitForPathDeleted(
vaultPath: string,
path: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (!(await pathExists(vaultPath, path))) {
return;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
}
async function writeNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string, content: string) {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(content)};`,
"const folder=path.split('/').slice(0,-1).join('/');",
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.modify(existing,content);",
"else await app.vault.create(path,content);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function deleteNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string) {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
overrides: Record<string, unknown> = {}
): Promise<ObsidianLiveSyncSession> {
const session = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await configureCouchDb(
context.cliBinary,
session.cliEnv,
{
uri: context.couchDb.uri,
username: context.couchDb.username,
password: context.couchDb.password,
dbName: context.dbName,
},
overrides
);
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await prepareRemote(context.cliBinary, session.cliEnv);
return session;
}
async function uploadNote(
context: RunnerContext,
session: ObsidianLiveSyncSession,
path: string
): Promise<LocalDatabaseEntry> {
const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
});
return entry;
}
async function syncAndApply(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
await pushLocalChanges(context.cliBinary, session.cliEnv);
}
async function storeFileRevision(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
content: string,
baseRev?: string
): Promise<string> {
const result = await evalObsidianJson<{ rev: string }>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(content)};`,
`const baseRev=${JSON.stringify(baseRev ?? "")};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const blob=new Blob([content],{type:'text/plain'});",
"const id=await core.services.path.path2id(path);",
"const now=Date.now();",
"const result=await core.localDatabase.putDBEntry({",
" _id:id,",
" path,",
" data:blob,",
" ctime:now,",
" mtime:now,",
" size:(await blob.arrayBuffer()).byteLength,",
" children:[],",
" datatype:'plain',",
" type:'plain',",
" eden:{},",
"},false,baseRev||undefined);",
"if(!result?.ok) throw new Error(`Could not store file revision: ${path}`);",
"return JSON.stringify({ok:true,rev:result.rev});",
"})()",
].join(""),
env
);
return result.rev;
}
async function createMarkdownConflict(
context: RunnerContext,
session: ObsidianLiveSyncSession,
vault: TemporaryVault,
path: string,
base: string,
left: string,
right: string
): Promise<void> {
const baseRev = await storeFileRevision(context.cliBinary, session.cliEnv, path, base);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path);
await storeFileRevision(context.cliBinary, session.cliEnv, path, left, baseRev);
await storeFileRevision(context.cliBinary, session.cliEnv, path, right, baseRev);
await writeVaultFile(vault.path, path, right);
}
async function autoMergeMarkdownConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const result=await core.localDatabase.managers.conflictManager.tryAutoMerge(path,true);",
"if(!('result' in result)){",
" throw new Error(`Markdown conflict was not auto-mergeable: ${path}; ${JSON.stringify(result)}`);",
"}",
"if(!(await core.databaseFileAccess.storeContent(path,result.result))){",
" throw new Error(`Could not store merged Markdown content: ${path}`);",
"}",
"if(!(await core.fileHandler.deleteRevisionFromDB(path,result.conflictedRev))){",
" throw new Error(`Could not delete conflicted revision: ${path}`);",
"}",
"if(!(await core.fileHandler.dbToStorage(path,path,true))){",
" throw new Error(`Could not reflect merged Markdown content: ${path}`);",
"}",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function runCreateUpdateDelete(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const createdContent = "# Created on A\n\nThis note should appear on B.\n";
let session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, createPath, createdContent);
await uploadNote(context, session, createPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
const createdOnB = await waitForPathContent(vaultB.path, createPath, (content) => content === createdContent);
await session.app.stop();
assertEqual(createdOnB, createdContent, "Created note did not round-trip to the second vault.");
const initialUpdateContent = "# Update target\n\nInitial content.\n";
const updatedContent = "# Update target\n\nUpdated content from A.\n";
session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, updatePath, initialUpdateContent);
await uploadNote(context, session, updatePath);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, updatePath, updatedContent);
await uploadNote(context, session, updatePath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
const updatedOnB = await waitForPathContent(vaultB.path, updatePath, (content) => content === updatedContent);
await session.app.stop();
assertEqual(updatedOnB, updatedContent, "Updated note content did not round-trip to the second vault.");
const deleteContent = "# Delete target\n\nThis note should be removed from B.\n";
session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, deletePath, deleteContent);
await uploadNote(context, session, deletePath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
await waitForPathContent(vaultB.path, deletePath, (content) => content === deleteContent);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await deleteNoteViaObsidian(context.cliBinary, session.cliEnv, deletePath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
await waitForPathDeleted(vaultB.path, deletePath);
await session.app.stop();
console.log("Two-vault note creation, update, and deletion round-tripped.");
}
async function runMarkdownAutoMerge(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const base = "# Conflict\n\nBase line\n\nShared tail\n";
const left = "# Conflict\n\nLeft line\n\nShared tail\n";
const right = "# Conflict\n\nBase line\n\nRight tail\n";
let session = await startConfiguredSession(context, vaultB);
await createMarkdownConflict(context, session, vaultB, conflictPath, base, left, right);
await autoMergeMarkdownConflict(context.cliBinary, session.cliEnv, conflictPath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
const mergedOnB = await waitForPathContent(
vaultB.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail")
);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await syncAndApply(context, session);
const mergedOnA = await waitForPathContent(
vaultA.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail")
);
await session.app.stop();
assertEqual(mergedOnA, mergedOnB, "Merged Markdown content was not consistent across both vaults.");
console.log("Markdown conflict was automatically merged and propagated by the next synchronisation.");
}
async function runTargetMismatch(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const ignoredContent = "# Target mismatch\n\nB should ignore this revision.\n";
const acceptedContent = "# Target mismatch\n\nB should accept this revision after its target filter changes.\n";
let session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, targetMismatchPath, ignoredContent);
await uploadNote(context, session, targetMismatchPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, {
syncOnlyRegEx: "^E2E/two-vault/allowed/.*",
});
await syncAndApply(context, session);
assertEqual(
await pathExists(vaultB.path, targetMismatchPath),
false,
"A note was reflected on a device where it was not a target file."
);
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, targetMismatchPath, acceptedContent);
await uploadNote(context, session, targetMismatchPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, {
syncOnlyRegEx: "",
});
await syncAndApply(context, session);
const received = await waitForPathContent(
vaultB.path,
targetMismatchPath,
(content) => content === acceptedContent
);
await session.app.stop();
assertEqual(received, acceptedContent, "Target file was not reflected after the device accepted the path.");
console.log("Two-vault target mismatch skipped a non-target note, then reflected it after enabling the target.");
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
await runCreateUpdateDelete(context, vaultA, vaultB);
await runMarkdownAutoMerge(context, vaultA, vaultB);
await runTargetMismatch(context, vaultA, vaultB);
} finally {
await vaultA.dispose();
await vaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -0,0 +1,122 @@
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
type CreatedNote = {
path: string;
read: string;
exists: boolean;
};
type ReadNote = {
exists: boolean;
read: string | null;
};
const notePath = "E2E/real-vault-reflection.md";
const noteContent = [
"# Real Obsidian E2E",
"",
"This note was created through Obsidian's own vault API.",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function waitForFileContent(vaultPath: string, path: string, expectedContent: string): Promise<void> {
const fullPath = join(vaultPath, path);
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000);
let lastError: unknown;
while (Date.now() < deadline) {
try {
const content = await readFile(fullPath, "utf-8");
if (content === expectedContent) {
return;
}
lastError = new Error(`Unexpected content in ${fullPath}`);
} catch (error) {
lastError = error;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for reflected vault file: ${fullPath}\nLast error: ${String(lastError)}`);
}
function assertEqual(actual: unknown, expected: unknown, message: string): void {
if (actual !== expected) {
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
}
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
const created = await evalObsidianJson<CreatedNote>(
cli.binary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
`const content=${JSON.stringify(noteContent)};`,
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"const file=await app.vault.create(path,content);",
"const read=await app.vault.read(file);",
"return JSON.stringify({path:file.path,read,exists:await app.vault.adapter.exists(path)});",
"})()",
].join(""),
session.cliEnv
);
assertEqual(created.path, notePath, "Obsidian created the note at an unexpected path.");
assertEqual(created.exists, true, "Obsidian adapter did not report the created note.");
assertEqual(created.read, noteContent, "Obsidian did not read back the created note content.");
await waitForFileContent(vault.path, notePath, noteContent);
const readBack = await evalObsidianJson<ReadNote>(
cli.binary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
"const file=app.vault.getAbstractFileByPath(path);",
"return JSON.stringify({exists:!!file,read:file?await app.vault.read(file):null});",
"})()",
].join(""),
session.cliEnv
);
assertEqual(readBack.exists, true, "Obsidian did not find the reflected note on read-back.");
assertEqual(readBack.read, noteContent, "Obsidian read-back content did not match the reflected file.");
console.log(`Created and verified reflected note: ${notePath}`);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});