(test): add local Obsidian E2E suite

This commit is contained in:
vorotamoroz
2026-06-30 08:36:29 +00:00
parent e916683b8d
commit 54e6a761e5
18 changed files with 975 additions and 119 deletions
+18 -6
View File
@@ -1,4 +1,5 @@
# Self-hosted LiveSync Development Guide
## Project Overview
Self-hosted LiveSync is an Obsidian plugin for synchronising vaults across devices using CouchDB, MinIO/S3, or peer-to-peer WebRTC. The codebase uses a modular architecture with TypeScript, Svelte, and PouchDB.
@@ -10,6 +11,7 @@ Self-hosted LiveSync is an Obsidian plugin for synchronising vaults across devic
#### First-time Setup
This repository uses submodules by convention. Therefore, you must use the `--recursive` flag when cloning it.
```bash
git clone --recursive https://github.com/vrtmrz/obsidian-livesync
npm ci
@@ -19,7 +21,9 @@ npm run build
Note: if you already cloned without submodules, run: `git submodule update --init --recursive`
#### Branch switching
When switching branches, please make sure to update submodules as well, since they may be updated in the new branch.
```bash
git checkout --recurse-submodules 0.25.70-patch1 # tag or branch name
npm ci
@@ -41,7 +45,7 @@ npm test # Run Harness based vitest tests (requires Docker services)
### Tips
We can use CLI's E2E test command instead of `npm test`.
Use CLI E2E tests or real Obsidian E2E scripts instead of `npm test` when the behaviour can be verified outside the browser harness.
### Auto-copy to test vaults
@@ -58,46 +62,54 @@ To facilitate development and testing, the build process can automatically copy
- **Unit Tests** (`vitest.config.unit.ts`): Unit tests run in Node.js (excluding harnesses and integration tests). Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`). Executed via `npm run test:unit`.
- **Integration Tests** (`vitest.config.integration.ts`): Tests run in Node.js against a real CouchDB instance. Integration tests should be `*.integration.spec.ts` or `*.integration.test.ts` and placed alongside the implementation file (e.g., `StreamingFetch.integration.spec.ts`). Executed via `npm run test:integration`.
- If you add a feature that interacts with the remote database (e.g., replication changes, custom changes feed parameters, or custom HTTP queries), you strongly expected to write an integration test to verify the behaviour against a real CouchDB server.
- **E2E Tests** (`vitest.config.ts`): End-to-end tests run in a browser-based harness using Playwright/Chromium to test full synchronisation scenarios. Executed via `npm run test`.
- **Browser Harness Tests** (`vitest.config.ts`): Transitional browser-based harness tests using Playwright/Chromium. Executed via `npm run test`. This layer is no longer the preferred destination for new broad E2E coverage because `test/harness` can drift from real Obsidian behaviour.
- **P2P Tests** (`vitest.config.p2p.ts`): Browser-based Peer-to-Peer replication tests. Executed via `npm run test:p2p`.
- **RPC Unit Tests** (`vitest.config.rpc-unit.ts`): RPC-specific unit tests with coverage thresholds.
- **Real Obsidian E2E** (`test/e2e-obsidian/`): Local-first scripts that launch real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in. Use these for boot-up sequence, vault reflection, RedFlag flows, Fast Setup (Simple Fetch), settings dialogues, restart-sensitive workflows, Object Storage regressions, and other behaviour that depends on Obsidian itself. Run focused scripts such as `npm run test:e2e:obsidian:two-vault-sync`, or use `npm run test:e2e:obsidian:local-suite:services` to run the broader local suite with CouchDB and MinIO fixtures managed by the wrapper.
- **Docker Services**: Tests require CouchDB, MinIO (S3), and P2P services:
```bash
npm run test:docker-all:start # Start all test services
npm run test:full # Run tests with coverage
npm run test:docker-all:stop # Stop services
```
If some services are not needed, start only required ones (e.g., `test:docker-couchdb:start`).
Note that if services are already running, starting script will fail. Please stop them first.
- **Test Structure**:
- `test/suite/` - E2E tests for sync operations (running in browser)
- `test/e2e-obsidian/` - Real Obsidian E2E scripts for local verification
- `test/suite/` - Transitional browser harness tests for sync operations
- `test/unit/` - Unit tests (via vitest, as harness is browser-based)
- `test/harness/` - Mock implementations (e.g., `obsidian-mock.ts`)
- `test/harness/` - Transitional mock implementations (e.g., `obsidian-mock.ts`). Avoid adding broad new E2E coverage here unless it is explicitly a compatibility bridge.
### Import Path Normalisation
The codebase uses `@/` and `@lib/` path aliases to keep import structures clean. To normalise imports and exports across files, use the following utility script:
```bash
npm run pretty:importpath
```
Under the hood, this runs Deno with the script [utilsdeno/normalise-imports.ts](file:///p:/plant25/obsidian/projects/obsidian-livesync/utilsdeno/normalise-imports.ts). You can pass additional flags to this script if required (by running it via Deno directly from the `utilsdeno` directory):
- `--run`: Applies the changes (the script runs in dry-run mode by default).
- `--all-alias`: Normalises sibling/child relative imports starting with `./` to use aliases.
### Type Generation
To generate fallback type definitions for the shared library and add appropriate Deno ignore comments (which suppresses Deno compilation warnings and linting warnings inside the `_types` directory), run:
```bash
npm run build:lib:types
```
This script executes:
1. TypeScript compilation (`tsconfig.types.json`) to output definitions to the `_types` directory.
2. The Deno script [utilsdeno/types-add-ignore.ts](file:///p:/plant25/obsidian/projects/obsidian-livesync/utilsdeno/types-add-ignore.ts) to prepend Deno ignore comments to the generated type files.
## Architecture
### Module System
+37 -17
View File
@@ -125,12 +125,12 @@ Initial discovery on Linux ARM64 found that:
Current implementation status:
- Added `test/e2e-obsidian/runner` helpers for Obsidian discovery, CLI discovery, temporary vault creation, plug-in installation, process launch, CLI execution, and readiness polling.
- Added `test:e2e:obsidian:discover`, `test:e2e:obsidian:cli-help`, `test:e2e:obsidian:smoke`, `test:e2e:obsidian:vault-reflection`, `test:e2e:obsidian:couchdb-upload`, `test:e2e:obsidian:startup-scan`, `test:e2e:obsidian:two-vault-sync`, `test:e2e:obsidian:hidden-file-snippet-sync`, `test:e2e:obsidian:customisation-sync`, `test:e2e:obsidian:setting-markdown-export`, and `test:e2e:obsidian:install-appimage`.
- Added `startObsidianLiveSyncSession()` so future workflows can reuse the launch, vault open, community plug-in enablement, plug-in reload, and readiness sequence without duplicating smoke runner code.
- Added `test:e2e:obsidian:discover`, `test:e2e:obsidian:cli-help`, `test:e2e:obsidian:smoke`, `test:e2e:obsidian:vault-reflection`, `test:e2e:obsidian:couchdb-upload`, `test:e2e:obsidian:minio-upload`, `test:e2e:obsidian:startup-scan`, `test:e2e:obsidian:two-vault-sync`, `test:e2e:obsidian:hidden-file-snippet-sync`, `test:e2e:obsidian:customisation-sync`, `test:e2e:obsidian:setting-markdown-export`, `test:e2e:obsidian:local-suite`, `test:e2e:obsidian:local-suite:services`, and `test:e2e:obsidian:install-appimage`.
- Added `startObsidianLiveSyncSession()` so future workflows can reuse the launch, trusted temporary vault state, vault open, community plug-in reload, and readiness sequence without duplicating smoke runner code.
- Added CouchDB runner utilities that reuse `.test.env`/process environment values, create unique temporary databases, query uploaded documents directly, and clean up the database unless `E2E_OBSIDIAN_KEEP_COUCHDB=true` is set.
- Added a manual AppImage installer that downloads Obsidian `1.12.7` for `arm64` or `x86_64`, stores it under `_testdata/obsidian`, and extracts it for FUSE-free execution.
- Confirmed the smoke runner on Linux ARM64 with the extracted Obsidian `1.12.7` AppImage, `xvfb-run`, and the built Self-hosted LiveSync bundle.
- Confirmed the runner can enable the Obsidian CLI through isolated `obsidian.json` state, open the temporary vault through `obsidian-cli`, enable community plug-ins through `app.plugins.setEnable(true)`, reload Self-hosted LiveSync, and verify readiness through `obsidian-cli eval`.
- Confirmed the runner can enable the Obsidian CLI through isolated `obsidian.json` state, pre-seed the temporary Chromium local storage so the generated vault ID is trusted for community plug-ins, open the temporary vault through `obsidian-cli`, reload Self-hosted LiveSync, and verify readiness through `obsidian-cli eval`.
- Removed the first test-only ready-marker bridge from the plug-in bundle. The current runner observes readiness from outside the plug-in through Obsidian's own CLI, so normal user vaults do not receive E2E marker files.
Current verification:
@@ -141,18 +141,22 @@ Current verification:
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS=1000 npm run test:e2e:obsidian:smoke` passes locally.
- `npm run test:e2e:obsidian:vault-reflection` creates a note through Obsidian's vault API, verifies the reflected file on disk, and reads it back through Obsidian.
- `npm run test:e2e:obsidian:couchdb-upload` configures a unique CouchDB database, creates a note through Obsidian, commits it into the local database, runs one-shot synchronisation, and verifies that CouchDB contains the metadata document and all referenced chunk documents.
- `npm run test:e2e:obsidian:minio-upload` configures a unique Object Storage bucket prefix, creates a note through Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written to the S3-compatible bucket.
- `npm run test:e2e:obsidian:startup-scan` verifies that a file written while Obsidian is stopped is picked up during the next real Obsidian boot and uploaded to CouchDB after one-shot synchronisation.
- `npm run test:e2e:obsidian:two-vault-sync` verifies two-vault note synchronisation: creation, update, deletion, Markdown conflict automatic merging with the merged result propagated by a second synchronisation, and per-device target-filter differences.
- `npm run test:e2e:obsidian:two-vault-sync` verifies two-vault note synchronisation: creation, update, rename, deletion, per-device target-filter differences, and a separate encrypted round-trip with Path Obfuscation enabled. The experimental Markdown conflict automatic merge check is available with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` but is not part of the default local suite.
- `npm run test:e2e:obsidian:hidden-file-snippet-sync` verifies hidden file synchronisation as a two-vault round-trip: creation, deletion, automatic JSON conflict merging with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target-pattern differences.
- `npm run test:e2e:obsidian:customisation-sync` verifies a two-vault Customisation Sync workflow: scan a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronise them through CouchDB, apply them on the second vault, assert the resulting `.obsidian` files, propagate a snippet update, and verify deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
- `npm run test:e2e:obsidian:setting-markdown-export` verifies that setting Markdown export creates a vault file and omits credentials when credential export is disabled.
- `npm run test:e2e:obsidian:install-appimage` reuses the existing AppImage and extracted binary when they are already present.
- `npm run test:e2e:obsidian:local-suite` runs the local verification sequence for the real Obsidian runner after CouchDB and MinIO have been started.
- `npm run test:e2e:obsidian:local-suite:services` stops leftover CouchDB and MinIO fixtures, starts fresh fixtures, runs the local suite, and stops the fixtures again.
- `npm run test:e2e:obsidian:local-suite:services` has been verified locally with real Obsidian, CouchDB, and MinIO. The run completed discovery, smoke, vault reflection, CouchDB upload, Object Storage upload, startup scan, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export. The build step still emits existing Svelte warnings.
Known limits:
- The smoke runner currently proves only one-vault launch and plug-in load readiness. Broader workflows are covered by separate real Obsidian scripts, including CouchDB upload, startup scan, two-vault note synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export.
- Cross-platform support is still discovery-level. The working path has been validated on Linux ARM64.
- CI wiring is not yet implemented. CI should use `OBSIDIAN_BINARY` or a cached `_testdata/obsidian/squashfs-root` rather than downloading the AppImage on every run.
- Cross-platform support is still discovery-level. The working path has been validated on Linux ARM64. macOS and Windows should be validated in their own environments as follow-up work.
- CI wiring is intentionally not implemented. The runner depends on a licensed desktop application and is treated as a local verification tool.
### Phase 2: First Real Workflow
@@ -169,6 +173,7 @@ Current implementation status:
- Added a pre-CouchDB workflow that creates a note through Obsidian's vault API, confirms the note is reflected as a real vault file, and reads the same note back through Obsidian. This covers the vault reflection part of the Phase 2 path before remote database setup is introduced.
- Added a first CouchDB-backed upload workflow, modelled after the CLI Deno tests: reuse the standard CouchDB environment variables, create a unique remote database, apply CouchDB settings through the plug-in's setting service, commit the note through the real Obsidian vault path, run one-shot synchronisation, and assert that remote metadata and chunks exist.
- Added an Object Storage-backed upload workflow against MinIO to exercise Journal Sync and the AWS SDK path from real Obsidian.
- Added Obsidian-specific workflows for boot-time vault scanning, two-vault note synchronisation, hidden `.obsidian/snippets` file round-tripping, hidden JSON conflict resolution, Customisation Sync application for snippets, config JSON files, and plug-in fixtures, per-device target-filter differences, and setting Markdown export. These scenarios assert against CouchDB documents, vault files, or real Obsidian UI outcomes instead of internal service state.
### Phase 3: Two-Vault Synchronisation
@@ -179,39 +184,53 @@ Current implementation status:
- Verify reflection in the other vault.
- Cover encrypted and non-encrypted configurations separately.
Current implementation status:
- `test:e2e:obsidian:two-vault-sync` covers creation, update, rename, deletion, and per-device target-filter behaviour for a non-encrypted CouchDB configuration. Markdown conflict automatic merging remains an optional check because it needs a dedicated, less timing-sensitive fixture.
- The same script creates a separate temporary CouchDB database and temporary vault pair for an encrypted two-vault round-trip with Path Obfuscation enabled.
### Phase 4: Harness Retirement
- Mark `test/harness` as deprecated in documentation.
- Stop adding new tests to `test/suite` unless they are explicitly transitional.
- Move critical existing scenarios from `test/suite` to real Obsidian E2E or lower-level integration tests.
- Do not mechanically port `test/suite` into real Obsidian E2E. Scenarios that can already be exercised and asserted through the CLI test layer should stay there or move to lower-level integration tests.
- Prioritise real Obsidian coverage for behaviours that the CLI cannot prove well, especially RedFlag flag-file recovery flows, Fast Setup (Simple Fetch), boot-up sequencing, restart-sensitive initial synchronisation, and user-visible recovery dialogues.
- Remove the harness only after the new runner covers the critical boot-up and synchronisation workflows.
## CI Strategy
Current implementation status:
Start with local-only execution. After the smoke runner is stable:
- `test/harness` is now documented as a transitional compatibility layer.
- New broad E2E work should target `test/e2e-obsidian/` when real Obsidian behaviour is the risk being tested.
- The next high-value scenarios are RedFlag variants and Fast Setup (Simple Fetch) variants, not a line-by-line migration of `test/suite`.
- Run the smoke test in CI on Linux.
- Keep full two-vault synchronisation scenarios as nightly or manually triggered jobs until runtime and flakiness are understood.
- Do not download the Obsidian AppImage on every CI run. Use a pre-installed Obsidian binary, a CI cache for `_testdata/obsidian/squashfs-root`, or a manually triggered preparation job.
- Capture Obsidian logs, plug-in logs, vault snapshots, and service logs on failure.
## Local Verification Strategy
Real Obsidian E2E is a local verification layer. It should not be wired into the default CI gate.
- Keep the scripts individually runnable for focused local debugging.
- Provide `test:e2e:obsidian:local-suite` for a broader local pass after the CouchDB and MinIO fixtures have been started.
- Provide `test:e2e:obsidian:local-suite:services` for a broader local pass that manages the CouchDB and MinIO fixtures itself.
- Use `OBSIDIAN_BINARY` when testing against an installed desktop application.
- Use `test:e2e:obsidian:install-appimage` on Linux when a local AppImage copy is needed, and reuse the extracted `_testdata/obsidian/squashfs-root` directory between local runs.
- Capture Obsidian logs, plug-in logs, vault snapshots, and service logs manually when investigating failures.
- Fail fast on launch failures, readiness timeouts, and cleanup failures with clear diagnostics.
## Risks and Mitigations
- **Obsidian licensing and installation**: CI may need a cached installer or a pre-installed binary. Keep the runner capable of using `OBSIDIAN_BINARY`.
- **Obsidian licensing and installation**: Keep the runner local-first and capable of using `OBSIDIAN_BINARY`.
- **Flakiness from UI timing**: Prefer a control channel and service-level probes over visual selectors.
- **Multiple instances**: Obsidian may not support multiple independent instances cleanly on all platforms. Start with one-instance smoke tests, then validate two-instance behaviour on Linux before expanding scope.
- **State leakage**: Isolate vault directories, Obsidian user data, remote database names, and bridge tokens per test.
- **Security of E2E controls**: Keep readiness and control outside the production plug-in bundle. Prefer Obsidian CLI probes over E2E-only plug-in code.
- **Runtime cost**: Keep the default PR gate small. Move slow synchronisation matrices to scheduled jobs.
- **Runtime cost**: Keep real Obsidian E2E out of the default PR gate. Use focused scripts or the local suite when a change touches real Obsidian integration.
## Open Questions
- Which launch mechanism is most reliable for Obsidian on Linux in this repository's CI environment?
- Which launch mechanism is most reliable for Obsidian on each supported desktop platform?
- Can two Obsidian instances run with isolated user data at the same time?
- Do future scenarios need a richer control channel than Obsidian CLI, or can CLI `eval` and developer commands cover the required workflows?
- Should any future E2E-only plug-in code live in a separate test build, or should the production bundle remain free of E2E controls?
- Which existing `test/suite` scenarios are critical enough to port before deprecating the harness?
- Which RedFlag and Fast Setup (Simple Fetch) variants should be added first?
## Initial Implementation Checklist
@@ -222,6 +241,7 @@ Start with local-only execution. After the smoke runner is stable:
5. Document required local environment variables, especially `OBSIDIAN_BINARY`.
6. Port one CouchDB-backed workflow after the smoke test is stable.
7. Mark `test/harness` as transitional and block new broad E2E work from targeting it.
8. Add the local suite script for broader local verification.
## Consequences
+4
View File
@@ -41,14 +41,18 @@
"test:e2e:obsidian:install-appimage": "tsx test/e2e-obsidian/scripts/install-appimage.ts",
"test:e2e:obsidian:discover": "tsx test/e2e-obsidian/scripts/discover.ts",
"test:e2e:obsidian:cli-help": "tsx test/e2e-obsidian/scripts/cli-help.ts",
"test:e2e:obsidian:debug-ui": "tsx test/e2e-obsidian/scripts/debug-ui.ts",
"test:e2e:obsidian:smoke": "tsx test/e2e-obsidian/scripts/smoke.ts",
"test:e2e:obsidian:vault-reflection": "tsx test/e2e-obsidian/scripts/vault-reflection.ts",
"test:e2e:obsidian:couchdb-upload": "tsx test/e2e-obsidian/scripts/couchdb-upload.ts",
"test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts",
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
"test:e2e:obsidian:setting-markdown-export": "tsx test/e2e-obsidian/scripts/setting-markdown-export.ts",
"test:e2e:obsidian:local-suite": "tsx test/e2e-obsidian/scripts/local-suite.ts",
"test:e2e:obsidian:local-suite:services": "tsx test/e2e-obsidian/scripts/local-suite.ts --manage-services",
"test:coverage": "vitest run --coverage",
"test:docker-couchdb:up": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-start.sh",
"test:docker-couchdb:init": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-init.sh",
+26 -6
View File
@@ -20,6 +20,8 @@ Obsidian 1.12 stores the global community plug-in switch outside `.obsidian/comm
Future workflows should use `startObsidianLiveSyncSession()` from `runner/session.ts` rather than repeating the launch and plug-in readiness sequence.
Each test vault uses an isolated Obsidian profile. The runner creates temporary directories for `HOME`, `XDG_CONFIG_HOME`, `XDG_CACHE_HOME`, `XDG_DATA_HOME`, and Electron `--user-data-dir`, writes the vault registry into those directories, pre-seeds the temporary Chromium local storage so community plug-ins are trusted for that generated vault ID, and passes the same environment to `obsidian-cli`. This is intended to keep real Obsidian E2E runs separate from a developer's daily Obsidian profile and vault registry.
## Local Setup
Set `OBSIDIAN_BINARY` when Obsidian is not installed in a standard location.
@@ -32,11 +34,11 @@ npm run test:e2e:obsidian:install-appimage
The script downloads Obsidian `1.12.7` for the current architecture, stores it in `_testdata/obsidian`, and extracts it to `_testdata/obsidian/squashfs-root`. The runner checks `_testdata/obsidian/squashfs-root/obsidian` before the AppImage path.
Do not download the AppImage on every CI run. Prefer one of these approaches:
These tests are intended for local verification, not the default CI gate. Reuse the installed Obsidian application, or reuse the extracted AppImage directory between local runs:
- set `OBSIDIAN_BINARY` to a pre-installed Obsidian executable,
- restore `_testdata/obsidian/squashfs-root` from a CI cache, or
- run `test:e2e:obsidian:install-appimage` only in a manually triggered preparation job.
- set `OBSIDIAN_BINARY` to an installed Obsidian executable,
- keep `_testdata/obsidian/squashfs-root` after running the AppImage installer, or
- run `test:e2e:obsidian:install-appimage` again only when the local Obsidian version should change.
## Commands
@@ -47,18 +49,25 @@ npm run test:e2e:obsidian:cli-help -- vaults verbose
npm run test:e2e:obsidian:smoke
npm run test:e2e:obsidian:vault-reflection
npm run test:e2e:obsidian:couchdb-upload
npm run test:e2e:obsidian:minio-upload
npm run test:e2e:obsidian:startup-scan
npm run test:e2e:obsidian:two-vault-sync
npm run test:e2e:obsidian:hidden-file-snippet-sync
npm run test:e2e:obsidian:customisation-sync
npm run test:e2e:obsidian:setting-markdown-export
npm run test:e2e:obsidian:local-suite
npm run test:e2e:obsidian:local-suite:services
```
`test:e2e:obsidian:local-suite` runs `npm run build`, discovery, smoke, vault reflection, CouchDB upload, Object Storage upload, startup scan, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export in sequence. Start the local CouchDB and MinIO fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run.
`test:e2e:obsidian:couchdb-upload` reuses the CouchDB variables from `.test.env` or the process environment. It expects a reachable CouchDB service, creates a unique database, configures Self-hosted LiveSync through `obsidian-cli eval`, creates a note in real Obsidian, commits the note into the local database, runs one-shot synchronisation, and verifies that the remote database contains both the metadata document and its chunk documents.
`test:e2e:obsidian:minio-upload` reuses the Object Storage variables from `.test.env` or the process environment. It expects a reachable S3-compatible service, configures Self-hosted LiveSync for Object Storage through `obsidian-cli eval`, creates a note in real Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written under a unique bucket prefix.
`test:e2e:obsidian:startup-scan` configures a temporary CouchDB database, stops Obsidian, writes a note directly into the vault, restarts Obsidian, and verifies from CouchDB that the boot-time scan picked up the offline file.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, deletion, Markdown conflict automatic merging with the merged result propagated by a second synchronisation, and per-device target filters where one vault ignores a note that the other vault synchronises.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, rename, deletion, per-device target filters where one vault ignores a note that the other vault synchronises, and a separate encrypted round-trip with Path Obfuscation enabled. The optional Markdown conflict automatic merge check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`, but it is not part of the default local suite.
`test:e2e:obsidian:hidden-file-snippet-sync` runs a two-vault hidden file round-trip. It verifies creation and deletion of a real `.obsidian/snippets/*.css` file, automatic JSON conflict merging for a hidden file with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target patterns where one vault ignores a hidden file that the other vault synchronises.
@@ -66,10 +75,18 @@ npm run test:e2e:obsidian:setting-markdown-export
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`.
Start the local CouchDB fixture first when one is not already running:
Start the local fixtures first when they are not already running:
```bash
npm run test:docker-couchdb:start
npm run test:docker-s3:start
npm run test:e2e:obsidian:local-suite
```
Or let the wrapper manage both fixtures:
```bash
npm run test:e2e:obsidian:local-suite:services
```
Useful environment variables:
@@ -89,10 +106,13 @@ Useful environment variables:
- `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready.
- `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database.
- `E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS`: timeout for waiting until CouchDB contains uploaded E2E documents.
- `E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS`: timeout for waiting until Object Storage contains uploaded E2E objects.
- `E2E_OBSIDIAN_KEEP_COUCHDB=true`: keep the temporary CouchDB database for inspection.
- `E2E_OBSIDIAN_KEEP_OBJECT_STORAGE=true`: keep the temporary Object Storage prefix for inspection.
- `E2E_OBSIDIAN_STARTUP_GRACE_MS`: early process-exit detection window in milliseconds.
- `E2E_OBSIDIAN_KEEP_VAULT=true`: keep the temporary vault for inspection.
- `E2E_OBSIDIAN_USE_XVFB=false`: disable automatic `xvfb-run` on headless Linux.
- `E2E_OBSIDIAN_USE_USER_DATA_DIR=false`: disable the isolated Electron `--user-data-dir` argument. This is not recommended for normal local testing.
- `E2E_OBSIDIAN_ARGS`: override the default Obsidian launch arguments.
On headless Linux, the runner automatically uses `/usr/bin/xvfb-run` when no `DISPLAY` or `WAYLAND_DISPLAY` is present.
+3 -2
View File
@@ -71,9 +71,10 @@ export async function openVaultWithObsidianCli(
export async function evalObsidianJson<T>(
cliBinary: string,
code: string,
env: NodeJS.ProcessEnv = process.env
env: NodeJS.ProcessEnv = process.env,
timeoutMs?: number
): Promise<T> {
const result = await runObsidianCli(cliBinary, ["eval", `code=${code}`], env);
const result = await runObsidianCli(cliBinary, ["eval", `code=${code}`], env, timeoutMs);
if (result.code !== 0) {
throw new Error(
[
+85 -8
View File
@@ -1,11 +1,13 @@
import { spawn, type ChildProcess } from "node:child_process";
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { once } from "node:events";
import { existsSync } from "node:fs";
import { dirname } from "node:path";
import { platform } from "node:process";
import { promisify } from "node:util";
export type ObsidianProcess = {
process: ChildProcess;
output: () => { stdout: string; stderr: string };
stop: () => Promise<void>;
};
@@ -14,10 +16,14 @@ export type LaunchObsidianOptions = {
vaultPath: string;
homePath?: string;
xdgConfigPath?: string;
xdgCachePath?: string;
xdgDataPath?: string;
userDataPath?: string;
startupGraceMs?: number;
};
const execFileAsync = promisify(execFile);
function splitArgs(args: string): string[] {
return args.split(" ").filter((arg) => arg.length > 0);
}
@@ -31,9 +37,13 @@ function launchArgs(options: LaunchObsidianOptions): string[] {
"--no-sandbox",
"--disable-gpu",
"--disable-software-rasterizer",
...(process.env.E2E_OBSIDIAN_USE_USER_DATA_DIR === "true" && options.userDataPath
...(process.env.E2E_OBSIDIAN_USE_USER_DATA_DIR !== "false" && options.userDataPath
? [`--user-data-dir=${options.userDataPath}`]
: []),
...(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT
? [`--remote-debugging-port=${process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT}`]
: []),
`obsidian://open?path=${encodeURIComponent(options.vaultPath)}`,
];
}
@@ -47,7 +57,63 @@ function shouldUseXvfb(): boolean {
return platform === "linux" && existsSync("/usr/bin/xvfb-run");
}
async function listChildPids(pid: number): Promise<number[]> {
if (platform === "win32") {
return [];
}
const { stdout } = await execFileAsync("ps", ["-o", "pid=", "--ppid", String(pid)]).catch(() => ({
stdout: "",
}));
const directChildren = stdout
.split("\n")
.map((line) => Number(line.trim()))
.filter((childPid) => Number.isInteger(childPid) && childPid > 0);
const descendants = await Promise.all(directChildren.map((childPid) => listChildPids(childPid)));
return [...directChildren, ...descendants.flat()];
}
async function killPids(pids: number[], signal: NodeJS.Signals): Promise<void> {
for (const pid of pids) {
if (pid === process.pid) {
continue;
}
try {
process.kill(pid, signal);
} catch {
// The process may have exited between discovery and signalling.
}
}
}
async function waitForExit(exitPromise: Promise<unknown>, timeoutMs: number): Promise<"exited" | "timeout"> {
const stopTimer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), timeoutMs);
});
const stopResult = await Promise.race([exitPromise.then(() => "exited" as const), stopTimer]);
return stopResult;
}
export async function cleanupStaleObsidianE2EProcesses(): Promise<void> {
if (process.env.E2E_OBSIDIAN_CLEANUP_STALE_PROCESSES === "false" || platform === "win32") {
return;
}
const { stdout } = await execFileAsync("pgrep", ["-f", "obsidian-livesync-e2e-state"]).catch(() => ({
stdout: "",
}));
const pids = stdout
.split("\n")
.map((line) => Number(line.trim()))
.filter((pid) => Number.isInteger(pid) && pid > 0 && pid !== process.pid);
if (pids.length === 0) {
return;
}
await killPids(pids, "SIGTERM");
await new Promise((resolve) => setTimeout(resolve, 1000));
await killPids(pids, "SIGKILL");
}
export async function launchObsidian(options: LaunchObsidianOptions): Promise<ObsidianProcess> {
await cleanupStaleObsidianE2EProcesses();
const startupGraceMs = options.startupGraceMs ?? 1000;
const args = launchArgs(options);
const useXvfb = shouldUseXvfb();
@@ -61,6 +127,8 @@ export async function launchObsidian(options: LaunchObsidianOptions): Promise<Ob
...process.env,
...(options.homePath ? { HOME: options.homePath } : {}),
...(options.xdgConfigPath ? { XDG_CONFIG_HOME: options.xdgConfigPath } : {}),
...(options.xdgCachePath ? { XDG_CACHE_HOME: options.xdgCachePath } : {}),
...(options.xdgDataPath ? { XDG_DATA_HOME: options.xdgDataPath } : {}),
OBSIDIAN_DISABLE_GPU: process.env.OBSIDIAN_DISABLE_GPU ?? "1",
},
});
@@ -93,25 +161,34 @@ export async function launchObsidian(options: LaunchObsidianOptions): Promise<Ob
return {
process: child,
output: () => ({ stdout, stderr }),
stop: async () => {
if (child.exitCode !== null || child.signalCode !== null) {
return;
}
const descendantPids = child.pid ? await listChildPids(child.pid) : [];
if (child.pid) {
process.kill(-child.pid, "SIGTERM");
try {
process.kill(-child.pid, "SIGTERM");
} catch {
child.kill("SIGTERM");
}
} else {
child.kill("SIGTERM");
}
const stopTimer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), 5000);
});
const stopResult = await Promise.race([exitPromise, stopTimer]);
await killPids(descendantPids.reverse(), "SIGTERM");
const stopResult = await waitForExit(exitPromise, 5000);
if (stopResult === "timeout") {
if (child.pid) {
process.kill(-child.pid, "SIGKILL");
try {
process.kill(-child.pid, "SIGKILL");
} catch {
child.kill("SIGKILL");
}
} else {
child.kill("SIGKILL");
}
await killPids(descendantPids, "SIGKILL");
await exitPromise;
}
},
+77 -12
View File
@@ -1,13 +1,18 @@
import { evalObsidianJson } from "./cli.ts";
import type { CouchDbConfig } from "./couchdb.ts";
import type { ObjectStorageConfig } from "./objectStorage.ts";
export type ConfiguredSettings = {
isConfigured: boolean;
liveSync: boolean;
syncOnStart: boolean;
syncOnSave: boolean;
remoteType: string;
couchDB_URI: string;
couchDB_DBNAME: string;
endpoint?: string;
bucket?: string;
bucketPrefix?: string;
};
export type CoreReadiness = {
@@ -23,6 +28,30 @@ export type LocalDatabaseEntry = {
children: string[];
};
function e2ePreferredSettingsSource(): string[] {
return [
"liveSync:false,",
"syncOnStart:false,",
"syncOnSave:false,",
"usePluginSync:false,",
"usePluginSyncV2:true,",
"useEden:false,",
"customChunkSize:60,",
"sendChunksBulk:false,",
"sendChunksBulkMaxSize:1,",
"chunkSplitterVersion:'v3-rabin-karp',",
"readChunksOnline:true,",
"disableCheckingConfigMismatch:false,",
"enableCompression:false,",
"hashAlg:'xxhash64',",
"handleFilenameCaseSensitive:false,",
"doNotUseFixedRevisionForChunks:true,",
"E2EEAlgorithm:'v2',",
"doctorProcessedVersion:'0.25.27',",
"isConfigured:true,",
];
}
export function assertEqual(actual: unknown, expected: unknown, message: string): void {
if (actual !== expected) {
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
@@ -47,18 +76,7 @@ export async function configureCouchDb(
`couchDB_PASSWORD:${JSON.stringify(settings.password)},`,
`couchDB_DBNAME:${JSON.stringify(settings.dbName)},`,
"remoteType:'',",
"liveSync:false,",
"syncOnStart:false,",
"syncOnSave:false,",
"usePluginSync:false,",
"usePluginSyncV2:true,",
"useEden:false,",
"customChunkSize:1,",
"sendChunksBulkMaxSize:1,",
"chunkSplitterVersion:'v3-rabin-karp',",
"readChunksOnline:false,",
"disableCheckingConfigMismatch:true,",
"isConfigured:true,",
...e2ePreferredSettingsSource(),
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
"};",
"await core.services.setting.applyExternalSettings(nextSettings,true);",
@@ -69,6 +87,7 @@ export async function configureCouchDb(
"liveSync:current.liveSync,",
"syncOnStart:current.syncOnStart,",
"syncOnSave:current.syncOnSave,",
"remoteType:current.remoteType,",
"couchDB_URI:current.couchDB_URI,",
"couchDB_DBNAME:current.couchDB_DBNAME,",
"});",
@@ -78,6 +97,52 @@ export async function configureCouchDb(
);
}
export async function configureObjectStorage(
cliBinary: string,
env: NodeJS.ProcessEnv,
settings: ObjectStorageConfig & { bucketPrefix: string },
overrides: Record<string, unknown> = {}
): Promise<ConfiguredSettings> {
return await evalObsidianJson<ConfiguredSettings>(
cliBinary,
[
"(async()=>{",
"const plugin=app.plugins.plugins['obsidian-livesync'];",
"const core=plugin.core;",
"const nextSettings={",
"remoteType:'MINIO',",
`endpoint:${JSON.stringify(settings.endpoint)},`,
`accessKey:${JSON.stringify(settings.accessKey)},`,
`secretKey:${JSON.stringify(settings.secretKey)},`,
`bucket:${JSON.stringify(settings.bucket)},`,
`region:${JSON.stringify(settings.region)},`,
`forcePathStyle:${JSON.stringify(settings.forcePathStyle)},`,
`bucketPrefix:${JSON.stringify(settings.bucketPrefix)},`,
"bucketCustomHeaders:'',",
...e2ePreferredSettingsSource(),
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
"};",
"await core.services.setting.applyExternalSettings(nextSettings,true);",
"await core.services.control.applySettings();",
"const current=core.services.setting.currentSettings();",
"return JSON.stringify({",
"isConfigured:current.isConfigured,",
"liveSync:current.liveSync,",
"syncOnStart:current.syncOnStart,",
"syncOnSave:current.syncOnSave,",
"remoteType:current.remoteType,",
"couchDB_URI:current.couchDB_URI,",
"couchDB_DBNAME:current.couchDB_DBNAME,",
"endpoint:current.endpoint,",
"bucket:current.bucket,",
"bucketPrefix:current.bucketPrefix,",
"});",
"})()",
].join(""),
env
);
}
export async function waitForLiveSyncCoreReady(
cliBinary: string,
env: NodeJS.ProcessEnv,
+142
View File
@@ -0,0 +1,142 @@
import {
CreateBucketCommand,
DeleteObjectsCommand,
ListObjectsV2Command,
S3Client,
type _Object,
} from "@aws-sdk/client-s3";
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
export type ObjectStorageConfig = {
endpoint: string;
accessKey: string;
secretKey: string;
bucket: string;
region: string;
forcePathStyle: boolean;
};
function parseEnvFile(content: string): Record<string, string> {
const entries = content
.split(/\r?\n/u)
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("#"))
.map((line) => {
const equalsAt = line.indexOf("=");
if (equalsAt < 0) {
return undefined;
}
const key = line.slice(0, equalsAt).trim();
const rawValue = line.slice(equalsAt + 1).trim();
const value = rawValue.replace(/^['"]|['"]$/gu, "");
return [key, value] as const;
})
.filter((entry): entry is readonly [string, string] => entry !== undefined);
return Object.fromEntries(entries);
}
function getEnvValue(values: Record<string, string | undefined>, ...keys: string[]): string {
for (const key of keys) {
const value = values[key]?.trim();
if (value) {
return value;
}
}
throw new Error(`Required Object Storage environment value is missing: ${keys.join(" or ")}`);
}
export async function loadObjectStorageConfig(envFile = ".test.env"): Promise<ObjectStorageConfig> {
let fileValues: Record<string, string> = {};
try {
fileValues = parseEnvFile(await readFile(resolve(envFile), "utf-8"));
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
throw error;
}
}
const values = { ...fileValues, ...process.env };
return {
endpoint: getEnvValue(values, "MINIO_ENDPOINT", "minioEndpoint").replace(/\/+$/u, ""),
accessKey: getEnvValue(values, "MINIO_ACCESS_KEY", "accessKey"),
secretKey: getEnvValue(values, "MINIO_SECRET_KEY", "secretKey"),
bucket: getEnvValue(values, "MINIO_BUCKET", "bucketName"),
region: values.MINIO_REGION?.trim() || values.region?.trim() || "us-east-1",
forcePathStyle: values.MINIO_FORCE_PATH_STYLE?.trim() !== "false",
};
}
export function makeUniqueBucketPrefix(label: string): string {
const random = Math.random().toString(36).slice(2, 8);
return `obsidian-e2e/${label}-${Date.now()}-${random}/`;
}
export function createObjectStorageClient(config: ObjectStorageConfig): S3Client {
return new S3Client({
endpoint: config.endpoint,
region: config.region,
forcePathStyle: config.forcePathStyle,
credentials: {
accessKeyId: config.accessKey,
secretAccessKey: config.secretKey,
},
});
}
export async function ensureObjectStorageBucket(config: ObjectStorageConfig): Promise<void> {
const client = createObjectStorageClient(config);
try {
await client.send(new CreateBucketCommand({ Bucket: config.bucket }));
} catch (error) {
const name = (error as { name?: string }).name;
if (name !== "BucketAlreadyOwnedByYou" && name !== "BucketAlreadyExists") {
throw error;
}
} finally {
client.destroy();
}
}
export async function listObjectStorageObjects(config: ObjectStorageConfig, prefix: string): Promise<_Object[]> {
const client = createObjectStorageClient(config);
try {
const objects: _Object[] = [];
let continuationToken: string | undefined;
do {
const response = await client.send(
new ListObjectsV2Command({
Bucket: config.bucket,
Prefix: prefix,
ContinuationToken: continuationToken,
})
);
objects.push(...(response.Contents ?? []));
continuationToken = response.NextContinuationToken;
} while (continuationToken);
return objects;
} finally {
client.destroy();
}
}
export async function deleteObjectStoragePrefix(config: ObjectStorageConfig, prefix: string): Promise<void> {
const client = createObjectStorageClient(config);
try {
const objects = await listObjectStorageObjects(config, prefix);
const keys = objects.flatMap((object) => (object.Key ? [{ Key: object.Key }] : []));
for (let index = 0; index < keys.length; index += 1000) {
await client.send(
new DeleteObjectsCommand({
Bucket: config.bucket,
Delete: {
Objects: keys.slice(index, index + 1000),
Quiet: true,
},
})
);
}
} finally {
client.destroy();
}
}
+32 -7
View File
@@ -1,8 +1,9 @@
import { openVaultWithObsidianCli, runObsidianCli } from "./cli.ts";
import { evalObsidianJson, openVaultWithObsidianCli, runObsidianCli } from "./cli.ts";
import { launchObsidian, type ObsidianProcess } from "./launch.ts";
import { installBuiltPlugin, type PluginInstallResult } from "./pluginInstaller.ts";
import { waitForPluginReady, type PluginReadiness } from "./readiness.ts";
import type { TemporaryVault } from "./vault.ts";
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, trustVaultIfPrompted } from "./ui.ts";
export type ObsidianLiveSyncSession = {
app: ObsidianProcess;
@@ -19,13 +20,21 @@ export type StartObsidianLiveSyncSessionOptions = {
};
async function waitForPluginCatalogue(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS ?? 15000);
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS ?? 60000);
let lastOutput = "";
while (Date.now() < deadline) {
const result = await runObsidianCli(cliBinary, ["plugins", "filter=community"], env);
lastOutput = [result.stdout, result.stderr].filter(Boolean).join("\n");
if (result.stdout.includes("obsidian-livesync")) {
return;
try {
const result = await evalObsidianJson<{ hasLiveSync: boolean }>(
cliBinary,
["JSON.stringify({", "hasLiveSync:!!app.plugins?.manifests?.['obsidian-livesync']", "})"].join(""),
env
);
lastOutput = JSON.stringify(result);
if (result.hasLiveSync) {
return;
}
} catch (error) {
lastOutput = error instanceof Error ? error.message : String(error);
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
@@ -66,11 +75,14 @@ export async function startObsidianLiveSyncSession(
options: StartObsidianLiveSyncSessionOptions
): Promise<ObsidianLiveSyncSession> {
const install = await installBuiltPlugin(options.vault.path);
const remoteDebuggingPort = obsidianRemoteDebuggingPort();
const app = await launchObsidian({
binary: options.binary,
vaultPath: options.vault.path,
homePath: options.vault.homePath,
xdgConfigPath: options.vault.xdgConfigPath,
xdgCachePath: options.vault.xdgCachePath,
xdgDataPath: options.vault.xdgDataPath,
userDataPath: options.vault.userDataPath,
startupGraceMs: options.startupGraceMs,
});
@@ -78,17 +90,30 @@ export async function startObsidianLiveSyncSession(
...process.env,
HOME: options.vault.homePath,
XDG_CONFIG_HOME: options.vault.xdgConfigPath,
XDG_CACHE_HOME: options.vault.xdgCachePath,
XDG_DATA_HOME: options.vault.xdgDataPath,
};
try {
await preseedTrustedVaultState(remoteDebuggingPort, options.vault.id);
await openVaultWithObsidianCli(options.cliBinary, options.vault.path, cliEnv);
await trustVaultIfPrompted(remoteDebuggingPort);
await waitForPluginCatalogue(options.cliBinary, cliEnv);
await enableCommunityPlugins(options.cliBinary, cliEnv);
await reloadLiveSyncPlugin(options.cliBinary, cliEnv);
const readiness = await waitForPluginReady(options.cliBinary, cliEnv);
return { app, cliEnv, install, readiness };
} catch (error) {
const output = app.output();
await app.stop();
throw error;
throw new Error(
[
error instanceof Error ? error.message : String(error),
output.stdout ? `Obsidian stdout:\n${output.stdout}` : undefined,
output.stderr ? `Obsidian stderr:\n${output.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
+83
View File
@@ -0,0 +1,83 @@
import { chromium, type Page } from "playwright";
export function obsidianRemoteDebuggingPort(): number {
const port = Number(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT ?? 9222);
process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT = String(port);
return port;
}
async function waitForCdp(port: number): Promise<void> {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CDP_TIMEOUT_MS ?? 30000);
while (Date.now() < deadline) {
try {
const response = await fetch(`http://127.0.0.1:${port}/json/version`);
if (response.ok) {
return;
}
} catch {
// Keep polling until Obsidian exposes the debugging endpoint.
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Obsidian DevTools endpoint on port ${port}`);
}
export async function withObsidianPage<T>(port: number, operation: (page: Page) => Promise<T>): Promise<T> {
await waitForCdp(port);
const browser = await chromium.connectOverCDP(`http://127.0.0.1:${port}`);
try {
const context = browser.contexts()[0];
const page = context.pages()[0] ?? (await context.waitForEvent("page", { timeout: 10000 }));
return await operation(page);
} finally {
await browser.close();
}
}
export async function preseedTrustedVaultState(port: number, vaultId: string): Promise<void> {
await withObsidianPage(port, async (page) => {
await page.evaluate((id) => {
localStorage.setItem(`enable-plugin-${id}`, "true");
}, vaultId);
await page.reload({ waitUntil: "domcontentloaded", timeout: 10000 }).catch(() => undefined);
await page.waitForTimeout(1000);
});
}
export async function trustVaultIfPrompted(port: number): Promise<void> {
await withObsidianPage(port, async (page) => {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_TRUST_PROMPT_TIMEOUT_MS ?? 30000);
while (Date.now() < deadline) {
const yesButton = page.getByRole("button", { name: "Yes" });
if (await yesButton.isVisible({ timeout: 1000 }).catch(() => false)) {
await yesButton.click();
await page.waitForTimeout(500);
continue;
}
const trustButton = page.getByText("Trust author and enable plugins");
if (await trustButton.isVisible({ timeout: 1000 }).catch(() => false)) {
await trustButton.click();
await page.waitForTimeout(500);
continue;
}
const workspace = page.locator(".workspace");
if (await workspace.isVisible({ timeout: 1000 }).catch(() => false)) {
return;
}
}
});
}
export async function clickJsonResolveOption(port: number, mode: "AB" | "BA"): Promise<void> {
await withObsidianPage(port, async (page) => {
const option = page.locator(`label:has(input[name="disp"][value="${mode}"])`);
await option.click({ timeout: 10000 });
const checked = await page.locator(`input[name="disp"][value="${mode}"]`).isChecked({ timeout: 10000 });
if (!checked) {
throw new Error(`JSON Resolve option was not selected: ${mode}`);
}
await page.getByRole("button", { name: "Apply" }).click({ timeout: 10000 });
});
}
+25 -8
View File
@@ -5,8 +5,11 @@ import { tmpdir } from "node:os";
export type TemporaryVault = {
path: string;
name: string;
id: string;
homePath: string;
xdgConfigPath: string;
xdgCachePath: string;
xdgDataPath: string;
userDataPath: string;
dispose: () => Promise<void>;
};
@@ -18,21 +21,33 @@ export async function createTemporaryVault(prefix = "obsidian-livesync-e2e-"): P
await mkdir(join(vaultPath, ".obsidian"), { recursive: true });
const homePath = join(statePath, "home");
const xdgConfigPath = join(statePath, "xdg-config");
const xdgCachePath = join(statePath, "xdg-cache");
const xdgDataPath = join(statePath, "xdg-data");
const userDataPath = join(statePath, "user-data");
const id = `livesync-e2e-${Date.now()}`;
await mkdir(homePath, { recursive: true });
await mkdir(xdgConfigPath, { recursive: true });
await mkdir(xdgCachePath, { recursive: true });
await mkdir(xdgDataPath, { recursive: true });
await mkdir(userDataPath, { recursive: true });
await writeFile(
join(vaultPath, ".obsidian", "app.json"),
JSON.stringify({ legacyEditor: false, safeMode: false }, null, 4)
);
await writeObsidianVaultRegistry(vaultPath, name, homePath, xdgConfigPath, userDataPath);
await writeFile(
join(vaultPath, ".obsidian", "community-plugins.json"),
JSON.stringify(["obsidian-livesync"], null, 4)
);
await writeObsidianVaultRegistry(id, vaultPath, name, homePath, xdgConfigPath, userDataPath);
return {
path: vaultPath,
name,
id,
homePath,
xdgConfigPath,
xdgCachePath,
xdgDataPath,
userDataPath,
dispose: async () => {
if (process.env.E2E_OBSIDIAN_KEEP_VAULT === "true") {
@@ -49,22 +64,23 @@ export async function createTemporaryVault(prefix = "obsidian-livesync-e2e-"): P
}
async function writeObsidianVaultRegistry(
vaultId: string,
vaultPath: string,
vaultName: string,
homePath: string,
xdgConfigPath: string,
userDataPath: string
): Promise<void> {
const vaultId = `livesync-e2e-${Date.now()}`;
const vaultRecord = {
path: vaultPath,
ts: Date.now(),
open: true,
name: vaultName,
};
const registry = {
cli: true,
vaults: {
[vaultId]: {
path: vaultPath,
ts: Date.now(),
open: true,
name: vaultName,
},
[vaultId]: vaultRecord,
},
};
const registryText = JSON.stringify(registry, null, 4);
@@ -74,4 +90,5 @@ async function writeObsidianVaultRegistry(
await writeFile(join(obsidianConfigDir, "obsidian.json"), registryText);
}
await writeFile(join(userDataPath, "obsidian.json"), registryText);
await writeFile(join(userDataPath, `${vaultId}.json`), JSON.stringify(vaultRecord, null, 4));
}
+2
View File
@@ -25,6 +25,8 @@ async function main(): Promise<void> {
...process.env,
HOME: vault.homePath,
XDG_CONFIG_HOME: vault.xdgConfigPath,
XDG_CACHE_HOME: vault.xdgCachePath,
XDG_DATA_HOME: vault.xdgDataPath,
};
await runObsidianCli(cli.binary, [`obsidian://open?path=${encodeURIComponent(vault.path)}`], cliEnv);
await new Promise((resolve) => setTimeout(resolve, 3000));
@@ -228,10 +228,12 @@ async function storeCustomisationFile(cliBinary: string, env: NodeJS.ProcessEnv,
"const result=await addOn.storeCustomizationFiles(path,term);",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>doc.path);",
"if(!result){",
"const filename=path.split('/').pop();",
"const existing=entries.some((entry)=>entry.startsWith(`ix:${term}/${category}/`)&&entry.endsWith(`%${filename}`));",
"if(!result&&!existing){",
" throw new Error(`Could not store Customisation Sync file: path=${path}; term=${term}; category=${category}; stat=${JSON.stringify(stat)}; result=${JSON.stringify(result)}; entries=${JSON.stringify(entries)}`);",
"}",
"return JSON.stringify({ok:true,path,term,category,entries});",
"return JSON.stringify({ok:true,path,term,category,result:!!result,existing,entries});",
"})()",
].join(""),
env
+60
View File
@@ -0,0 +1,60 @@
import { launchObsidian } from "../runner/launch.ts";
import { installBuiltPlugin } from "../runner/pluginInstaller.ts";
import { createTemporaryVault } from "../runner/vault.ts";
import { requireObsidianBinary } from "../runner/environment.ts";
import { writeFile } from "node:fs/promises";
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, withObsidianPage } from "../runner/ui.ts";
const port = obsidianRemoteDebuggingPort();
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const vault = await createTemporaryVault();
await installBuiltPlugin(vault.path);
const app = await launchObsidian({
binary,
vaultPath: vault.path,
homePath: vault.homePath,
xdgConfigPath: vault.xdgConfigPath,
xdgCachePath: vault.xdgCachePath,
xdgDataPath: vault.xdgDataPath,
userDataPath: vault.userDataPath,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
try {
await preseedTrustedVaultState(port, vault.id);
const { screenshotPath, textPath } = await withObsidianPage(port, async (page) => {
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_WAIT_MS ?? 5000));
const title = await page.title().catch((error: unknown) => `title error: ${String(error)}`);
const url = page.url();
const text = await page
.locator("body")
.innerText({ timeout: 5000 })
.catch((error: unknown) => {
return `body text error: ${String(error)}`;
});
if (process.env.E2E_OBSIDIAN_DEBUG_CLICK_TRUST === "true") {
await page.getByText("Trust author and enable plugins").click({ timeout: 10000 });
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_AFTER_CLICK_WAIT_MS ?? 3000));
}
const screenshotPath = process.env.E2E_OBSIDIAN_DEBUG_SCREENSHOT ?? "/tmp/obsidian-e2e-debug.png";
const textPath = process.env.E2E_OBSIDIAN_DEBUG_TEXT ?? "/tmp/obsidian-e2e-debug.txt";
await page.screenshot({ path: screenshotPath, fullPage: true });
await writeFile(textPath, [`title: ${title}`, `url: ${url}`, "", text].join("\n"), "utf-8");
return { screenshotPath, textPath };
});
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary Obsidian state: ${vault.userDataPath}`);
console.log(`Debug text: ${textPath}`);
console.log(`Debug screenshot: ${screenshotPath}`);
} finally {
await app.stop();
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -21,6 +21,7 @@ import {
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { clickJsonResolveOption, obsidianRemoteDebuggingPort } from "../runner/ui.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
@@ -41,6 +42,7 @@ const snippetContent = [
const mergeJsonPath = ".obsidian/livesync-e2e-merge.json";
const manualMergeJsonPath = ".obsidian/livesync-e2e-manual-merge.json";
const targetPath = ".obsidian/livesync-targeted/only-a.json";
const hiddenFileCliTimeoutMs = Number(process.env.E2E_OBSIDIAN_HIDDEN_FILE_CLI_TIMEOUT_MS ?? 90000);
type RunnerContext = {
binary: string;
@@ -145,7 +147,8 @@ async function scanHiddenDatabase(cliBinary: string, env: NodeJS.ProcessEnv): Pr
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
env,
hiddenFileCliTimeoutMs
);
}
@@ -161,7 +164,8 @@ async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv)
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
env,
hiddenFileCliTimeoutMs
);
}
@@ -233,34 +237,6 @@ async function openHiddenJsonResolveModal(cliBinary: string, env: NodeJS.Process
);
}
async function clickJsonResolveOption(cliBinary: string, env: NodeJS.ProcessEnv, mode: "AB" | "BA"): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const mode=${JSON.stringify(mode)};`,
"const deadline=Date.now()+10000;",
"while(Date.now()<deadline){",
" const input=[...document.querySelectorAll('input[name=\"disp\"]')].find((candidate)=>candidate.value===mode);",
" const apply=[...document.querySelectorAll('button')].find((button)=>button.textContent?.trim()==='Apply');",
" if(input&&apply){",
" input.click();",
" input.dispatchEvent(new Event('change',{bubbles:true}));",
" await new Promise((resolve)=>setTimeout(resolve,100));",
" apply.click();",
" return JSON.stringify({ok:true});",
" }",
" await new Promise((resolve)=>setTimeout(resolve,250));",
"}",
"const buttons=[...document.querySelectorAll('button')].map((button)=>button.textContent?.trim()).filter(Boolean);",
"const inputs=[...document.querySelectorAll('input[name=\"disp\"]')].map((input)=>input.value);",
"throw new Error(`Timed out waiting for JSON resolve modal; buttons=${JSON.stringify(buttons)}; inputs=${JSON.stringify(inputs)}`);",
"})()",
].join(""),
env
);
}
async function storeHiddenFileAsConflict(
cliBinary: string,
env: NodeJS.ProcessEnv,
@@ -368,9 +344,15 @@ async function uploadHiddenFile(
return entry;
}
async function pullAndApplyHiddenFiles(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
async function pullAndApplyHiddenFiles(
context: RunnerContext,
session: ObsidianLiveSyncSession,
options: { resolveConflicts?: boolean } = {}
): Promise<void> {
await pushLocalChanges(context.cliBinary, session.cliEnv);
await resolveHiddenConflicts(context.cliBinary, session.cliEnv);
if (options.resolveConflicts === true) {
await resolveHiddenConflicts(context.cliBinary, session.cliEnv);
}
await scanHiddenDatabase(context.cliBinary, session.cliEnv);
}
@@ -449,7 +431,7 @@ async function runJsonManualConflictResolution(context: RunnerContext, vault: Te
const session = await startConfiguredSession(context, vault);
await createHiddenJsonConflict(context, session, vault, manualMergeJsonPath, base, left, right);
await openHiddenJsonResolveModal(context.cliBinary, session.cliEnv, manualMergeJsonPath);
await clickJsonResolveOption(context.cliBinary, session.cliEnv, "AB");
await clickJsonResolveOption(obsidianRemoteDebuggingPort(), "AB");
const merged = await waitForPathContent(vault.path, manualMergeJsonPath, (content) =>
hasJsonValues(content, { shared: "right", fromA: true, fromB: true })
@@ -472,26 +454,36 @@ async function runTargetMismatch(
await writeVaultFile(vaultA.path, targetPath, targetContent);
let session = await startConfiguredSession(context, vaultA);
await uploadHiddenFile(context, session, targetPath);
await session.app.stop();
try {
await uploadHiddenFile(context, session, targetPath);
} finally {
await session.app.stop();
}
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "snippets",
});
await pullAndApplyHiddenFiles(context, session);
assertEqual(
await pathExists(vaultB.path, targetPath),
false,
"Hidden file was applied on a device where it was not a target file."
);
await session.app.stop();
try {
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
assertEqual(
await pathExists(vaultB.path, targetPath),
false,
"Hidden file was applied on a device where it was not a target file."
);
} finally {
await session.app.stop();
}
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "",
});
await pullAndApplyHiddenFiles(context, session);
const received = await waitForPathContent(vaultB.path, targetPath, (content) => content === targetContent);
await session.app.stop();
let received = "";
try {
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
received = await waitForPathContent(vaultB.path, targetPath, (content) => content === targetContent);
} finally {
await session.app.stop();
}
assertEqual(received, targetContent, "Hidden file was not applied after it became a target file.");
console.log("Hidden target mismatch respected per-device target patterns, then applied after enabling the target.");
+105
View File
@@ -0,0 +1,105 @@
import { spawn } from "node:child_process";
type Step = {
name: string;
args: string[];
optional?: boolean;
};
const testSteps: Step[] = [
{ name: "build", args: ["run", "build"] },
{ name: "discover", args: ["run", "test:e2e:obsidian:discover"] },
{ name: "smoke", args: ["run", "test:e2e:obsidian:smoke"] },
{ name: "vault reflection", args: ["run", "test:e2e:obsidian:vault-reflection"] },
{ name: "CouchDB upload", args: ["run", "test:e2e:obsidian:couchdb-upload"] },
{ name: "Object Storage upload", args: ["run", "test:e2e:obsidian:minio-upload"] },
{ name: "startup scan", args: ["run", "test:e2e:obsidian:startup-scan"] },
{ name: "two-vault synchronisation", args: ["run", "test:e2e:obsidian:two-vault-sync"] },
{ name: "hidden file snippet synchronisation", args: ["run", "test:e2e:obsidian:hidden-file-snippet-sync"] },
{ name: "Customisation Sync", args: ["run", "test:e2e:obsidian:customisation-sync"] },
{ name: "setting Markdown export", args: ["run", "test:e2e:obsidian:setting-markdown-export"] },
];
const manageCouchDb = process.argv.includes("--manage-couchdb") || process.argv.includes("--manage-services");
const manageMinio = process.argv.includes("--manage-minio") || process.argv.includes("--manage-services");
const keepServices = process.argv.includes("--keep-services");
const keepCouchDb = keepServices || process.argv.includes("--keep-couchdb");
const keepMinio = keepServices || process.argv.includes("--keep-minio");
function npmBinary(): string {
return process.platform === "win32" ? "npm.cmd" : "npm";
}
function runStep(step: Step): Promise<void> {
return new Promise((resolve, reject) => {
console.log(`\n# ${step.name}`);
const child = spawn(npmBinary(), step.args, {
cwd: process.cwd(),
env: process.env,
stdio: "inherit",
});
child.on("error", reject);
child.on("exit", (code, signal) => {
if (code === 0) {
resolve();
return;
}
const message = `${step.name} failed with ${signal ? `signal ${signal}` : `exit code ${code}`}`;
if (step.optional) {
console.warn(message);
resolve();
return;
}
reject(new Error(message));
});
});
}
async function stopManagedCouchDb(): Promise<void> {
await runStep({
name: "stop CouchDB fixture",
args: ["run", "test:docker-couchdb:stop"],
optional: true,
});
}
async function stopManagedMinio(): Promise<void> {
await runStep({
name: "stop MinIO fixture",
args: ["run", "test:docker-s3:stop"],
optional: true,
});
}
async function main(): Promise<void> {
let shouldStopCouchDb = false;
let shouldStopMinio = false;
try {
if (manageCouchDb) {
await stopManagedCouchDb();
await runStep({ name: "start CouchDB fixture", args: ["run", "test:docker-couchdb:start"] });
shouldStopCouchDb = !keepCouchDb;
}
if (manageMinio) {
await stopManagedMinio();
await runStep({ name: "start MinIO fixture", args: ["run", "test:docker-s3:start"] });
shouldStopMinio = !keepMinio;
}
for (const step of testSteps) {
await runStep(step);
}
} finally {
if (shouldStopMinio) {
await stopManagedMinio();
}
if (shouldStopCouchDb) {
await stopManagedCouchDb();
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+142
View File
@@ -0,0 +1,142 @@
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureObjectStorage,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import {
deleteObjectStoragePrefix,
ensureObjectStorageBucket,
listObjectStorageObjects,
loadObjectStorageConfig,
makeUniqueBucketPrefix,
} from "../runner/objectStorage.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/minio-upload.md";
const noteContent = [
"# Object Storage upload from real Obsidian",
"",
"This note is created through Obsidian and uploaded by Self-hosted LiveSync to S3-compatible Object Storage.",
"The test is intentionally small, but it crosses the real Obsidian, Journal Sync, and AWS SDK boundary.",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.ProcessEnv): Promise<LocalDatabaseEntry> {
return await evalObsidianJson<LocalDatabaseEntry>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
`const content=${JSON.stringify(noteContent)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"await app.vault.create(path,content);",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"for(let i=0;i<40;i++){",
"await core.services.fileProcessing.commitPendingFileEvents();",
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true).catch(()=>false);",
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
"await sleep(250);",
"}",
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
"})()",
].join(""),
env
);
}
async function waitForObjectStorageObjects(prefix: string): Promise<string[]> {
const objectStorage = await loadObjectStorageConfig();
const timeoutMs = Number(process.env.E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS ?? 20000);
const deadline = Date.now() + timeoutMs;
let keys: string[] = [];
while (Date.now() < deadline) {
const objects = await listObjectStorageObjects(objectStorage, prefix);
keys = objects.flatMap((object) => (object.Key ? [object.Key] : []));
if (keys.length > 0) {
return keys;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Object Storage objects under ${prefix}. Last keys: ${keys.join(", ")}`);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const objectStorage = await loadObjectStorageConfig();
const bucketPrefix = makeUniqueBucketPrefix("minio-upload");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await ensureObjectStorageBucket(objectStorage);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary Object Storage bucket: ${objectStorage.bucket}`);
console.log(`Temporary Object Storage prefix: ${bucketPrefix}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureObjectStorage(cli.binary, session.cliEnv, {
...objectStorage,
bucketPrefix,
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
assertEqual(configured.remoteType, "MINIO", "Remote type was not Object Storage.");
assertEqual(configured.endpoint, objectStorage.endpoint, "Configured Object Storage endpoint did not match.");
assertEqual(configured.bucket, objectStorage.bucket, "Configured Object Storage bucket did not match.");
assertEqual(configured.bucketPrefix, bucketPrefix, "Configured Object Storage bucket prefix did not match.");
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during this one-shot workflow.");
await prepareRemote(cli.binary, session.cliEnv);
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
await pushLocalChanges(cli.binary, session.cliEnv);
const keys = await waitForObjectStorageObjects(bucketPrefix);
console.log(
`Uploaded ${localEntry.path} through Journal Sync to ${objectStorage.bucket}/${bucketPrefix} (${keys.length} object(s))`
);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_OBJECT_STORAGE !== "true") {
await deleteObjectStoragePrefix(objectStorage, bucketPrefix).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+93 -6
View File
@@ -29,8 +29,11 @@ process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const createPath = "E2E/two-vault/create.md";
const updatePath = "E2E/two-vault/update.md";
const deletePath = "E2E/two-vault/delete.md";
const renameFromPath = "E2E/two-vault/rename-source.md";
const renameToPath = "E2E/two-vault/renamed/rename-target.md";
const conflictPath = "E2E/two-vault/conflict.md";
const targetMismatchPath = "E2E/two-vault/target-mismatch.md";
const encryptedPath = "E2E/two-vault/encrypted.md";
type RunnerContext = {
binary: string;
@@ -134,6 +137,25 @@ async function deleteNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv,
);
}
async function renameNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, fromPath: string, toPath: string) {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const fromPath=${JSON.stringify(fromPath)};`,
`const toPath=${JSON.stringify(toPath)};`,
"const folder=toPath.split('/').slice(0,-1).join('/');",
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
"const existing=app.vault.getAbstractFileByPath(fromPath);",
"if(!existing) throw new Error(`Could not find note to rename: ${fromPath}`);",
"await app.vault.rename(existing,toPath);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
@@ -319,14 +341,62 @@ async function runCreateUpdateDelete(
console.log("Two-vault note creation, update, and deletion round-tripped.");
}
async function runRename(context: RunnerContext, vaultA: TemporaryVault, vaultB: TemporaryVault): Promise<void> {
const renamedContent = "# Rename target\n\nThis note should move from A to B.\n";
let session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renamedContent);
await uploadNote(context, session, renameFromPath);
await renameNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renameToPath);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, renameToPath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
const renamedOnB = await waitForPathContent(vaultB.path, renameToPath, (content) => content === renamedContent);
await waitForPathDeleted(vaultB.path, renameFromPath);
await session.app.stop();
assertEqual(renamedOnB, renamedContent, "Renamed note content did not round-trip to the second vault.");
console.log("Two-vault note rename round-tripped.");
}
async function runEncryptedRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const encryptedContent = "# Encrypted round-trip\n\nThis note should synchronise with E2EE enabled.\n";
const encryptedOverrides = {
encrypt: true,
passphrase: "real-obsidian-e2e-passphrase",
usePathObfuscation: true,
E2EEAlgorithm: "v2",
};
let session = await startConfiguredSession(context, vaultA, encryptedOverrides);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, encryptedPath, encryptedContent);
await uploadNote(context, session, encryptedPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, encryptedOverrides);
await syncAndApply(context, session);
const received = await waitForPathContent(vaultB.path, encryptedPath, (content) => content === encryptedContent);
await session.app.stop();
assertEqual(received, encryptedContent, "Encrypted note did not round-trip to the second vault.");
console.log("Two-vault encrypted note synchronisation round-tripped.");
}
async function runMarkdownAutoMerge(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const base = "# Conflict\n\nBase line\n\nShared tail\n";
const left = "# Conflict\n\nLeft line\n\nShared tail\n";
const right = "# Conflict\n\nBase line\n\nRight tail\n";
const base = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nBottom anchor\n";
const left = "# Conflict\n\nTop anchor\n\nLeft line\n\nMiddle anchor\n\nBottom anchor\n";
const right = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nRight tail\n\nBottom anchor\n";
let session = await startConfiguredSession(context, vaultB);
await createMarkdownConflict(context, session, vaultB, conflictPath, base, left, right);
@@ -335,7 +405,8 @@ async function runMarkdownAutoMerge(
const mergedOnB = await waitForPathContent(
vaultB.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail")
(content) => content.includes("Left line") && content.includes("Right tail"),
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
);
await session.app.stop();
@@ -344,7 +415,8 @@ async function runMarkdownAutoMerge(
const mergedOnA = await waitForPathContent(
vaultA.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail")
(content) => content.includes("Left line") && content.includes("Right tail"),
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
);
await session.app.stop();
@@ -405,29 +477,44 @@ async function main(): Promise<void> {
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync");
const encryptedDbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync-e2ee");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const encryptedVaultA = await createTemporaryVault();
const encryptedVaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
const encryptedContext: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName: encryptedDbName };
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
await createCouchDbDatabase(couchDb, encryptedDbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
console.log(`Temporary encrypted CouchDB database: ${encryptedDbName}`);
await runCreateUpdateDelete(context, vaultA, vaultB);
await runMarkdownAutoMerge(context, vaultA, vaultB);
await runRename(context, vaultA, vaultB);
if (process.env.E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT === "true") {
await runMarkdownAutoMerge(context, vaultA, vaultB);
}
await runTargetMismatch(context, vaultA, vaultB);
await runEncryptedRoundTrip(encryptedContext, encryptedVaultA, encryptedVaultB);
} finally {
await vaultA.dispose();
await vaultB.dispose();
await encryptedVaultA.dispose();
await encryptedVaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
await deleteCouchDbDatabase(couchDb, encryptedDbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}