mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-06-30 18:05:20 +00:00
Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 vorotamoroz
|
f97888572b | ||
|
vorotamoroz
|
9d9b512f3f | ||
|
vorotamoroz
|
54e6a761e5 | ||
|
vorotamoroz
|
d37ed51660 | ||
|
vorotamoroz
|
e916683b8d | ||
|
vorotamoroz
|
7028a0857e | ||
|
vorotamoroz
|
150f0700b0 | ||
|
vorotamoroz
|
36590ee762 | ||
|
vorotamoroz
|
4a33236c8f | ||
|
vorotamoroz
|
be23fa51a1 |
@@ -1,4 +1,5 @@
|
||||
# Self-hosted LiveSync Development Guide
|
||||
|
||||
## Project Overview
|
||||
|
||||
Self-hosted LiveSync is an Obsidian plugin for synchronising vaults across devices using CouchDB, MinIO/S3, or peer-to-peer WebRTC. The codebase uses a modular architecture with TypeScript, Svelte, and PouchDB.
|
||||
@@ -10,6 +11,7 @@ Self-hosted LiveSync is an Obsidian plugin for synchronising vaults across devic
|
||||
#### First-time Setup
|
||||
|
||||
This repository uses submodules by convention. Therefore, you must use the `--recursive` flag when cloning it.
|
||||
|
||||
```bash
|
||||
git clone --recursive https://github.com/vrtmrz/obsidian-livesync
|
||||
npm ci
|
||||
@@ -19,7 +21,9 @@ npm run build
|
||||
Note: if you already cloned without submodules, run: `git submodule update --init --recursive`
|
||||
|
||||
#### Branch switching
|
||||
|
||||
When switching branches, please make sure to update submodules as well, since they may be updated in the new branch.
|
||||
|
||||
```bash
|
||||
git checkout --recurse-submodules 0.25.70-patch1 # tag or branch name
|
||||
npm ci
|
||||
@@ -41,7 +45,7 @@ npm test # Run Harness based vitest tests (requires Docker services)
|
||||
|
||||
### Tips
|
||||
|
||||
We can use CLI's E2E test command instead of `npm test`.
|
||||
Use CLI E2E tests or real Obsidian E2E scripts instead of `npm test` when the behaviour can be verified outside the browser harness.
|
||||
|
||||
### Auto-copy to test vaults
|
||||
|
||||
@@ -58,46 +62,54 @@ To facilitate development and testing, the build process can automatically copy
|
||||
- **Unit Tests** (`vitest.config.unit.ts`): Unit tests run in Node.js (excluding harnesses and integration tests). Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`). Executed via `npm run test:unit`.
|
||||
- **Integration Tests** (`vitest.config.integration.ts`): Tests run in Node.js against a real CouchDB instance. Integration tests should be `*.integration.spec.ts` or `*.integration.test.ts` and placed alongside the implementation file (e.g., `StreamingFetch.integration.spec.ts`). Executed via `npm run test:integration`.
|
||||
- If you add a feature that interacts with the remote database (e.g., replication changes, custom changes feed parameters, or custom HTTP queries), you strongly expected to write an integration test to verify the behaviour against a real CouchDB server.
|
||||
- **E2E Tests** (`vitest.config.ts`): End-to-end tests run in a browser-based harness using Playwright/Chromium to test full synchronisation scenarios. Executed via `npm run test`.
|
||||
- **Browser Harness Tests** (`vitest.config.ts`): Transitional browser-based harness tests using Playwright/Chromium. Executed via `npm run test`. This layer is no longer the preferred destination for new broad E2E coverage because `test/harness` can drift from real Obsidian behaviour.
|
||||
- **P2P Tests** (`vitest.config.p2p.ts`): Browser-based Peer-to-Peer replication tests. Executed via `npm run test:p2p`.
|
||||
- **RPC Unit Tests** (`vitest.config.rpc-unit.ts`): RPC-specific unit tests with coverage thresholds.
|
||||
- **Real Obsidian E2E** (`test/e2e-obsidian/`): Local-first scripts that launch real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in. Use these for boot-up sequence, vault reflection, RedFlag flows, Fast Setup (Simple Fetch), settings dialogues, restart-sensitive workflows, Object Storage regressions, and other behaviour that depends on Obsidian itself. Run focused scripts such as `npm run test:e2e:obsidian:two-vault-sync`, or use `npm run test:e2e:obsidian:local-suite:services` to run the broader local suite with CouchDB and MinIO fixtures managed by the wrapper.
|
||||
|
||||
- **Docker Services**: Tests require CouchDB, MinIO (S3), and P2P services:
|
||||
|
||||
```bash
|
||||
npm run test:docker-all:start # Start all test services
|
||||
npm run test:full # Run tests with coverage
|
||||
npm run test:docker-all:stop # Stop services
|
||||
```
|
||||
|
||||
If some services are not needed, start only required ones (e.g., `test:docker-couchdb:start`).
|
||||
Note that if services are already running, starting script will fail. Please stop them first.
|
||||
|
||||
- **Test Structure**:
|
||||
- `test/suite/` - E2E tests for sync operations (running in browser)
|
||||
- `test/e2e-obsidian/` - Real Obsidian E2E scripts for local verification
|
||||
- `test/suite/` - Transitional browser harness tests for sync operations
|
||||
- `test/unit/` - Unit tests (via vitest, as harness is browser-based)
|
||||
- `test/harness/` - Mock implementations (e.g., `obsidian-mock.ts`)
|
||||
- `test/harness/` - Transitional mock implementations (e.g., `obsidian-mock.ts`). Avoid adding broad new E2E coverage here unless it is explicitly a compatibility bridge.
|
||||
|
||||
### Import Path Normalisation
|
||||
|
||||
The codebase uses `@/` and `@lib/` path aliases to keep import structures clean. To normalise imports and exports across files, use the following utility script:
|
||||
|
||||
```bash
|
||||
npm run pretty:importpath
|
||||
```
|
||||
|
||||
Under the hood, this runs Deno with the script [utilsdeno/normalise-imports.ts](file:///p:/plant25/obsidian/projects/obsidian-livesync/utilsdeno/normalise-imports.ts). You can pass additional flags to this script if required (by running it via Deno directly from the `utilsdeno` directory):
|
||||
|
||||
- `--run`: Applies the changes (the script runs in dry-run mode by default).
|
||||
- `--all-alias`: Normalises sibling/child relative imports starting with `./` to use aliases.
|
||||
|
||||
### Type Generation
|
||||
|
||||
To generate fallback type definitions for the shared library and add appropriate Deno ignore comments (which suppresses Deno compilation warnings and linting warnings inside the `_types` directory), run:
|
||||
|
||||
```bash
|
||||
npm run build:lib:types
|
||||
```
|
||||
|
||||
This script executes:
|
||||
|
||||
1. TypeScript compilation (`tsconfig.types.json`) to output definitions to the `_types` directory.
|
||||
2. The Deno script [utilsdeno/types-add-ignore.ts](file:///p:/plant25/obsidian/projects/obsidian-livesync/utilsdeno/types-add-ignore.ts) to prepend Deno ignore comments to the generated type files.
|
||||
|
||||
|
||||
|
||||
## Architecture
|
||||
|
||||
### Module System
|
||||
|
||||
@@ -0,0 +1,251 @@
|
||||
# Architectural Decision Record: Real Obsidian End-to-End Test Runner
|
||||
|
||||
## Status
|
||||
|
||||
Proposed / Spike Implemented
|
||||
|
||||
## Release
|
||||
|
||||
Not yet. Planned after the serviceFeature refactoring branch is reviewed.
|
||||
|
||||
## Context
|
||||
|
||||
The current end-to-end tests run through Vitest browser mode and a mocked Obsidian environment in `test/harness`. This has been useful for exercising synchronisation flows without launching Obsidian, but it is no longer a reliable final signal for plug-in behaviour.
|
||||
|
||||
The main issues are:
|
||||
|
||||
- The harness reimplements a large part of the Obsidian API surface, including vault files, workspace events, settings, and lifecycle behaviour. This mock can drift from real Obsidian behaviour without failing.
|
||||
- The tests run inside a browser-style environment, while the desktop plug-in runs inside Obsidian's Electron environment with its own application lifecycle, storage paths, command registry, and event ordering.
|
||||
- Several high-value regressions are about integration boundaries: boot-up sequence timing, real vault file reflection, Obsidian command registration, settings persistence, restart prompts, and file watcher behaviour. These are precisely the areas where a mock harness gives weak confidence.
|
||||
- Maintaining the harness competes with maintaining the plug-in. Adding behaviour to the plug-in often requires teaching the mock another Obsidian detail before the actual regression can be tested.
|
||||
|
||||
The current harness should therefore stop being treated as the primary E2E layer.
|
||||
|
||||
## Decision
|
||||
|
||||
Introduce a new E2E layer that launches real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in installed into those vaults.
|
||||
|
||||
The long-term test pyramid should be:
|
||||
|
||||
1. Unit tests for deterministic operations and serviceFeature boundaries.
|
||||
2. Integration tests for CouchDB, Object Storage, P2P services, database operations, and replication protocols.
|
||||
3. Real Obsidian E2E tests for boot-up sequence, vault reflection, command registration, settings dialogues, restart scheduling, and user-visible workflows.
|
||||
|
||||
The existing `test/harness` should be demoted to a transitional compatibility layer. It may remain temporarily while the real Obsidian runner reaches parity for critical flows, but new high-level E2E coverage should target the real runner.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not replace unit or integration tests with slow UI tests.
|
||||
- Do not keep extending the Obsidian mock to cover new Obsidian APIs unless a short-term compatibility bridge is required.
|
||||
- Do not require real Obsidian E2E for every pull request initially. The first CI integration should be opt-in or nightly until stability is proven.
|
||||
- Do not test every setting dialogue through UI clicks if the behaviour is already covered by unit or integration tests. Use UI automation only for workflows whose risk is in real Obsidian integration.
|
||||
|
||||
## Proposed Architecture
|
||||
|
||||
### Runner
|
||||
|
||||
Create a dedicated runner under `test/e2e-obsidian/`.
|
||||
|
||||
The runner should:
|
||||
|
||||
- Create one or more temporary vault directories.
|
||||
- Build the plug-in once with `npm run build` or a narrower production build command.
|
||||
- Install `main.js`, `manifest.json`, and `styles.css` when present into `.obsidian/plugins/obsidian-livesync/`.
|
||||
- Prepare `.obsidian/community-plugins.json` and `.obsidian/plugins/obsidian-livesync/data.json` as needed.
|
||||
- Launch Obsidian against the temporary vault.
|
||||
- Wait until the plug-in reports readiness through a deterministic probe.
|
||||
- Drive assertions through a narrow control channel rather than fragile visual selectors wherever possible.
|
||||
- Dispose of Obsidian and temporary vaults after each scenario.
|
||||
|
||||
### Obsidian Launch
|
||||
|
||||
The preferred desktop target is the installed Obsidian application. The launch mechanism should be platform-specific but hidden behind a small adapter:
|
||||
|
||||
- Linux: launch the Obsidian executable with a vault path or Obsidian URI, depending on what is most reliable. If an AppImage is used and FUSE is not available, extract it with `--appimage-extract` and launch the extracted `squashfs-root/obsidian` binary.
|
||||
- macOS: launch the app bundle through `open` or the executable inside the bundle.
|
||||
- Windows: launch the installed executable or the registered application protocol.
|
||||
|
||||
The first implementation can support Linux only if that is the local and CI target. Cross-platform support can be added after the runner contract is stable.
|
||||
|
||||
In headless Linux environments, launch through `xvfb-run`, pass Electron flags such as `--no-sandbox` and `--disable-gpu`, and isolate `HOME`, `XDG_CONFIG_HOME`, and `--user-data-dir` per temporary vault.
|
||||
|
||||
### Control Channel
|
||||
|
||||
The runner needs a stable way to observe readiness and issue test commands. Prefer a test-only plug-in bridge compiled only in test builds or enabled only by an environment variable.
|
||||
|
||||
Possible bridge options:
|
||||
|
||||
- The official Obsidian CLI, using the installed `obsidian-cli` helper to open vaults, reload the plug-in, run `eval`, and call developer commands.
|
||||
- A local HTTP/WebSocket bridge bound to `127.0.0.1` with a random port and token.
|
||||
- A file-based bridge in the vault, where Obsidian writes status files and consumes command files.
|
||||
- A DevTools protocol bridge if Obsidian exposes a stable debugging port in the test environment.
|
||||
|
||||
The first implementation uses Obsidian's CLI for orchestration and readiness checks. The CLI handles vault opening through `obsidian://open?path=...`, enables community plug-ins through `app.plugins.setEnable(true)`, reloads Self-hosted LiveSync through `plugin:reload id=obsidian-livesync`, and verifies that `app.plugins.plugins['obsidian-livesync']` is loaded.
|
||||
|
||||
This keeps E2E-only behaviour out of the production plug-in bundle. The runner should not require Self-hosted LiveSync to write marker files or expose a test server merely to prove that Obsidian loaded it.
|
||||
|
||||
The DevTools protocol remains useful for diagnostics. Obsidian's CLI exposes developer commands such as `dev:cdp`, `dev:errors`, and `dev:console`, so the runner should prefer the CLI path first and fall back to direct DevTools attachment only if the CLI cannot provide the required signal.
|
||||
|
||||
### Test Data and Services
|
||||
|
||||
Keep the existing Docker scripts for CouchDB, MinIO, and P2P services. The real Obsidian runner should reuse these service fixtures instead of creating another service orchestration stack.
|
||||
|
||||
Each test should use unique database names, bucket prefixes, vault names, and P2P room IDs. This prevents tests from depending on cleanup and makes interrupted runs less harmful.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### Phase 0: Discovery
|
||||
|
||||
- Confirm how Obsidian can be launched reliably on the local development environment.
|
||||
- Confirm whether Obsidian accepts a vault path directly, requires an Obsidian URI, or needs a pre-existing vault registry.
|
||||
- Identify where Obsidian stores per-user state in the test environment and decide how to isolate it.
|
||||
- Decide whether the first bridge is file-based or HTTP/WebSocket.
|
||||
|
||||
Initial discovery on Linux ARM64 found that:
|
||||
|
||||
- `Obsidian-1.12.7-arm64.AppImage` requires `libfuse.so.2` for direct AppImage execution.
|
||||
- Extracting the AppImage with `--appimage-extract` works without FUSE.
|
||||
- Launching the extracted `squashfs-root/obsidian` binary under `xvfb-run` with isolated user data stays alive for the smoke timeout.
|
||||
- No missing shared libraries were reported by `ldd` for the extracted binary in the tested environment.
|
||||
- Obsidian's CLI is disabled unless the global `obsidian.json` contains `cli: true`.
|
||||
- Passing only `.obsidian/community-plugins.json` is not enough to load community plug-ins on Obsidian 1.12. The runner also has to enable the global community plug-in switch through `app.plugins.setEnable(true)`.
|
||||
- The reliable launch sequence is: start Obsidian, send `obsidian://open?path=...` through `obsidian-cli`, wait until the vault-side CLI exposes the plug-in catalogue, enable community plug-ins, reload Self-hosted LiveSync, and verify plug-in readiness through `obsidian-cli eval`.
|
||||
|
||||
### Phase 1: Smoke Runner
|
||||
|
||||
- Add `test/e2e-obsidian/runner` utilities for temporary vault creation, plug-in installation, launch, readiness wait, and cleanup.
|
||||
- Add one smoke test:
|
||||
- launch Obsidian with an empty vault,
|
||||
- load Self-hosted LiveSync,
|
||||
- wait for the boot-up sequence to become ready,
|
||||
- read the plug-in version or status through the control channel,
|
||||
- close Obsidian cleanly.
|
||||
- Add an npm script such as `test:e2e:obsidian`.
|
||||
|
||||
Current implementation status:
|
||||
|
||||
- Added `test/e2e-obsidian/runner` helpers for Obsidian discovery, CLI discovery, temporary vault creation, plug-in installation, process launch, CLI execution, and readiness polling.
|
||||
- Added `test:e2e:obsidian:discover`, `test:e2e:obsidian:cli-help`, `test:e2e:obsidian:smoke`, `test:e2e:obsidian:vault-reflection`, `test:e2e:obsidian:couchdb-upload`, `test:e2e:obsidian:minio-upload`, `test:e2e:obsidian:startup-scan`, `test:e2e:obsidian:two-vault-sync`, `test:e2e:obsidian:hidden-file-snippet-sync`, `test:e2e:obsidian:customisation-sync`, `test:e2e:obsidian:setting-markdown-export`, `test:e2e:obsidian:local-suite`, `test:e2e:obsidian:local-suite:services`, and `test:e2e:obsidian:install-appimage`.
|
||||
- Added `startObsidianLiveSyncSession()` so future workflows can reuse the launch, trusted temporary vault state, vault open, community plug-in reload, and readiness sequence without duplicating smoke runner code.
|
||||
- Added CouchDB runner utilities that reuse `.test.env`/process environment values, create unique temporary databases, query uploaded documents directly, and clean up the database unless `E2E_OBSIDIAN_KEEP_COUCHDB=true` is set.
|
||||
- Added a manual AppImage installer that downloads Obsidian `1.12.7` for `arm64` or `x86_64`, stores it under `_testdata/obsidian`, and extracts it for FUSE-free execution.
|
||||
- Confirmed the smoke runner on Linux ARM64 with the extracted Obsidian `1.12.7` AppImage, `xvfb-run`, and the built Self-hosted LiveSync bundle.
|
||||
- Confirmed the runner can enable the Obsidian CLI through isolated `obsidian.json` state, pre-seed the temporary Chromium local storage so the generated vault ID is trusted for community plug-ins, open the temporary vault through `obsidian-cli`, reload Self-hosted LiveSync, and verify readiness through `obsidian-cli eval`.
|
||||
- Removed the first test-only ready-marker bridge from the plug-in bundle. The current runner observes readiness from outside the plug-in through Obsidian's own CLI, so normal user vaults do not receive E2E marker files.
|
||||
|
||||
Current verification:
|
||||
|
||||
- `npm run tsc-check` passes.
|
||||
- `npm run build` passes with existing Svelte warnings.
|
||||
- `npm run test:e2e:obsidian:discover` finds `_testdata/obsidian/squashfs-root/obsidian` when the extracted AppImage is present.
|
||||
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS=1000 npm run test:e2e:obsidian:smoke` passes locally.
|
||||
- `npm run test:e2e:obsidian:vault-reflection` creates a note through Obsidian's vault API, verifies the reflected file on disk, and reads it back through Obsidian.
|
||||
- `npm run test:e2e:obsidian:couchdb-upload` configures a unique CouchDB database, creates a note through Obsidian, commits it into the local database, runs one-shot synchronisation, and verifies that CouchDB contains the metadata document and all referenced chunk documents.
|
||||
- `npm run test:e2e:obsidian:minio-upload` configures a unique Object Storage bucket prefix, creates a note through Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written to the S3-compatible bucket.
|
||||
- `npm run test:e2e:obsidian:startup-scan` verifies that a file written while Obsidian is stopped is picked up during the next real Obsidian boot and uploaded to CouchDB after one-shot synchronisation.
|
||||
- `npm run test:e2e:obsidian:two-vault-sync` verifies two-vault note synchronisation: creation, update, rename, deletion, per-device target-filter differences, and a separate encrypted round-trip with Path Obfuscation enabled. The experimental Markdown conflict automatic merge check is available with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` but is not part of the default local suite.
|
||||
- `npm run test:e2e:obsidian:hidden-file-snippet-sync` verifies hidden file synchronisation as a two-vault round-trip: creation, deletion, automatic JSON conflict merging with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target-pattern differences.
|
||||
- `npm run test:e2e:obsidian:customisation-sync` verifies a two-vault Customisation Sync workflow: scan a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronise them through CouchDB, apply them on the second vault, assert the resulting `.obsidian` files, propagate a snippet update, and verify deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
|
||||
- `npm run test:e2e:obsidian:setting-markdown-export` verifies that setting Markdown export creates a vault file and omits credentials when credential export is disabled.
|
||||
- `npm run test:e2e:obsidian:install-appimage` reuses the existing AppImage and extracted binary when they are already present.
|
||||
- `npm run test:e2e:obsidian:local-suite` runs the local verification sequence for the real Obsidian runner after CouchDB and MinIO have been started.
|
||||
- `npm run test:e2e:obsidian:local-suite:services` stops leftover CouchDB and MinIO fixtures, starts fresh fixtures, runs the local suite, and stops the fixtures again.
|
||||
- `npm run test:e2e:obsidian:local-suite:services` has been verified locally with real Obsidian, CouchDB, and MinIO. The run completed discovery, smoke, vault reflection, CouchDB upload, Object Storage upload, startup scan, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export. The build step still emits existing Svelte warnings.
|
||||
|
||||
Known limits:
|
||||
|
||||
- The smoke runner currently proves only one-vault launch and plug-in load readiness. Broader workflows are covered by separate real Obsidian scripts, including CouchDB upload, startup scan, two-vault note synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export.
|
||||
- Cross-platform support is still discovery-level. The working path has been validated on Linux ARM64. macOS and Windows should be validated in their own environments as follow-up work.
|
||||
- CI wiring is intentionally not implemented. The runner depends on a licensed desktop application and is treated as a local verification tool.
|
||||
|
||||
### Phase 2: First Real Workflow
|
||||
|
||||
- Add a one-vault local workflow:
|
||||
- configure a temporary CouchDB database,
|
||||
- create a note in the real vault,
|
||||
- wait for metadata and chunks to be stored,
|
||||
- restart Obsidian,
|
||||
- verify that the plug-in loads and the note remains consistent.
|
||||
|
||||
This validates real boot-up, settings persistence, vault file access, database writes, and restart-sensitive state.
|
||||
|
||||
Current implementation status:
|
||||
|
||||
- Added a pre-CouchDB workflow that creates a note through Obsidian's vault API, confirms the note is reflected as a real vault file, and reads the same note back through Obsidian. This covers the vault reflection part of the Phase 2 path before remote database setup is introduced.
|
||||
- Added a first CouchDB-backed upload workflow, modelled after the CLI Deno tests: reuse the standard CouchDB environment variables, create a unique remote database, apply CouchDB settings through the plug-in's setting service, commit the note through the real Obsidian vault path, run one-shot synchronisation, and assert that remote metadata and chunks exist.
|
||||
- Added an Object Storage-backed upload workflow against MinIO to exercise Journal Sync and the AWS SDK path from real Obsidian.
|
||||
- Added Obsidian-specific workflows for boot-time vault scanning, two-vault note synchronisation, hidden `.obsidian/snippets` file round-tripping, hidden JSON conflict resolution, Customisation Sync application for snippets, config JSON files, and plug-in fixtures, per-device target-filter differences, and setting Markdown export. These scenarios assert against CouchDB documents, vault files, or real Obsidian UI outcomes instead of internal service state.
|
||||
|
||||
### Phase 3: Two-Vault Synchronisation
|
||||
|
||||
- Launch two Obsidian instances with two temporary vaults.
|
||||
- Configure both against the same temporary remote database.
|
||||
- Create, modify, rename, and delete notes in one vault.
|
||||
- Verify reflection in the other vault.
|
||||
- Cover encrypted and non-encrypted configurations separately.
|
||||
|
||||
Current implementation status:
|
||||
|
||||
- `test:e2e:obsidian:two-vault-sync` covers creation, update, rename, deletion, and per-device target-filter behaviour for a non-encrypted CouchDB configuration. Markdown conflict automatic merging remains an optional check because it needs a dedicated, less timing-sensitive fixture.
|
||||
- The same script creates a separate temporary CouchDB database and temporary vault pair for an encrypted two-vault round-trip with Path Obfuscation enabled.
|
||||
|
||||
### Phase 4: Harness Retirement
|
||||
|
||||
- Mark `test/harness` as deprecated in documentation.
|
||||
- Stop adding new tests to `test/suite` unless they are explicitly transitional.
|
||||
- Do not mechanically port `test/suite` into real Obsidian E2E. Scenarios that can already be exercised and asserted through the CLI test layer should stay there or move to lower-level integration tests.
|
||||
- Prioritise real Obsidian coverage for behaviours that the CLI cannot prove well, especially RedFlag flag-file recovery flows, Fast Setup (Simple Fetch), boot-up sequencing, restart-sensitive initial synchronisation, and user-visible recovery dialogues.
|
||||
- Remove the harness only after the new runner covers the critical boot-up and synchronisation workflows.
|
||||
|
||||
Current implementation status:
|
||||
|
||||
- `test/harness` is now documented as a transitional compatibility layer.
|
||||
- New broad E2E work should target `test/e2e-obsidian/` when real Obsidian behaviour is the risk being tested.
|
||||
- The next high-value scenarios are RedFlag variants and Fast Setup (Simple Fetch) variants, not a line-by-line migration of `test/suite`.
|
||||
|
||||
## Local Verification Strategy
|
||||
|
||||
Real Obsidian E2E is a local verification layer. It should not be wired into the default CI gate.
|
||||
|
||||
- Keep the scripts individually runnable for focused local debugging.
|
||||
- Provide `test:e2e:obsidian:local-suite` for a broader local pass after the CouchDB and MinIO fixtures have been started.
|
||||
- Provide `test:e2e:obsidian:local-suite:services` for a broader local pass that manages the CouchDB and MinIO fixtures itself.
|
||||
- Use `OBSIDIAN_BINARY` when testing against an installed desktop application.
|
||||
- Use `test:e2e:obsidian:install-appimage` on Linux when a local AppImage copy is needed, and reuse the extracted `_testdata/obsidian/squashfs-root` directory between local runs.
|
||||
- Capture Obsidian logs, plug-in logs, vault snapshots, and service logs manually when investigating failures.
|
||||
- Fail fast on launch failures, readiness timeouts, and cleanup failures with clear diagnostics.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
- **Obsidian licensing and installation**: Keep the runner local-first and capable of using `OBSIDIAN_BINARY`.
|
||||
- **Flakiness from UI timing**: Prefer a control channel and service-level probes over visual selectors.
|
||||
- **Multiple instances**: Obsidian may not support multiple independent instances cleanly on all platforms. Start with one-instance smoke tests, then validate two-instance behaviour on Linux before expanding scope.
|
||||
- **State leakage**: Isolate vault directories, Obsidian user data, remote database names, and bridge tokens per test.
|
||||
- **Security of E2E controls**: Keep readiness and control outside the production plug-in bundle. Prefer Obsidian CLI probes over E2E-only plug-in code.
|
||||
- **Runtime cost**: Keep real Obsidian E2E out of the default PR gate. Use focused scripts or the local suite when a change touches real Obsidian integration.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Which launch mechanism is most reliable for Obsidian on each supported desktop platform?
|
||||
- Can two Obsidian instances run with isolated user data at the same time?
|
||||
- Do future scenarios need a richer control channel than Obsidian CLI, or can CLI `eval` and developer commands cover the required workflows?
|
||||
- Should any future E2E-only plug-in code live in a separate test build, or should the production bundle remain free of E2E controls?
|
||||
- Which RedFlag and Fast Setup (Simple Fetch) variants should be added first?
|
||||
|
||||
## Initial Implementation Checklist
|
||||
|
||||
1. Add an Obsidian launch discovery script that prints the detected executable, version, and launch mode.
|
||||
2. Add temporary vault and plug-in installation helpers.
|
||||
3. Add CLI-based plug-in readiness polling.
|
||||
4. Add `test:e2e:obsidian:smoke` for one-vault plug-in load.
|
||||
5. Document required local environment variables, especially `OBSIDIAN_BINARY`.
|
||||
6. Port one CouchDB-backed workflow after the smoke test is stable.
|
||||
7. Mark `test/harness` as transitional and block new broad E2E work from targeting it.
|
||||
8. Add the local suite script for broader local verification.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Real Obsidian E2E becomes the source of truth for plug-in lifecycle and vault integration.
|
||||
- Unit and integration tests remain the primary fast feedback loops.
|
||||
- The old browser harness can be deleted once the new runner covers the critical workflows.
|
||||
- The project will gain slower but higher-confidence tests for the behaviours most likely to differ between mocks and Obsidian itself.
|
||||
+1
-1
@@ -16,7 +16,7 @@ try {
|
||||
|
||||
try {
|
||||
console.log("[Postbuild] Type definitions generated successfully. Adding ignore comments...");
|
||||
execSync("Deno run -A ./utilsdeno/types-add-ignore.ts", { stdio: "inherit" });
|
||||
execSync("deno run -A ./utilsdeno/types-add-ignore.ts", { stdio: "inherit" });
|
||||
console.log("[Postbuild] Ignore comments added successfully.");
|
||||
} catch (error) {
|
||||
console.warn("[Postbuild] Failed to add ignore comments to type definitions.");
|
||||
|
||||
@@ -38,6 +38,21 @@
|
||||
"test:unit:coverage": "vitest run --config vitest.config.unit.ts --coverage",
|
||||
"test:install-playwright": "npx playwright install chromium",
|
||||
"test:install-dependencies": "npm run test:install-playwright",
|
||||
"test:e2e:obsidian:install-appimage": "tsx test/e2e-obsidian/scripts/install-appimage.ts",
|
||||
"test:e2e:obsidian:discover": "tsx test/e2e-obsidian/scripts/discover.ts",
|
||||
"test:e2e:obsidian:cli-help": "tsx test/e2e-obsidian/scripts/cli-help.ts",
|
||||
"test:e2e:obsidian:debug-ui": "tsx test/e2e-obsidian/scripts/debug-ui.ts",
|
||||
"test:e2e:obsidian:smoke": "tsx test/e2e-obsidian/scripts/smoke.ts",
|
||||
"test:e2e:obsidian:vault-reflection": "tsx test/e2e-obsidian/scripts/vault-reflection.ts",
|
||||
"test:e2e:obsidian:couchdb-upload": "tsx test/e2e-obsidian/scripts/couchdb-upload.ts",
|
||||
"test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts",
|
||||
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
|
||||
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
|
||||
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
|
||||
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
|
||||
"test:e2e:obsidian:setting-markdown-export": "tsx test/e2e-obsidian/scripts/setting-markdown-export.ts",
|
||||
"test:e2e:obsidian:local-suite": "tsx test/e2e-obsidian/scripts/local-suite.ts",
|
||||
"test:e2e:obsidian:local-suite:services": "tsx test/e2e-obsidian/scripts/local-suite.ts --manage-services",
|
||||
"test:coverage": "vitest run --coverage",
|
||||
"test:docker-couchdb:up": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-start.sh",
|
||||
"test:docker-couchdb:init": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-init.sh",
|
||||
|
||||
+634
-634
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,118 @@
|
||||
# Real Obsidian E2E Runner
|
||||
|
||||
This directory contains the experimental real Obsidian end-to-end runner.
|
||||
|
||||
The current smoke runner verifies only the launch path:
|
||||
|
||||
1. create a temporary vault,
|
||||
2. install the built Self-hosted LiveSync plug-in artifacts,
|
||||
3. launch real Obsidian,
|
||||
4. open the temporary vault through `obsidian-cli`,
|
||||
5. enable Obsidian community plug-ins for the temporary app profile,
|
||||
6. reload Self-hosted LiveSync through `obsidian-cli`,
|
||||
7. verify through `obsidian-cli eval` that the plug-in is loaded,
|
||||
8. optionally drive a real vault or CouchDB workflow through Obsidian's own API,
|
||||
9. terminate Obsidian and remove the temporary vault.
|
||||
|
||||
The runner does not require Self-hosted LiveSync to expose an E2E-only bridge. Readiness is checked from outside the plug-in through Obsidian's own CLI.
|
||||
|
||||
Obsidian 1.12 stores the global community plug-in switch outside `.obsidian/community-plugins.json`. The smoke runner enables it through `app.plugins.setEnable(true)` after the vault window is available.
|
||||
|
||||
Future workflows should use `startObsidianLiveSyncSession()` from `runner/session.ts` rather than repeating the launch and plug-in readiness sequence.
|
||||
|
||||
Each test vault uses an isolated Obsidian profile. The runner creates temporary directories for `HOME`, `XDG_CONFIG_HOME`, `XDG_CACHE_HOME`, `XDG_DATA_HOME`, and Electron `--user-data-dir`, writes the vault registry into those directories, pre-seeds the temporary Chromium local storage so community plug-ins are trusted for that generated vault ID, and passes the same environment to `obsidian-cli`. This is intended to keep real Obsidian E2E runs separate from a developer's daily Obsidian profile and vault registry.
|
||||
|
||||
## Local Setup
|
||||
|
||||
Set `OBSIDIAN_BINARY` when Obsidian is not installed in a standard location.
|
||||
|
||||
For an AppImage on Linux without FUSE, use the helper script:
|
||||
|
||||
```bash
|
||||
npm run test:e2e:obsidian:install-appimage
|
||||
```
|
||||
|
||||
The script downloads Obsidian `1.12.7` for the current architecture, stores it in `_testdata/obsidian`, and extracts it to `_testdata/obsidian/squashfs-root`. The runner checks `_testdata/obsidian/squashfs-root/obsidian` before the AppImage path.
|
||||
|
||||
These tests are intended for local verification, not the default CI gate. Reuse the installed Obsidian application, or reuse the extracted AppImage directory between local runs:
|
||||
|
||||
- set `OBSIDIAN_BINARY` to an installed Obsidian executable,
|
||||
- keep `_testdata/obsidian/squashfs-root` after running the AppImage installer, or
|
||||
- run `test:e2e:obsidian:install-appimage` again only when the local Obsidian version should change.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
npm run test:e2e:obsidian:install-appimage
|
||||
npm run test:e2e:obsidian:discover
|
||||
npm run test:e2e:obsidian:cli-help -- vaults verbose
|
||||
npm run test:e2e:obsidian:smoke
|
||||
npm run test:e2e:obsidian:vault-reflection
|
||||
npm run test:e2e:obsidian:couchdb-upload
|
||||
npm run test:e2e:obsidian:minio-upload
|
||||
npm run test:e2e:obsidian:startup-scan
|
||||
npm run test:e2e:obsidian:two-vault-sync
|
||||
npm run test:e2e:obsidian:hidden-file-snippet-sync
|
||||
npm run test:e2e:obsidian:customisation-sync
|
||||
npm run test:e2e:obsidian:setting-markdown-export
|
||||
npm run test:e2e:obsidian:local-suite
|
||||
npm run test:e2e:obsidian:local-suite:services
|
||||
```
|
||||
|
||||
`test:e2e:obsidian:local-suite` runs `npm run build`, discovery, smoke, vault reflection, CouchDB upload, Object Storage upload, startup scan, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export in sequence. Start the local CouchDB and MinIO fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run.
|
||||
|
||||
`test:e2e:obsidian:couchdb-upload` reuses the CouchDB variables from `.test.env` or the process environment. It expects a reachable CouchDB service, creates a unique database, configures Self-hosted LiveSync through `obsidian-cli eval`, creates a note in real Obsidian, commits the note into the local database, runs one-shot synchronisation, and verifies that the remote database contains both the metadata document and its chunk documents.
|
||||
|
||||
`test:e2e:obsidian:minio-upload` reuses the Object Storage variables from `.test.env` or the process environment. It expects a reachable S3-compatible service, configures Self-hosted LiveSync for Object Storage through `obsidian-cli eval`, creates a note in real Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written under a unique bucket prefix.
|
||||
|
||||
`test:e2e:obsidian:startup-scan` configures a temporary CouchDB database, stops Obsidian, writes a note directly into the vault, restarts Obsidian, and verifies from CouchDB that the boot-time scan picked up the offline file.
|
||||
|
||||
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, rename, deletion, per-device target filters where one vault ignores a note that the other vault synchronises, and a separate encrypted round-trip with Path Obfuscation enabled. The optional Markdown conflict automatic merge check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`, but it is not part of the default local suite.
|
||||
|
||||
`test:e2e:obsidian:hidden-file-snippet-sync` runs a two-vault hidden file round-trip. It verifies creation and deletion of a real `.obsidian/snippets/*.css` file, automatic JSON conflict merging for a hidden file with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target patterns where one vault ignores a hidden file that the other vault synchronises.
|
||||
|
||||
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
|
||||
|
||||
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`.
|
||||
|
||||
Start the local fixtures first when they are not already running:
|
||||
|
||||
```bash
|
||||
npm run test:docker-couchdb:start
|
||||
npm run test:docker-s3:start
|
||||
npm run test:e2e:obsidian:local-suite
|
||||
```
|
||||
|
||||
Or let the wrapper manage both fixtures:
|
||||
|
||||
```bash
|
||||
npm run test:e2e:obsidian:local-suite:services
|
||||
```
|
||||
|
||||
Useful environment variables:
|
||||
|
||||
- `OBSIDIAN_BINARY`: explicit Obsidian executable path.
|
||||
- `E2E_OBSIDIAN_VERSION`: Obsidian AppImage version for `test:e2e:obsidian:install-appimage`; default is `1.12.7`.
|
||||
- `E2E_OBSIDIAN_APPIMAGE_ARCH`: AppImage architecture override, such as `arm64` or `x86_64`.
|
||||
- `E2E_OBSIDIAN_APPIMAGE_URL`: explicit AppImage URL override.
|
||||
- `E2E_OBSIDIAN_DOWNLOAD_DIR`: AppImage download and extraction directory; default is `_testdata/obsidian`.
|
||||
- `E2E_OBSIDIAN_FORCE_DOWNLOAD=true`: re-download the AppImage even when it exists.
|
||||
- `E2E_OBSIDIAN_SKIP_EXTRACT=true`: download the AppImage without extracting it.
|
||||
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS`: smoke timeout in milliseconds.
|
||||
- `E2E_OBSIDIAN_READY_TIMEOUT_MS`: plug-in readiness timeout in milliseconds.
|
||||
- `E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS`: timeout for waiting until the vault-side Obsidian CLI exposes the plug-in catalogue.
|
||||
- `E2E_OBSIDIAN_CLI_TIMEOUT_MS`: timeout for each `obsidian-cli` invocation.
|
||||
- `E2E_OBSIDIAN_FILE_TIMEOUT_MS`: timeout for waiting until a note created through Obsidian's vault API is reflected to disk.
|
||||
- `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready.
|
||||
- `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database.
|
||||
- `E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS`: timeout for waiting until CouchDB contains uploaded E2E documents.
|
||||
- `E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS`: timeout for waiting until Object Storage contains uploaded E2E objects.
|
||||
- `E2E_OBSIDIAN_KEEP_COUCHDB=true`: keep the temporary CouchDB database for inspection.
|
||||
- `E2E_OBSIDIAN_KEEP_OBJECT_STORAGE=true`: keep the temporary Object Storage prefix for inspection.
|
||||
- `E2E_OBSIDIAN_STARTUP_GRACE_MS`: early process-exit detection window in milliseconds.
|
||||
- `E2E_OBSIDIAN_KEEP_VAULT=true`: keep the temporary vault for inspection.
|
||||
- `E2E_OBSIDIAN_USE_XVFB=false`: disable automatic `xvfb-run` on headless Linux.
|
||||
- `E2E_OBSIDIAN_USE_USER_DATA_DIR=false`: disable the isolated Electron `--user-data-dir` argument. This is not recommended for normal local testing.
|
||||
- `E2E_OBSIDIAN_ARGS`: override the default Obsidian launch arguments.
|
||||
|
||||
On headless Linux, the runner automatically uses `/usr/bin/xvfb-run` when no `DISPLAY` or `WAYLAND_DISPLAY` is present.
|
||||
@@ -0,0 +1,103 @@
|
||||
import { spawn } from "node:child_process";
|
||||
|
||||
export type ObsidianCliResult = {
|
||||
code: number | null;
|
||||
signal: NodeJS.Signals | null;
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
};
|
||||
|
||||
function parseEvalJson(stdout: string): unknown {
|
||||
const marker = "=> ";
|
||||
const markerIndex = stdout.indexOf(marker);
|
||||
const text = markerIndex >= 0 ? stdout.slice(markerIndex + marker.length) : stdout;
|
||||
return JSON.parse(text.trim());
|
||||
}
|
||||
|
||||
export async function runObsidianCli(
|
||||
cliBinary: string,
|
||||
args: string[],
|
||||
env: NodeJS.ProcessEnv = process.env,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ?? 10000)
|
||||
): Promise<ObsidianCliResult> {
|
||||
return await new Promise((resolve, reject) => {
|
||||
const child = spawn(cliBinary, args, {
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
env,
|
||||
});
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const timeout = setTimeout(() => {
|
||||
child.kill("SIGKILL");
|
||||
reject(new Error(`Obsidian CLI timed out: ${cliBinary} ${args.join(" ")}`));
|
||||
}, timeoutMs);
|
||||
|
||||
child.stdout?.on("data", (chunk: Buffer) => {
|
||||
stdout += chunk.toString();
|
||||
});
|
||||
child.stderr?.on("data", (chunk: Buffer) => {
|
||||
stderr += chunk.toString();
|
||||
});
|
||||
child.on("error", (error) => {
|
||||
clearTimeout(timeout);
|
||||
reject(error);
|
||||
});
|
||||
child.on("exit", (code, signal) => {
|
||||
clearTimeout(timeout);
|
||||
resolve({ code, signal, stdout, stderr });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
export async function openVaultWithObsidianCli(
|
||||
cliBinary: string,
|
||||
vaultPath: string,
|
||||
env: NodeJS.ProcessEnv = process.env
|
||||
): Promise<void> {
|
||||
const result = await runObsidianCli(cliBinary, [`obsidian://open?path=${encodeURIComponent(vaultPath)}`], env);
|
||||
if (result.code !== 0) {
|
||||
throw new Error(
|
||||
[
|
||||
`Failed to open Obsidian vault through CLI. code=${result.code}, signal=${result.signal}`,
|
||||
result.stdout ? `stdout:\n${result.stdout}` : undefined,
|
||||
result.stderr ? `stderr:\n${result.stderr}` : undefined,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export async function evalObsidianJson<T>(
|
||||
cliBinary: string,
|
||||
code: string,
|
||||
env: NodeJS.ProcessEnv = process.env,
|
||||
timeoutMs?: number
|
||||
): Promise<T> {
|
||||
const result = await runObsidianCli(cliBinary, ["eval", `code=${code}`], env, timeoutMs);
|
||||
if (result.code !== 0) {
|
||||
throw new Error(
|
||||
[
|
||||
`Failed to evaluate Obsidian JavaScript through CLI. code=${result.code}, signal=${result.signal}`,
|
||||
result.stdout ? `stdout:\n${result.stdout}` : undefined,
|
||||
result.stderr ? `stderr:\n${result.stderr}` : undefined,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
);
|
||||
}
|
||||
try {
|
||||
return parseEvalJson(result.stdout) as T;
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
[
|
||||
`Failed to parse Obsidian CLI eval JSON. code=${result.code}, signal=${result.signal}`,
|
||||
error instanceof Error ? `parse error: ${error.message}` : undefined,
|
||||
result.stdout ? `stdout:\n${result.stdout}` : undefined,
|
||||
result.stderr ? `stderr:\n${result.stderr}` : undefined,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
export type CouchDbConfig = {
|
||||
uri: string;
|
||||
username: string;
|
||||
password: string;
|
||||
dbPrefix: string;
|
||||
};
|
||||
|
||||
export type CouchDbDocument = {
|
||||
_id: string;
|
||||
_rev?: string;
|
||||
type?: string;
|
||||
path?: string;
|
||||
children?: string[];
|
||||
[key: string]: unknown;
|
||||
};
|
||||
|
||||
export type CouchDbAllDocsResponse = {
|
||||
rows: Array<{
|
||||
id: string;
|
||||
key: string;
|
||||
value: { rev: string; deleted?: boolean };
|
||||
doc?: CouchDbDocument;
|
||||
}>;
|
||||
};
|
||||
|
||||
function parseEnvFile(content: string): Record<string, string> {
|
||||
const entries = content
|
||||
.split(/\r?\n/u)
|
||||
.map((line) => line.trim())
|
||||
.filter((line) => line && !line.startsWith("#"))
|
||||
.map((line) => {
|
||||
const equalsAt = line.indexOf("=");
|
||||
if (equalsAt < 0) {
|
||||
return undefined;
|
||||
}
|
||||
const key = line.slice(0, equalsAt).trim();
|
||||
const rawValue = line.slice(equalsAt + 1).trim();
|
||||
const value = rawValue.replace(/^['"]|['"]$/gu, "");
|
||||
return [key, value] as const;
|
||||
})
|
||||
.filter((entry): entry is readonly [string, string] => entry !== undefined);
|
||||
return Object.fromEntries(entries);
|
||||
}
|
||||
|
||||
function getEnvValue(values: Record<string, string | undefined>, ...keys: string[]): string {
|
||||
for (const key of keys) {
|
||||
const value = values[key]?.trim();
|
||||
if (value) {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
throw new Error(`Required CouchDB environment value is missing: ${keys.join(" or ")}`);
|
||||
}
|
||||
|
||||
function authHeader(config: Pick<CouchDbConfig, "username" | "password">): string {
|
||||
return `Basic ${Buffer.from(`${config.username}:${config.password}`).toString("base64")}`;
|
||||
}
|
||||
|
||||
function databaseUrl(config: Pick<CouchDbConfig, "uri">, dbName: string, suffix = ""): string {
|
||||
return `${config.uri.replace(/\/+$/u, "")}/${encodeURIComponent(dbName)}${suffix}`;
|
||||
}
|
||||
|
||||
async function couchDbRequest(
|
||||
config: Pick<CouchDbConfig, "uri" | "username" | "password">,
|
||||
path: string,
|
||||
init: RequestInit = {}
|
||||
): Promise<Response> {
|
||||
const response = await fetch(`${config.uri.replace(/\/+$/u, "")}${path}`, {
|
||||
...init,
|
||||
headers: {
|
||||
authorization: authHeader(config),
|
||||
...init.headers,
|
||||
},
|
||||
});
|
||||
return response;
|
||||
}
|
||||
|
||||
export async function loadCouchDbConfig(envFile = ".test.env"): Promise<CouchDbConfig> {
|
||||
let fileValues: Record<string, string> = {};
|
||||
try {
|
||||
fileValues = parseEnvFile(await readFile(resolve(envFile), "utf-8"));
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
const values = { ...fileValues, ...process.env };
|
||||
return {
|
||||
uri: getEnvValue(values, "COUCHDB_URI", "hostname").replace(/\/+$/u, ""),
|
||||
username: getEnvValue(values, "COUCHDB_USER", "username"),
|
||||
password: getEnvValue(values, "COUCHDB_PASSWORD", "password"),
|
||||
dbPrefix: getEnvValue(values, "COUCHDB_DBNAME", "dbname"),
|
||||
};
|
||||
}
|
||||
|
||||
export function makeUniqueDatabaseName(prefix: string, label: string): string {
|
||||
const safePrefix = prefix
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9_$()+/-]+/gu, "-")
|
||||
.replace(/^-+/u, "")
|
||||
.slice(0, 80);
|
||||
const random = Math.random().toString(36).slice(2, 8);
|
||||
return `${safePrefix || "livesync-e2e"}-${label}-${Date.now()}-${random}`;
|
||||
}
|
||||
|
||||
export async function assertCouchDbReachable(config: CouchDbConfig): Promise<void> {
|
||||
const response = await couchDbRequest(config, "/_up");
|
||||
if (!response.ok) {
|
||||
throw new Error(`CouchDB is not reachable at ${config.uri}. HTTP ${response.status}: ${await response.text()}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function createCouchDbDatabase(config: CouchDbConfig, dbName: string): Promise<void> {
|
||||
const response = await fetch(databaseUrl(config, dbName), {
|
||||
method: "PUT",
|
||||
headers: { authorization: authHeader(config) },
|
||||
});
|
||||
if (!response.ok && response.status !== 412) {
|
||||
throw new Error(
|
||||
`Failed to create CouchDB database ${dbName}. HTTP ${response.status}: ${await response.text()}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export async function deleteCouchDbDatabase(config: CouchDbConfig, dbName: string): Promise<void> {
|
||||
const response = await fetch(databaseUrl(config, dbName), {
|
||||
method: "DELETE",
|
||||
headers: { authorization: authHeader(config) },
|
||||
});
|
||||
if (!response.ok && response.status !== 404) {
|
||||
throw new Error(
|
||||
`Failed to delete CouchDB database ${dbName}. HTTP ${response.status}: ${await response.text()}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchAllCouchDbDocs(config: CouchDbConfig, dbName: string): Promise<CouchDbAllDocsResponse> {
|
||||
const response = await fetch(databaseUrl(config, dbName, "/_all_docs?include_docs=true"), {
|
||||
headers: { authorization: authHeader(config) },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`Failed to read CouchDB documents from ${dbName}. HTTP ${response.status}: ${await response.text()}`
|
||||
);
|
||||
}
|
||||
return (await response.json()) as CouchDbAllDocsResponse;
|
||||
}
|
||||
|
||||
export async function waitForCouchDbDocs(
|
||||
config: CouchDbConfig,
|
||||
dbName: string,
|
||||
predicate: (docs: CouchDbDocument[]) => boolean,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ?? 15000)
|
||||
): Promise<CouchDbDocument[]> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastDocs: CouchDbDocument[] = [];
|
||||
while (Date.now() < deadline) {
|
||||
const response = await fetchAllCouchDbDocs(config, dbName);
|
||||
lastDocs = response.rows.flatMap((row) => (row.doc ? [row.doc] : []));
|
||||
if (predicate(lastDocs)) {
|
||||
return lastDocs;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
}
|
||||
throw new Error(
|
||||
`Timed out waiting for CouchDB documents in ${dbName}. Last document IDs: ${lastDocs
|
||||
.map((doc) => doc._id)
|
||||
.join(", ")}`
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
import { accessSync, constants, existsSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { platform } from "node:process";
|
||||
|
||||
export type ObsidianDiscoveryResult = {
|
||||
binary?: string;
|
||||
source?: string;
|
||||
checked: string[];
|
||||
};
|
||||
|
||||
const defaultCandidatesByPlatform: Record<NodeJS.Platform, string[]> = {
|
||||
aix: [],
|
||||
android: [],
|
||||
darwin: [
|
||||
"/Applications/Obsidian.app/Contents/MacOS/Obsidian",
|
||||
"/Applications/Obsidian.app/Contents/MacOS/obsidian",
|
||||
],
|
||||
freebsd: [],
|
||||
haiku: [],
|
||||
linux: [
|
||||
"_testdata/obsidian/squashfs-root/obsidian",
|
||||
"_testdata/obsidian/squashfs-root/AppRun",
|
||||
"_testdata/obsidian/Obsidian-1.12.7-arm64.AppImage",
|
||||
"_testdata/obsidian/Obsidian-1.12.7-x86_64.AppImage",
|
||||
"/usr/bin/obsidian",
|
||||
"/usr/local/bin/obsidian",
|
||||
"/snap/bin/obsidian",
|
||||
"/opt/Obsidian/obsidian",
|
||||
"/opt/obsidian/obsidian",
|
||||
"/app/bin/obsidian",
|
||||
],
|
||||
openbsd: [],
|
||||
sunos: [],
|
||||
win32: ["C:\\Program Files\\Obsidian\\Obsidian.exe", "C:\\Program Files (x86)\\Obsidian\\Obsidian.exe"],
|
||||
cygwin: [],
|
||||
netbsd: [],
|
||||
};
|
||||
|
||||
const defaultCliCandidatesByPlatform: Record<NodeJS.Platform, string[]> = {
|
||||
aix: [],
|
||||
android: [],
|
||||
darwin: [
|
||||
"/Applications/Obsidian.app/Contents/MacOS/obsidian-cli",
|
||||
"/Applications/Obsidian.app/Contents/Resources/obsidian-cli",
|
||||
],
|
||||
freebsd: [],
|
||||
haiku: [],
|
||||
linux: [
|
||||
"_testdata/obsidian/squashfs-root/obsidian-cli",
|
||||
"/usr/bin/obsidian-cli",
|
||||
"/usr/local/bin/obsidian-cli",
|
||||
"/snap/bin/obsidian-cli",
|
||||
"/opt/Obsidian/obsidian-cli",
|
||||
"/opt/obsidian/obsidian-cli",
|
||||
],
|
||||
openbsd: [],
|
||||
sunos: [],
|
||||
win32: ["C:\\Program Files\\Obsidian\\obsidian-cli.exe", "C:\\Program Files (x86)\\Obsidian\\obsidian-cli.exe"],
|
||||
cygwin: [],
|
||||
netbsd: [],
|
||||
};
|
||||
|
||||
function isUsableFile(path: string): boolean {
|
||||
const resolvedPath = resolve(path);
|
||||
if (!existsSync(resolvedPath)) {
|
||||
return false;
|
||||
}
|
||||
if (platform === "win32") {
|
||||
return true;
|
||||
}
|
||||
try {
|
||||
accessSync(resolvedPath, constants.X_OK);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function discoverObsidianBinary(env: NodeJS.ProcessEnv = process.env): ObsidianDiscoveryResult {
|
||||
const checked: string[] = [];
|
||||
const envBinary = env.OBSIDIAN_BINARY?.trim();
|
||||
if (envBinary) {
|
||||
checked.push(envBinary);
|
||||
if (isUsableFile(envBinary)) {
|
||||
return {
|
||||
binary: resolve(envBinary),
|
||||
source: "OBSIDIAN_BINARY",
|
||||
checked,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const candidates = defaultCandidatesByPlatform[platform] ?? [];
|
||||
for (const candidate of candidates) {
|
||||
checked.push(candidate);
|
||||
if (isUsableFile(candidate)) {
|
||||
return {
|
||||
binary: resolve(candidate),
|
||||
source: "default-path",
|
||||
checked,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return { checked };
|
||||
}
|
||||
|
||||
export function requireObsidianBinary(env: NodeJS.ProcessEnv = process.env): string {
|
||||
const result = discoverObsidianBinary(env);
|
||||
if (!result.binary) {
|
||||
throw new Error(
|
||||
[
|
||||
"Could not find an Obsidian executable.",
|
||||
"Set OBSIDIAN_BINARY to the installed Obsidian executable path.",
|
||||
`Checked paths: ${result.checked.length > 0 ? result.checked.join(", ") : "(none)"}`,
|
||||
].join("\n")
|
||||
);
|
||||
}
|
||||
return result.binary;
|
||||
}
|
||||
|
||||
export function discoverObsidianCli(env: NodeJS.ProcessEnv = process.env): ObsidianDiscoveryResult {
|
||||
const checked: string[] = [];
|
||||
const envBinary = env.OBSIDIAN_CLI?.trim();
|
||||
if (envBinary) {
|
||||
checked.push(envBinary);
|
||||
if (isUsableFile(envBinary)) {
|
||||
return {
|
||||
binary: resolve(envBinary),
|
||||
source: "OBSIDIAN_CLI",
|
||||
checked,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const candidates = defaultCliCandidatesByPlatform[platform] ?? [];
|
||||
for (const candidate of candidates) {
|
||||
checked.push(candidate);
|
||||
if (isUsableFile(candidate)) {
|
||||
return {
|
||||
binary: resolve(candidate),
|
||||
source: "default-path",
|
||||
checked,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return { checked };
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
import { execFile, spawn, type ChildProcess } from "node:child_process";
|
||||
import { once } from "node:events";
|
||||
import { existsSync } from "node:fs";
|
||||
import { dirname } from "node:path";
|
||||
import { platform } from "node:process";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
export type ObsidianProcess = {
|
||||
process: ChildProcess;
|
||||
output: () => { stdout: string; stderr: string };
|
||||
stop: () => Promise<void>;
|
||||
};
|
||||
|
||||
export type LaunchObsidianOptions = {
|
||||
binary: string;
|
||||
vaultPath: string;
|
||||
homePath?: string;
|
||||
xdgConfigPath?: string;
|
||||
xdgCachePath?: string;
|
||||
xdgDataPath?: string;
|
||||
userDataPath?: string;
|
||||
startupGraceMs?: number;
|
||||
};
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
function splitArgs(args: string): string[] {
|
||||
return args.split(" ").filter((arg) => arg.length > 0);
|
||||
}
|
||||
|
||||
function launchArgs(options: LaunchObsidianOptions): string[] {
|
||||
const explicitArgs = process.env.E2E_OBSIDIAN_ARGS;
|
||||
if (explicitArgs) {
|
||||
return splitArgs(explicitArgs);
|
||||
}
|
||||
return [
|
||||
"--no-sandbox",
|
||||
"--disable-gpu",
|
||||
"--disable-software-rasterizer",
|
||||
...(process.env.E2E_OBSIDIAN_USE_USER_DATA_DIR !== "false" && options.userDataPath
|
||||
? [`--user-data-dir=${options.userDataPath}`]
|
||||
: []),
|
||||
...(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT
|
||||
? [`--remote-debugging-port=${process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT}`]
|
||||
: []),
|
||||
`obsidian://open?path=${encodeURIComponent(options.vaultPath)}`,
|
||||
];
|
||||
}
|
||||
|
||||
function shouldUseXvfb(): boolean {
|
||||
if (process.env.E2E_OBSIDIAN_USE_XVFB === "false") {
|
||||
return false;
|
||||
}
|
||||
if (process.env.DISPLAY || process.env.WAYLAND_DISPLAY) {
|
||||
return false;
|
||||
}
|
||||
return platform === "linux" && existsSync("/usr/bin/xvfb-run");
|
||||
}
|
||||
|
||||
async function listChildPids(pid: number): Promise<number[]> {
|
||||
if (platform === "win32") {
|
||||
return [];
|
||||
}
|
||||
const { stdout } = await execFileAsync("ps", ["-o", "pid=", "--ppid", String(pid)]).catch(() => ({
|
||||
stdout: "",
|
||||
}));
|
||||
const directChildren = stdout
|
||||
.split("\n")
|
||||
.map((line) => Number(line.trim()))
|
||||
.filter((childPid) => Number.isInteger(childPid) && childPid > 0);
|
||||
const descendants = await Promise.all(directChildren.map((childPid) => listChildPids(childPid)));
|
||||
return [...directChildren, ...descendants.flat()];
|
||||
}
|
||||
|
||||
async function killPids(pids: number[], signal: NodeJS.Signals): Promise<void> {
|
||||
for (const pid of pids) {
|
||||
if (pid === process.pid) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
process.kill(pid, signal);
|
||||
} catch {
|
||||
// The process may have exited between discovery and signalling.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForExit(exitPromise: Promise<unknown>, timeoutMs: number): Promise<"exited" | "timeout"> {
|
||||
const stopTimer = new Promise<"timeout">((resolve) => {
|
||||
setTimeout(() => resolve("timeout"), timeoutMs);
|
||||
});
|
||||
const stopResult = await Promise.race([exitPromise.then(() => "exited" as const), stopTimer]);
|
||||
return stopResult;
|
||||
}
|
||||
|
||||
export async function cleanupStaleObsidianE2EProcesses(): Promise<void> {
|
||||
if (process.env.E2E_OBSIDIAN_CLEANUP_STALE_PROCESSES === "false" || platform === "win32") {
|
||||
return;
|
||||
}
|
||||
const { stdout } = await execFileAsync("pgrep", ["-f", "obsidian-livesync-e2e-state"]).catch(() => ({
|
||||
stdout: "",
|
||||
}));
|
||||
const pids = stdout
|
||||
.split("\n")
|
||||
.map((line) => Number(line.trim()))
|
||||
.filter((pid) => Number.isInteger(pid) && pid > 0 && pid !== process.pid);
|
||||
if (pids.length === 0) {
|
||||
return;
|
||||
}
|
||||
await killPids(pids, "SIGTERM");
|
||||
await new Promise((resolve) => setTimeout(resolve, 1000));
|
||||
await killPids(pids, "SIGKILL");
|
||||
}
|
||||
|
||||
export async function launchObsidian(options: LaunchObsidianOptions): Promise<ObsidianProcess> {
|
||||
await cleanupStaleObsidianE2EProcesses();
|
||||
const startupGraceMs = options.startupGraceMs ?? 1000;
|
||||
const args = launchArgs(options);
|
||||
const useXvfb = shouldUseXvfb();
|
||||
const command = useXvfb ? "/usr/bin/xvfb-run" : options.binary;
|
||||
const commandArgs = useXvfb ? ["-a", options.binary, ...args] : args;
|
||||
const child = spawn(command, commandArgs, {
|
||||
cwd: dirname(options.binary),
|
||||
detached: true,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
env: {
|
||||
...process.env,
|
||||
...(options.homePath ? { HOME: options.homePath } : {}),
|
||||
...(options.xdgConfigPath ? { XDG_CONFIG_HOME: options.xdgConfigPath } : {}),
|
||||
...(options.xdgCachePath ? { XDG_CACHE_HOME: options.xdgCachePath } : {}),
|
||||
...(options.xdgDataPath ? { XDG_DATA_HOME: options.xdgDataPath } : {}),
|
||||
OBSIDIAN_DISABLE_GPU: process.env.OBSIDIAN_DISABLE_GPU ?? "1",
|
||||
},
|
||||
});
|
||||
|
||||
let stderr = "";
|
||||
let stdout = "";
|
||||
child.stderr?.on("data", (chunk: Buffer) => {
|
||||
stderr += chunk.toString();
|
||||
});
|
||||
child.stdout?.on("data", (chunk: Buffer) => {
|
||||
stdout += chunk.toString();
|
||||
});
|
||||
|
||||
const exitPromise = once(child, "exit").then(([code, signal]) => ({ code, signal }));
|
||||
const timer = new Promise<"timeout">((resolve) => {
|
||||
setTimeout(() => resolve("timeout"), startupGraceMs);
|
||||
});
|
||||
const firstResult = await Promise.race([exitPromise, timer]);
|
||||
if (firstResult !== "timeout") {
|
||||
throw new Error(
|
||||
[
|
||||
`Obsidian exited before the smoke timeout. code=${firstResult.code}, signal=${firstResult.signal}`,
|
||||
stdout ? `stdout:\n${stdout}` : undefined,
|
||||
stderr ? `stderr:\n${stderr}` : undefined,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
process: child,
|
||||
output: () => ({ stdout, stderr }),
|
||||
stop: async () => {
|
||||
if (child.exitCode !== null || child.signalCode !== null) {
|
||||
return;
|
||||
}
|
||||
const descendantPids = child.pid ? await listChildPids(child.pid) : [];
|
||||
if (child.pid) {
|
||||
try {
|
||||
process.kill(-child.pid, "SIGTERM");
|
||||
} catch {
|
||||
child.kill("SIGTERM");
|
||||
}
|
||||
} else {
|
||||
child.kill("SIGTERM");
|
||||
}
|
||||
await killPids(descendantPids.reverse(), "SIGTERM");
|
||||
const stopResult = await waitForExit(exitPromise, 5000);
|
||||
if (stopResult === "timeout") {
|
||||
if (child.pid) {
|
||||
try {
|
||||
process.kill(-child.pid, "SIGKILL");
|
||||
} catch {
|
||||
child.kill("SIGKILL");
|
||||
}
|
||||
} else {
|
||||
child.kill("SIGKILL");
|
||||
}
|
||||
await killPids(descendantPids, "SIGKILL");
|
||||
await exitPromise;
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,247 @@
|
||||
import { evalObsidianJson } from "./cli.ts";
|
||||
import type { CouchDbConfig } from "./couchdb.ts";
|
||||
import type { ObjectStorageConfig } from "./objectStorage.ts";
|
||||
|
||||
export type ConfiguredSettings = {
|
||||
isConfigured: boolean;
|
||||
liveSync: boolean;
|
||||
syncOnStart: boolean;
|
||||
syncOnSave: boolean;
|
||||
remoteType: string;
|
||||
couchDB_URI: string;
|
||||
couchDB_DBNAME: string;
|
||||
endpoint?: string;
|
||||
bucket?: string;
|
||||
bucketPrefix?: string;
|
||||
};
|
||||
|
||||
export type CoreReadiness = {
|
||||
databaseReady: boolean;
|
||||
appReady: boolean;
|
||||
};
|
||||
|
||||
export type LocalDatabaseEntry = {
|
||||
id: string;
|
||||
rev: string;
|
||||
path: string;
|
||||
type: string;
|
||||
children: string[];
|
||||
};
|
||||
|
||||
function e2ePreferredSettingsSource(): string[] {
|
||||
return [
|
||||
"liveSync:false,",
|
||||
"syncOnStart:false,",
|
||||
"syncOnSave:false,",
|
||||
"usePluginSync:false,",
|
||||
"usePluginSyncV2:true,",
|
||||
"useEden:false,",
|
||||
"customChunkSize:60,",
|
||||
"sendChunksBulk:false,",
|
||||
"sendChunksBulkMaxSize:1,",
|
||||
"chunkSplitterVersion:'v3-rabin-karp',",
|
||||
"readChunksOnline:true,",
|
||||
"disableCheckingConfigMismatch:false,",
|
||||
"enableCompression:false,",
|
||||
"hashAlg:'xxhash64',",
|
||||
"handleFilenameCaseSensitive:false,",
|
||||
"doNotUseFixedRevisionForChunks:true,",
|
||||
"E2EEAlgorithm:'v2',",
|
||||
"doctorProcessedVersion:'0.25.27',",
|
||||
"isConfigured:true,",
|
||||
];
|
||||
}
|
||||
|
||||
export function assertEqual(actual: unknown, expected: unknown, message: string): void {
|
||||
if (actual !== expected) {
|
||||
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function configureCouchDb(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
settings: Pick<CouchDbConfig, "uri" | "username" | "password"> & { dbName: string },
|
||||
overrides: Record<string, unknown> = {}
|
||||
): Promise<ConfiguredSettings> {
|
||||
return await evalObsidianJson<ConfiguredSettings>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const plugin=app.plugins.plugins['obsidian-livesync'];",
|
||||
"const core=plugin.core;",
|
||||
"const nextSettings={",
|
||||
`couchDB_URI:${JSON.stringify(settings.uri)},`,
|
||||
`couchDB_USER:${JSON.stringify(settings.username)},`,
|
||||
`couchDB_PASSWORD:${JSON.stringify(settings.password)},`,
|
||||
`couchDB_DBNAME:${JSON.stringify(settings.dbName)},`,
|
||||
"remoteType:'',",
|
||||
...e2ePreferredSettingsSource(),
|
||||
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
|
||||
"};",
|
||||
"await core.services.setting.applyExternalSettings(nextSettings,true);",
|
||||
"await core.services.control.applySettings();",
|
||||
"const current=core.services.setting.currentSettings();",
|
||||
"return JSON.stringify({",
|
||||
"isConfigured:current.isConfigured,",
|
||||
"liveSync:current.liveSync,",
|
||||
"syncOnStart:current.syncOnStart,",
|
||||
"syncOnSave:current.syncOnSave,",
|
||||
"remoteType:current.remoteType,",
|
||||
"couchDB_URI:current.couchDB_URI,",
|
||||
"couchDB_DBNAME:current.couchDB_DBNAME,",
|
||||
"});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
export async function configureObjectStorage(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
settings: ObjectStorageConfig & { bucketPrefix: string },
|
||||
overrides: Record<string, unknown> = {}
|
||||
): Promise<ConfiguredSettings> {
|
||||
return await evalObsidianJson<ConfiguredSettings>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const plugin=app.plugins.plugins['obsidian-livesync'];",
|
||||
"const core=plugin.core;",
|
||||
"const nextSettings={",
|
||||
"remoteType:'MINIO',",
|
||||
`endpoint:${JSON.stringify(settings.endpoint)},`,
|
||||
`accessKey:${JSON.stringify(settings.accessKey)},`,
|
||||
`secretKey:${JSON.stringify(settings.secretKey)},`,
|
||||
`bucket:${JSON.stringify(settings.bucket)},`,
|
||||
`region:${JSON.stringify(settings.region)},`,
|
||||
`forcePathStyle:${JSON.stringify(settings.forcePathStyle)},`,
|
||||
`bucketPrefix:${JSON.stringify(settings.bucketPrefix)},`,
|
||||
"bucketCustomHeaders:'',",
|
||||
...e2ePreferredSettingsSource(),
|
||||
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
|
||||
"};",
|
||||
"await core.services.setting.applyExternalSettings(nextSettings,true);",
|
||||
"await core.services.control.applySettings();",
|
||||
"const current=core.services.setting.currentSettings();",
|
||||
"return JSON.stringify({",
|
||||
"isConfigured:current.isConfigured,",
|
||||
"liveSync:current.liveSync,",
|
||||
"syncOnStart:current.syncOnStart,",
|
||||
"syncOnSave:current.syncOnSave,",
|
||||
"remoteType:current.remoteType,",
|
||||
"couchDB_URI:current.couchDB_URI,",
|
||||
"couchDB_DBNAME:current.couchDB_DBNAME,",
|
||||
"endpoint:current.endpoint,",
|
||||
"bucket:current.bucket,",
|
||||
"bucketPrefix:current.bucketPrefix,",
|
||||
"});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
export async function waitForLiveSyncCoreReady(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS ?? 20000)
|
||||
): Promise<CoreReadiness> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastReadiness: CoreReadiness | undefined;
|
||||
while (Date.now() < deadline) {
|
||||
lastReadiness = await evalObsidianJson<CoreReadiness>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"return JSON.stringify({",
|
||||
"databaseReady:core.services.database.isDatabaseReady(),",
|
||||
"appReady:core.services.appLifecycle.isReady(),",
|
||||
"});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
if (lastReadiness.databaseReady && lastReadiness.appReady) {
|
||||
return lastReadiness;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
}
|
||||
throw new Error(`Timed out waiting for Self-hosted LiveSync core readiness: ${JSON.stringify(lastReadiness)}`);
|
||||
}
|
||||
|
||||
export async function prepareRemote(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const settings=core.services.setting.currentSettings();",
|
||||
"const replicator=core.services.replicator.getActiveReplicator();",
|
||||
"await replicator.tryCreateRemoteDatabase(settings);",
|
||||
"await replicator.markRemoteResolved(settings);",
|
||||
"const status=await replicator.getRemoteStatus(settings);",
|
||||
"return JSON.stringify({status});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
export async function pushLocalChanges(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"await core.services.fileProcessing.commitPendingFileEvents();",
|
||||
"const result=await core.services.replication.replicate(true);",
|
||||
"return JSON.stringify({result:!!result});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
export async function waitForLocalDatabaseEntry(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
path: string,
|
||||
options: { hidden?: boolean; timeoutMs?: number } = {}
|
||||
): Promise<LocalDatabaseEntry> {
|
||||
const timeoutMs = options.timeoutMs ?? Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000);
|
||||
return await evalObsidianJson<LocalDatabaseEntry>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
`const hidden=${JSON.stringify(options.hidden === true)};`,
|
||||
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const deadline=Date.now()+timeoutMs;",
|
||||
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
|
||||
"let entry=false;",
|
||||
"while(Date.now()<deadline){",
|
||||
"await core.services.fileProcessing.commitPendingFileEvents();",
|
||||
"const dbPath=hidden?`i:${path}`:path;",
|
||||
"entry=await core.localDatabase.getDBEntry(dbPath,undefined,false,true).catch(()=>false);",
|
||||
"if(!entry||!entry._id){",
|
||||
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
"entry=rows.map((row)=>row.doc).find((doc)=>doc&&(",
|
||||
"doc._id===dbPath||doc._id===path||doc.path===dbPath||doc.path===path||",
|
||||
"(typeof doc.path==='string'&&doc.path.endsWith(path))||",
|
||||
"(typeof doc._id==='string'&&doc._id.endsWith(path))",
|
||||
"))||false;",
|
||||
"}",
|
||||
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
|
||||
"await sleep(250);",
|
||||
"}",
|
||||
"if(!entry||!entry._id) throw new Error(`Timed out waiting for local database entry: ${path}`);",
|
||||
"return JSON.stringify({id:entry._id,rev:entry._rev,path:entry.path,type:entry.type,children:entry.children||[]});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
import {
|
||||
CreateBucketCommand,
|
||||
DeleteObjectsCommand,
|
||||
ListObjectsV2Command,
|
||||
S3Client,
|
||||
type _Object,
|
||||
} from "@aws-sdk/client-s3";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
export type ObjectStorageConfig = {
|
||||
endpoint: string;
|
||||
accessKey: string;
|
||||
secretKey: string;
|
||||
bucket: string;
|
||||
region: string;
|
||||
forcePathStyle: boolean;
|
||||
};
|
||||
|
||||
function parseEnvFile(content: string): Record<string, string> {
|
||||
const entries = content
|
||||
.split(/\r?\n/u)
|
||||
.map((line) => line.trim())
|
||||
.filter((line) => line && !line.startsWith("#"))
|
||||
.map((line) => {
|
||||
const equalsAt = line.indexOf("=");
|
||||
if (equalsAt < 0) {
|
||||
return undefined;
|
||||
}
|
||||
const key = line.slice(0, equalsAt).trim();
|
||||
const rawValue = line.slice(equalsAt + 1).trim();
|
||||
const value = rawValue.replace(/^['"]|['"]$/gu, "");
|
||||
return [key, value] as const;
|
||||
})
|
||||
.filter((entry): entry is readonly [string, string] => entry !== undefined);
|
||||
return Object.fromEntries(entries);
|
||||
}
|
||||
|
||||
function getEnvValue(values: Record<string, string | undefined>, ...keys: string[]): string {
|
||||
for (const key of keys) {
|
||||
const value = values[key]?.trim();
|
||||
if (value) {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
throw new Error(`Required Object Storage environment value is missing: ${keys.join(" or ")}`);
|
||||
}
|
||||
|
||||
export async function loadObjectStorageConfig(envFile = ".test.env"): Promise<ObjectStorageConfig> {
|
||||
let fileValues: Record<string, string> = {};
|
||||
try {
|
||||
fileValues = parseEnvFile(await readFile(resolve(envFile), "utf-8"));
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
const values = { ...fileValues, ...process.env };
|
||||
return {
|
||||
endpoint: getEnvValue(values, "MINIO_ENDPOINT", "minioEndpoint").replace(/\/+$/u, ""),
|
||||
accessKey: getEnvValue(values, "MINIO_ACCESS_KEY", "accessKey"),
|
||||
secretKey: getEnvValue(values, "MINIO_SECRET_KEY", "secretKey"),
|
||||
bucket: getEnvValue(values, "MINIO_BUCKET", "bucketName"),
|
||||
region: values.MINIO_REGION?.trim() || values.region?.trim() || "us-east-1",
|
||||
forcePathStyle: values.MINIO_FORCE_PATH_STYLE?.trim() !== "false",
|
||||
};
|
||||
}
|
||||
|
||||
export function makeUniqueBucketPrefix(label: string): string {
|
||||
const random = Math.random().toString(36).slice(2, 8);
|
||||
return `obsidian-e2e/${label}-${Date.now()}-${random}/`;
|
||||
}
|
||||
|
||||
export function createObjectStorageClient(config: ObjectStorageConfig): S3Client {
|
||||
return new S3Client({
|
||||
endpoint: config.endpoint,
|
||||
region: config.region,
|
||||
forcePathStyle: config.forcePathStyle,
|
||||
credentials: {
|
||||
accessKeyId: config.accessKey,
|
||||
secretAccessKey: config.secretKey,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export async function ensureObjectStorageBucket(config: ObjectStorageConfig): Promise<void> {
|
||||
const client = createObjectStorageClient(config);
|
||||
try {
|
||||
await client.send(new CreateBucketCommand({ Bucket: config.bucket }));
|
||||
} catch (error) {
|
||||
const name = (error as { name?: string }).name;
|
||||
if (name !== "BucketAlreadyOwnedByYou" && name !== "BucketAlreadyExists") {
|
||||
throw error;
|
||||
}
|
||||
} finally {
|
||||
client.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
export async function listObjectStorageObjects(config: ObjectStorageConfig, prefix: string): Promise<_Object[]> {
|
||||
const client = createObjectStorageClient(config);
|
||||
try {
|
||||
const objects: _Object[] = [];
|
||||
let continuationToken: string | undefined;
|
||||
do {
|
||||
const response = await client.send(
|
||||
new ListObjectsV2Command({
|
||||
Bucket: config.bucket,
|
||||
Prefix: prefix,
|
||||
ContinuationToken: continuationToken,
|
||||
})
|
||||
);
|
||||
objects.push(...(response.Contents ?? []));
|
||||
continuationToken = response.NextContinuationToken;
|
||||
} while (continuationToken);
|
||||
return objects;
|
||||
} finally {
|
||||
client.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
export async function deleteObjectStoragePrefix(config: ObjectStorageConfig, prefix: string): Promise<void> {
|
||||
const client = createObjectStorageClient(config);
|
||||
try {
|
||||
const objects = await listObjectStorageObjects(config, prefix);
|
||||
const keys = objects.flatMap((object) => (object.Key ? [{ Key: object.Key }] : []));
|
||||
for (let index = 0; index < keys.length; index += 1000) {
|
||||
await client.send(
|
||||
new DeleteObjectsCommand({
|
||||
Bucket: config.bucket,
|
||||
Delete: {
|
||||
Objects: keys.slice(index, index + 1000),
|
||||
Quiet: true,
|
||||
},
|
||||
})
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
client.destroy();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
import { copyFile, mkdir, writeFile } from "node:fs/promises";
|
||||
import { existsSync } from "node:fs";
|
||||
import { join, resolve } from "node:path";
|
||||
|
||||
export type PluginInstallResult = {
|
||||
pluginDir: string;
|
||||
copied: string[];
|
||||
};
|
||||
|
||||
const pluginId = "obsidian-livesync";
|
||||
|
||||
export async function installBuiltPlugin(vaultPath: string, rootDir = process.cwd()): Promise<PluginInstallResult> {
|
||||
const pluginDir = join(vaultPath, ".obsidian", "plugins", pluginId);
|
||||
const copied: string[] = [];
|
||||
await mkdir(pluginDir, { recursive: true });
|
||||
|
||||
const requiredArtifacts = ["main.js", "manifest.json"];
|
||||
for (const artifact of requiredArtifacts) {
|
||||
const source = resolve(rootDir, artifact);
|
||||
if (!existsSync(source)) {
|
||||
throw new Error(`Required plug-in artifact is missing: ${source}`);
|
||||
}
|
||||
await copyFile(source, join(pluginDir, artifact));
|
||||
copied.push(artifact);
|
||||
}
|
||||
|
||||
const optionalArtifacts = ["styles.css"];
|
||||
for (const artifact of optionalArtifacts) {
|
||||
const source = resolve(rootDir, artifact);
|
||||
if (!existsSync(source)) {
|
||||
continue;
|
||||
}
|
||||
await copyFile(source, join(pluginDir, artifact));
|
||||
copied.push(artifact);
|
||||
}
|
||||
|
||||
await writeFile(join(vaultPath, ".obsidian", "community-plugins.json"), JSON.stringify([pluginId], null, 4));
|
||||
return { pluginDir, copied };
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
import { evalObsidianJson } from "./cli.ts";
|
||||
|
||||
export type PluginReadiness = {
|
||||
status: "ready";
|
||||
pluginId: string;
|
||||
pluginVersion: string;
|
||||
vaultName: string;
|
||||
};
|
||||
|
||||
export async function waitForPluginReady(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_READY_TIMEOUT_MS ?? 20000)
|
||||
): Promise<PluginReadiness> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastOutput = "";
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const readiness = await evalObsidianJson<PluginReadiness>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>JSON.stringify({",
|
||||
"status:!!app.plugins.plugins['obsidian-livesync']?'ready':'pending',",
|
||||
"pluginId:'obsidian-livesync',",
|
||||
"pluginVersion:app.plugins.manifests['obsidian-livesync']?.version,",
|
||||
"vaultName:app.vault.getName()",
|
||||
"}))()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
if (readiness.status === "ready") {
|
||||
return readiness;
|
||||
}
|
||||
} catch (error) {
|
||||
lastOutput = error instanceof Error ? error.message : String(error);
|
||||
// Keep polling until Obsidian exposes the vault-side CLI and plug-in state.
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
}
|
||||
throw new Error(`Timed out waiting for Self-hosted LiveSync readiness through Obsidian CLI.\n${lastOutput}`);
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
import { evalObsidianJson, openVaultWithObsidianCli, runObsidianCli } from "./cli.ts";
|
||||
import { launchObsidian, type ObsidianProcess } from "./launch.ts";
|
||||
import { installBuiltPlugin, type PluginInstallResult } from "./pluginInstaller.ts";
|
||||
import { waitForPluginReady, type PluginReadiness } from "./readiness.ts";
|
||||
import type { TemporaryVault } from "./vault.ts";
|
||||
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, trustVaultIfPrompted } from "./ui.ts";
|
||||
|
||||
export type ObsidianLiveSyncSession = {
|
||||
app: ObsidianProcess;
|
||||
cliEnv: NodeJS.ProcessEnv;
|
||||
install: PluginInstallResult;
|
||||
readiness: PluginReadiness;
|
||||
};
|
||||
|
||||
export type StartObsidianLiveSyncSessionOptions = {
|
||||
binary: string;
|
||||
cliBinary: string;
|
||||
vault: TemporaryVault;
|
||||
startupGraceMs?: number;
|
||||
};
|
||||
|
||||
async function waitForPluginCatalogue(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS ?? 60000);
|
||||
let lastOutput = "";
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const result = await evalObsidianJson<{ hasLiveSync: boolean }>(
|
||||
cliBinary,
|
||||
["JSON.stringify({", "hasLiveSync:!!app.plugins?.manifests?.['obsidian-livesync']", "})"].join(""),
|
||||
env
|
||||
);
|
||||
lastOutput = JSON.stringify(result);
|
||||
if (result.hasLiveSync) {
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
lastOutput = error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
}
|
||||
throw new Error(`Timed out waiting for Obsidian plug-in catalogue through CLI.\n${lastOutput}`);
|
||||
}
|
||||
|
||||
async function enableCommunityPlugins(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
const result = await runObsidianCli(cliBinary, ["eval", "code=(async()=>app.plugins.setEnable(true))()"], env);
|
||||
if (result.code !== 0 || result.stdout.includes("Error:")) {
|
||||
throw new Error(
|
||||
[
|
||||
`Failed to enable Obsidian community plug-ins through CLI. code=${result.code}, signal=${result.signal}`,
|
||||
result.stdout ? `stdout:\n${result.stdout}` : undefined,
|
||||
result.stderr ? `stderr:\n${result.stderr}` : undefined,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function reloadLiveSyncPlugin(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
const reload = await runObsidianCli(cliBinary, ["plugin:reload", "id=obsidian-livesync"], env);
|
||||
if (reload.code !== 0 || !reload.stdout.includes("Reloaded: obsidian-livesync")) {
|
||||
throw new Error(
|
||||
[
|
||||
`Failed to reload Self-hosted LiveSync through Obsidian CLI. code=${reload.code}, signal=${reload.signal}`,
|
||||
reload.stdout ? `stdout:\n${reload.stdout}` : undefined,
|
||||
reload.stderr ? `stderr:\n${reload.stderr}` : undefined,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export async function startObsidianLiveSyncSession(
|
||||
options: StartObsidianLiveSyncSessionOptions
|
||||
): Promise<ObsidianLiveSyncSession> {
|
||||
const install = await installBuiltPlugin(options.vault.path);
|
||||
const remoteDebuggingPort = obsidianRemoteDebuggingPort();
|
||||
const app = await launchObsidian({
|
||||
binary: options.binary,
|
||||
vaultPath: options.vault.path,
|
||||
homePath: options.vault.homePath,
|
||||
xdgConfigPath: options.vault.xdgConfigPath,
|
||||
xdgCachePath: options.vault.xdgCachePath,
|
||||
xdgDataPath: options.vault.xdgDataPath,
|
||||
userDataPath: options.vault.userDataPath,
|
||||
startupGraceMs: options.startupGraceMs,
|
||||
});
|
||||
const cliEnv = {
|
||||
...process.env,
|
||||
HOME: options.vault.homePath,
|
||||
XDG_CONFIG_HOME: options.vault.xdgConfigPath,
|
||||
XDG_CACHE_HOME: options.vault.xdgCachePath,
|
||||
XDG_DATA_HOME: options.vault.xdgDataPath,
|
||||
};
|
||||
|
||||
try {
|
||||
await preseedTrustedVaultState(remoteDebuggingPort, options.vault.id);
|
||||
await openVaultWithObsidianCli(options.cliBinary, options.vault.path, cliEnv);
|
||||
await trustVaultIfPrompted(remoteDebuggingPort);
|
||||
await waitForPluginCatalogue(options.cliBinary, cliEnv);
|
||||
await enableCommunityPlugins(options.cliBinary, cliEnv);
|
||||
await reloadLiveSyncPlugin(options.cliBinary, cliEnv);
|
||||
const readiness = await waitForPluginReady(options.cliBinary, cliEnv);
|
||||
return { app, cliEnv, install, readiness };
|
||||
} catch (error) {
|
||||
const output = app.output();
|
||||
await app.stop();
|
||||
throw new Error(
|
||||
[
|
||||
error instanceof Error ? error.message : String(error),
|
||||
output.stdout ? `Obsidian stdout:\n${output.stdout}` : undefined,
|
||||
output.stderr ? `Obsidian stderr:\n${output.stderr}` : undefined,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
import { chromium, type Page } from "playwright";
|
||||
|
||||
export function obsidianRemoteDebuggingPort(): number {
|
||||
const port = Number(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT ?? 9222);
|
||||
process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT = String(port);
|
||||
return port;
|
||||
}
|
||||
|
||||
async function waitForCdp(port: number): Promise<void> {
|
||||
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CDP_TIMEOUT_MS ?? 30000);
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${port}/json/version`);
|
||||
if (response.ok) {
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
// Keep polling until Obsidian exposes the debugging endpoint.
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
}
|
||||
throw new Error(`Timed out waiting for Obsidian DevTools endpoint on port ${port}`);
|
||||
}
|
||||
|
||||
export async function withObsidianPage<T>(port: number, operation: (page: Page) => Promise<T>): Promise<T> {
|
||||
await waitForCdp(port);
|
||||
const browser = await chromium.connectOverCDP(`http://127.0.0.1:${port}`);
|
||||
try {
|
||||
const context = browser.contexts()[0];
|
||||
const page = context.pages()[0] ?? (await context.waitForEvent("page", { timeout: 10000 }));
|
||||
return await operation(page);
|
||||
} finally {
|
||||
await browser.close();
|
||||
}
|
||||
}
|
||||
|
||||
export async function preseedTrustedVaultState(port: number, vaultId: string): Promise<void> {
|
||||
await withObsidianPage(port, async (page) => {
|
||||
await page.evaluate((id) => {
|
||||
localStorage.setItem(`enable-plugin-${id}`, "true");
|
||||
}, vaultId);
|
||||
await page.reload({ waitUntil: "domcontentloaded", timeout: 10000 }).catch(() => undefined);
|
||||
await page.waitForTimeout(1000);
|
||||
});
|
||||
}
|
||||
|
||||
export async function trustVaultIfPrompted(port: number): Promise<void> {
|
||||
await withObsidianPage(port, async (page) => {
|
||||
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_TRUST_PROMPT_TIMEOUT_MS ?? 30000);
|
||||
while (Date.now() < deadline) {
|
||||
const yesButton = page.getByRole("button", { name: "Yes" });
|
||||
if (await yesButton.isVisible({ timeout: 1000 }).catch(() => false)) {
|
||||
await yesButton.click();
|
||||
await page.waitForTimeout(500);
|
||||
continue;
|
||||
}
|
||||
|
||||
const trustButton = page.getByText("Trust author and enable plugins");
|
||||
if (await trustButton.isVisible({ timeout: 1000 }).catch(() => false)) {
|
||||
await trustButton.click();
|
||||
await page.waitForTimeout(500);
|
||||
continue;
|
||||
}
|
||||
|
||||
const workspace = page.locator(".workspace");
|
||||
if (await workspace.isVisible({ timeout: 1000 }).catch(() => false)) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
export async function clickJsonResolveOption(port: number, mode: "AB" | "BA"): Promise<void> {
|
||||
await withObsidianPage(port, async (page) => {
|
||||
const option = page.locator(`label:has(input[name="disp"][value="${mode}"])`);
|
||||
await option.click({ timeout: 10000 });
|
||||
const checked = await page.locator(`input[name="disp"][value="${mode}"]`).isChecked({ timeout: 10000 });
|
||||
if (!checked) {
|
||||
throw new Error(`JSON Resolve option was not selected: ${mode}`);
|
||||
}
|
||||
await page.getByRole("button", { name: "Apply" }).click({ timeout: 10000 });
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
export type TemporaryVault = {
|
||||
path: string;
|
||||
name: string;
|
||||
id: string;
|
||||
homePath: string;
|
||||
xdgConfigPath: string;
|
||||
xdgCachePath: string;
|
||||
xdgDataPath: string;
|
||||
userDataPath: string;
|
||||
dispose: () => Promise<void>;
|
||||
};
|
||||
|
||||
export async function createTemporaryVault(prefix = "obsidian-livesync-e2e-"): Promise<TemporaryVault> {
|
||||
const vaultPath = await mkdtemp(join(tmpdir(), prefix));
|
||||
const statePath = await mkdtemp(join(tmpdir(), `${prefix}state-`));
|
||||
const name = vaultPath.split(/[\\/]/).pop() ?? "obsidian-livesync-e2e";
|
||||
await mkdir(join(vaultPath, ".obsidian"), { recursive: true });
|
||||
const homePath = join(statePath, "home");
|
||||
const xdgConfigPath = join(statePath, "xdg-config");
|
||||
const xdgCachePath = join(statePath, "xdg-cache");
|
||||
const xdgDataPath = join(statePath, "xdg-data");
|
||||
const userDataPath = join(statePath, "user-data");
|
||||
const id = `livesync-e2e-${Date.now()}`;
|
||||
await mkdir(homePath, { recursive: true });
|
||||
await mkdir(xdgConfigPath, { recursive: true });
|
||||
await mkdir(xdgCachePath, { recursive: true });
|
||||
await mkdir(xdgDataPath, { recursive: true });
|
||||
await mkdir(userDataPath, { recursive: true });
|
||||
await writeFile(
|
||||
join(vaultPath, ".obsidian", "app.json"),
|
||||
JSON.stringify({ legacyEditor: false, safeMode: false }, null, 4)
|
||||
);
|
||||
await writeFile(
|
||||
join(vaultPath, ".obsidian", "community-plugins.json"),
|
||||
JSON.stringify(["obsidian-livesync"], null, 4)
|
||||
);
|
||||
await writeObsidianVaultRegistry(id, vaultPath, name, homePath, xdgConfigPath, userDataPath);
|
||||
|
||||
return {
|
||||
path: vaultPath,
|
||||
name,
|
||||
id,
|
||||
homePath,
|
||||
xdgConfigPath,
|
||||
xdgCachePath,
|
||||
xdgDataPath,
|
||||
userDataPath,
|
||||
dispose: async () => {
|
||||
if (process.env.E2E_OBSIDIAN_KEEP_VAULT === "true") {
|
||||
console.log(`Keeping temporary vault: ${vaultPath}`);
|
||||
console.log(`Keeping temporary Obsidian state: ${statePath}`);
|
||||
return;
|
||||
}
|
||||
await Promise.all([
|
||||
rm(vaultPath, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 }),
|
||||
rm(statePath, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 }),
|
||||
]);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function writeObsidianVaultRegistry(
|
||||
vaultId: string,
|
||||
vaultPath: string,
|
||||
vaultName: string,
|
||||
homePath: string,
|
||||
xdgConfigPath: string,
|
||||
userDataPath: string
|
||||
): Promise<void> {
|
||||
const vaultRecord = {
|
||||
path: vaultPath,
|
||||
ts: Date.now(),
|
||||
open: true,
|
||||
name: vaultName,
|
||||
};
|
||||
const registry = {
|
||||
cli: true,
|
||||
vaults: {
|
||||
[vaultId]: vaultRecord,
|
||||
},
|
||||
};
|
||||
const registryText = JSON.stringify(registry, null, 4);
|
||||
for (const configRoot of [join(homePath, ".config"), xdgConfigPath]) {
|
||||
const obsidianConfigDir = join(configRoot, "obsidian");
|
||||
await mkdir(obsidianConfigDir, { recursive: true });
|
||||
await writeFile(join(obsidianConfigDir, "obsidian.json"), registryText);
|
||||
}
|
||||
await writeFile(join(userDataPath, "obsidian.json"), registryText);
|
||||
await writeFile(join(userDataPath, `${vaultId}.json`), JSON.stringify(vaultRecord, null, 4));
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import { launchObsidian } from "../runner/launch.ts";
|
||||
import { runObsidianCli } from "../runner/cli.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
import { installBuiltPlugin } from "../runner/pluginInstaller.ts";
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
const vault = await createTemporaryVault();
|
||||
let app;
|
||||
try {
|
||||
await installBuiltPlugin(vault.path);
|
||||
app = await launchObsidian({
|
||||
binary,
|
||||
vaultPath: vault.path,
|
||||
homePath: vault.homePath,
|
||||
xdgConfigPath: vault.xdgConfigPath,
|
||||
userDataPath: vault.userDataPath,
|
||||
});
|
||||
const cliEnv = {
|
||||
...process.env,
|
||||
HOME: vault.homePath,
|
||||
XDG_CONFIG_HOME: vault.xdgConfigPath,
|
||||
XDG_CACHE_HOME: vault.xdgCachePath,
|
||||
XDG_DATA_HOME: vault.xdgDataPath,
|
||||
};
|
||||
await runObsidianCli(cli.binary, [`obsidian://open?path=${encodeURIComponent(vault.path)}`], cliEnv);
|
||||
await new Promise((resolve) => setTimeout(resolve, 3000));
|
||||
if (process.env.E2E_OBSIDIAN_RELOAD_PLUGIN === "true") {
|
||||
await runObsidianCli(cli.binary, ["eval", "code=(async()=>app.plugins.setEnable(true))()"], cliEnv);
|
||||
await runObsidianCli(cli.binary, ["plugin:reload", "id=obsidian-livesync"], cliEnv);
|
||||
}
|
||||
const cliArgs = process.argv.slice(2);
|
||||
const result = await runObsidianCli(cli.binary, cliArgs.length > 0 ? cliArgs : ["--help"], cliEnv);
|
||||
console.log(result.stdout);
|
||||
console.error(result.stderr);
|
||||
process.exitCode = result.code ?? 1;
|
||||
} finally {
|
||||
if (app) {
|
||||
await app.stop();
|
||||
}
|
||||
await vault.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,140 @@
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import {
|
||||
assertCouchDbReachable,
|
||||
createCouchDbDatabase,
|
||||
deleteCouchDbDatabase,
|
||||
loadCouchDbConfig,
|
||||
makeUniqueDatabaseName,
|
||||
waitForCouchDbDocs,
|
||||
} from "../runner/couchdb.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
configureCouchDb,
|
||||
prepareRemote,
|
||||
pushLocalChanges,
|
||||
waitForLiveSyncCoreReady,
|
||||
type LocalDatabaseEntry,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
|
||||
const notePath = "E2E/couchdb-upload.md";
|
||||
const noteContent = [
|
||||
"# CouchDB upload from real Obsidian",
|
||||
"",
|
||||
"This note is created through Obsidian and uploaded by Self-hosted LiveSync.",
|
||||
"The content is intentionally long enough to require chunk metadata in the local database.",
|
||||
"0123456789 abcdefghijklmnopqrstuvwxyz 0123456789 abcdefghijklmnopqrstuvwxyz",
|
||||
"0123456789 abcdefghijklmnopqrstuvwxyz 0123456789 abcdefghijklmnopqrstuvwxyz",
|
||||
`Created at: ${new Date().toISOString()}`,
|
||||
"",
|
||||
].join("\n");
|
||||
|
||||
async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.ProcessEnv): Promise<LocalDatabaseEntry> {
|
||||
return await evalObsidianJson<LocalDatabaseEntry>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(notePath)};`,
|
||||
`const content=${JSON.stringify(noteContent)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
|
||||
"const existing=app.vault.getAbstractFileByPath(path);",
|
||||
"if(existing) await app.vault.delete(existing);",
|
||||
"await app.vault.create(path,content);",
|
||||
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
|
||||
"let entry=false;",
|
||||
"for(let i=0;i<40;i++){",
|
||||
"await core.services.fileProcessing.commitPendingFileEvents();",
|
||||
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true).catch(()=>false);",
|
||||
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
|
||||
"await sleep(250);",
|
||||
"}",
|
||||
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
|
||||
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const couchDb = await loadCouchDbConfig();
|
||||
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "obsidian-upload");
|
||||
const vault = await createTemporaryVault();
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
|
||||
try {
|
||||
await assertCouchDbReachable(couchDb);
|
||||
await createCouchDbDatabase(couchDb, dbName);
|
||||
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault: ${vault.path}`);
|
||||
console.log(`Temporary CouchDB database: ${dbName}`);
|
||||
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
|
||||
const configured = await configureCouchDb(cli.binary, session.cliEnv, {
|
||||
uri: couchDb.uri,
|
||||
username: couchDb.username,
|
||||
password: couchDb.password,
|
||||
dbName,
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
|
||||
assertEqual(configured.couchDB_URI, couchDb.uri, "Configured CouchDB URI did not match.");
|
||||
assertEqual(configured.couchDB_DBNAME, dbName, "Configured CouchDB database name did not match.");
|
||||
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during this one-shot workflow.");
|
||||
assertEqual(configured.syncOnStart, false, "Sync on start should remain disabled during this workflow.");
|
||||
assertEqual(configured.syncOnSave, false, "Sync on save should remain disabled during this workflow.");
|
||||
|
||||
await prepareRemote(cli.binary, session.cliEnv);
|
||||
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
|
||||
await pushLocalChanges(cli.binary, session.cliEnv);
|
||||
|
||||
const remoteDocs = await waitForCouchDbDocs(couchDb, dbName, (docs) => {
|
||||
const ids = new Set(docs.map((doc) => doc._id));
|
||||
return ids.has(localEntry.id) && localEntry.children.every((childId) => ids.has(childId));
|
||||
});
|
||||
const remoteMetadata = remoteDocs.find((doc) => doc._id === localEntry.id);
|
||||
assertEqual(
|
||||
remoteMetadata?.path,
|
||||
localEntry.path,
|
||||
"Remote metadata path did not match the local database entry."
|
||||
);
|
||||
|
||||
console.log(
|
||||
`Uploaded metadata ${localEntry.id} and ${localEntry.children.length} chunk(s) to CouchDB database ${dbName}`
|
||||
);
|
||||
} finally {
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
}
|
||||
await vault.dispose();
|
||||
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
|
||||
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,603 @@
|
||||
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import {
|
||||
assertCouchDbReachable,
|
||||
createCouchDbDatabase,
|
||||
deleteCouchDbDatabase,
|
||||
loadCouchDbConfig,
|
||||
makeUniqueDatabaseName,
|
||||
waitForCouchDbDocs,
|
||||
type CouchDbConfig,
|
||||
} from "../runner/couchdb.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
configureCouchDb,
|
||||
prepareRemote,
|
||||
pushLocalChanges,
|
||||
waitForLiveSyncCoreReady,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
|
||||
|
||||
const snippetPath = ".obsidian/snippets/livesync-customisation-e2e.css";
|
||||
const snippetContent = [
|
||||
"body {",
|
||||
" --livesync-customisation-e2e-colour: #3d6f54;",
|
||||
"}",
|
||||
"",
|
||||
".livesync-customisation-e2e {",
|
||||
" color: var(--livesync-customisation-e2e-colour);",
|
||||
"}",
|
||||
"",
|
||||
].join("\n");
|
||||
const snippetUpdatedContent = [
|
||||
"body {",
|
||||
" --livesync-customisation-e2e-colour: #73548f;",
|
||||
"}",
|
||||
"",
|
||||
".livesync-customisation-e2e {",
|
||||
" background-color: var(--livesync-customisation-e2e-colour);",
|
||||
"}",
|
||||
"",
|
||||
].join("\n");
|
||||
const configPath = ".obsidian/livesync-customisation-e2e.json";
|
||||
const configContent = JSON.stringify({ source: "customisation-sync", enabled: true }, null, 4) + "\n";
|
||||
const pluginDir = ".obsidian/plugins/livesync-e2e-sample";
|
||||
const pluginManifestPath = `${pluginDir}/manifest.json`;
|
||||
const pluginMainPath = `${pluginDir}/main.js`;
|
||||
const pluginStylesPath = `${pluginDir}/styles.css`;
|
||||
const pluginManifestContent =
|
||||
JSON.stringify(
|
||||
{
|
||||
id: "livesync-e2e-sample",
|
||||
name: "LiveSync E2E Sample",
|
||||
version: "0.0.1",
|
||||
minAppVersion: "1.0.0",
|
||||
description: "A sample plug-in fixture for real Obsidian E2E.",
|
||||
author: "Self-hosted LiveSync",
|
||||
isDesktopOnly: false,
|
||||
},
|
||||
null,
|
||||
4
|
||||
) + "\n";
|
||||
const pluginMainContent = [
|
||||
"module.exports = class LiveSyncE2ESamplePlugin extends Plugin {",
|
||||
" async onload() {",
|
||||
" this.register(() => undefined);",
|
||||
" }",
|
||||
"};",
|
||||
"",
|
||||
].join("\n");
|
||||
const pluginStylesContent = ".livesync-e2e-sample { color: #73548f; }\n";
|
||||
const sourceDeviceName = "customisation-sync-a";
|
||||
const targetDeviceName = "customisation-sync-b";
|
||||
|
||||
type RunnerContext = {
|
||||
binary: string;
|
||||
cliBinary: string;
|
||||
couchDb: CouchDbConfig;
|
||||
dbName: string;
|
||||
};
|
||||
|
||||
type CustomisationEntry = {
|
||||
id: string;
|
||||
path: string;
|
||||
children: string[];
|
||||
};
|
||||
|
||||
type CustomisationScanResult = {
|
||||
enabled: boolean;
|
||||
useV2: boolean;
|
||||
device: string;
|
||||
configDir: string;
|
||||
files: string[];
|
||||
};
|
||||
|
||||
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
|
||||
const fullPath = join(vaultPath, path);
|
||||
await mkdir(dirname(fullPath), { recursive: true });
|
||||
await writeFile(fullPath, content, "utf-8");
|
||||
}
|
||||
|
||||
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
|
||||
await rm(join(vaultPath, path), { force: true });
|
||||
}
|
||||
|
||||
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
|
||||
return await readFile(join(vaultPath, path), "utf-8");
|
||||
}
|
||||
|
||||
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
|
||||
try {
|
||||
await readFile(join(vaultPath, path));
|
||||
return true;
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
|
||||
return false;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForPathContent(
|
||||
vaultPath: string,
|
||||
path: string,
|
||||
predicate: (content: string) => boolean,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
|
||||
): Promise<string> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastContent = "";
|
||||
while (Date.now() < deadline) {
|
||||
if (await pathExists(vaultPath, path)) {
|
||||
lastContent = await readVaultFile(vaultPath, path);
|
||||
if (predicate(lastContent)) {
|
||||
return lastContent;
|
||||
}
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
|
||||
}
|
||||
|
||||
async function startConfiguredSession(
|
||||
context: RunnerContext,
|
||||
vault: TemporaryVault,
|
||||
deviceName: string
|
||||
): Promise<ObsidianLiveSyncSession> {
|
||||
const session = await startObsidianLiveSyncSession({
|
||||
binary: context.binary,
|
||||
cliBinary: context.cliBinary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
||||
await configureCouchDb(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
{
|
||||
uri: context.couchDb.uri,
|
||||
username: context.couchDb.username,
|
||||
password: context.couchDb.password,
|
||||
dbName: context.dbName,
|
||||
},
|
||||
{
|
||||
deviceAndVaultName: deviceName,
|
||||
usePluginSync: true,
|
||||
usePluginSyncV2: true,
|
||||
autoSweepPlugins: false,
|
||||
autoSweepPluginsPeriodic: false,
|
||||
syncInternalFiles: false,
|
||||
}
|
||||
);
|
||||
await evalObsidianJson<unknown>(
|
||||
context.cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
`core.services.setting.setDeviceAndVaultName(${JSON.stringify(deviceName)});`,
|
||||
"await core.services.setting.saveSettingData();",
|
||||
"return JSON.stringify({device:core.services.setting.getDeviceAndVaultName()});",
|
||||
"})()",
|
||||
].join(""),
|
||||
session.cliEnv
|
||||
);
|
||||
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
||||
await prepareRemote(context.cliBinary, session.cliEnv);
|
||||
return session;
|
||||
}
|
||||
|
||||
async function scanCustomisations(cliBinary: string, env: NodeJS.ProcessEnv): Promise<CustomisationScanResult> {
|
||||
return await evalObsidianJson<CustomisationScanResult>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('ConfigSync');",
|
||||
"const before=await addOn.scanInternalFiles();",
|
||||
"await addOn.scanAllConfigFiles(false);",
|
||||
"return JSON.stringify({",
|
||||
"ok:true,",
|
||||
"enabled:core.settings.usePluginSync,",
|
||||
"useV2:core.settings.usePluginSyncV2,",
|
||||
"device:core.services.setting.getDeviceAndVaultName(),",
|
||||
"configDir:addOn.configDir,",
|
||||
"files:before,",
|
||||
"});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function storeCustomisationFile(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('ConfigSync');",
|
||||
"const term=core.services.setting.getDeviceAndVaultName();",
|
||||
"const stat=await core.storageAccess.statHidden(path);",
|
||||
"const category=addOn.getFileCategory(path);",
|
||||
"const result=await addOn.storeCustomizationFiles(path,term);",
|
||||
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>doc.path);",
|
||||
"const filename=path.split('/').pop();",
|
||||
"const existing=entries.some((entry)=>entry.startsWith(`ix:${term}/${category}/`)&&entry.endsWith(`%${filename}`));",
|
||||
"if(!result&&!existing){",
|
||||
" throw new Error(`Could not store Customisation Sync file: path=${path}; term=${term}; category=${category}; stat=${JSON.stringify(stat)}; result=${JSON.stringify(result)}; entries=${JSON.stringify(entries)}`);",
|
||||
"}",
|
||||
"return JSON.stringify({ok:true,path,term,category,result:!!result,existing,entries});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function deleteCustomisationSyncEntry(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
||||
name: string,
|
||||
term?: string
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const category=${JSON.stringify(category)};`,
|
||||
`const name=${JSON.stringify(name)};`,
|
||||
`const term=${JSON.stringify(term ?? "")};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('ConfigSync');",
|
||||
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
"const entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&!doc.deleted&&!doc._deleted)||false;",
|
||||
"if(!entry) throw new Error(`Could not find customisation sync entry to delete: ${category}/${name}`);",
|
||||
"if(!(await addOn.deleteConfigOnDatabase(entry.path))){",
|
||||
" throw new Error(`Could not delete Customisation Sync entry: ${entry.path}`);",
|
||||
"}",
|
||||
"return JSON.stringify({ok:true,path:entry.path});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function waitForCustomisationEntry(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
||||
name: string,
|
||||
term?: string,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
|
||||
): Promise<CustomisationEntry> {
|
||||
const entries = await waitForCustomisationEntries(cliBinary, env, category, name, 1, term, timeoutMs);
|
||||
return entries[0];
|
||||
}
|
||||
|
||||
async function waitForCustomisationEntries(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
||||
name: string,
|
||||
count: number,
|
||||
term?: string,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
|
||||
): Promise<CustomisationEntry[]> {
|
||||
return await evalObsidianJson<CustomisationEntry[]>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const category=${JSON.stringify(category)};`,
|
||||
`const name=${JSON.stringify(name)};`,
|
||||
`const count=${JSON.stringify(count)};`,
|
||||
`const term=${JSON.stringify(term ?? "")};`,
|
||||
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const deadline=Date.now()+timeoutMs;",
|
||||
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
|
||||
"let entries=[];",
|
||||
"while(Date.now()<deadline){",
|
||||
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
" entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&Array.isArray(doc.children)&&doc.children.length>0);",
|
||||
" if(entries.length>=count) break;",
|
||||
" await sleep(250);",
|
||||
"}",
|
||||
"if(entries.length<count){",
|
||||
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
" const found=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>({id:doc._id,path:doc.path,children:doc.children?.length??0}));",
|
||||
" throw new Error(`Timed out waiting for customisation sync entries: ${category}/${name}; expected=${count}; entries=${JSON.stringify(found)}`);",
|
||||
"}",
|
||||
"return JSON.stringify(entries.map((entry)=>({id:entry._id,path:entry.path,children:entry.children||[]})));",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function waitForCustomisationEntryAbsent(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
||||
name: string,
|
||||
term?: string,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000)
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const category=${JSON.stringify(category)};`,
|
||||
`const name=${JSON.stringify(name)};`,
|
||||
`const term=${JSON.stringify(term ?? "")};`,
|
||||
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const deadline=Date.now()+timeoutMs;",
|
||||
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
|
||||
"let entry=false;",
|
||||
"while(Date.now()<deadline){",
|
||||
" const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
" entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`))&&!doc.deleted&&!doc._deleted)||false;",
|
||||
" if(!entry) return JSON.stringify({ok:true});",
|
||||
" await sleep(250);",
|
||||
"}",
|
||||
"throw new Error(`Timed out waiting for customisation sync entry deletion: ${category}/${name}; entry=${JSON.stringify(entry)}`);",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function applyRemoteCustomisationEntry(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
category: "CONFIG" | "SNIPPET" | "PLUGIN_MAIN",
|
||||
name: string,
|
||||
term?: string
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const category=${JSON.stringify(category)};`,
|
||||
`const name=${JSON.stringify(name)};`,
|
||||
`const term=${JSON.stringify(term ?? "")};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('ConfigSync');",
|
||||
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
"const entry=rows.map((row)=>row.doc).find((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`)))||false;",
|
||||
"if(!entry) throw new Error(`Could not find remote customisation entry: ${category}/${name}`);",
|
||||
"const display=addOn.createPluginDataFromV2(entry.path);",
|
||||
"if(!display) throw new Error(`Could not create Customisation Sync display entry: ${entry.path}`);",
|
||||
"const file=await addOn.createPluginDataExFileV2(entry.path);",
|
||||
"if(!file) throw new Error(`Could not load Customisation Sync file entry: ${entry.path}`);",
|
||||
"await display.setFile(file);",
|
||||
"if(!(await addOn.applyDataV2(display))){",
|
||||
" throw new Error(`Could not apply Customisation Sync entry: ${entry.path}`);",
|
||||
"}",
|
||||
"return JSON.stringify({ok:true,path:entry.path});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function applyRemoteCustomisationGroup(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
category: "PLUGIN_MAIN",
|
||||
name: string,
|
||||
term?: string
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const category=${JSON.stringify(category)};`,
|
||||
`const name=${JSON.stringify(name)};`,
|
||||
`const term=${JSON.stringify(term ?? "")};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('ConfigSync');",
|
||||
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
|
||||
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.includes(`/${category}/`)&&doc.path?.includes(`/${name}%`)&&(!term||doc.path?.startsWith(`ix:${term}/`)));",
|
||||
"if(entries.length===0) throw new Error(`Could not find remote customisation entries: ${category}/${name}`);",
|
||||
"const display=addOn.createPluginDataFromV2(entries[0].path);",
|
||||
"if(!display) throw new Error(`Could not create Customisation Sync display entry: ${entries[0].path}`);",
|
||||
"for(const entry of entries){",
|
||||
" const file=await addOn.createPluginDataExFileV2(entry.path);",
|
||||
" if(!file) throw new Error(`Could not load Customisation Sync file entry: ${entry.path}`);",
|
||||
" await display.setFile(file);",
|
||||
"}",
|
||||
"if(!(await addOn.applyDataV2(display))){",
|
||||
" throw new Error(`Could not apply Customisation Sync group: ${category}/${name}`);",
|
||||
"}",
|
||||
"return JSON.stringify({ok:true,count:entries.length});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const couchDb = await loadCouchDbConfig();
|
||||
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "customisation-sync");
|
||||
const vaultA = await createTemporaryVault();
|
||||
const vaultB = await createTemporaryVault();
|
||||
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
|
||||
const snippetPathParts = snippetPath.split("/");
|
||||
const snippetName = snippetPathParts[snippetPathParts.length - 1] ?? snippetPath;
|
||||
const configName = configPath.split("/").pop() ?? configPath;
|
||||
const pluginName = pluginDir.split("/").pop() ?? pluginDir;
|
||||
|
||||
try {
|
||||
await assertCouchDbReachable(couchDb);
|
||||
await createCouchDbDatabase(couchDb, dbName);
|
||||
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault A: ${vaultA.path}`);
|
||||
console.log(`Temporary vault B: ${vaultB.path}`);
|
||||
console.log(`Temporary CouchDB database: ${dbName}`);
|
||||
|
||||
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
|
||||
await writeVaultFile(vaultA.path, configPath, configContent);
|
||||
await writeVaultFile(vaultA.path, pluginManifestPath, pluginManifestContent);
|
||||
await writeVaultFile(vaultA.path, pluginMainPath, pluginMainContent);
|
||||
await writeVaultFile(vaultA.path, pluginStylesPath, pluginStylesContent);
|
||||
|
||||
let session = await startConfiguredSession(context, vaultA, sourceDeviceName);
|
||||
const scanResult = await scanCustomisations(context.cliBinary, session.cliEnv);
|
||||
console.log(`Customisation scan files: ${scanResult.files.join(", ") || "(none)"}`);
|
||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, snippetPath);
|
||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, configPath);
|
||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginManifestPath);
|
||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginMainPath);
|
||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, pluginStylesPath);
|
||||
const entry = await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName);
|
||||
const configEntry = await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "CONFIG", configName);
|
||||
const pluginEntries = await waitForCustomisationEntries(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
"PLUGIN_MAIN",
|
||||
pluginName,
|
||||
3
|
||||
);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
|
||||
const ids = new Set(docs.map((doc) => doc._id));
|
||||
const entries = [entry, configEntry, ...pluginEntries];
|
||||
return entries.every(
|
||||
(target) => ids.has(target.id) && target.children.every((childId) => ids.has(childId))
|
||||
);
|
||||
});
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, targetDeviceName);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
|
||||
assertEqual(
|
||||
await pathExists(vaultB.path, snippetPath),
|
||||
false,
|
||||
"Customisation Sync snippet was reflected before explicit application."
|
||||
);
|
||||
await applyRemoteCustomisationEntry(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
"SNIPPET",
|
||||
snippetName,
|
||||
sourceDeviceName
|
||||
);
|
||||
const applied = await waitForPathContent(vaultB.path, snippetPath, (content) => content === snippetContent);
|
||||
await applyRemoteCustomisationEntry(context.cliBinary, session.cliEnv, "CONFIG", configName, sourceDeviceName);
|
||||
const appliedConfig = await waitForPathContent(vaultB.path, configPath, (content) => content === configContent);
|
||||
await applyRemoteCustomisationGroup(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
"PLUGIN_MAIN",
|
||||
pluginName,
|
||||
sourceDeviceName
|
||||
);
|
||||
const appliedPluginManifest = await waitForPathContent(
|
||||
vaultB.path,
|
||||
pluginManifestPath,
|
||||
(content) => content === pluginManifestContent
|
||||
);
|
||||
const appliedPluginMain = await waitForPathContent(
|
||||
vaultB.path,
|
||||
pluginMainPath,
|
||||
(content) => content === pluginMainContent
|
||||
);
|
||||
const appliedPluginStyles = await waitForPathContent(
|
||||
vaultB.path,
|
||||
pluginStylesPath,
|
||||
(content) => content === pluginStylesContent
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
assertEqual(applied, snippetContent, "Customisation Sync snippet content did not match after application.");
|
||||
assertEqual(appliedConfig, configContent, "Customisation Sync config content did not match after application.");
|
||||
assertEqual(
|
||||
appliedPluginManifest,
|
||||
pluginManifestContent,
|
||||
"Customisation Sync plug-in manifest did not match after application."
|
||||
);
|
||||
assertEqual(appliedPluginMain, pluginMainContent, "Customisation Sync plug-in main file did not match.");
|
||||
assertEqual(appliedPluginStyles, pluginStylesContent, "Customisation Sync plug-in stylesheet did not match.");
|
||||
|
||||
await writeVaultFile(vaultA.path, snippetPath, snippetUpdatedContent);
|
||||
session = await startConfiguredSession(context, vaultA, sourceDeviceName);
|
||||
await storeCustomisationFile(context.cliBinary, session.cliEnv, snippetPath);
|
||||
await waitForCustomisationEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, targetDeviceName);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await applyRemoteCustomisationEntry(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
"SNIPPET",
|
||||
snippetName,
|
||||
sourceDeviceName
|
||||
);
|
||||
const updated = await waitForPathContent(
|
||||
vaultB.path,
|
||||
snippetPath,
|
||||
(content) => content === snippetUpdatedContent
|
||||
);
|
||||
await session.app.stop();
|
||||
assertEqual(updated, snippetUpdatedContent, "Updated Customisation Sync snippet did not apply.");
|
||||
|
||||
await removeVaultFile(vaultA.path, snippetPath);
|
||||
session = await startConfiguredSession(context, vaultA, sourceDeviceName);
|
||||
await deleteCustomisationSyncEntry(context.cliBinary, session.cliEnv, "SNIPPET", snippetName, sourceDeviceName);
|
||||
await waitForCustomisationEntryAbsent(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
"SNIPPET",
|
||||
snippetName,
|
||||
sourceDeviceName
|
||||
);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, targetDeviceName);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await waitForCustomisationEntryAbsent(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
"SNIPPET",
|
||||
snippetName,
|
||||
sourceDeviceName
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
console.log(
|
||||
`Customisation Sync applied snippet, config, and plug-in fixtures, then propagated snippet update and sync-data deletion.`
|
||||
);
|
||||
} finally {
|
||||
await vaultA.dispose();
|
||||
await vaultB.dispose();
|
||||
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
|
||||
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,60 @@
|
||||
import { launchObsidian } from "../runner/launch.ts";
|
||||
import { installBuiltPlugin } from "../runner/pluginInstaller.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
import { requireObsidianBinary } from "../runner/environment.ts";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, withObsidianPage } from "../runner/ui.ts";
|
||||
|
||||
const port = obsidianRemoteDebuggingPort();
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const vault = await createTemporaryVault();
|
||||
await installBuiltPlugin(vault.path);
|
||||
const app = await launchObsidian({
|
||||
binary,
|
||||
vaultPath: vault.path,
|
||||
homePath: vault.homePath,
|
||||
xdgConfigPath: vault.xdgConfigPath,
|
||||
xdgCachePath: vault.xdgCachePath,
|
||||
xdgDataPath: vault.xdgDataPath,
|
||||
userDataPath: vault.userDataPath,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
|
||||
try {
|
||||
await preseedTrustedVaultState(port, vault.id);
|
||||
const { screenshotPath, textPath } = await withObsidianPage(port, async (page) => {
|
||||
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_WAIT_MS ?? 5000));
|
||||
const title = await page.title().catch((error: unknown) => `title error: ${String(error)}`);
|
||||
const url = page.url();
|
||||
const text = await page
|
||||
.locator("body")
|
||||
.innerText({ timeout: 5000 })
|
||||
.catch((error: unknown) => {
|
||||
return `body text error: ${String(error)}`;
|
||||
});
|
||||
if (process.env.E2E_OBSIDIAN_DEBUG_CLICK_TRUST === "true") {
|
||||
await page.getByText("Trust author and enable plugins").click({ timeout: 10000 });
|
||||
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_AFTER_CLICK_WAIT_MS ?? 3000));
|
||||
}
|
||||
const screenshotPath = process.env.E2E_OBSIDIAN_DEBUG_SCREENSHOT ?? "/tmp/obsidian-e2e-debug.png";
|
||||
const textPath = process.env.E2E_OBSIDIAN_DEBUG_TEXT ?? "/tmp/obsidian-e2e-debug.txt";
|
||||
await page.screenshot({ path: screenshotPath, fullPage: true });
|
||||
await writeFile(textPath, [`title: ${title}`, `url: ${url}`, "", text].join("\n"), "utf-8");
|
||||
return { screenshotPath, textPath };
|
||||
});
|
||||
console.log(`Temporary vault: ${vault.path}`);
|
||||
console.log(`Temporary Obsidian state: ${vault.userDataPath}`);
|
||||
console.log(`Debug text: ${textPath}`);
|
||||
console.log(`Debug screenshot: ${screenshotPath}`);
|
||||
} finally {
|
||||
await app.stop();
|
||||
await vault.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,13 @@
|
||||
import { discoverObsidianBinary } from "../runner/environment.ts";
|
||||
|
||||
const result = discoverObsidianBinary();
|
||||
if (result.binary) {
|
||||
console.log(`Obsidian executable: ${result.binary}`);
|
||||
console.log(`Source: ${result.source}`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
console.error("Obsidian executable was not found.");
|
||||
console.error("Set OBSIDIAN_BINARY to the installed Obsidian executable path.");
|
||||
console.error(`Checked paths: ${result.checked.length > 0 ? result.checked.join(", ") : "(none)"}`);
|
||||
process.exit(1);
|
||||
@@ -0,0 +1,533 @@
|
||||
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import {
|
||||
assertCouchDbReachable,
|
||||
createCouchDbDatabase,
|
||||
deleteCouchDbDatabase,
|
||||
loadCouchDbConfig,
|
||||
makeUniqueDatabaseName,
|
||||
waitForCouchDbDocs,
|
||||
type CouchDbConfig,
|
||||
} from "../runner/couchdb.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
configureCouchDb,
|
||||
prepareRemote,
|
||||
pushLocalChanges,
|
||||
waitForLiveSyncCoreReady,
|
||||
waitForLocalDatabaseEntry,
|
||||
type LocalDatabaseEntry,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { clickJsonResolveOption, obsidianRemoteDebuggingPort } from "../runner/ui.ts";
|
||||
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
|
||||
|
||||
const snippetPath = ".obsidian/snippets/livesync-e2e.css";
|
||||
const snippetContent = [
|
||||
"body {",
|
||||
" --livesync-e2e-snippet-colour: #245a70;",
|
||||
"}",
|
||||
"",
|
||||
".livesync-e2e-snippet {",
|
||||
" color: var(--livesync-e2e-snippet-colour);",
|
||||
"}",
|
||||
"",
|
||||
].join("\n");
|
||||
|
||||
const mergeJsonPath = ".obsidian/livesync-e2e-merge.json";
|
||||
const manualMergeJsonPath = ".obsidian/livesync-e2e-manual-merge.json";
|
||||
const targetPath = ".obsidian/livesync-targeted/only-a.json";
|
||||
const hiddenFileCliTimeoutMs = Number(process.env.E2E_OBSIDIAN_HIDDEN_FILE_CLI_TIMEOUT_MS ?? 90000);
|
||||
|
||||
type RunnerContext = {
|
||||
binary: string;
|
||||
cliBinary: string;
|
||||
couchDb: CouchDbConfig;
|
||||
dbName: string;
|
||||
};
|
||||
|
||||
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
|
||||
const fullPath = join(vaultPath, path);
|
||||
await mkdir(dirname(fullPath), { recursive: true });
|
||||
await writeFile(fullPath, content, "utf-8");
|
||||
}
|
||||
|
||||
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
|
||||
await rm(join(vaultPath, path), { force: true });
|
||||
}
|
||||
|
||||
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
|
||||
return await readFile(join(vaultPath, path), "utf-8");
|
||||
}
|
||||
|
||||
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
|
||||
try {
|
||||
await readFile(join(vaultPath, path));
|
||||
return true;
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
|
||||
return false;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForPathContent(
|
||||
vaultPath: string,
|
||||
path: string,
|
||||
predicate: (content: string) => boolean,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
|
||||
): Promise<string> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastContent = "";
|
||||
while (Date.now() < deadline) {
|
||||
if (await pathExists(vaultPath, path)) {
|
||||
lastContent = await readVaultFile(vaultPath, path);
|
||||
if (predicate(lastContent)) {
|
||||
return lastContent;
|
||||
}
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
|
||||
}
|
||||
|
||||
async function waitForPathDeleted(
|
||||
vaultPath: string,
|
||||
path: string,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
|
||||
): Promise<void> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
while (Date.now() < deadline) {
|
||||
if (!(await pathExists(vaultPath, path))) {
|
||||
return;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
|
||||
}
|
||||
|
||||
function hasJsonValues(content: string, values: Record<string, unknown>): boolean {
|
||||
try {
|
||||
const parsed = JSON.parse(content) as Record<string, unknown>;
|
||||
return Object.entries(values).every(([key, value]) => parsed[key] === value);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function scanHiddenStorage(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('HiddenFileSync');",
|
||||
"await addOn.scanAllStorageChanges(true);",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function scanHiddenDatabase(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('HiddenFileSync');",
|
||||
"await addOn.scanAllDatabaseChanges(true);",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env,
|
||||
hiddenFileCliTimeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('HiddenFileSync');",
|
||||
"await addOn.resolveConflictOnInternalFiles();",
|
||||
"await addOn.scanAllDatabaseChanges(true);",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env,
|
||||
hiddenFileCliTimeoutMs
|
||||
);
|
||||
}
|
||||
|
||||
async function autoMergeHiddenJsonConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
"const prefixedPath=`i:${path}`;",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('HiddenFileSync');",
|
||||
"let doc=false;",
|
||||
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
|
||||
" if(entry.path===prefixedPath){ doc=entry; break; }",
|
||||
"}",
|
||||
"if(!doc) throw new Error(`Could not find hidden conflict candidate: ${path}`);",
|
||||
"if(!doc._conflicts?.length) throw new Error(`Hidden file has no conflicts: ${path}`);",
|
||||
"const conflicts=doc._conflicts.sort((a,b)=>Number(a.split('-')[0])-Number(b.split('-')[0]));",
|
||||
"const conflictedRev=conflicts[0];",
|
||||
"const conflictedRevNo=Number(conflictedRev.split('-')[0]);",
|
||||
"const revFrom=await core.localDatabase.getRaw(doc._id,{revs_info:true});",
|
||||
"const commonBase=(revFrom._revs_info||[])",
|
||||
" .filter((rev)=>rev.status==='available'&&Number(rev.rev.split('-')[0])<conflictedRevNo)",
|
||||
" .map((rev)=>rev.rev)[0]||'';",
|
||||
"const result=await core.localDatabase.managers.conflictManager.mergeObject(",
|
||||
" doc.path, commonBase, doc._rev, conflictedRev",
|
||||
");",
|
||||
"if(!result){",
|
||||
" throw new Error(`Hidden JSON conflict was not auto-mergeable: ${path}; base=${commonBase}; current=${doc._rev}; conflict=${conflictedRev}`);",
|
||||
"}",
|
||||
"await addOn.ensureDir(path);",
|
||||
"const stat=await addOn.writeFile(path,result);",
|
||||
"if(!stat) throw new Error(`Could not write merged hidden file: ${path}`);",
|
||||
"await addOn.storeInternalFileToDatabase({path,mtime:stat.mtime,ctime:stat.ctime,size:stat.size},true);",
|
||||
"await core.localDatabase.removeRevision(doc._id,conflictedRev);",
|
||||
"await addOn.extractInternalFileFromDatabase(path);",
|
||||
"await addOn.scanAllDatabaseChanges(true);",
|
||||
"return JSON.stringify({ok:true,merged:JSON.parse(result)});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function openHiddenJsonResolveModal(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
"const prefixedPath=`i:${path}`;",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('HiddenFileSync');",
|
||||
"let doc=false;",
|
||||
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
|
||||
" if(entry.path===prefixedPath){ doc=entry; break; }",
|
||||
"}",
|
||||
"if(!doc?._conflicts?.length) throw new Error(`Could not find hidden JSON conflict: ${path}`);",
|
||||
"const conflicts=doc._conflicts.sort((a,b)=>Number(a.split('-')[0])-Number(b.split('-')[0]));",
|
||||
"const docA=await core.localDatabase.getDBEntry(prefixedPath,{rev:doc._rev});",
|
||||
"const docB=await core.localDatabase.getDBEntry(prefixedPath,{rev:conflicts[0]});",
|
||||
"if(docA===false||docB===false) throw new Error(`Could not load conflicted hidden JSON entries: ${path}`);",
|
||||
"void addOn.showJSONMergeDialogAndMerge(docA,docB);",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function storeHiddenFileAsConflict(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
path: string,
|
||||
baseRev: string
|
||||
): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
`const baseRev=${JSON.stringify(baseRev)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const addOn=core.getAddOn('HiddenFileSync');",
|
||||
"const fileInfo=await addOn.loadFileWithInfo(path);",
|
||||
"if(fileInfo.deleted) throw new Error(`Hidden file was unexpectedly deleted: ${path}`);",
|
||||
"const baseData=await addOn.__loadBaseSaveData(path,true);",
|
||||
"if(baseData===false) throw new Error(`Could not load base save data: ${path}`);",
|
||||
"const saveData={",
|
||||
" ...baseData,",
|
||||
" data:fileInfo.body,",
|
||||
" mtime:fileInfo.stat.mtime,",
|
||||
" ctime:fileInfo.stat.ctime,",
|
||||
" size:fileInfo.stat.size,",
|
||||
" children:[],",
|
||||
" deleted:false,",
|
||||
" type:baseData.datatype,",
|
||||
"};",
|
||||
"const result=await core.localDatabase.putDBEntry(saveData,false,baseRev);",
|
||||
"if(!result?.ok) throw new Error(`Could not store conflicted hidden file: ${path}`);",
|
||||
"return JSON.stringify({ok:true,rev:result.rev});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function createHiddenJsonConflict(
|
||||
context: RunnerContext,
|
||||
session: ObsidianLiveSyncSession,
|
||||
vault: TemporaryVault,
|
||||
path: string,
|
||||
base: string,
|
||||
left: string,
|
||||
right: string
|
||||
): Promise<void> {
|
||||
await writeVaultFile(vault.path, path, base);
|
||||
await scanHiddenStorage(context.cliBinary, session.cliEnv);
|
||||
const baseEntry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
|
||||
|
||||
await writeVaultFile(vault.path, path, left);
|
||||
await scanHiddenStorage(context.cliBinary, session.cliEnv);
|
||||
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
|
||||
|
||||
await writeVaultFile(vault.path, path, right);
|
||||
await storeHiddenFileAsConflict(context.cliBinary, session.cliEnv, path, baseEntry.rev);
|
||||
}
|
||||
|
||||
async function startConfiguredSession(
|
||||
context: RunnerContext,
|
||||
vault: TemporaryVault,
|
||||
overrides: Record<string, unknown> = {}
|
||||
): Promise<ObsidianLiveSyncSession> {
|
||||
const session = await startObsidianLiveSyncSession({
|
||||
binary: context.binary,
|
||||
cliBinary: context.cliBinary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
||||
await configureCouchDb(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
{
|
||||
uri: context.couchDb.uri,
|
||||
username: context.couchDb.username,
|
||||
password: context.couchDb.password,
|
||||
dbName: context.dbName,
|
||||
},
|
||||
{
|
||||
syncInternalFiles: true,
|
||||
syncInternalFilesBeforeReplication: true,
|
||||
watchInternalFileChanges: false,
|
||||
syncInternalFilesTargetPatterns: "",
|
||||
...overrides,
|
||||
}
|
||||
);
|
||||
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
||||
await prepareRemote(context.cliBinary, session.cliEnv);
|
||||
return session;
|
||||
}
|
||||
|
||||
async function uploadHiddenFile(
|
||||
context: RunnerContext,
|
||||
session: ObsidianLiveSyncSession,
|
||||
path: string
|
||||
): Promise<LocalDatabaseEntry> {
|
||||
await scanHiddenStorage(context.cliBinary, session.cliEnv);
|
||||
const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path, { hidden: true });
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
|
||||
const ids = new Set(docs.map((doc) => doc._id));
|
||||
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
|
||||
});
|
||||
return entry;
|
||||
}
|
||||
|
||||
async function pullAndApplyHiddenFiles(
|
||||
context: RunnerContext,
|
||||
session: ObsidianLiveSyncSession,
|
||||
options: { resolveConflicts?: boolean } = {}
|
||||
): Promise<void> {
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
if (options.resolveConflicts === true) {
|
||||
await resolveHiddenConflicts(context.cliBinary, session.cliEnv);
|
||||
}
|
||||
await scanHiddenDatabase(context.cliBinary, session.cliEnv);
|
||||
}
|
||||
|
||||
async function runCreateRoundTrip(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
|
||||
let session = await startConfiguredSession(context, vaultA);
|
||||
const entry = await uploadHiddenFile(context, session, snippetPath);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB);
|
||||
await pullAndApplyHiddenFiles(context, session);
|
||||
const received = await waitForPathContent(vaultB.path, snippetPath, (content) => content === snippetContent);
|
||||
await session.app.stop();
|
||||
|
||||
assertEqual(received, snippetContent, "Hidden snippet content did not round-trip to the second vault.");
|
||||
console.log(`Hidden create round-trip copied ${entry.id} to the second vault.`);
|
||||
}
|
||||
|
||||
async function runDeleteRoundTrip(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
await removeVaultFile(vaultA.path, snippetPath);
|
||||
let session = await startConfiguredSession(context, vaultA);
|
||||
await scanHiddenStorage(context.cliBinary, session.cliEnv);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB);
|
||||
await pullAndApplyHiddenFiles(context, session);
|
||||
await waitForPathDeleted(vaultB.path, snippetPath);
|
||||
await session.app.stop();
|
||||
|
||||
console.log("Hidden delete round-trip removed the snippet from the second vault.");
|
||||
}
|
||||
|
||||
async function runJsonConflictRoundTrip(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
const base = JSON.stringify({ base: true, fromA: false, fromB: false }, null, 4) + "\n";
|
||||
const left = JSON.stringify({ base: true, fromA: true, fromB: false }, null, 4) + "\n";
|
||||
const right = JSON.stringify({ base: true, fromA: false, fromB: true }, null, 4) + "\n";
|
||||
|
||||
let session = await startConfiguredSession(context, vaultB);
|
||||
await createHiddenJsonConflict(context, session, vaultB, mergeJsonPath, base, left, right);
|
||||
await autoMergeHiddenJsonConflict(context.cliBinary, session.cliEnv, mergeJsonPath);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
const mergedOnB = await waitForPathContent(vaultB.path, mergeJsonPath, (content) =>
|
||||
hasJsonValues(content, { fromA: true, fromB: true })
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultA);
|
||||
await pullAndApplyHiddenFiles(context, session);
|
||||
const mergedOnA = await waitForPathContent(vaultA.path, mergeJsonPath, (content) =>
|
||||
hasJsonValues(content, { fromA: true, fromB: true })
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
assertEqual(mergedOnA, mergedOnB, "Merged hidden JSON content was not consistent across both vaults.");
|
||||
console.log("Hidden JSON conflict was automatically merged and round-tripped.");
|
||||
}
|
||||
|
||||
async function runJsonManualConflictResolution(context: RunnerContext, vault: TemporaryVault): Promise<void> {
|
||||
const base = JSON.stringify({ shared: "base" }, null, 4) + "\n";
|
||||
const left = JSON.stringify({ shared: "left", fromA: true }, null, 4) + "\n";
|
||||
const right = JSON.stringify({ shared: "right", fromB: true }, null, 4) + "\n";
|
||||
|
||||
const session = await startConfiguredSession(context, vault);
|
||||
await createHiddenJsonConflict(context, session, vault, manualMergeJsonPath, base, left, right);
|
||||
await openHiddenJsonResolveModal(context.cliBinary, session.cliEnv, manualMergeJsonPath);
|
||||
await clickJsonResolveOption(obsidianRemoteDebuggingPort(), "AB");
|
||||
|
||||
const merged = await waitForPathContent(vault.path, manualMergeJsonPath, (content) =>
|
||||
hasJsonValues(content, { shared: "right", fromA: true, fromB: true })
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
const parsed = JSON.parse(merged);
|
||||
assertEqual(parsed.shared, "right", "Manual JSON conflict resolution did not apply the selected merged result.");
|
||||
assertEqual(parsed.fromA, true, "Manual JSON conflict resolution lost the first-side value.");
|
||||
assertEqual(parsed.fromB, true, "Manual JSON conflict resolution lost the second-side value.");
|
||||
console.log("Hidden JSON conflict modal applied the selected merged result.");
|
||||
}
|
||||
|
||||
async function runTargetMismatch(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
const targetContent = JSON.stringify({ onlyA: true, targetMismatch: true }, null, 4) + "\n";
|
||||
await writeVaultFile(vaultA.path, targetPath, targetContent);
|
||||
|
||||
let session = await startConfiguredSession(context, vaultA);
|
||||
try {
|
||||
await uploadHiddenFile(context, session, targetPath);
|
||||
} finally {
|
||||
await session.app.stop();
|
||||
}
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, {
|
||||
syncInternalFilesTargetPatterns: "snippets",
|
||||
});
|
||||
try {
|
||||
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
|
||||
assertEqual(
|
||||
await pathExists(vaultB.path, targetPath),
|
||||
false,
|
||||
"Hidden file was applied on a device where it was not a target file."
|
||||
);
|
||||
} finally {
|
||||
await session.app.stop();
|
||||
}
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, {
|
||||
syncInternalFilesTargetPatterns: "",
|
||||
});
|
||||
let received = "";
|
||||
try {
|
||||
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
|
||||
received = await waitForPathContent(vaultB.path, targetPath, (content) => content === targetContent);
|
||||
} finally {
|
||||
await session.app.stop();
|
||||
}
|
||||
|
||||
assertEqual(received, targetContent, "Hidden file was not applied after it became a target file.");
|
||||
console.log("Hidden target mismatch respected per-device target patterns, then applied after enabling the target.");
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const couchDb = await loadCouchDbConfig();
|
||||
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "hidden-roundtrip");
|
||||
const vaultA = await createTemporaryVault();
|
||||
const vaultB = await createTemporaryVault();
|
||||
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
|
||||
|
||||
try {
|
||||
await assertCouchDbReachable(couchDb);
|
||||
await createCouchDbDatabase(couchDb, dbName);
|
||||
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault A: ${vaultA.path}`);
|
||||
console.log(`Temporary vault B: ${vaultB.path}`);
|
||||
console.log(`Temporary CouchDB database: ${dbName}`);
|
||||
|
||||
await runCreateRoundTrip(context, vaultA, vaultB);
|
||||
await runDeleteRoundTrip(context, vaultA, vaultB);
|
||||
await runJsonConflictRoundTrip(context, vaultA, vaultB);
|
||||
await runJsonManualConflictResolution(context, vaultB);
|
||||
await runTargetMismatch(context, vaultA, vaultB);
|
||||
} finally {
|
||||
await vaultA.dispose();
|
||||
await vaultB.dispose();
|
||||
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
|
||||
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,121 @@
|
||||
import { createWriteStream, existsSync } from "node:fs";
|
||||
import { chmod, mkdir } from "node:fs/promises";
|
||||
import { get } from "node:https";
|
||||
import { arch } from "node:process";
|
||||
import { basename, join, resolve } from "node:path";
|
||||
import { spawn } from "node:child_process";
|
||||
|
||||
const defaultVersion = "1.12.7";
|
||||
|
||||
function appImageArch(): string {
|
||||
const requestedArch = process.env.E2E_OBSIDIAN_APPIMAGE_ARCH?.trim();
|
||||
if (requestedArch) {
|
||||
return requestedArch;
|
||||
}
|
||||
if (arch === "arm64") {
|
||||
return "arm64";
|
||||
}
|
||||
if (arch === "x64") {
|
||||
return "x86_64";
|
||||
}
|
||||
throw new Error(`Unsupported architecture for Obsidian AppImage: ${arch}`);
|
||||
}
|
||||
|
||||
function appImageUrl(version: string, imageArch: string): string {
|
||||
return `https://github.com/obsidianmd/obsidian-releases/releases/download/v${version}/Obsidian-${version}-${imageArch}.AppImage`;
|
||||
}
|
||||
|
||||
function download(url: string, destination: string, redirectsLeft = 5): Promise<void> {
|
||||
return new Promise((resolveDownload, reject) => {
|
||||
const request = get(url, (response) => {
|
||||
const statusCode = response.statusCode ?? 0;
|
||||
const location = response.headers.location;
|
||||
if (statusCode >= 300 && statusCode < 400 && location) {
|
||||
response.resume();
|
||||
if (redirectsLeft <= 0) {
|
||||
reject(new Error(`Too many redirects while downloading ${url}`));
|
||||
return;
|
||||
}
|
||||
download(new URL(location, url).toString(), destination, redirectsLeft - 1)
|
||||
.then(resolveDownload)
|
||||
.catch(reject);
|
||||
return;
|
||||
}
|
||||
if (statusCode !== 200) {
|
||||
response.resume();
|
||||
reject(new Error(`Failed to download ${url}: HTTP ${statusCode}`));
|
||||
return;
|
||||
}
|
||||
|
||||
const file = createWriteStream(destination, { mode: 0o755 });
|
||||
response.pipe(file);
|
||||
file.on("finish", () => {
|
||||
file.close((error) => {
|
||||
if (error) {
|
||||
reject(error);
|
||||
} else {
|
||||
resolveDownload();
|
||||
}
|
||||
});
|
||||
});
|
||||
file.on("error", reject);
|
||||
});
|
||||
request.on("error", reject);
|
||||
});
|
||||
}
|
||||
|
||||
function extractAppImage(appImagePath: string, cwd: string): Promise<void> {
|
||||
return new Promise((resolveExtract, reject) => {
|
||||
const child = spawn(appImagePath, ["--appimage-extract"], {
|
||||
cwd,
|
||||
stdio: "inherit",
|
||||
});
|
||||
child.on("error", reject);
|
||||
child.on("exit", (code, signal) => {
|
||||
if (code === 0) {
|
||||
resolveExtract();
|
||||
return;
|
||||
}
|
||||
reject(new Error(`AppImage extraction failed. code=${code}, signal=${signal}`));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const version = process.env.E2E_OBSIDIAN_VERSION?.trim() || defaultVersion;
|
||||
const imageArch = appImageArch();
|
||||
const targetDir = resolve(process.env.E2E_OBSIDIAN_DOWNLOAD_DIR?.trim() || "_testdata/obsidian");
|
||||
const url = process.env.E2E_OBSIDIAN_APPIMAGE_URL?.trim() || appImageUrl(version, imageArch);
|
||||
const appImagePath = join(targetDir, basename(new URL(url).pathname));
|
||||
const extractedBinary = join(targetDir, "squashfs-root", "obsidian");
|
||||
const forceDownload = process.env.E2E_OBSIDIAN_FORCE_DOWNLOAD === "true";
|
||||
const skipExtract = process.env.E2E_OBSIDIAN_SKIP_EXTRACT === "true";
|
||||
|
||||
await mkdir(targetDir, { recursive: true });
|
||||
|
||||
if (!existsSync(appImagePath) || forceDownload) {
|
||||
console.log(`Downloading Obsidian AppImage: ${url}`);
|
||||
console.log(`Destination: ${appImagePath}`);
|
||||
await download(url, appImagePath);
|
||||
await chmod(appImagePath, 0o755);
|
||||
} else {
|
||||
console.log(`Using existing Obsidian AppImage: ${appImagePath}`);
|
||||
}
|
||||
|
||||
if (!skipExtract) {
|
||||
if (existsSync(extractedBinary)) {
|
||||
console.log(`Using existing extracted Obsidian binary: ${extractedBinary}`);
|
||||
} else {
|
||||
console.log(`Extracting Obsidian AppImage in ${targetDir}`);
|
||||
await extractAppImage(appImagePath, targetDir);
|
||||
console.log(`Extracted Obsidian binary: ${extractedBinary}`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`Set OBSIDIAN_BINARY=${extractedBinary} to use the extracted binary explicitly.`);
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,105 @@
|
||||
import { spawn } from "node:child_process";
|
||||
|
||||
type Step = {
|
||||
name: string;
|
||||
args: string[];
|
||||
optional?: boolean;
|
||||
};
|
||||
|
||||
const testSteps: Step[] = [
|
||||
{ name: "build", args: ["run", "build"] },
|
||||
{ name: "discover", args: ["run", "test:e2e:obsidian:discover"] },
|
||||
{ name: "smoke", args: ["run", "test:e2e:obsidian:smoke"] },
|
||||
{ name: "vault reflection", args: ["run", "test:e2e:obsidian:vault-reflection"] },
|
||||
{ name: "CouchDB upload", args: ["run", "test:e2e:obsidian:couchdb-upload"] },
|
||||
{ name: "Object Storage upload", args: ["run", "test:e2e:obsidian:minio-upload"] },
|
||||
{ name: "startup scan", args: ["run", "test:e2e:obsidian:startup-scan"] },
|
||||
{ name: "two-vault synchronisation", args: ["run", "test:e2e:obsidian:two-vault-sync"] },
|
||||
{ name: "hidden file snippet synchronisation", args: ["run", "test:e2e:obsidian:hidden-file-snippet-sync"] },
|
||||
{ name: "Customisation Sync", args: ["run", "test:e2e:obsidian:customisation-sync"] },
|
||||
{ name: "setting Markdown export", args: ["run", "test:e2e:obsidian:setting-markdown-export"] },
|
||||
];
|
||||
|
||||
const manageCouchDb = process.argv.includes("--manage-couchdb") || process.argv.includes("--manage-services");
|
||||
const manageMinio = process.argv.includes("--manage-minio") || process.argv.includes("--manage-services");
|
||||
const keepServices = process.argv.includes("--keep-services");
|
||||
const keepCouchDb = keepServices || process.argv.includes("--keep-couchdb");
|
||||
const keepMinio = keepServices || process.argv.includes("--keep-minio");
|
||||
|
||||
function npmBinary(): string {
|
||||
return process.platform === "win32" ? "npm.cmd" : "npm";
|
||||
}
|
||||
|
||||
function runStep(step: Step): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
console.log(`\n# ${step.name}`);
|
||||
const child = spawn(npmBinary(), step.args, {
|
||||
cwd: process.cwd(),
|
||||
env: process.env,
|
||||
stdio: "inherit",
|
||||
});
|
||||
child.on("error", reject);
|
||||
child.on("exit", (code, signal) => {
|
||||
if (code === 0) {
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
const message = `${step.name} failed with ${signal ? `signal ${signal}` : `exit code ${code}`}`;
|
||||
if (step.optional) {
|
||||
console.warn(message);
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
reject(new Error(message));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function stopManagedCouchDb(): Promise<void> {
|
||||
await runStep({
|
||||
name: "stop CouchDB fixture",
|
||||
args: ["run", "test:docker-couchdb:stop"],
|
||||
optional: true,
|
||||
});
|
||||
}
|
||||
|
||||
async function stopManagedMinio(): Promise<void> {
|
||||
await runStep({
|
||||
name: "stop MinIO fixture",
|
||||
args: ["run", "test:docker-s3:stop"],
|
||||
optional: true,
|
||||
});
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
let shouldStopCouchDb = false;
|
||||
let shouldStopMinio = false;
|
||||
try {
|
||||
if (manageCouchDb) {
|
||||
await stopManagedCouchDb();
|
||||
await runStep({ name: "start CouchDB fixture", args: ["run", "test:docker-couchdb:start"] });
|
||||
shouldStopCouchDb = !keepCouchDb;
|
||||
}
|
||||
if (manageMinio) {
|
||||
await stopManagedMinio();
|
||||
await runStep({ name: "start MinIO fixture", args: ["run", "test:docker-s3:start"] });
|
||||
shouldStopMinio = !keepMinio;
|
||||
}
|
||||
|
||||
for (const step of testSteps) {
|
||||
await runStep(step);
|
||||
}
|
||||
} finally {
|
||||
if (shouldStopMinio) {
|
||||
await stopManagedMinio();
|
||||
}
|
||||
if (shouldStopCouchDb) {
|
||||
await stopManagedCouchDb();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,142 @@
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
configureObjectStorage,
|
||||
prepareRemote,
|
||||
pushLocalChanges,
|
||||
waitForLiveSyncCoreReady,
|
||||
type LocalDatabaseEntry,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import {
|
||||
deleteObjectStoragePrefix,
|
||||
ensureObjectStorageBucket,
|
||||
listObjectStorageObjects,
|
||||
loadObjectStorageConfig,
|
||||
makeUniqueBucketPrefix,
|
||||
} from "../runner/objectStorage.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
|
||||
const notePath = "E2E/minio-upload.md";
|
||||
const noteContent = [
|
||||
"# Object Storage upload from real Obsidian",
|
||||
"",
|
||||
"This note is created through Obsidian and uploaded by Self-hosted LiveSync to S3-compatible Object Storage.",
|
||||
"The test is intentionally small, but it crosses the real Obsidian, Journal Sync, and AWS SDK boundary.",
|
||||
`Created at: ${new Date().toISOString()}`,
|
||||
"",
|
||||
].join("\n");
|
||||
|
||||
async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.ProcessEnv): Promise<LocalDatabaseEntry> {
|
||||
return await evalObsidianJson<LocalDatabaseEntry>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(notePath)};`,
|
||||
`const content=${JSON.stringify(noteContent)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
|
||||
"const existing=app.vault.getAbstractFileByPath(path);",
|
||||
"if(existing) await app.vault.delete(existing);",
|
||||
"await app.vault.create(path,content);",
|
||||
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
|
||||
"let entry=false;",
|
||||
"for(let i=0;i<40;i++){",
|
||||
"await core.services.fileProcessing.commitPendingFileEvents();",
|
||||
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true).catch(()=>false);",
|
||||
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
|
||||
"await sleep(250);",
|
||||
"}",
|
||||
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
|
||||
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function waitForObjectStorageObjects(prefix: string): Promise<string[]> {
|
||||
const objectStorage = await loadObjectStorageConfig();
|
||||
const timeoutMs = Number(process.env.E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS ?? 20000);
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let keys: string[] = [];
|
||||
while (Date.now() < deadline) {
|
||||
const objects = await listObjectStorageObjects(objectStorage, prefix);
|
||||
keys = objects.flatMap((object) => (object.Key ? [object.Key] : []));
|
||||
if (keys.length > 0) {
|
||||
return keys;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
}
|
||||
throw new Error(`Timed out waiting for Object Storage objects under ${prefix}. Last keys: ${keys.join(", ")}`);
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const objectStorage = await loadObjectStorageConfig();
|
||||
const bucketPrefix = makeUniqueBucketPrefix("minio-upload");
|
||||
const vault = await createTemporaryVault();
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
|
||||
try {
|
||||
await ensureObjectStorageBucket(objectStorage);
|
||||
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault: ${vault.path}`);
|
||||
console.log(`Temporary Object Storage bucket: ${objectStorage.bucket}`);
|
||||
console.log(`Temporary Object Storage prefix: ${bucketPrefix}`);
|
||||
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
|
||||
const configured = await configureObjectStorage(cli.binary, session.cliEnv, {
|
||||
...objectStorage,
|
||||
bucketPrefix,
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
|
||||
assertEqual(configured.remoteType, "MINIO", "Remote type was not Object Storage.");
|
||||
assertEqual(configured.endpoint, objectStorage.endpoint, "Configured Object Storage endpoint did not match.");
|
||||
assertEqual(configured.bucket, objectStorage.bucket, "Configured Object Storage bucket did not match.");
|
||||
assertEqual(configured.bucketPrefix, bucketPrefix, "Configured Object Storage bucket prefix did not match.");
|
||||
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during this one-shot workflow.");
|
||||
|
||||
await prepareRemote(cli.binary, session.cliEnv);
|
||||
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
|
||||
await pushLocalChanges(cli.binary, session.cliEnv);
|
||||
|
||||
const keys = await waitForObjectStorageObjects(bucketPrefix);
|
||||
|
||||
console.log(
|
||||
`Uploaded ${localEntry.path} through Journal Sync to ${objectStorage.bucket}/${bucketPrefix} (${keys.length} object(s))`
|
||||
);
|
||||
} finally {
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
}
|
||||
await vault.dispose();
|
||||
if (process.env.E2E_OBSIDIAN_KEEP_OBJECT_STORAGE !== "true") {
|
||||
await deleteObjectStoragePrefix(objectStorage, bucketPrefix).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,106 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import { assertEqual, waitForLiveSyncCoreReady } from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
|
||||
const settingPath = "LiveSync/settings-export.md";
|
||||
|
||||
async function waitForFileContaining(
|
||||
vaultPath: string,
|
||||
path: string,
|
||||
predicates: ((content: string) => boolean)[],
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
|
||||
): Promise<string> {
|
||||
const fullPath = join(vaultPath, path);
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastContent = "";
|
||||
let lastError: unknown;
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
lastContent = await readFile(fullPath, "utf-8");
|
||||
if (predicates.every((predicate) => predicate(lastContent))) {
|
||||
return lastContent;
|
||||
}
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for setting Markdown: ${fullPath}\nLast error: ${String(lastError)}`);
|
||||
}
|
||||
|
||||
async function configureSettingMarkdown(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"await core.services.setting.applyExternalSettings({",
|
||||
`settingSyncFile:${JSON.stringify(settingPath)},`,
|
||||
"writeCredentialsForSettingSync:false,",
|
||||
"couchDB_USER:'e2e-user',",
|
||||
"couchDB_PASSWORD:'e2e-password',",
|
||||
"passphrase:'e2e-passphrase',",
|
||||
"showVerboseLog:true,",
|
||||
"},true);",
|
||||
"await core.services.setting.saveSettingData();",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const vault = await createTemporaryVault();
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
try {
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault: ${vault.path}`);
|
||||
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
|
||||
await configureSettingMarkdown(cli.binary, session.cliEnv);
|
||||
const content = await waitForFileContaining(vault.path, settingPath, [
|
||||
(value) => value.includes("````yaml:livesync-setting"),
|
||||
(value) => value.includes(`settingSyncFile: ${settingPath}`),
|
||||
(value) => value.includes("showVerboseLog: true"),
|
||||
]);
|
||||
|
||||
assertEqual(
|
||||
content.includes("couchDB_PASSWORD: e2e-password"),
|
||||
false,
|
||||
"Credential leaked into setting Markdown."
|
||||
);
|
||||
assertEqual(content.includes("passphrase: e2e-passphrase"), false, "Passphrase leaked into setting Markdown.");
|
||||
|
||||
console.log(`Generated setting Markdown without credentials: ${settingPath}`);
|
||||
} finally {
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
}
|
||||
await vault.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,41 @@
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
const vault = await createTemporaryVault();
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
try {
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault: ${vault.path}`);
|
||||
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
console.log(`Installed plug-in artifacts: ${session.install.copied.join(", ")}`);
|
||||
const { readiness } = session;
|
||||
console.log(
|
||||
`Obsidian plug-in ready: ${readiness.pluginId}@${readiness.pluginVersion} in ${readiness.vaultName}`
|
||||
);
|
||||
await new Promise((resolve) => setTimeout(resolve, Number(process.env.E2E_OBSIDIAN_SMOKE_TIMEOUT_MS ?? 1000)));
|
||||
console.log("Obsidian stayed alive after the plug-in readiness check.");
|
||||
} finally {
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
}
|
||||
await vault.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,116 @@
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import {
|
||||
assertCouchDbReachable,
|
||||
createCouchDbDatabase,
|
||||
deleteCouchDbDatabase,
|
||||
loadCouchDbConfig,
|
||||
makeUniqueDatabaseName,
|
||||
waitForCouchDbDocs,
|
||||
} from "../runner/couchdb.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
configureCouchDb,
|
||||
prepareRemote,
|
||||
pushLocalChanges,
|
||||
waitForLiveSyncCoreReady,
|
||||
waitForLocalDatabaseEntry,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
|
||||
const notePath = "E2E/startup-scan.md";
|
||||
const noteContent = [
|
||||
"# Startup scan",
|
||||
"",
|
||||
"This note was written while Obsidian was stopped.",
|
||||
"The test verifies that the next real Obsidian boot scans it into the local database.",
|
||||
`Created at: ${new Date().toISOString()}`,
|
||||
"",
|
||||
].join("\n");
|
||||
|
||||
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
|
||||
const fullPath = join(vaultPath, path);
|
||||
await mkdir(dirname(fullPath), { recursive: true });
|
||||
await writeFile(fullPath, content, "utf-8");
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const couchDb = await loadCouchDbConfig();
|
||||
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "startup-scan");
|
||||
const vault = await createTemporaryVault();
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
|
||||
try {
|
||||
await assertCouchDbReachable(couchDb);
|
||||
await createCouchDbDatabase(couchDb, dbName);
|
||||
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault: ${vault.path}`);
|
||||
console.log(`Temporary CouchDB database: ${dbName}`);
|
||||
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
const configured = await configureCouchDb(cli.binary, session.cliEnv, {
|
||||
uri: couchDb.uri,
|
||||
username: couchDb.username,
|
||||
password: couchDb.password,
|
||||
dbName,
|
||||
});
|
||||
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not configured.");
|
||||
await prepareRemote(cli.binary, session.cliEnv);
|
||||
await session.app.stop();
|
||||
session = undefined;
|
||||
|
||||
await writeVaultFile(vault.path, notePath, noteContent);
|
||||
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
|
||||
|
||||
const localEntry = await waitForLocalDatabaseEntry(cli.binary, session.cliEnv, notePath);
|
||||
await pushLocalChanges(cli.binary, session.cliEnv);
|
||||
|
||||
const remoteDocs = await waitForCouchDbDocs(couchDb, dbName, (docs) => {
|
||||
const ids = new Set(docs.map((doc) => doc._id));
|
||||
return ids.has(localEntry.id) && localEntry.children.every((childId) => ids.has(childId));
|
||||
});
|
||||
const remoteMetadata = remoteDocs.find((doc) => doc._id === localEntry.id);
|
||||
assertEqual(remoteMetadata?.path, localEntry.path, "Startup-scanned remote metadata path did not match.");
|
||||
|
||||
console.log(`Startup scan uploaded metadata ${localEntry.id} and ${localEntry.children.length} chunk(s).`);
|
||||
} finally {
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
}
|
||||
await vault.dispose();
|
||||
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
|
||||
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,525 @@
|
||||
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import {
|
||||
assertCouchDbReachable,
|
||||
createCouchDbDatabase,
|
||||
deleteCouchDbDatabase,
|
||||
loadCouchDbConfig,
|
||||
makeUniqueDatabaseName,
|
||||
waitForCouchDbDocs,
|
||||
type CouchDbConfig,
|
||||
} from "../runner/couchdb.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import {
|
||||
assertEqual,
|
||||
configureCouchDb,
|
||||
prepareRemote,
|
||||
pushLocalChanges,
|
||||
waitForLiveSyncCoreReady,
|
||||
waitForLocalDatabaseEntry,
|
||||
type LocalDatabaseEntry,
|
||||
} from "../runner/liveSyncWorkflow.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
|
||||
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
|
||||
|
||||
const createPath = "E2E/two-vault/create.md";
|
||||
const updatePath = "E2E/two-vault/update.md";
|
||||
const deletePath = "E2E/two-vault/delete.md";
|
||||
const renameFromPath = "E2E/two-vault/rename-source.md";
|
||||
const renameToPath = "E2E/two-vault/renamed/rename-target.md";
|
||||
const conflictPath = "E2E/two-vault/conflict.md";
|
||||
const targetMismatchPath = "E2E/two-vault/target-mismatch.md";
|
||||
const encryptedPath = "E2E/two-vault/encrypted.md";
|
||||
|
||||
type RunnerContext = {
|
||||
binary: string;
|
||||
cliBinary: string;
|
||||
couchDb: CouchDbConfig;
|
||||
dbName: string;
|
||||
};
|
||||
|
||||
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
|
||||
const fullPath = join(vaultPath, path);
|
||||
await mkdir(dirname(fullPath), { recursive: true });
|
||||
await writeFile(fullPath, content, "utf-8");
|
||||
}
|
||||
|
||||
async function removeVaultFile(vaultPath: string, path: string): Promise<void> {
|
||||
await rm(join(vaultPath, path), { force: true });
|
||||
}
|
||||
|
||||
async function readVaultFile(vaultPath: string, path: string): Promise<string> {
|
||||
return await readFile(join(vaultPath, path), "utf-8");
|
||||
}
|
||||
|
||||
async function pathExists(vaultPath: string, path: string): Promise<boolean> {
|
||||
try {
|
||||
await readFile(join(vaultPath, path));
|
||||
return true;
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
|
||||
return false;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForPathContent(
|
||||
vaultPath: string,
|
||||
path: string,
|
||||
predicate: (content: string) => boolean,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
|
||||
): Promise<string> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastContent = "";
|
||||
while (Date.now() < deadline) {
|
||||
if (await pathExists(vaultPath, path)) {
|
||||
lastContent = await readVaultFile(vaultPath, path);
|
||||
if (predicate(lastContent)) {
|
||||
return lastContent;
|
||||
}
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
|
||||
}
|
||||
|
||||
async function waitForPathDeleted(
|
||||
vaultPath: string,
|
||||
path: string,
|
||||
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
|
||||
): Promise<void> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
while (Date.now() < deadline) {
|
||||
if (!(await pathExists(vaultPath, path))) {
|
||||
return;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
|
||||
}
|
||||
|
||||
async function writeNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string, content: string) {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
`const content=${JSON.stringify(content)};`,
|
||||
"const folder=path.split('/').slice(0,-1).join('/');",
|
||||
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
|
||||
"const existing=app.vault.getAbstractFileByPath(path);",
|
||||
"if(existing) await app.vault.modify(existing,content);",
|
||||
"else await app.vault.create(path,content);",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function deleteNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string) {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
"const existing=app.vault.getAbstractFileByPath(path);",
|
||||
"if(existing) await app.vault.delete(existing);",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function renameNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, fromPath: string, toPath: string) {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const fromPath=${JSON.stringify(fromPath)};`,
|
||||
`const toPath=${JSON.stringify(toPath)};`,
|
||||
"const folder=toPath.split('/').slice(0,-1).join('/');",
|
||||
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
|
||||
"const existing=app.vault.getAbstractFileByPath(fromPath);",
|
||||
"if(!existing) throw new Error(`Could not find note to rename: ${fromPath}`);",
|
||||
"await app.vault.rename(existing,toPath);",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function startConfiguredSession(
|
||||
context: RunnerContext,
|
||||
vault: TemporaryVault,
|
||||
overrides: Record<string, unknown> = {}
|
||||
): Promise<ObsidianLiveSyncSession> {
|
||||
const session = await startObsidianLiveSyncSession({
|
||||
binary: context.binary,
|
||||
cliBinary: context.cliBinary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
||||
await configureCouchDb(
|
||||
context.cliBinary,
|
||||
session.cliEnv,
|
||||
{
|
||||
uri: context.couchDb.uri,
|
||||
username: context.couchDb.username,
|
||||
password: context.couchDb.password,
|
||||
dbName: context.dbName,
|
||||
},
|
||||
overrides
|
||||
);
|
||||
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
|
||||
await prepareRemote(context.cliBinary, session.cliEnv);
|
||||
return session;
|
||||
}
|
||||
|
||||
async function uploadNote(
|
||||
context: RunnerContext,
|
||||
session: ObsidianLiveSyncSession,
|
||||
path: string
|
||||
): Promise<LocalDatabaseEntry> {
|
||||
const entry = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) => {
|
||||
const ids = new Set(docs.map((doc) => doc._id));
|
||||
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
|
||||
});
|
||||
return entry;
|
||||
}
|
||||
|
||||
async function syncAndApply(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
}
|
||||
|
||||
async function storeFileRevision(
|
||||
cliBinary: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
path: string,
|
||||
content: string,
|
||||
baseRev?: string
|
||||
): Promise<string> {
|
||||
const result = await evalObsidianJson<{ rev: string }>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
`const content=${JSON.stringify(content)};`,
|
||||
`const baseRev=${JSON.stringify(baseRev ?? "")};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const blob=new Blob([content],{type:'text/plain'});",
|
||||
"const id=await core.services.path.path2id(path);",
|
||||
"const now=Date.now();",
|
||||
"const result=await core.localDatabase.putDBEntry({",
|
||||
" _id:id,",
|
||||
" path,",
|
||||
" data:blob,",
|
||||
" ctime:now,",
|
||||
" mtime:now,",
|
||||
" size:(await blob.arrayBuffer()).byteLength,",
|
||||
" children:[],",
|
||||
" datatype:'plain',",
|
||||
" type:'plain',",
|
||||
" eden:{},",
|
||||
"},false,baseRev||undefined);",
|
||||
"if(!result?.ok) throw new Error(`Could not store file revision: ${path}`);",
|
||||
"return JSON.stringify({ok:true,rev:result.rev});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
return result.rev;
|
||||
}
|
||||
|
||||
async function createMarkdownConflict(
|
||||
context: RunnerContext,
|
||||
session: ObsidianLiveSyncSession,
|
||||
vault: TemporaryVault,
|
||||
path: string,
|
||||
base: string,
|
||||
left: string,
|
||||
right: string
|
||||
): Promise<void> {
|
||||
const baseRev = await storeFileRevision(context.cliBinary, session.cliEnv, path, base);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, path);
|
||||
await storeFileRevision(context.cliBinary, session.cliEnv, path, left, baseRev);
|
||||
await storeFileRevision(context.cliBinary, session.cliEnv, path, right, baseRev);
|
||||
await writeVaultFile(vault.path, path, right);
|
||||
}
|
||||
|
||||
async function autoMergeMarkdownConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
|
||||
await evalObsidianJson<unknown>(
|
||||
cliBinary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(path)};`,
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
"const result=await core.localDatabase.managers.conflictManager.tryAutoMerge(path,true);",
|
||||
"if(!('result' in result)){",
|
||||
" throw new Error(`Markdown conflict was not auto-mergeable: ${path}; ${JSON.stringify(result)}`);",
|
||||
"}",
|
||||
"if(!(await core.databaseFileAccess.storeContent(path,result.result))){",
|
||||
" throw new Error(`Could not store merged Markdown content: ${path}`);",
|
||||
"}",
|
||||
"if(!(await core.fileHandler.deleteRevisionFromDB(path,result.conflictedRev))){",
|
||||
" throw new Error(`Could not delete conflicted revision: ${path}`);",
|
||||
"}",
|
||||
"if(!(await core.fileHandler.dbToStorage(path,path,true))){",
|
||||
" throw new Error(`Could not reflect merged Markdown content: ${path}`);",
|
||||
"}",
|
||||
"return JSON.stringify({ok:true});",
|
||||
"})()",
|
||||
].join(""),
|
||||
env
|
||||
);
|
||||
}
|
||||
|
||||
async function runCreateUpdateDelete(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
const createdContent = "# Created on A\n\nThis note should appear on B.\n";
|
||||
let session = await startConfiguredSession(context, vaultA);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, createPath, createdContent);
|
||||
await uploadNote(context, session, createPath);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB);
|
||||
await syncAndApply(context, session);
|
||||
const createdOnB = await waitForPathContent(vaultB.path, createPath, (content) => content === createdContent);
|
||||
await session.app.stop();
|
||||
assertEqual(createdOnB, createdContent, "Created note did not round-trip to the second vault.");
|
||||
|
||||
const initialUpdateContent = "# Update target\n\nInitial content.\n";
|
||||
const updatedContent = "# Update target\n\nUpdated content from A.\n";
|
||||
session = await startConfiguredSession(context, vaultA);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, updatePath, initialUpdateContent);
|
||||
await uploadNote(context, session, updatePath);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, updatePath, updatedContent);
|
||||
await uploadNote(context, session, updatePath);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB);
|
||||
await syncAndApply(context, session);
|
||||
const updatedOnB = await waitForPathContent(vaultB.path, updatePath, (content) => content === updatedContent);
|
||||
await session.app.stop();
|
||||
assertEqual(updatedOnB, updatedContent, "Updated note content did not round-trip to the second vault.");
|
||||
|
||||
const deleteContent = "# Delete target\n\nThis note should be removed from B.\n";
|
||||
session = await startConfiguredSession(context, vaultA);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, deletePath, deleteContent);
|
||||
await uploadNote(context, session, deletePath);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB);
|
||||
await syncAndApply(context, session);
|
||||
await waitForPathContent(vaultB.path, deletePath, (content) => content === deleteContent);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultA);
|
||||
await deleteNoteViaObsidian(context.cliBinary, session.cliEnv, deletePath);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB);
|
||||
await syncAndApply(context, session);
|
||||
await waitForPathDeleted(vaultB.path, deletePath);
|
||||
await session.app.stop();
|
||||
|
||||
console.log("Two-vault note creation, update, and deletion round-tripped.");
|
||||
}
|
||||
|
||||
async function runRename(context: RunnerContext, vaultA: TemporaryVault, vaultB: TemporaryVault): Promise<void> {
|
||||
const renamedContent = "# Rename target\n\nThis note should move from A to B.\n";
|
||||
|
||||
let session = await startConfiguredSession(context, vaultA);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renamedContent);
|
||||
await uploadNote(context, session, renameFromPath);
|
||||
await renameNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renameToPath);
|
||||
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, renameToPath);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB);
|
||||
await syncAndApply(context, session);
|
||||
const renamedOnB = await waitForPathContent(vaultB.path, renameToPath, (content) => content === renamedContent);
|
||||
await waitForPathDeleted(vaultB.path, renameFromPath);
|
||||
await session.app.stop();
|
||||
|
||||
assertEqual(renamedOnB, renamedContent, "Renamed note content did not round-trip to the second vault.");
|
||||
console.log("Two-vault note rename round-tripped.");
|
||||
}
|
||||
|
||||
async function runEncryptedRoundTrip(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
const encryptedContent = "# Encrypted round-trip\n\nThis note should synchronise with E2EE enabled.\n";
|
||||
const encryptedOverrides = {
|
||||
encrypt: true,
|
||||
passphrase: "real-obsidian-e2e-passphrase",
|
||||
usePathObfuscation: true,
|
||||
E2EEAlgorithm: "v2",
|
||||
};
|
||||
|
||||
let session = await startConfiguredSession(context, vaultA, encryptedOverrides);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, encryptedPath, encryptedContent);
|
||||
await uploadNote(context, session, encryptedPath);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, encryptedOverrides);
|
||||
await syncAndApply(context, session);
|
||||
const received = await waitForPathContent(vaultB.path, encryptedPath, (content) => content === encryptedContent);
|
||||
await session.app.stop();
|
||||
|
||||
assertEqual(received, encryptedContent, "Encrypted note did not round-trip to the second vault.");
|
||||
console.log("Two-vault encrypted note synchronisation round-tripped.");
|
||||
}
|
||||
|
||||
async function runMarkdownAutoMerge(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
const base = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nBottom anchor\n";
|
||||
const left = "# Conflict\n\nTop anchor\n\nLeft line\n\nMiddle anchor\n\nBottom anchor\n";
|
||||
const right = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nRight tail\n\nBottom anchor\n";
|
||||
|
||||
let session = await startConfiguredSession(context, vaultB);
|
||||
await createMarkdownConflict(context, session, vaultB, conflictPath, base, left, right);
|
||||
await autoMergeMarkdownConflict(context.cliBinary, session.cliEnv, conflictPath);
|
||||
await pushLocalChanges(context.cliBinary, session.cliEnv);
|
||||
const mergedOnB = await waitForPathContent(
|
||||
vaultB.path,
|
||||
conflictPath,
|
||||
(content) => content.includes("Left line") && content.includes("Right tail"),
|
||||
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultA);
|
||||
await syncAndApply(context, session);
|
||||
const mergedOnA = await waitForPathContent(
|
||||
vaultA.path,
|
||||
conflictPath,
|
||||
(content) => content.includes("Left line") && content.includes("Right tail"),
|
||||
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
assertEqual(mergedOnA, mergedOnB, "Merged Markdown content was not consistent across both vaults.");
|
||||
console.log("Markdown conflict was automatically merged and propagated by the next synchronisation.");
|
||||
}
|
||||
|
||||
async function runTargetMismatch(
|
||||
context: RunnerContext,
|
||||
vaultA: TemporaryVault,
|
||||
vaultB: TemporaryVault
|
||||
): Promise<void> {
|
||||
const ignoredContent = "# Target mismatch\n\nB should ignore this revision.\n";
|
||||
const acceptedContent = "# Target mismatch\n\nB should accept this revision after its target filter changes.\n";
|
||||
|
||||
let session = await startConfiguredSession(context, vaultA);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, targetMismatchPath, ignoredContent);
|
||||
await uploadNote(context, session, targetMismatchPath);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, {
|
||||
syncOnlyRegEx: "^E2E/two-vault/allowed/.*",
|
||||
});
|
||||
await syncAndApply(context, session);
|
||||
assertEqual(
|
||||
await pathExists(vaultB.path, targetMismatchPath),
|
||||
false,
|
||||
"A note was reflected on a device where it was not a target file."
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultA);
|
||||
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, targetMismatchPath, acceptedContent);
|
||||
await uploadNote(context, session, targetMismatchPath);
|
||||
await session.app.stop();
|
||||
|
||||
session = await startConfiguredSession(context, vaultB, {
|
||||
syncOnlyRegEx: "",
|
||||
});
|
||||
await syncAndApply(context, session);
|
||||
const received = await waitForPathContent(
|
||||
vaultB.path,
|
||||
targetMismatchPath,
|
||||
(content) => content === acceptedContent
|
||||
);
|
||||
await session.app.stop();
|
||||
|
||||
assertEqual(received, acceptedContent, "Target file was not reflected after the device accepted the path.");
|
||||
console.log("Two-vault target mismatch skipped a non-target note, then reflected it after enabling the target.");
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const couchDb = await loadCouchDbConfig();
|
||||
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync");
|
||||
const encryptedDbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync-e2ee");
|
||||
const vaultA = await createTemporaryVault();
|
||||
const vaultB = await createTemporaryVault();
|
||||
const encryptedVaultA = await createTemporaryVault();
|
||||
const encryptedVaultB = await createTemporaryVault();
|
||||
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
|
||||
const encryptedContext: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName: encryptedDbName };
|
||||
|
||||
try {
|
||||
await assertCouchDbReachable(couchDb);
|
||||
await createCouchDbDatabase(couchDb, dbName);
|
||||
await createCouchDbDatabase(couchDb, encryptedDbName);
|
||||
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault A: ${vaultA.path}`);
|
||||
console.log(`Temporary vault B: ${vaultB.path}`);
|
||||
console.log(`Temporary CouchDB database: ${dbName}`);
|
||||
console.log(`Temporary encrypted CouchDB database: ${encryptedDbName}`);
|
||||
|
||||
await runCreateUpdateDelete(context, vaultA, vaultB);
|
||||
await runRename(context, vaultA, vaultB);
|
||||
if (process.env.E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT === "true") {
|
||||
await runMarkdownAutoMerge(context, vaultA, vaultB);
|
||||
}
|
||||
await runTargetMismatch(context, vaultA, vaultB);
|
||||
await runEncryptedRoundTrip(encryptedContext, encryptedVaultA, encryptedVaultB);
|
||||
} finally {
|
||||
await vaultA.dispose();
|
||||
await vaultB.dispose();
|
||||
await encryptedVaultA.dispose();
|
||||
await encryptedVaultB.dispose();
|
||||
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
|
||||
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
await deleteCouchDbDatabase(couchDb, encryptedDbName).catch((error: unknown) => {
|
||||
console.warn(error instanceof Error ? error.message : error);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,122 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { evalObsidianJson } from "../runner/cli.ts";
|
||||
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
|
||||
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
|
||||
import { createTemporaryVault } from "../runner/vault.ts";
|
||||
|
||||
type CreatedNote = {
|
||||
path: string;
|
||||
read: string;
|
||||
exists: boolean;
|
||||
};
|
||||
|
||||
type ReadNote = {
|
||||
exists: boolean;
|
||||
read: string | null;
|
||||
};
|
||||
|
||||
const notePath = "E2E/real-vault-reflection.md";
|
||||
const noteContent = [
|
||||
"# Real Obsidian E2E",
|
||||
"",
|
||||
"This note was created through Obsidian's own vault API.",
|
||||
`Created at: ${new Date().toISOString()}`,
|
||||
"",
|
||||
].join("\n");
|
||||
|
||||
async function waitForFileContent(vaultPath: string, path: string, expectedContent: string): Promise<void> {
|
||||
const fullPath = join(vaultPath, path);
|
||||
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000);
|
||||
let lastError: unknown;
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const content = await readFile(fullPath, "utf-8");
|
||||
if (content === expectedContent) {
|
||||
return;
|
||||
}
|
||||
lastError = new Error(`Unexpected content in ${fullPath}`);
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
throw new Error(`Timed out waiting for reflected vault file: ${fullPath}\nLast error: ${String(lastError)}`);
|
||||
}
|
||||
|
||||
function assertEqual(actual: unknown, expected: unknown, message: string): void {
|
||||
if (actual !== expected) {
|
||||
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const binary = requireObsidianBinary();
|
||||
const cli = discoverObsidianCli();
|
||||
if (!cli.binary) {
|
||||
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
|
||||
}
|
||||
|
||||
const vault = await createTemporaryVault();
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
try {
|
||||
console.log(`Using Obsidian executable: ${binary}`);
|
||||
console.log(`Temporary vault: ${vault.path}`);
|
||||
|
||||
session = await startObsidianLiveSyncSession({
|
||||
binary,
|
||||
cliBinary: cli.binary,
|
||||
vault,
|
||||
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
|
||||
});
|
||||
|
||||
const created = await evalObsidianJson<CreatedNote>(
|
||||
cli.binary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(notePath)};`,
|
||||
`const content=${JSON.stringify(noteContent)};`,
|
||||
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
|
||||
"const existing=app.vault.getAbstractFileByPath(path);",
|
||||
"if(existing) await app.vault.delete(existing);",
|
||||
"const file=await app.vault.create(path,content);",
|
||||
"const read=await app.vault.read(file);",
|
||||
"return JSON.stringify({path:file.path,read,exists:await app.vault.adapter.exists(path)});",
|
||||
"})()",
|
||||
].join(""),
|
||||
session.cliEnv
|
||||
);
|
||||
|
||||
assertEqual(created.path, notePath, "Obsidian created the note at an unexpected path.");
|
||||
assertEqual(created.exists, true, "Obsidian adapter did not report the created note.");
|
||||
assertEqual(created.read, noteContent, "Obsidian did not read back the created note content.");
|
||||
|
||||
await waitForFileContent(vault.path, notePath, noteContent);
|
||||
|
||||
const readBack = await evalObsidianJson<ReadNote>(
|
||||
cli.binary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(notePath)};`,
|
||||
"const file=app.vault.getAbstractFileByPath(path);",
|
||||
"return JSON.stringify({exists:!!file,read:file?await app.vault.read(file):null});",
|
||||
"})()",
|
||||
].join(""),
|
||||
session.cliEnv
|
||||
);
|
||||
assertEqual(readBack.exists, true, "Obsidian did not find the reflected note on read-back.");
|
||||
assertEqual(readBack.read, noteContent, "Obsidian read-back content did not match the reflected file.");
|
||||
|
||||
console.log(`Created and verified reflected note: ${notePath}`);
|
||||
} finally {
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
}
|
||||
await vault.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error: unknown) => {
|
||||
console.error(error instanceof Error ? error.stack : error);
|
||||
process.exit(1);
|
||||
});
|
||||
Reference in New Issue
Block a user