Compare commits

..
Author SHA1 Message Date
vorotamoroz fa97f96960 Group pending synchronisation fixes in Unreleased notes 2026-09-18 08:14:38 +00:00
vorotamoroz 916ec4f276 Document restart conflict protection in 1.0.29 notes 2026-09-18 07:03:34 +00:00
vorotamoroz 8ea656a7da Use Commonlib 0.1.27 for Journal transfer readiness 2026-09-18 07:01:07 +00:00
vorotamoroz a3231b0906 Reuse Security Seed reads within each resource 2026-09-18 03:34:31 +00:00
vorotamoroz a8fe11e0e0 Merge pull request #1203 from vrtmrz/test/cli-mirror-provenance
Align CLI mirror tests with file provenance
2026-09-18 11:02:11 +09:00
vorotamoroz e8bb05e884 Test ordinary mirror edits and preserve unknown local content 2026-09-18 01:14:35 +00:00
vorotamoroz c60323e11f Merge pull request #1201 from vrtmrz/fix/stale-file-revision-provenance
Prevent stale local files from replacing newer database content
2026-09-18 01:31:08 +09:00
vorotamoroz ce4eb7f9ac Document test procedures and show Community lint warnings 2026-09-17 16:26:24 +00:00
vorotamoroz 1c8db1a2b2 Test stale-file protection across multiple devices 2026-09-17 16:12:43 +00:00
vorotamoroz f449292792 Keep clean installation compatible with npm 10 2026-09-17 15:04:28 +00:00
vorotamoroz dee5b69689 Use compatible timers for TURN credential requests
Use compatGlobal and CompatTimeoutHandle for request deadlines. Enforce the community no-global-this rule for integrations and make violations fail quiet lint checks.
2026-09-17 14:26:08 +00:00
vorotamoroz 1044dca94f Merge main into stale-file protection integration 2026-09-17 12:46:32 +00:00
vorotamoroz 63c2811d47 Use published Commonlib 0.1.26 for stale-file protection 2026-09-17 12:45:34 +00:00
vorotamoroz 00abaf2668 Reject direct global access in community lint checks
Make no-global-this an error so quiet validation detects violations of Obsidian window compatibility rules.
2026-09-17 11:06:53 +00:00
vorotamoroz ef99cd8499 Cover stale-file recovery and bulk folder operations
Document exact file revision provenance and add real Obsidian scenarios for stale files after restart, parent-folder renames, and parent-folder deletion. Characterise replication queue delays during repeated writes to one document.

Companion to Commonlib c0a84a4. Validation used its packed artefact; the local file dependency remains uncommitted until a published Commonlib version is available.
2026-09-17 11:06:53 +00:00
vorotamoroz 85a12e311c Merge pull request #1198 from vrtmrz/feature/turn-credential-sources
Add Cloudflare TURN credentials to P2P settings
2026-09-16 20:24:23 +09:00
vorotamoroz db805fd70c Merge pull request #1199 from vrtmrz/1_0_29
Releasing 1.0.29
2026-09-16 20:17:10 +09:00
22 changed files with 1115 additions and 56 deletions
+3
View File
@@ -121,6 +121,9 @@ jobs:
node-version: '24.x'
cache: 'npm'
- name: Verify clean installation with npm 10
run: npx --yes npm@10.9.4 ci --ignore-scripts --no-audit --no-fund
- name: Install dependencies
run: npm ci
+7 -14
View File
@@ -29,18 +29,7 @@ npm run build
#### Community Review dependency installation
Community Review installs dependencies independently before applying type-aware source rules. A successful installation with the npm version bundled with the repository's current Node.js CI does not prove that the lockfile is accepted by the scanner's npm version.
After changing `package.json`, a workspace manifest, or `package-lock.json`, verify both installation paths:
```bash
npm ci --ignore-scripts
npx --yes npm@10.9.2 ci --ignore-scripts
```
The npm 10.9.2 command is the current project-side compatibility check for the Community Review installation path. Update this check when the scanner runtime changes.
If Community Review reports widespread TypeScript `error` types across unrelated external packages, confirm that dependency installation completed successfully before changing source imports, declarations, or lint rules. An installation failure can make every unresolved external type appear as downstream unsafe-type findings.
After changing a dependency manifest or lockfile, follow the [npm 10 clean-installation check](test/README.md#npm-10-clean-installation-check) before the normal source and unit checks. The test guide records the command used by CI and the distinction between installation failures and source diagnostics.
### Commands
@@ -77,6 +66,8 @@ To facilitate development and testing, the build process can automatically copy
### Testing Infrastructure
See the [test procedures](test/README.md) for clean-installation checks, local validation commands, and links to each runtime suite.
- **Vitest**:
- **Unit Tests** (`vitest.config.unit.ts`): Unit tests run in Node.js (excluding harnesses and integration tests). Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`). Executed via `npm run test:unit`.
- **Integration Tests** (`vitest.config.integration.ts`): Tests run in Node.js against a real CouchDB instance. Integration tests should be `*.integration.spec.ts` or `*.integration.test.ts` and placed alongside the implementation file (e.g., `StreamingFetch.integration.spec.ts`). Executed via `npm run test:integration`.
@@ -197,9 +188,9 @@ Markdown conflict auto-merge should behave like a conservative three-way merge.
When in doubt, prefer the safer outcome: preserve data, keep the conflict visible, and ask the user rather than silently discarding content or choosing one side.
The detailed contract is documented in [Conflict resolution and revision provenance](docs/specs_conflict_resolution.md). Determine the merge base by intersecting the exact `available` revision IDs from both leaf histories and selecting the nearest shared revision. Do not infer ancestry from revision generation numbers. When a remote resolution reaches a Vault which still contains the exact content of a deleted losing branch, treat that content as known synchronised history so the resolution can be reflected without recreating the conflict.
The detailed contract is documented in [Conflict resolution and revision provenance](docs/specs_conflict_resolution.md). Determine the merge base by intersecting the exact `available` revision IDs from both leaf histories and selecting the nearest shared revision. Do not infer ancestry from revision generation numbers. An unchanged file is recognised by comparing its bytes with its exact device-local file-reflection provenance, including when that revision belongs to a deleted losing branch.
File operations made while a conflict is active must use the device-local file-reflection provenance injected into `ServiceFileHandlerBase`. Treat its exact revision as authoritative; use byte equality only to reconstruct a missing record when exactly one available revision matches. If branch identity remains unknown, preserve data and leave the conflict visible. Do not hide key-value database readiness behind an implicit wait: maintained hosts open it through the sequential settings lifecycle before file events or replication begin.
Ordinary file saves and incoming reflection use that provenance even before a conflict exists. An unchanged stale file must not become a child of the current winner; a genuine edit extends the recorded revision. Without a readable recorded base, compare only current live leaves to avoid duplicate content. Otherwise, preserve the file as a fresh independent root under the same document ID, leaving ancestry unknown. Historical byte equality cannot distinguish an unchanged file from an intentional revert. Explicit reconciliation, deletion, and rename retain their separate contracts. Do not hide key-value database readiness behind an implicit wait: maintained hosts open it through the sequential settings lifecycle before file events or replication begin.
- If one side deletes a line and the other side leaves that same line unchanged, treat it as a safe deletion. The deleted line must not be reintroduced into the merged result.
- If one side inserts new content in a different region while the other side deletes an unchanged old region, preserve the insertion and the deletion.
@@ -209,6 +200,8 @@ File operations made while a conflict is active must use the device-local file-r
This policy is intentionally aligned with the conflict checkboxes and compatibility settings: automatic merge should remove avoidable prompts, but it must not silently choose between overlapping user intentions.
The [multiple-device conflict test procedure](test/README.md#multiple-device-conflict-regression-tests) documents the five CouchDB-backed cases, execution steps, expected results, and coverage boundaries.
### File Structure Conventions
- **Platform-specific code**: Use `.platform.ts` suffix (replaced with `.obsidian.ts` in production builds via esbuild)
+27 -15
View File
@@ -28,24 +28,27 @@ The modifiers defined under [Revision](glossary.md#revision) describe independen
| The Vault displays conflict leaf `C` | `W` | `C` | `C` |
| The database advances before Vault reflection | new winner `W2` | previous revision `R`, while the Vault is unchanged | `R` |
| A local edit of displayed revision `R` is pending | independent | none, or a coincidental content match | `R`, as the branch which the edit must extend |
| Provenance is missing and exactly one revision fits | independent | `M` | none, then `M` after safe reconstruction |
| Provenance is missing and exactly one current non-deleted leaf fits | independent | `M` | none, then `M` after safe reconstruction |
| Provenance is missing and several revisions fit | independent | every matching revision | none |
| A logical-deletion winner agrees with an absent file | deleted winner `D` | `D`, and possibly other logical-deletion revisions | none; an absent file retains no displayed provenance |
At most one revision is the winner, more than one revision can be Vault-matching, and at most one revision can be displayed for a path on one device. A displayed revision may stop matching the Vault while a local edit is pending, but its branch identity remains authoritative until that edit is stored or the relationship is safely reconstructed.
## Implemented 1.0 guarantees
## File saving and reflection guarantees
- Automatic text and structured-data merge uses the nearest `available` revision ID which is present in both leaf histories.
- Missing or compacted history stops conservative automatic merge instead of guessing a base.
- A receiving Vault file which exactly matches any available revision in the document tree is treated as previously synchronised content. This includes an ancestor below a deleted losing leaf.
- A receiving Vault file whose bytes do not match any available revision is preserved as an unsynchronised local change.
- A Vault file which still matches its exact recorded revision is unchanged. An ordinary save does not append those stale bytes to a newer database revision; a newer, unconflicted database result is reflected through the existing file-reflection path.
- A file which differs from its readable recorded revision is an edit of that revision, even if its bytes match another historical revision. Saving and incoming overwrite protection use the same rule.
- Without a readable recorded revision, current non-deleted leaves are checked for duplicate content. If none matches, the file is preserved as a fresh independent root under the same document ID. Its unknown ancestry cannot supply a three-way merge base.
- File bytes, rather than path, size, modification time, or revision generation, determine whether content is known.
- Three or more current versions are reviewed one pair at a time in a deterministic order, with each completed pair committed before the next pair is read.
- Each device records the exact revision most recently reflected in each Vault file. An edit, deletion, or case-only rename made while a conflict is active extends that displayed branch rather than the deterministic database winner.
- Each device records the exact revision most recently reflected in each Vault file. An ordinary edit extends that displayed branch even before a conflict exists. Conflict-time deletion and case-only rename retain their separate displayed-branch contracts.
- A cross-path rename stores the target before logically deleting only the displayed source branch.
The all-branch history check prevents a resolved conflict from being recreated merely because the receiving Vault still contains the known losing version. If the user has edited that version again, its bytes differ and the overwrite guard preserves it.
The recorded revision can belong to a deleted losing branch. If its readable body still matches the Vault, the propagated resolution can be reflected without recreating the conflict. A historical byte match without that record does not establish that the file is unchanged: it may be an intentional revert. Existing Vaults can lack records, so an upgrade, reset, or unavailable old body can expose additional conflicts requiring review.
The explicit **Always overwrite with a newer file** option retains its existing modification-time policy. An independent branch prevents an inferred three-way merge; it does not disable the user's selected conflict-resolution option. Metadata and Chunks retain their existing format, and matching chunks can be shared between branches.
## Resolution patterns
@@ -55,8 +58,10 @@ The all-branch history check prevents a resolved conflict from being recreated m
| Text or structured data has an available shared base and non-overlapping changes | Perform a conservative three-way merge. |
| One side deletes content which the other leaves unchanged | Preserve the deletion. |
| One side deletes content which the other modifies | Ask the user. |
| A receiving file matches a revision available anywhere in the tree | Apply the propagated database result. |
| A receiving file matches no available revision | Preserve it and ask the user. |
| A receiving file matches its exact readable recorded revision | Apply the propagated database result under the existing conflict policy. |
| A receiving file differs from its readable recorded revision | Preserve the edit as a child of that exact revision. |
| Provenance is unknown and no current non-deleted leaf matches the file | Preserve a fresh independent branch for conflict resolution. |
| Provenance is unknown and current non-deleted leaves already hold the file bytes | Avoid duplicate storage; infer provenance only for a unique match. |
| A required body or shared ancestor is missing or compacted | Ask the user. |
| Binary contents differ | Prefer an explicit user selection; semantic merge is unavailable. |
@@ -138,13 +143,17 @@ LiveSync composes Commonlib's injected `FileReflectionProvenance` with its local
path -> { revision, observedStorageMtime? }
```
`revision` identifies the exact database revision which most recently produced the displayed Vault file. `observedStorageMtime` is the raw local modification time observed after reflection. It is not rounded, combined with another device's value, or used as proof of branch identity. No content hash is persisted.
`revision` identifies the exact database revision most recently saved from or reflected in this device's Vault. It is the base for subsequent local edits, rather than a certificate that the current file still contains those bytes. `observedStorageMtime` is the raw local modification time of the saved snapshot or the file observed after reflection. It is not rounded, combined with another device's value, or used as proof of branch identity. No content hash is persisted.
The record changes only after a successful database-to-Vault reflection or Vault-to-database write. Reading a file does not change it. The recorded revision remains authoritative even if the user edits the file to bytes which equal another branch; otherwise content equality could silently move the edit to a branch which was not displayed.
Saving and reflection for the same Metadata document run one at a time, including the final provenance update. An ordinary save holds one captured file body and its base until the database write completes. An edit made while that save is running belongs to the next operation; the save does not reread the file to prove that it remained unchanged. Different files retain their existing concurrency limits, and the handler acquires the lock before loading a file body from storage. Conflict checking runs after the lock is released so that an immediate resolution can safely call the file handler again. The host queues count document-lock waiters against their concurrency limits, so a burst for one document can temporarily delay unrelated files.
The common lock does not stop Obsidian edits, external filesystem writes, or replication into the database. Incoming overwrite and deletion protection still checks current storage. Pending events restored at startup retain bounded rechecks because they run before file watching begins and cannot rely on another change notification.
LiveSync creates the namespaced store handle during service composition, before the key-value database is open. The sequential `onSettingLoaded` lifecycle opens that database before Vault scanning, watching, or replication starts. Store operations do not wait for implicit readiness: a lifecycle violation fails promptly, avoiding an indefinite or self-referential initialisation wait. Local database reset is a transient unavailable boundary, after which scanning reconstructs derived state.
When no record exists, LiveSync may reconstruct the displayed revision only if the current Vault bytes match exactly one available revision body. No match, or identical content in multiple revisions, cannot prove branch identity.
For ordinary saves and incoming reflection, a missing or unreadable recorded base permits reconstruction only from exactly one matching current non-deleted leaf. Matching several current leaves avoids duplicate storage but does not identify a displayed branch. No current match creates an independent branch, even when an older ancestor has the same bytes. Deletion and rename retain their existing provenance-recovery contracts.
## Operations while a conflict exists
@@ -231,7 +240,7 @@ If the user renames `draft.md` to `published.md`, LiveSync stores `published.md`
### A remote resolution reaches a device which still shows the losing content
Android may resolve a conflict and continue editing while Mac still shows the losing revision. When Mac receives the resolved tree, LiveSync searches every available branch and recognises Mac's unchanged bytes as content which was already synchronised below the deleted losing leaf. It can apply Android's resolution without asking Mac to resolve the same unchanged conflict again.
Android may resolve a conflict and continue editing while Mac still shows the losing revision. When Mac receives the resolved tree, LiveSync compares Mac's bytes with the exact revision recorded for its Vault. If that body remains readable and matches, it can apply Android's resolution without asking Mac to resolve the same unchanged conflict again.
If the user edited the file on Mac before the resolution arrived, the bytes no longer match that historical revision. LiveSync preserves the Mac edit as an unsynchronised conflict instead of overwriting it.
@@ -243,15 +252,15 @@ The first decision has already changed the ordinary revision tree. On restart, L
### The device-local record is missing
A local-database reset removes revision provenance. On the next scan, if the Vault file matches exactly one available revision, LiveSync can reconstruct which branch was displayed and continue from it. If the bytes match multiple revisions, or no available revision, the branch remains unproved.
A local-database reset removes revision provenance. When an ordinary save or incoming reflection examines the file, exactly one matching current non-deleted leaf can reconstruct the record. Multiple current matches prevent duplicate storage but leave branch identity unproved. A match only in past history is insufficient; differing current content is preserved as an independent branch. An unchanged-time scan alone does not guarantee that a record is created.
In that unproved state, an edit is retained as another manual-resolution branch. A deletion leaves all existing branches intact. A cross-path rename stores the target but leaves every source branch for review. The result can require an extra decision, but it does not discard data by guessing the winner.
If no current non-deleted leaf contains the file bytes, an ordinary save retains them as another independent branch. An unproven deletion leaves all existing branches intact. A cross-path rename stores the target but leaves every unproven source branch for review. The result can require an extra decision, but it does not discard data by guessing the winner.
### Start-up or reset overlaps a provenance operation
LiveSync creates the provenance handle during composition, then opens its backing store during the sequential settings lifecycle before starting scans, watchers, or replication. If the store cannot open, start-up stops rather than leaving file processing waiting indefinitely.
During reset, the store can be temporarily unavailable. A racing provenance lookup fails promptly and follows the same conservative missing-record behaviour. After reopen, scanning can reconstruct a record when one exact revision body matches the Vault file.
During reset, the store can be temporarily unavailable. A racing provenance lookup fails promptly and follows the same conservative missing-record behaviour. After reopen, ordinary saving or reflection can reconstruct a record from a unique matching current non-deleted leaf.
## Unsafe shortcuts
@@ -260,6 +269,7 @@ Do not:
- infer a common ancestor from generation numbers alone;
- assume that the PouchDB winner is the version currently displayed in the Vault;
- replace recorded displayed provenance merely because current bytes match another branch;
- classify a file as unchanged solely because it matches an ancestor somewhere in history;
- discard local content when revision-history lookup fails;
- infer revision identity from path, size, modification time, or content hash without a revision ID;
- select the newest modification time unless the user has explicitly chosen that destructive policy; or
@@ -267,7 +277,9 @@ Do not:
## Verification
Commonlib's real-PouchDB and injected-boundary unit tests cover unequal branch lengths, exact shared ancestry, deterministic ordering of multiple current leaves, a sensible stage followed by reconstruction of a manual pair, content below a deleted losing leaf, recorded and reconstructed branch identity, ambiguous matches, conflict-time editing, missing-body preservation when parent metadata is available, refusal to invent a parent for a generation-one revision, logical deletion, case-only rename, cross-path rename, and safe unproven fallbacks.
LiveSync also exercises three and four independently editing devices through real CouchDB, using the installed Commonlib package and the CLI conflict-resolution command dispatcher. These tests check unchanged losing files before and after resolution, genuine edits on a losing branch, missing provenance, compacted bases, independent-root deduplication, and propagation of the selected result. See the [multiple-device regression procedure and coverage boundaries](../test/README.md#multiple-device-conflict-regression-tests).
Commonlib owns the real-PouchDB and injected-boundary tests for revision ancestry, content preservation, provenance, independent branches, and repeated file events. LiveSync owns persistent host composition and actual Obsidian restart coverage. The focused `test:e2e:obsidian:stale-file-restart` scenario advances the local DB while old Vault bytes remain, persists pending file events, and restarts the same isolated profile. It requires an unchanged recorded file to reflect the DB without a new revision, an unknown file to remain on an independent branch alongside the DB content, and repeated processing after provenance loss to leave those branches unchanged. It uses real local storage and startup processing; transport replication and mobile lifecycle coverage are separate.
LiveSync's optional real-Obsidian two-Vault checks have two scopes. `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` resolves and edits a Markdown conflict, propagates it to a Vault which still displays the deleted losing content, and requires one current result to remain. `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` edits, deletes, case-renames, and cross-path-renames files while conflicts remain active; it verifies the parent revision of each resulting branch, replicates those exact trees, and confirms that the other conflict branches remain intact.
+2 -7
View File
@@ -52,6 +52,8 @@ export default defineConfig(
"obsidianmd/rule-custom-message": "off",
"no-console": "warn",
"obsidianmd/no-unsupported-api": "error",
// Treat direct globalThis access as an error so the CI gate rejects it.
"obsidianmd/no-global-this": "error",
// Keep legacy type-safety debt visible while reserving errors for directory-review blockers.
"@typescript-eslint/no-unsafe-argument": "warn",
"@typescript-eslint/no-unsafe-assignment": "warn",
@@ -63,13 +65,6 @@ export default defineConfig(
"@typescript-eslint/no-unnecessary-type-assertion": "warn",
},
},
{
files: ["src/integrations/**/*.ts"],
rules: {
// External-service integrations also run in Node and do not own window UI.
"obsidianmd/no-global-this": "off",
},
},
{
files: ["src/apps/**/*.{ts,js,mjs}"],
rules: {
+4 -4
View File
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.25",
"@vrtmrz/livesync-commonlib": "0.1.27",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4567,9 +4567,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.25",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.25.tgz",
"integrity": "sha512-uWlzcXi32EvrEx6OgKsSEuNQsY+PQDHPY3eq4Xd9W9lHKu2LNh5n1f2z+bMsfZJh1AGMuTkjjFW26X2Bvv/iog==",
"version": "0.1.27",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.27.tgz",
"integrity": "sha512-n/AIqXPGN9Z7zmq0QSKnlE5uY5hTZgXW6VM6NJERR0/ggx3TTv1b3WsSbiLpCeU7Mu6KbPqC2BSjr4jCGvSdCA==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
+4 -2
View File
@@ -23,7 +23,7 @@
"prettyNoWrite": "prettier --config ./.prettierrc.mjs \"**/*.js\" \"**/*.ts\" \"**/*.json\" ",
"precheck:compatibility": "npm run build",
"check:compatibility": "node utils/check-compatibility.js --file main.js --ios 15",
"check": "npm run tsc-check && npm run tsc-check:apps && npm run lint && npm run lint:community -- --quiet && npm run lint:community:tools && npm run svelte-check && npm run check:compatibility",
"check": "npm run tsc-check && npm run tsc-check:apps && npm run lint && npm run lint:community && npm run lint:community:tools && npm run svelte-check && npm run check:compatibility",
"i18n:bake": "npm run i18n:yaml2json && npm run i18n:bakejson && npm run i18n:format",
"i18n:bakejson": "tsx _tools/bakei18n.ts",
"i18n:format": "prettier --config .prettierrc.mjs --write --log-level error 'src/common/messagesJson/*.json' 'src/common/messages/*.ts'",
@@ -78,6 +78,8 @@
"test:e2e:obsidian:p2p-connection-check:services": "npm run test:e2e:obsidian:p2p-connection-check -- --manage-p2p",
"test:e2e:obsidian:partial-startup-file-failure": "tsx test/e2e-obsidian/scripts/partial-startup-file-failure.ts",
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
"test:e2e:obsidian:stale-file-restart": "tsx test/e2e-obsidian/scripts/stale-file-restart.ts",
"test:e2e:obsidian:folder-batch": "tsx test/e2e-obsidian/scripts/folder-batch.ts",
"test:e2e:obsidian:setup-uri-workflow": "tsx test/e2e-obsidian/scripts/setup-uri-workflow.ts",
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
"test:e2e:obsidian:security-seed-reconnect": "tsx test/e2e-obsidian/scripts/security-seed-reconnect.ts",
@@ -181,7 +183,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.25",
"@vrtmrz/livesync-commonlib": "0.1.27",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
+60 -1
View File
@@ -7,6 +7,8 @@
# 3. DB-deleted file → NOT restored to storage (UPDATE STORAGE skip)
# 4. Both, storage newer → DB updated (SYNC: STORAGE → DB)
# 5. Both, DB newer → storage updated (SYNC: DB → STORAGE)
# 6. Compatibility mode → omitted vault-path works
# 7. Unknown local origin → conflict preserved, deduplicated, and resolved
#
# Not covered (require precise mtime control or artificial conflict injection):
# - Both, equal mtime → no-op (EVEN)
@@ -43,7 +45,8 @@ cli_test_init_settings_file "$SETTINGS_FILE"
# isConfigured=true is required for mirror (canProceedScan checks this)
cli_test_mark_settings_configured "$SETTINGS_FILE"
# Enable writeDocumentsIfConflicted to resolve unsynced conflicts during mirror
# Allow incoming DB content to be reflected when conflicts exist (Case 5).
# This does not resolve conflicts or authorise overwriting DB content.
node -e '
const fs = require("fs");
const file = process.argv[1];
@@ -181,6 +184,11 @@ echo "=== Case 4: storage newer → DB updated (Separated Paths) ==="
# Seed DB with old content (mtime ≈ now)
printf 'old content\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put test/sync-storage-newer.md
# Establish the file's recorded base before making an ordinary local edit.
# A direct put followed by unrelated local content has unknown provenance.
run_mirror_test
cli_test_assert_equal "old content" "$(cat "$VAULT_DIR/test/sync-storage-newer.md")" "Case 4 base was not reflected"
# Write new content to storage with a timestamp 1 hour in the future
printf 'new content\n' > "$VAULT_DIR/test/sync-storage-newer.md"
touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/test/sync-storage-newer.md"
@@ -188,6 +196,8 @@ touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/test/sync-storage-n
run_mirror_test
DB_RESULT_FILE="$WORK_DIR/case4-pull.txt"
CASE4_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info test/sync-storage-newer.md)"
cli_test_assert_equal "N/A" "$(printf '%s' "$CASE4_INFO" | cli_test_json_string_field_from_stdin conflicts)" "Ordinary local edit unexpectedly created a conflict"
run_cli "$DB_DIR" --settings "$DB_SETTINGS" pull test/sync-storage-newer.md "$DB_RESULT_FILE"
if cmp -s "$VAULT_DIR/test/sync-storage-newer.md" "$DB_RESULT_FILE"; then
assert_pass "DB updated to match newer storage file"
@@ -238,6 +248,55 @@ else
assert_fail "Compatibility mode failed to sync file into DB"
fi
# ─────────────────────────────────────────────────────────────────────────────
# Case 7: Unknown local origin must preserve both contents, regardless of mtime
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 7: unknown local origin → preserve and resolve conflict ==="
UNKNOWN_PATH="test/unknown-origin.md"
printf 'original DB content\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put "$UNKNOWN_PATH"
printf 'unrelated local content\n' > "$VAULT_DIR/$UNKNOWN_PATH"
touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/$UNKNOWN_PATH"
run_mirror_test
UNKNOWN_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info "$UNKNOWN_PATH")"
WINNER="$(printf '%s' "$UNKNOWN_INFO" | cli_test_json_string_field_from_stdin revision)"
CONFLICT="$(printf '%s' "$UNKNOWN_INFO" | cli_test_json_string_field_from_stdin conflicts)"
if [[ ! "$WINNER" =~ ^1-[[:xdigit:]]+$ || ! "$CONFLICT" =~ ^1-[[:xdigit:]]+$ || "$WINNER" == "$CONFLICT" ]]; then
echo "[FAIL] Expected two independent non-deleted root revisions: $UNKNOWN_INFO" >&2
exit 1
fi
# Do not assume which randomly identified root PouchDB selects as the winner.
LOCAL_REV=""
DB_REV=""
for revision in "$WINNER" "$CONFLICT"; do
CONTENT="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" cat-rev "$UNKNOWN_PATH" "$revision" | cli_test_sanitise_cat_stdout)"
case "$CONTENT" in
'unrelated local content') LOCAL_REV="$revision" ;;
'original DB content') DB_REV="$revision" ;;
*) echo "[FAIL] Unexpected content for $revision: $CONTENT" >&2; exit 1 ;;
esac
done
[[ -n "$LOCAL_REV" && -n "$DB_REV" ]] || { echo "[FAIL] Both contents must remain readable" >&2; exit 1; }
# Force another ordinary save of the same unknown bytes, even if incoming
# reflection replaced the file under writeDocumentsIfConflicted.
printf 'unrelated local content\n' > "$VAULT_DIR/$UNKNOWN_PATH"
touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/$UNKNOWN_PATH"
run_mirror_test
REPEATED_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info "$UNKNOWN_PATH")"
cli_test_assert_equal "$WINNER" "$(printf '%s' "$REPEATED_INFO" | cli_test_json_string_field_from_stdin revision)" "Repeated mirror changed the winning revision"
cli_test_assert_equal "$CONFLICT" "$(printf '%s' "$REPEATED_INFO" | cli_test_json_string_field_from_stdin conflicts)" "Repeated mirror created another conflict"
run_cli "$DB_DIR" --vault "$VAULT_DIR" --settings "$DB_SETTINGS" resolve "$UNKNOWN_PATH" "$LOCAL_REV"
RESOLVED_INFO="$(run_cli "$DB_DIR" --settings "$DB_SETTINGS" info "$UNKNOWN_PATH")"
cli_test_assert_equal "N/A" "$(printf '%s' "$RESOLVED_INFO" | cli_test_json_string_field_from_stdin conflicts)" "CLI resolve left a conflict"
cli_test_assert_equal "$LOCAL_REV" "$(printf '%s' "$RESOLVED_INFO" | cli_test_json_string_field_from_stdin revision)" "CLI resolve selected the wrong revision"
cli_test_assert_equal "unrelated local content" "$(cat "$VAULT_DIR/$UNKNOWN_PATH")" "CLI resolve did not reflect the selected content"
assert_pass "Unknown local content was preserved, deduplicated, and resolved through the CLI"
# ─────────────────────────────────────────────────────────────────────────────
# Summary
# ─────────────────────────────────────────────────────────────────────────────
+50 -2
View File
@@ -11,6 +11,7 @@
* 4. Both, storage newer -> DB updated (SYNC: STORAGE -> DB)
* 5. Both, DB newer -> storage updated (SYNC: DB -> STORAGE)
* 6. Compatibility mode -> omitted vault-path works (same DB + vault path)
* 7. Unknown local origin -> conflict preserved, deduplicated, and resolved
*
* No external services are required.
*
@@ -18,9 +19,9 @@
* deno test -A test-mirror.ts
*/
import { assert } from "@std/assert";
import { assert, assertEquals } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { runCliOrFail } from "./helpers/cli.ts";
import { runCliOrFail, runCliWithInputOrFail } from "./helpers/cli.ts";
import { initSettingsFile, markSettingsConfigured } from "./helpers/settings.ts";
Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
@@ -130,6 +131,10 @@ Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
await Deno.writeTextFile(seedFile, "old content\n");
await dbRun("push", seedFile, "test/sync-storage-newer.md");
// Reflect the shared base into the actual Vault before editing it.
await runMirror();
assertEquals(await Deno.readTextFile(workDir.join("vault", "test", "sync-storage-newer.md")), "old content\n");
// Write new content to storage with a timestamp 1 hour in the future
const storageFile = workDir.join("vault", "test", "sync-storage-newer.md");
await Deno.writeTextFile(storageFile, "new content\n");
@@ -138,6 +143,8 @@ Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
await runMirror();
const resultFile = workDir.join("case4-pull.txt");
const info = JSON.parse(await dbRun("info", "test/sync-storage-newer.md"));
assertEquals(info.conflicts, "N/A", "An ordinary local edit must not create a conflict");
await dbRun("pull", "test/sync-storage-newer.md", resultFile);
const storageContent = await Deno.readTextFile(storageFile);
const pulledContent = await Deno.readTextFile(resultFile);
@@ -184,6 +191,47 @@ Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
assert(pulled === "compat-content\n", `Compatibility mode failed to sync file into DB (got: '${pulled}')`);
console.log("[PASS] case 6: compatibility mode works");
});
// -------------------------------------------------------------------
// Case 7: unknown local origin must preserve both contents regardless of mtime.
// This deliberately uses put: push would record a file provenance entry.
// -------------------------------------------------------------------
await t.step("case 7: unknown local content is preserved, deduplicated, and resolved", async () => {
const path = "test/unknown-origin.md";
const storageFile = workDir.join("vault", "test", "unknown-origin.md");
await runCliWithInputOrFail("original DB content\n", dbDir, "--settings", dbSettings, "put", path);
const writeUnknownFile = async () => {
await Deno.writeTextFile(storageFile, "unrelated local content\n");
await Deno.utime(storageFile, new Date(), new Date(Date.now() + 3600_000));
};
await writeUnknownFile();
await runMirror();
const info = JSON.parse(await dbRun("info", path));
assert(/^1-[\da-f]+$/.test(info.revision), "Expected an independent winning root");
assert(/^1-[\da-f]+$/.test(info.conflicts), "Expected exactly one independent conflicting root");
assert(info.revision !== info.conflicts, "Expected two distinct revisions");
const contents = new Map<string, string>();
for (const revision of [info.revision, info.conflicts]) {
contents.set(await dbRun("cat-rev", path, revision), revision);
}
assertEquals([...contents.keys()].sort(), ["original DB content\n", "unrelated local content\n"]);
// Re-submit identical local bytes even if incoming reflection replaced
// the file under writeDocumentsIfConflicted; no third branch is needed.
await writeUnknownFile();
await runMirror();
const repeated = JSON.parse(await dbRun("info", path));
assertEquals(repeated.revision, info.revision);
assertEquals(repeated.conflicts, info.conflicts);
const localRevision = contents.get("unrelated local content\n")!;
await runCliOrFail(dbDir, "--vault", vaultDir, "--settings", dbSettings, "resolve", path, localRevision);
const resolved = JSON.parse(await dbRun("info", path));
assertEquals(resolved.conflicts, "N/A");
assertEquals(resolved.revision, localRevision);
assertEquals(await Deno.readTextFile(storageFile), "unrelated local content\n");
});
});
// ---------------------------------------------------------------------------
+2 -1
View File
@@ -3,6 +3,7 @@ import {
CAPABILITY_NOT_APPLICABLE,
CENTRAL_REMOTE_REPLICATION_READINESS,
NO_INTERACTION,
PROVIDER_OWNED_CENTRAL_REMOTE_REPLICATION_READINESS,
REPLICATION_PROGRESS_PRESENTATIONS,
REMOTE_RESOURCE_KINDS,
defineReplicatorProviderDefinitions,
@@ -134,7 +135,7 @@ export function createCentralReplicatorProviderDefinitions(
[REMOTE_MINIO]: {
kind: REMOTE_MINIO,
diagnosticName: "Object Storage",
readiness: CENTRAL_REMOTE_REPLICATION_READINESS,
readiness: PROVIDER_OWNED_CENTRAL_REMOTE_REPLICATION_READINESS,
isConfigured: (settings) =>
settings.remoteType === REMOTE_MINIO && !!settings.endpoint?.trim() && !!settings.bucket?.trim(),
configurationIdentity: getObjectStorageReplicatorConfigurationIdentity,
@@ -47,6 +47,12 @@ describe("central Replicator provider definitions", () => {
.toEqual(["connection", "preferred-tweak", "security-seed", "synchronisation-information"].sort());
});
it("lets Journal prepare its own fresh Security Seed while CouchDB uses central preparation", () => {
const definitions = createCentralReplicatorProviderDefinitions({} as never);
expect(definitions.get(REMOTE_COUCHDB)?.readiness.centralRemotePreparation).toBe("required");
expect(definitions.get(REMOTE_MINIO)?.readiness.centralRemotePreparation).toBe("provider-owned");
});
it("composes CouchDB and Object Storage policies outside LiveSyncBaseCore", async () => {
const host = {} as Parameters<typeof createCentralReplicatorProviderDefinitions>[0];
const definitions = createCentralReplicatorProviderDefinitions(host);
@@ -285,6 +285,46 @@ describe("replicator probe factories", () => {
expect(objectReplicator.closeReplication).toHaveBeenCalledOnce();
});
it("reads the Security Seed once per resource, including concurrent reads, and refreshes for a new resource", async () => {
const settings = createSettings();
const factory = createCouchDBSecuritySeedResourceFactory({} as never);
const firstResource = await factory(settings);
const firstReplicator = mocks.couchDB[0];
const firstSeed = new Uint8Array([1]);
firstReplicator.getReplicationPBKDF2Salt.mockResolvedValue(firstSeed);
const [first, concurrent] = await Promise.all([firstResource.read(), firstResource.read()]);
expect(first).toBe(firstSeed);
expect(concurrent).toBe(firstSeed);
await expect(firstResource.read()).resolves.toBe(firstSeed);
expect(firstReplicator.getReplicationPBKDF2Salt).toHaveBeenCalledOnce();
expect(firstReplicator.getReplicationPBKDF2Salt).toHaveBeenCalledWith({ ...settings }, true);
await firstResource.dispose();
const nextResource = await factory(settings);
const nextReplicator = mocks.couchDB[1];
const nextSeed = new Uint8Array([2]);
nextReplicator.getReplicationPBKDF2Salt.mockResolvedValue(nextSeed);
await expect(nextResource.read()).resolves.toBe(nextSeed);
expect(nextReplicator.getReplicationPBKDF2Salt).toHaveBeenCalledOnce();
expect(nextReplicator.getReplicationPBKDF2Salt).toHaveBeenCalledWith({ ...settings }, true);
await nextResource.dispose();
});
it("retries a failed Security Seed read within the same resource", async () => {
const resource = await createCouchDBSecuritySeedResourceFactory({} as never)(createSettings());
const replicator = mocks.couchDB[0];
const failure = new Error("connection interrupted");
const seed = new Uint8Array([1]);
replicator.getReplicationPBKDF2Salt.mockRejectedValueOnce(failure).mockResolvedValueOnce(seed);
await expect(resource.read()).rejects.toBe(failure);
await expect(resource.read()).resolves.toBe(seed);
await expect(resource.read()).resolves.toBe(seed);
expect(replicator.getReplicationPBKDF2Salt).toHaveBeenCalledTimes(2);
await resource.dispose();
});
it("checks synchronisation information through an owned connection and disposes the private Replicator", async () => {
const settings = createSettings();
const snapshot = { ...settings };
+12 -1
View File
@@ -21,8 +21,19 @@ function createSecuritySeedResourceFactory(
return (setting) => {
const snapshot = snapshotRemoteSettings(setting);
const replicator = createReplicator();
let readPromise: Promise<Uint8Array<ArrayBuffer>> | undefined;
const read = () => {
if (readPromise) return readPromise;
const pending = Promise.resolve().then(() => replicator.getReplicationPBKDF2Salt(snapshot, true));
readPromise = pending;
// A failed read must not poison a later retry within the same resource.
void pending.catch(() => {
if (readPromise === pending) readPromise = undefined;
});
return pending;
};
return Promise.resolve({
read: () => replicator.getReplicationPBKDF2Salt(snapshot, true),
read,
dispose: createReplicatorDisposer(replicator),
});
};
@@ -4,6 +4,7 @@ import {
type CloudflareTurnConfiguration,
validateCloudflareTurnConfiguration,
} from "./settings";
import { compatGlobal, type CompatTimeoutHandle } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
/** Fetch-compatible function supplied by the host composition. */
export type CloudflareTurnFetch = (input: string | Request, init?: RequestInit) => Promise<Response>;
@@ -283,9 +284,9 @@ export async function acquireCloudflareTurnCredentials(
requestController.abort();
throw abortError();
}
let timeoutId: ReturnType<typeof setTimeout> | undefined;
let timeoutId: CompatTimeoutHandle | undefined;
const deadline = new Promise<never>((_resolve, reject) => {
timeoutId = globalThis.setTimeout(() => {
timeoutId = compatGlobal.setTimeout(() => {
timedOut = true;
requestController.abort();
reject(credentialFailure("unavailable", true));
@@ -293,7 +294,7 @@ export async function acquireCloudflareTurnCredentials(
});
const cleanup = () => {
if (timeoutId !== undefined) globalThis.clearTimeout(timeoutId);
if (timeoutId !== undefined) compatGlobal.clearTimeout(timeoutId);
signal.removeEventListener("abort", onAbort);
};
@@ -83,6 +83,40 @@ function setup(options: SetupOptions = {}) {
}
describe("ReplicateResultProcessor", () => {
it("resumes another document after in-flight updates to one document fill the application slots", async () => {
const hotGate = promiseWithResolvers<boolean>();
const { processor, processSynchroniseResult } = setup({
processSynchroniseResult: async (entry) => {
if ((entry as { _id: string })._id === "hot-queue") return await hotGate.promise;
return true;
},
});
try {
for (let index = 1; index <= 10; index++) {
// A queued duplicate is coalesced; a new notification for a document
// already being processed can occupy another application slot.
processor.enqueueAll([note("hot-queue")]);
await vi.waitFor(() => expect(processor["_processingChanges"]).toHaveLength(index));
}
processor.enqueueAll([note("unrelated-queue")]);
await vi.waitFor(() => {
expect(processor["_semaphore"].waiting).toBeGreaterThan(0);
expect(processSynchroniseResult).toHaveBeenCalledTimes(1);
});
expect(processor["_queuedChanges"].map((entry) => entry._id)).toEqual(["unrelated-queue"]);
} finally {
hotGate.resolve(true);
await vi.waitFor(() => {
expect(processor["_processingChanges"]).toHaveLength(0);
expect(processor["_queuedChanges"]).toHaveLength(0);
});
}
expect(processSynchroniseResult).toHaveBeenCalledTimes(11);
expect(processSynchroniseResult.mock.calls.some(([entry]) =>
(entry as { _id: string })._id === "unrelated-queue"
)).toBe(true);
});
it("suspends result application while the application is not ready", () => {
const { isReady, processor } = setup({ applicationReady: false });
@@ -0,0 +1,334 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import PouchDB from "pouchdb-core";
import MemoryAdapter from "pouchdb-adapter-memory";
import HttpAdapter from "pouchdb-adapter-http";
import replication from "pouchdb-replication";
import type { EntryDoc, FilePathWithPrefix, UXFileInfo } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { compareMTime, createTextBlob, readContent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { createLiveSyncEventHub } from "@vrtmrz/livesync-commonlib/context";
import { LiveSyncLocalDB, type LiveSyncLocalDBEnv } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import {
ServiceDatabaseFileAccessBase,
type ServiceDatabaseFileAccessDependencies,
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/ServiceDatabaseFileAccessBase";
import type { ServiceFileHandlerDependencies } from "@vrtmrz/livesync-commonlib/compat/serviceModules/ServiceFileHandlerBase";
import { ServiceFileHandler } from "./FileHandler";
import {
createConflictResolutionOperations,
type ConflictResolutionOperationsDependencies,
} from "@/serviceFeatures/conflictResolution/operations";
import { runCommand } from "@/apps/cli/commands/runCommand";
import type { CLICommandContext } from "@/apps/cli/commands/types";
PouchDB.plugin(MemoryAdapter).plugin(HttpAdapter).plugin(replication);
const path = "multi-device.txt" as FilePathWithPrefix;
const old = "Original content\n";
const oldTime = 1_000_000;
class TestHandler extends ServiceFileHandler {}
function makeFile(body: string, mtime = oldTime): UXFileInfo {
return {
name: path,
path,
stat: { type: "file", ctime: oldTime, mtime, size: new Blob([body]).size },
body: createTextBlob(body),
};
}
async function makeDevice(name: string) {
const db = new PouchDB<EntryDoc>(name, { adapter: "memory" });
const reflection = new Map<FilePathWithPrefix, { revision: string; observedStorageMtime?: number }>();
let storage = makeFile(old);
const settings = { ...DEFAULT_SETTINGS, useOnlyLocalChunk: true, writeDocumentsIfConflicted: false };
const setting = { currentSettings: () => settings };
const pathService = {
path2id: (value: string) => Promise.resolve(value),
id2path: (id: string, entry?: { path?: string }) => entry?.path ?? id,
getPath: (entry: { path: FilePathWithPrefix }) => entry.path,
compareFileFreshness: (file: UXFileInfo, entry: { mtime: number }) =>
compareMTime(file.stat.mtime, entry.mtime),
markChangesAreSame: vi.fn(),
};
const events = createLiveSyncEventHub();
const API = { addLog: vi.fn() };
const localDatabase = new LiveSyncLocalDB(name, {
services: {
API,
setting,
path: pathService,
context: { events },
database: { createPouchDBInstance: () => db },
databaseEvents: {
onDatabaseInitialisation: () => Promise.resolve(true),
onDatabaseHasReady: () => Promise.resolve(true),
onCloseDatabase: () => Promise.resolve(true),
onUnloadDatabase: () => Promise.resolve(true),
},
replicator: { onCloseActiveReplication: () => Promise.resolve(true) },
},
} as unknown as LiveSyncLocalDBEnv);
await expect(localDatabase.initializeDatabase()).resolves.toBe(true);
const storageAccess = {
getStub: () => Promise.resolve(storage),
getFileStub: () => Promise.resolve(storage),
readStubContent: () => Promise.resolve(storage),
ensureDir: () => Promise.resolve(true),
writeFileAuto: vi.fn((_path: string, body: string, times: { mtime: number }) => {
storage = makeFile(body, times.mtime);
return Promise.resolve(true);
}),
stat: () => Promise.resolve(storage.stat),
touched: () => Promise.resolve(),
triggerFileEvent: vi.fn(),
};
const conflict = { queueCheckFor: vi.fn(), queueCheckForIfOpen: vi.fn() };
const services = {
API,
path: pathService,
setting,
events,
database: { localDatabase },
vault: { isTargetFile: () => Promise.resolve(true), isFileSizeTooLarge: () => false },
storageAccess,
conflict,
fileReflectionProvenance: {
get: (value: FilePathWithPrefix) => Promise.resolve(reflection.get(value)),
set: (value: FilePathWithPrefix, record: { revision: string }) => {
reflection.set(value, record);
return Promise.resolve();
},
delete: (value: FilePathWithPrefix) => {
reflection.delete(value);
return Promise.resolve();
},
},
fileProcessing: { processFileEvent: { addHandler: vi.fn() } },
replication: { processSynchroniseResult: { addHandler: vi.fn() } },
} as unknown as ServiceFileHandlerDependencies & ServiceDatabaseFileAccessDependencies;
const access = new ServiceDatabaseFileAccessBase(services);
(services as ServiceFileHandlerDependencies).databaseFileAccess = access;
const handler = new TestHandler(services);
return {
db,
localDatabase,
access,
handler,
conflict,
reflection,
storageAccess,
settings,
services,
getStorage: () => storage,
setStorage: (file: UXFileInfo) => {
storage = file;
},
};
}
type Device = Awaited<ReturnType<typeof makeDevice>>;
function requiredEnvironment(name: "hostname" | "username" | "password"): string {
const value = process.env[name];
if (!value) throw new Error(`Missing integration-test environment variable: ${name}`);
return value;
}
/** Read every non-deleted leaf, including branches which are not the winner. */
async function leaves(db: PouchDB.Database<EntryDoc>) {
const docs = await db.get(path, { open_revs: "all", revs: true });
return docs
.flatMap((result) => ("ok" in result && !result.ok._deleted ? [result.ok] : []))
.sort((left, right) => left._rev.localeCompare(right._rev));
}
async function revisionContent(device: Device, rev: string) {
const entry = await device.access.fetchEntry(path, rev, true);
if (!entry) throw new Error(`Missing content for ${rev}`);
return readContent(entry);
}
/** Exercise the real CLI dispatcher and conflict operations with the fixture's real DB services. */
async function resolveFromCLI(device: Device, keep: string) {
const operations = createConflictResolutionOperations({
events: device.services.events,
databaseFileAccess: device.access,
fileHandler: device.handler,
log: vi.fn(),
} as unknown as ConflictResolutionOperationsDependencies);
const context = {
databasePath: "/fixture",
vaultPath: "/fixture",
core: {
services: {
context: { standardIo: { writeStdout: vi.fn(), writeStderr: vi.fn() } },
control: { activated: Promise.resolve() },
conflict: { resolveByDeletingRevision: operations.resolveByDeletingRevision },
},
serviceModules: { databaseFileAccess: device.access, fileHandler: device.handler },
},
} as unknown as CLICommandContext;
await expect(runCommand({ command: "resolve", commandArgs: [path, keep] }, context)).resolves.toBe(true);
}
describe("file provenance across multiple devices and real CouchDB", () => {
const databases: PouchDB.Database<EntryDoc>[] = [];
const owners: LiveSyncLocalDB[] = [];
afterEach(async () => {
for (const owner of owners.splice(0)) {
owner.offRemoteChunkFetchedHandler?.();
await owner.managers.teardownManagers();
}
const results = await Promise.allSettled(databases.splice(0).map((db) => db.destroy()));
for (const result of results) if (result.status === "rejected") throw result.reason;
});
/** Replication deliberately precedes file reflection, modelling a delayed storage event. */
async function conflictedDevices(count: number) {
const name = `livesync-provenance-${crypto.randomUUID()}`;
const remote = new PouchDB<EntryDoc>(`${requiredEnvironment("hostname").replace(/\/+$/u, "")}/${name}`, {
adapter: "http",
auth: { username: requiredEnvironment("username"), password: requiredEnvironment("password") },
});
databases.push(remote);
await remote.info();
const devices: Device[] = [];
for (let i = 0; i < count; i++) {
const device = await makeDevice(`${name}-${i}`);
devices.push(device);
owners.push(device.localDatabase);
databases.push(device.db);
}
const root = await devices[0].access.storeWithBaseRevision(makeFile(old), undefined, true);
if (!root) throw new Error("Could not create the shared original revision");
await devices[0].db.replicate.to(remote);
const revisions: string[] = [];
for (const [index, device] of devices.entries()) {
await device.db.replicate.from(remote);
device.reflection.set(path, { revision: root });
// All devices edit while disconnected. Equal mtimes rule out timestamp-based detection.
device.setStorage(makeFile(`Edited on device ${index}\n`));
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
revisions.push((await device.db.get(path))._rev);
}
// Reverse upload order so the fixture does not rely on the first writer winning.
for (const device of [...devices].reverse()) await device.db.replicate.to(remote);
for (const device of devices) {
await device.db.replicate.from(remote);
expect((await leaves(device.db)).map((doc) => doc._rev)).toEqual([...revisions].sort());
expect(await Promise.all(revisions.map((rev) => revisionContent(device, rev)))).toEqual(
devices.map((_, index) => `Edited on device ${index}\n`)
);
}
expect(await leaves(remote)).toHaveLength(count);
return { devices, remote, root, revisions };
}
async function resolveAndReplicate(f: Awaited<ReturnType<typeof conflictedDevices>>) {
const winner = (await f.devices[0].db.get(path))._rev;
const keepIndex = f.revisions.findIndex((rev) => rev !== winner);
const keep = f.revisions[keepIndex];
const content = await revisionContent(f.devices[0], keep);
await resolveFromCLI(f.devices[0], keep);
expect((await leaves(f.devices[0].db)).map((doc) => doc._rev)).toEqual([keep]);
expect(await f.devices[0].getStorage().body.text()).toBe(content);
expect(f.devices[0].reflection.get(path)?.revision).toBe(keep);
await f.devices[0].db.replicate.to(f.remote);
for (const device of f.devices) await device.db.replicate.from(f.remote);
return { keep, content };
}
it.each([3, 4])(
"does not resurrect unchanged files before or after CLI resolution with %i editing devices",
async (count) => {
const f = await conflictedDevices(count);
for (const [index, device] of f.devices.entries()) {
const before = (await device.db.info()).update_seq;
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
expect((await device.db.info()).update_seq).toBe(before);
expect(await device.getStorage().body.text()).toBe(`Edited on device ${index}\n`);
expect(device.conflict.queueCheckFor).toHaveBeenCalled();
}
const { keep, content } = await resolveAndReplicate(f);
for (const device of f.devices) {
const before = (await device.db.info()).update_seq;
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
expect((await device.db.info()).update_seq).toBe(before);
expect(await device.getStorage().body.text()).toBe(content);
expect(device.reflection.get(path)?.revision).toBe(keep);
await device.db.replicate.to(f.remote);
}
for (const device of f.devices) {
await device.db.replicate.from(f.remote);
expect((await leaves(device.db)).map((doc) => doc._rev)).toEqual([keep]);
}
expect((await leaves(f.remote)).map((doc) => doc._rev)).toEqual([keep]);
},
60_000
);
it("preserves a real edit made on a losing device after three-way resolution", async () => {
const f = await conflictedDevices(3);
const { keep, content } = await resolveAndReplicate(f);
const index = f.revisions.findIndex((rev, i) => i > 0 && rev !== keep);
const device = f.devices[index];
const edit = `${await device.getStorage().body.text()}A further offline edit\n`;
device.setStorage(makeFile(edit));
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
const editedRevision = device.reflection.get(path)!.revision;
const edited = await device.db.get(path, { rev: editedRevision, revs: true });
expect(edited._revisions?.ids[1]).toBe(f.revisions[index].split("-")[1]);
await device.db.replicate.to(f.remote);
for (const peer of f.devices) {
await peer.db.replicate.from(f.remote);
expect((await leaves(peer.db)).map((doc) => doc._rev)).toEqual([keep, editedRevision].sort());
expect(await revisionContent(peer, keep)).toBe(content);
expect(await revisionContent(peer, editedRevision)).toBe(edit);
}
}, 60_000);
it.each(["missing record", "compacted base"] as const)(
"preserves uncertain storage as one independent conflict with four devices: %s",
async (reason) => {
const f = await conflictedDevices(4);
const { keep, content } = await resolveAndReplicate(f);
const index = f.revisions.findIndex((rev, i) => i > 0 && rev !== keep);
const device = f.devices[index];
if (reason === "missing record") {
device.reflection.delete(path);
// Matching an old ancestor must not be mistaken for an unchanged current branch.
device.setStorage(makeFile(old));
} else {
await device.db.compact();
await expect(device.db.get(path, { rev: f.revisions[index] })).rejects.toMatchObject({ status: 404 });
}
const uncertainContent = await device.getStorage().body.text();
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
const independent = device.reflection.get(path)!.revision;
expect(independent).toMatch(/^1-/u);
expect(independent).not.toBe(f.root);
expect((await device.db.get(path, { rev: independent, revs: true }))._revisions?.ids).toHaveLength(1);
const before = (await device.db.info()).update_seq;
device.reflection.delete(path);
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
expect((await device.db.info()).update_seq).toBe(before);
await device.db.replicate.to(f.remote);
for (const peer of f.devices) {
await peer.db.replicate.from(f.remote);
expect((await leaves(peer.db)).map((doc) => doc._rev)).toEqual([keep, independent].sort());
expect(await revisionContent(peer, keep)).toBe(content);
expect(await revisionContent(peer, independent)).toBe(uncertainContent);
}
// The CLI must also accept the independent root as the selected conflict.
await resolveFromCLI(f.devices[0], independent);
await f.devices[0].db.replicate.to(f.remote);
for (const peer of f.devices) {
await peer.db.replicate.from(f.remote);
expect((await leaves(peer.db)).map((doc) => doc._rev)).toEqual([independent]);
}
expect(await f.devices[0].getStorage().body.text()).toBe(uncertainContent);
},
60_000
);
});
+125
View File
@@ -0,0 +1,125 @@
# Test procedures
Run the commands below from the repository root. Test ownership and source layout are described in the [development guide](../devs.md#testing-infrastructure).
## npm 10 clean-installation check
Run this check after changing `package.json`, a workspace manifest, or `package-lock.json`, including a Commonlib dependency update. A successful installation with the npm version bundled with Node.js does not prove that npm 10 accepts the lockfile.
The following sequence matches the installation steps in [unit-ci](../.github/workflows/unit-ci.yml). Use Node.js 24, as configured in that workflow:
```bash
npx --yes npm@10.9.4 ci --ignore-scripts --no-audit --no-fund
npm ci
```
Both commands must complete successfully without changing the lockfile. The first checks npm 10 lockfile compatibility; the second prepares the normal development installation, including lifecycle scripts, for the source checks and tests below. Keep the pinned npm version and command here aligned with CI. This project-side check does not establish the runtime version used by the external Community Review service or replace its authenticated review result.
If Community Review reports widespread TypeScript `error` types across unrelated external packages, first confirm that dependency installation completed successfully. An installation failure can leave external types unresolved and produce misleading source warnings.
## Source and unit checks
Run broad checks sequentially. These examples bound the Node.js heap and Vitest workers for machines with limited memory:
```bash
NODE_OPTIONS=--max-old-space-size=3072 npm run check
NODE_OPTIONS=--max-old-space-size=3072 npm run test:unit -- --maxWorkers=1
```
`npm run check` includes TypeScript, ESLint, the Community rules, Svelte checks, a production build, and bundle compatibility checks. Inspect installation and source-check failures before interpreting later test results. Add the relevant service or runtime suite for the boundary changed:
| Boundary | Procedure |
| --- | --- |
| Multiple-device file conflicts and stale-file protection | [CouchDB procedure below](#multiple-device-conflict-regression-tests) |
| Obsidian startup, file watching, persistence, and UI | [Real Obsidian E2E](e2e-obsidian/README.md) |
| CLI subprocesses, filesystem workflows, and P2P | [CLI Deno tests](../src/apps/cli/testdeno/test_dev_deno.md) and [test authoring](../src/apps/cli/testdeno/CONTRIBUTING_TESTS.md) |
| WebApp, WebPeer, and browser interoperability | [Browser application tests](browser-apps/README.md) |
## Community Review checks and CI confirmation
Before requesting review or merging source or dependency changes, run the project-side Community checks after dependency installation:
```bash
NODE_OPTIONS=--max-old-space-size=3072 npm run lint:community
NODE_OPTIONS=--max-old-space-size=3072 npm run lint:community:tools
```
The source check uses the official `eslint-plugin-obsidianmd` rules with the repository's [Community configuration](../eslint.community.config.mjs). Run it without `--quiet` so warnings remain visible. Review new warnings as well as errors, and distinguish existing warnings from those introduced by the change. A successful exit alone does not establish that the source has no warnings. The tooling check requires zero warnings.
The [unit-ci workflow](../.github/workflows/unit-ci.yml), in its `Unit Tests` job, runs the npm 10 installation check and then `npm run check`. That script includes `lint:community` and `lint:community:tools`, so source warnings are visible in the CI log as well as during local checks. Source errors fail the gate; source warnings remain visible for review without failing it. The explicit commands above can run these checks independently of the full source-check sequence.
After pushing, confirm that the `Unit Tests` job passed for the exact commit being reviewed, including its `Verify clean installation with npm 10` and `Run source checks` steps. For service-backed changes, also confirm the integration-test job and the relevant runtime checks. A successful run for an earlier commit does not validate later changes.
The local checks and CI use the project's installed rule versions and configured file scope. Record the authenticated external Community Review result separately when that review is required; the project-side checks do not replace it. When the external review reports additional findings, retain its relevant output and investigate differences in installation, type resolution, scope, or rules.
## CLI mirror regression tests
Run the native CLI subprocess suite after building the CLI:
```bash
NODE_OPTIONS=--max-old-space-size=3072 npm run build --workspace self-hosted-livesync-cli
cd src/apps/cli/testdeno
deno test -A --no-check test-mirror.ts
```
The expected result is seven passing steps. These cover storage-only and database-only files, database deletion, an ordinary local edit, incoming database content, an omitted Vault path, and local content with unknown provenance.
For an ordinary local edit, first reflect the database content into the Vault, confirm its bytes, and then edit that file. The next `mirror` must store the edit without a conflict. For unknown provenance, use `put` to seed only the database and independently create different local content with a newer modification time. The next `mirror` must preserve two independent non-deleted revisions. Read both with `cat-rev`, submit the same local bytes again to check that no additional revision appears, and use `resolve` to select the local content. Confirm that the conflict is gone and the selected content is reflected into the Vault.
`put` deliberately bypasses file provenance, whereas `push` records it. Substituting one for the other changes the scenario. The tests enable `writeDocumentsIfConflicted` for incoming reflection; this setting does not authorise an ordinary save to replace unrelated database content or resolve the conflict. Do not assume which independent root PouchDB selects as the winner.
The [CLI Docker workflow](../.github/workflows/cli-docker.yml) runs the corresponding Bash suite. From the repository root, build and check that path with:
```bash
NODE_OPTIONS=--max-old-space-size=3072 npm run build:docker --workspace self-hosted-livesync-cli
npm run test:e2e:docker:mirror --workspace self-hosted-livesync-cli
```
The expected result is `PASS=7 FAIL=0`. These mirror suites use temporary local databases and need no CouchDB service. The complete `test:e2e:docker:all` command also runs the other Docker CLI suites and manages a disposable CouchDB fixture; use it to check the full Docker CI gate. Keep the ordinary-edit and unknown-provenance scenarios aligned between the Deno and Bash suites.
## Multiple-device conflict regression tests
### Preparation and execution
The suite is [FileHandler.multidevice.integration.spec.ts](../src/serviceModules/FileHandler.multidevice.integration.spec.ts). It exercises the installed Commonlib package through LiveSync's shared file handler, the CLI `resolve` command dispatcher, and the shared conflict-resolution operations.
Use a disposable CouchDB service. To use the repository's Docker fixture, set the following values in `.test.env` and ensure `.env` exists. The fixture uses the container name `couchdb-test` and host port `5989`:
```dotenv
hostname=http://127.0.0.1:5989/
username=admin
password=testpassword
```
Start the fixture, then run the focused suite:
```bash
npm run test:docker-couchdb:start
NODE_OPTIONS=--max-old-space-size=3072 npm run test:integration -- src/serviceModules/FileHandler.multidevice.integration.spec.ts --maxWorkers=1
```
The expected result is five passing tests. Each case creates a uniquely named remote database and removes it during teardown. Stop the fixture after the run, including when the test command fails:
```bash
npm run test:docker-couchdb:stop
```
When using an already running disposable CouchDB service, configure its endpoint and credentials in `.test.env` and run only the test command. The start and stop commands manage the repository's Docker fixture. This suite needs no Object Storage, P2P relay, or Obsidian application. It is also discovered by the existing integration-test CI job.
### Scenarios and expected results
Each simulated device owns a separate real PouchDB database, `LiveSyncLocalDB` managers, file content, and provenance record. Devices first share one revision, edit while disconnected, and then replicate their Metadata and Chunks through real CouchDB. File reflection is deliberately delayed after replication to reproduce the interval in which the database has advanced but the file still contains older content. All edits use equal modification times, so the tests require revision and content checks rather than timestamp ordering.
| Case | Setup and action | Required result |
| --- | --- | --- |
| Three editing devices | Replicate three conflicting edits, reprocess unchanged files, select a non-winning revision through CLI `resolve`, and replicate the resolution before reprocessing the other files. | All three contents are initially readable on every replica. Unchanged saves make no database writes. After resolution, all files and replicas converge to the selected revision without creating revisions or restoring conflicts. |
| Four editing devices | Repeat the same sequence with four independently edited branches. | All four contents are initially preserved, and the same unchanged-save and convergence guarantees hold. |
| A genuine edit on a losing device | After three-way resolution reaches its DB, a device adds content to its still-unreflected losing file, then saves and replicates it. | The new revision extends that device's recorded branch. The selected result and the new edit remain readable on every replica. |
| Missing provenance | After four-way resolution, remove a losing device's provenance record and set its file to the original historical ancestor's content. Save, remove provenance again, and repeat the save. | The file becomes one fresh independent root, distinct from the historical root. Both contents remain readable after replication, and repeated saves create no duplicates. CLI `resolve` can select the independent root and propagate its resolution. |
| Compacted base | After four-way resolution, compact a losing device's local DB and confirm that its recorded revision body is unavailable. Save the remaining file, then repeat after removing provenance. | The file is preserved as one independent conflict rather than discarded. Both contents remain readable on every replica, repeated saves create no duplicates, and CLI `resolve` can select the independent root and propagate its resolution. |
### Coverage boundaries
This is a service integration test, not a CLI subprocess or Obsidian runtime test. The file and provenance stores are in-memory fixtures; automatic conflict callbacks are observed without running interactive or automatic merge policies. PouchDB revision creation, chunk storage and retrieval, local compaction, CouchDB replication, the CLI command dispatcher, and its resolution operations are real.
Use the CLI and real-Obsidian procedures linked above for argument parsing, persistent host stores, file watchers, and dialogues. In particular, the real-Obsidian `stale-file-restart` scenario exercises persisted pending events and restart, and `folder-batch` exercises bulk Vault rename and deletion. These scenarios do not simulate a mobile operating system suspending the application.
+6
View File
@@ -75,11 +75,17 @@ After changing plug-in source, use the focused wrapper rather than invoking a sc
```bash
npm run test:e2e:obsidian:focused -- settings-ui
npm run test:e2e:obsidian:focused -- two-vault-sync
npm run test:e2e:obsidian:focused -- stale-file-restart
npm run test:e2e:obsidian:focused -- folder-batch
npm run test:e2e:obsidian:focused -- security-seed-reconnect
```
The wrapper accepts only maintained real-Obsidian scenario names; run it with `--help` for the current list. It deliberately does not manage CouchDB, Object Storage, or the P2P signalling relay. Start the required fixture first, or use the complete service-managed suite.
`folder-batch` needs no remote service. It creates 24 notes in nested folders, renames and deletes the parent through the Obsidian Vault API, and checks descendant events, content, Chunks, deletion markers, and provenance. A note outside the parent must remain writable.
`stale-file-restart` needs no remote service. It advances the local database while old Vault bytes remain, persists pending storage events, and restarts the same isolated Vault and profile. It checks that an unchanged file with exact provenance receives the newer database content without creating a revision, that unknown-origin content is preserved on a fresh independent branch, and that losing provenance and processing the file again does not duplicate or automatically merge that branch. The database advance and pending snapshot are controlled fixtures; startup processing, persistence, file reflection, and conflict checking run in real Obsidian. The scenario does not simulate a mobile operating system suspending the application.
The principal entry points are:
```bash
+167
View File
@@ -0,0 +1,167 @@
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { createE2eObsidianDeviceLocalState, waitForLiveSyncCoreReady } from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
const originalRoot = "batch/original";
const renamedRoot = "batch/renamed";
const outsidePath = "batch/outside.md";
const folders = ["alpha", "alpha/deep", "beta"];
const notes = Array.from({ length: 24 }, (_, index) => ({
relativePath: `${folders[index % folders.length]}/note-${index}.md`,
body: `# Descendant ${index}\n\nThis body must survive a parent folder rename.\n`,
}));
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked: ${cli.checked.join(", ")}`);
const cliBinary = cli.binary;
const vault = await createTemporaryVault("obsidian-livesync-folder-batch-");
let session: ObsidianLiveSyncSession | undefined;
try {
session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
pluginData: {
doctorProcessedVersion: "1.0.0",
isConfigured: true,
liveSync: false,
remoteType: "",
couchDB_URI: "http://127.0.0.1:5984",
couchDB_DBNAME: "folder-batch",
notifyThresholdOfRemoteStorageSize: -1,
periodicReplication: false,
syncOnStart: false,
syncOnSave: false,
syncOnFileOpen: false,
syncOnEditorSave: false,
syncAfterMerge: false,
useEden: false,
},
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
const result = await evalObsidianJson<{ descendants: number; renamed: number; deleted: number }>(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const provenance=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
const notes=${JSON.stringify(notes)};
const originalRoot=${JSON.stringify(originalRoot)};
const renamedRoot=${JSON.stringify(renamedRoot)};
const outsidePath=${JSON.stringify(outsidePath)};
const renamed=new Set(), deleted=new Set();
const refs=[
app.vault.on('rename',(file,oldPath)=>{
if(file.stat) renamed.add(oldPath+' -> '+file.path);
}),
app.vault.on('delete',(file)=>{if(file.stat) deleted.add(file.path);}),
];
const meta=(path)=>core.localDatabase.getDBEntryMeta(path,{conflicts:true},true);
const isDeleted=(entry)=>entry && (entry.deleted || entry._deleted);
const getContent=(entry)=>Array.isArray(entry.data)?entry.data.join(''):entry.data;
async function liveErrors(path,body){
const errors=[];
const file=app.vault.getAbstractFileByPath(path);
const entry=await meta(path);
if(!file?.stat || file.path!==path || await app.vault.read(file)!==body)
errors.push('Vault content: '+path);
if(!entry || isDeleted(entry) || entry.path!==path || !entry.children.length){
errors.push('DB metadata: '+path);
}else{
const loaded=await core.localDatabase.getDBEntry(path,{rev:entry._rev},false,true,true);
if(!loaded || getContent(loaded)!==body) errors.push('DB content: '+path);
if(entry._conflicts?.length) errors.push('Unexpected conflict: '+path);
if((await provenance.get(path))?.revision!==entry._rev)
errors.push('Provenance: '+path);
}
return errors;
}
async function deletedErrors(path){
const errors=[];
const entry=await meta(path);
if(app.vault.getAbstractFileByPath(path)) errors.push('File remains: '+path);
if(!isDeleted(entry)) errors.push('Missing tombstone: '+path);
if(entry?._conflicts?.length) errors.push('Deletion conflict: '+path);
if(await provenance.get(path)) errors.push('Old provenance remains: '+path);
return errors;
}
async function waitFor(phase,check){
const deadline=Date.now()+20000;
let errors=[];
do{
await core.services.fileProcessing.commitPendingFileEvents();
errors=await check();
if(!errors.length) return;
await new Promise(resolve=>setTimeout(resolve,50));
}while(Date.now()<deadline);
throw new Error(phase+': '+errors.slice(0,8).join('; '));
}
const liveBatch=(root)=>Promise.all(notes.map(note=>
liveErrors(root+'/'+note.relativePath,note.body))).then(results=>results.flat());
const deletedBatch=(root)=>Promise.all(notes.map(note=>
deletedErrors(root+'/'+note.relativePath))).then(results=>results.flat());
try{
await app.vault.createFolder('batch');
await app.vault.createFolder(originalRoot);
for(const folder of ${JSON.stringify(folders)})
await app.vault.createFolder(originalRoot+'/'+folder);
await Promise.all(notes.map(note=>app.vault.create(originalRoot+'/'+note.relativePath,note.body)));
await app.vault.create(outsidePath,'Outside note');
await waitFor('Initial batch',async()=>[
...await liveBatch(originalRoot), ...await liveErrors(outsidePath,'Outside note'),
]);
const originalIds=await Promise.all(notes.map(async note=>(await meta(originalRoot+'/'+note.relativePath))._id));
// Rename the parent once: Obsidian must emit every descendant event.
await app.vault.rename(app.vault.getAbstractFileByPath(originalRoot),renamedRoot);
await waitFor('Renamed batch',async()=>[
...await liveBatch(renamedRoot), ...await deletedBatch(originalRoot),
...await liveErrors(outsidePath,'Outside note'),
]);
for(const [index,note] of notes.entries()){
const from=originalRoot+'/'+note.relativePath, to=renamedRoot+'/'+note.relativePath;
if(!renamed.has(from+' -> '+to)) throw new Error('Missing descendant rename: '+from);
if((await meta(to))._id===originalIds[index]) throw new Error('Rename reused the source ID: '+to);
}
// Delete the parent once, without synthesising individual file events.
await app.vault.delete(app.vault.getAbstractFileByPath(renamedRoot),true);
await waitFor('Deleted batch',async()=>[
...await deletedBatch(renamedRoot), ...await deletedBatch(originalRoot),
...await liveErrors(outsidePath,'Outside note'),
]);
for(const note of notes){
const path=renamedRoot+'/'+note.relativePath;
if(!deleted.has(path)) throw new Error('Missing descendant deletion: '+path);
}
if(app.vault.getAbstractFileByPath(renamedRoot)) throw new Error('Deleted folder remains');
await app.vault.modify(app.vault.getAbstractFileByPath(outsidePath),'Outside note updated');
await waitFor('Outside update',()=>liveErrors(outsidePath,'Outside note updated'));
return JSON.stringify({descendants:notes.length,renamed:renamed.size,deleted:deleted.size});
}finally{
for(const ref of refs) app.vault.offref(ref);
}
})()`,
session.cliEnv
);
console.log(
`Folder batch: ${result.descendants} descendants persisted, renamed, and deleted; ` +
`${result.renamed} rename and ${result.deleted} delete events observed; outside note remained writable.`
);
} finally {
if (session) await session.app.stop();
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exitCode = 1;
});
+2
View File
@@ -25,6 +25,8 @@ const focusedScenarios = new Set([
"p2p-setup-uri-workflow",
"partial-startup-file-failure",
"startup-scan",
"stale-file-restart",
"folder-batch",
"setup-uri-workflow",
"two-vault-sync",
"security-seed-reconnect",
@@ -0,0 +1,188 @@
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
createE2eObsidianDeviceLocalState,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
const paths = ["stale-known.md", "stale-unknown.md"];
const oldContent = "# Note\nKeep\n\nTail\n\nFooter\n";
const newContent = oldContent.replace(
"Footer\n",
Array.from({ length: 50 }, (_, index) => `Remote addition ${index}\n`).join("") + "Footer\n"
);
type Branch = { rev: string; content: string; history: string[] };
type FileState = { path: string; content: string; rev: string; branches: Branch[]; provenance: string | null };
async function readState(cliBinary: string, env: NodeJS.ProcessEnv): Promise<FileState[]> {
return await evalObsidianJson<FileState[]>(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const store=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
const states=[];
for(const path of ${JSON.stringify(paths)}){
const meta=await core.localDatabase.getDBEntryMeta(path,{conflicts:true},true);
const branches=[];
for(const rev of [meta._rev,...(meta._conflicts??[])]){
const entry=await core.localDatabase.getDBEntry(path,{rev,revs:true},false,true,true);
const raw=await core.localDatabase.getRaw(meta._id,{rev,revs:true});
branches.push({rev,content:Array.isArray(entry.data)?entry.data.join(''):entry.data,
history:raw._revisions.ids.map((id,i)=>(raw._revisions.start-i)+'-'+id)});
}
const file=app.vault.getAbstractFileByPath(path);
states.push({path,content:await app.vault.read(file),rev:meta._rev,branches,
provenance:(await store.get(path))?.revision??null});
}
return JSON.stringify(states);
})()`,
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked: ${cli.checked.join(", ")}`);
const cliBinary = cli.binary;
const vault = await createTemporaryVault("obsidian-livesync-stale-file-");
let session: ObsidianLiveSyncSession | undefined;
try {
session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
pluginData: {
doctorProcessedVersion: "1.0.0",
isConfigured: true,
liveSync: false,
remoteType: "",
couchDB_URI: "http://127.0.0.1:5984",
couchDB_DBNAME: "stale-file-restart",
notifyThresholdOfRemoteStorageSize: -1,
periodicReplication: false,
syncAfterMerge: false,
syncOnEditorSave: false,
syncOnFileOpen: false,
syncOnSave: false,
syncOnStart: false,
disableMarkdownAutoMerge: false,
resolveConflictsByNewerFile: false,
checkConflictOnlyOnOpen: true,
showMergeDialogOnlyOnActive: true,
},
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
await evalObsidianJson(
cliBinary,
`(async()=>{
for(const path of ${JSON.stringify(paths)}) await app.vault.create(path,${JSON.stringify(oldContent)});
return JSON.stringify(true);
})()`,
session.cliEnv
);
for (const path of paths) await waitForLocalDatabaseEntry(cliBinary, session.cliEnv, path);
// Drain real Vault events before creating a persisted pending-event fixture.
// The DB advances without reflecting it in the Vault, as on an offline device.
const fixture = await evalObsidianJson<{ current: string[]; original: string[] }>(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const store=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
await core.services.fileProcessing.commitPendingFileEvents();
const snapshot=[], current=[], original=[];
for(const [index,path] of ${JSON.stringify(paths)}.entries()){
const meta=await core.localDatabase.getDBEntryMeta(path,{},true);
const file=await core.storageAccess.getFileStub(path);
const data=new Blob([${JSON.stringify(newContent)}],{type:'text/plain'});
const result=await core.localDatabase.putDBEntry({...meta,data,mtime:file.stat.mtime+60000,
size:data.size,children:[]},false,meta._rev);
if(!result?.ok) throw new Error('Could not advance '+path);
current.push(result.rev); original.push(meta._rev);
if(index===0) await store.set(path,{revision:meta._rev,observedStorageMtime:file.stat.mtime});
else await store.delete(path);
snapshot.push({type:'CHANGED',key:'CHANGED-'+path,args:{file}});
}
await core.kvDB.set('storage-event-manager-snapshot',snapshot);
return JSON.stringify({current,original});
})()`,
session.cliEnv
);
await session.app.stop();
session = undefined;
session = await startObsidianLiveSyncSession({ binary, cliBinary, vault });
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
const [known, unknown] = await readState(cliBinary, session.cliEnv);
assertEqual(known.rev, fixture.current[0], "An unchanged stale file created a revision during restart.");
assertEqual(known.branches.length, 1, "An unchanged stale file created a conflict.");
assertEqual(known.content, newContent, "The newer DB content was not reflected after suppressing the save.");
assertEqual(known.provenance, fixture.current[0], "The reflected revision was not recorded.");
assertEqual(unknown.branches.length, 2, "Unknown local content was not preserved as a conflict.");
assertEqual(unknown.content, oldContent, "Unknown local content was overwritten.");
const independent = unknown.branches.find((branch) => branch.content === oldContent);
if (!independent) throw new Error("The old local content is missing from the current branches.");
assertEqual(independent.history.length, 1, "Unknown content was attached to an inferred ancestor.");
if (independent.rev === fixture.original[1]) throw new Error("The historical root was reused.");
if (!unknown.branches.some((branch) => branch.content === newContent)) {
throw new Error("The remote additions were lost.");
}
await evalObsidianJson(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const path=${JSON.stringify(paths[1])};
await core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1').delete(path);
if(!await core.fileHandler.storeFileToDB(path)) throw new Error('Repeated save failed');
await app.workspace.getLeaf(false).openFile(app.vault.getAbstractFileByPath(${JSON.stringify(paths[0])}));
await core.services.conflict.resolve(path);
return JSON.stringify(true);
})()`,
session.cliEnv
);
const [, repeated] = await readState(cliBinary, session.cliEnv);
assertEqual(
repeated.branches
.map((branch) => branch.rev)
.sort()
.join(","),
unknown.branches
.map((branch) => branch.rev)
.sort()
.join(","),
"Losing provenance and reprocessing added or auto-merged a branch."
);
await evalObsidianJson(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
core.settings.resolveConflictsByNewerFile=true;
await core.services.conflict.resolve(${JSON.stringify(paths[1])});
return JSON.stringify(true);
})()`,
session.cliEnv
);
const [, resolved] = await readState(cliBinary, session.cliEnv);
assertEqual(resolved.branches.length, 1, "The explicit newer-file option did not resolve the conflict.");
assertEqual(resolved.content, newContent, "The newer-file option did not reflect the newer DB version.");
console.log(
"Stale-file restart: known content reflected; unknown content preserved without duplicate branches; explicit newer-file resolution retained."
);
} finally {
if (session) await session.app.stop();
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exitCode = 1;
});
+28 -6
View File
@@ -997,14 +997,26 @@ async function runMarkdownAutoMerge(
session = await startConfiguredSession(context, vaultA, conflictOverrides);
const baseOnA = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, conflictPath);
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, left, baseOnA.rev);
await writeVaultFile(vaultA.path, conflictPath, left);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, left);
const storedLeft = await waitForConflictBranch(
context.cliBinary,
session.cliEnv,
conflictPath,
(branch) => branch.content === left
);
assertEqual(storedLeft.parentRev, baseOnA.rev, "Vault A's edit did not extend its displayed base.");
await pushLocalChanges(context.cliBinary, session.cliEnv);
await stopTrackedSession(context, session);
session = await startConfiguredSession(context, vaultB, conflictOverrides);
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, right, baseOnB.rev);
await writeVaultFile(vaultB.path, conflictPath, right);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, right);
const storedRight = await waitForConflictBranch(
context.cliBinary,
session.cliEnv,
conflictPath,
(branch) => branch.content === right
);
assertEqual(storedRight.parentRev, baseOnB.rev, "Vault B's edit did not extend its displayed base.");
await pushLocalChanges(context.cliBinary, session.cliEnv);
const conflict = await waitForFileConflict(context.cliBinary, session.cliEnv, conflictPath);
const leftBranch = conflict.branches.find((branch) => branch.content === left);
@@ -1028,8 +1040,18 @@ async function runMarkdownAutoMerge(
);
const afterResolution = `${merged.trimEnd()}\n\nPost-resolution edit on B.\n`;
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, afterResolution, mergedRev);
await writeVaultFile(vaultB.path, conflictPath, afterResolution);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, afterResolution);
const storedAfterResolution = await waitForConflictBranch(
context.cliBinary,
session.cliEnv,
conflictPath,
(branch) => branch.content === afterResolution
);
assertEqual(
storedAfterResolution.parentRev,
mergedRev,
"The post-resolution edit did not extend the merged revision."
);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await stopTrackedSession(context, session);
+10
View File
@@ -12,6 +12,16 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
## Unreleased
### Synchronisation
#### Fixed
- Fixed a case where unchanged local files could overwrite newer synchronised content after a restart. (#994)
- Local content whose origin cannot be established is now preserved as a conflict, allowing you to review and choose which version to keep.
- The same protection applies to ordinary file synchronisation in the command-line tool.
- Fast Fetch now avoids repeated requests for the same Security Seed during a transfer, reducing network traffic during initial setup.
- Object Storage synchronisation now reads fresh synchronisation parameters before it can write to the remote, then reuses them for that transfer. A failed parameter read stops the transfer before writing.
## 1.0.29
16th September, 2026