Compare commits

..

12 Commits

Author SHA1 Message Date
github-actions[bot] 809caffed9 Releasing 1.0.0-beta.4 2026-07-25 05:08:39 +00:00
vorotamoroz bea0e68091 Keep translation details out of startup path 2026-07-25 04:34:53 +00:00
vorotamoroz 99eb9cd46f Record beta.3 and current validation 2026-07-25 03:49:39 +00:00
vorotamoroz a7bc337fd1 Test Security Seed refresh in Real Obsidian 2026-07-25 03:49:18 +00:00
vorotamoroz 0d68b6530f Test Garbage Collection V3 against CouchDB 2026-07-25 03:48:51 +00:00
vorotamoroz c24fc1dc82 Update Commonlib to 0.1.0-rc.12 2026-07-24 19:06:53 +00:00
vorotamoroz c8162fd0cd Make dialogue prose selectable 2026-07-24 17:53:55 +00:00
vorotamoroz 0a7a3b1635 Harden Garbage Collection V3 validation 2026-07-24 17:53:22 +00:00
vorotamoroz 20fb027055 Protect conflict chunks during garbage collection 2026-07-24 17:09:22 +00:00
vorotamoroz 07cba4ce83 Keep unreadable revisions for explicit repair 2026-07-24 16:15:46 +00:00
vorotamoroz 658ad6dfe4 Describe saved connections consistently 2026-07-24 16:15:02 +00:00
vorotamoroz 0e5475b7e3 Limit commands to applicable contexts 2026-07-24 16:13:49 +00:00
63 changed files with 4682 additions and 361 deletions
@@ -65,7 +65,7 @@ The special meaning would duplicate `activeConfigurationId`, make a user-visible
### Add profile naming and full list editing to onboarding
That would make the first-run path longer and duplicate the established Remote Databases interface. Automatic descriptive names and later renaming keep this change limited to data integrity and consistent selection.
That would make the first-run path longer and duplicate the established Saved connections interface. Automatic descriptive names and later renaming keep this change limited to data integrity and consistent selection.
### Replace the compatibility fields immediately
@@ -77,11 +77,11 @@ Commonlib unit tests cover preserving existing profiles, opaque-ID insertion, ge
Self-hosted LiveSync unit tests cover preserving modern Setup URI profiles and their active selection, retaining legacy Setup URI and QR migration, adding CouchDB and Object Storage profiles beside an existing profile, independent P2P selection, fresh P2P selection as both main and P2P remote, and cancellation without mutation.
The real-Obsidian onboarding E2E owns the invitation, dialogue presentation, safe-area and touch-target checks, cancellation, and command reopening. It does not contact a remote or submit credentials. Remote connection correctness remains owned by the CouchDB, Object Storage, P2P, and two-Vault suites. The end-to-end Setup URI and provisioning acceptance workflow remains a separate release gate.
The real-Obsidian onboarding E2E owns the invitation, dialogue presentation, safe-area and touch-target checks, cancellation, and reopening from the Setup pane. It does not contact a remote or submit credentials. Remote connection correctness remains owned by the CouchDB, Object Storage, P2P, and two-Vault suites. The end-to-end Setup URI and provisioning acceptance workflow remains a separate release gate.
## Consequences
- Manual onboarding and the Remote Databases pane share one Commonlib profile contract.
- Manual onboarding and the Saved connections list share one Commonlib profile contract.
- Existing profiles survive reconfiguration, and a newly configured connection becomes explicitly selectable.
- Modern imports retain user-assigned profile identity and names.
- Legacy Setup URIs continue to work through an isolated compatibility boundary.
@@ -60,7 +60,7 @@ Keep configured-state inference separate from new-Vault initialisation. If an ex
### Onboarding activation and initialisation
- Keep an unconfigured Vault outside database initialisation, offline scanning, and configured-only checks. Offer setup through the long-lived onboarding Notice and the permanent command instead of opening a competing dialogue automatically.
- Keep an unconfigured Vault outside database initialisation, offline scanning, and configured-only checks. Offer setup through the long-lived onboarding Notice, and allow the wizard to be reopened from the Setup pane instead of opening a competing dialogue automatically.
- For new-device onboarding, reserve Rebuild before enabling and saving the accepted settings.
- For an unconfigured existing device, reserve Fetch before enabling and saving imported or manually confirmed settings.
- Suspend the current runtime after the flag has been written, apply the accepted settings through the scheduler's preparation callback, and request restart only after that callback succeeds.
+1 -1
View File
@@ -48,7 +48,7 @@ A TURN provider cannot read LiveSync's encrypted Vault contents, but it can obse
The **P2P Status** pane is the current Obsidian interface for P2P connections.
- The command **Self-hosted LiveSync: P2P Sync : Open P2P Status** remains available from the command palette.
- After a P2P configuration exists, the command **Self-hosted LiveSync: P2P Sync : Open P2P Status** is available from the command palette.
- The P2P ribbon icon appears only after a P2P configuration exists.
- LiveSync does not open the pane merely because Obsidian has started. If the pane was already part of the saved Obsidian workspace, Obsidian may restore it.
- Workspaces containing the retired P2P pane are migrated to the current status pane. The retired command is no longer exposed.
+1 -1
View File
@@ -69,7 +69,7 @@ Garbage Collection removes unreferenced chunks while preserving the current data
- all relevant devices have synchronised; and
- the remaining historical and deletion state is understood.
Deleted documents and tombstones are not free, and historical revisions may keep chunks reachable. Garbage Collection therefore cannot promise the smallest possible remote.
Deleted documents, tombstones, live conflicts, and retained metadata are not free. Live conflict branches keep the chunks needed for review, while an ordinary superseded linear revision does not protect its former chunks. Garbage Collection can therefore make old content unreadable and cannot promise the smallest possible remote. Review the [Garbage Collection V3 specification](specs_garbage_collection.md) before using it.
Rebuild is a different operation. It reconstructs the database from a chosen authoritative state and is the more certain way to remove unwanted history or repair a damaged remote, but it is also more disruptive and can discard changes which exist only elsewhere.
+23 -9
View File
@@ -12,7 +12,7 @@ The following status applies to optional and compatibility features in the 1.0 l
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Supported, opt-in | Peer-to-Peer Synchronisation, Hidden File Sync, and Customisation Sync | Maintained and covered by focused real-runtime tests. Enable them only where their separate setup and operational constraints are acceptable. |
| Maintained, advanced | Data Compression | Available as an explicit storage and bandwidth trade-off. It remains disabled by default because the measured processing and memory costs outweigh the mixed-dataset saving. |
| Beta or experimental | JWT authentication, ignore files, automatic newer-file conflict resolution, and Garbage Collection V3 | Retained for explicit testing and specialised use. They remain disabled by default and are not part of the minimum supported setup. |
| Beta or experimental | JWT authentication, ignore files, automatic newer-file conflict resolution, and Garbage Collection V3 for CouchDB | Retained for explicit testing and specialised use. They remain disabled by default and are not part of the minimum supported setup. |
| Compatibility only | V1 dynamic iteration counts, the old IndexedDB adapter, non-current hash algorithms, Eden chunks, and the stored `doNotUseFixedRevisionForChunks` key | Existing settings and data remain readable. New Vaults use the current defaults, and compatibility controls are shown only where a migration or recovery path still needs them. |
| Icon | Description |
@@ -40,7 +40,7 @@ Internal database or settings compatibility reviews use a separate safety dialog
This pane is used for setting up Self-hosted LiveSync. There are several options to set up Self-hosted LiveSync.
An unconfigured installation does not open the onboarding dialogue automatically or scan the Vault into the local database. A long-lived Notice offers the onboarding action, and **Open onboarding wizard** remains available from the command palette after that Notice closes.
An unconfigured installation does not open the onboarding dialogue automatically or scan the Vault into the local database. A long-lived Notice offers the onboarding action. If the Notice is dismissed, open **Self-hosted LiveSync settings****Setup****Rerun Onboarding Wizard**.
Choose the new-device path when this device owns the files which should initialise synchronisation. Choose the existing-device path when it should receive an established remote state. The wizard reserves Rebuild or Fetch respectively before enabling the settings and requesting a restart, so the selected initialisation runs before the ordinary start-up scan.
@@ -169,9 +169,9 @@ Show verbose log. Please enable when you report the logs
## 3. Remote Configuration
### 1. Remote Server
### 1. Connection settings
Self-hosted LiveSync supports multiple remote connection profiles under **Remote Server** -> **Remote Databases**. This allows you to save and switch between multiple databases or bucket configurations in a single vault.
Self-hosted LiveSync stores multiple remote connection profiles under **Connection settings** **Saved connections**. Each profile represents a CouchDB database, an Object Storage connection, or a P2P configuration, and several profiles can be kept in one Vault.
Each profile has an opaque identifier and a presentation name. The name does not need to be unique and is not used to select the profile. The main remote and the P2P remote are selected independently, so code and settings imports must preserve both selections rather than relying on a special identifier such as `default`.
@@ -185,7 +185,7 @@ Each profile has an opaque identifier and a presentation name. The name does not
Setting key: remoteType
The active remote server type. This is automatically projected to the legacy configuration when you activate a connection profile.
The active connection type. This is automatically projected to the legacy configuration when you activate a connection profile.
### 2. Notification
@@ -698,6 +698,12 @@ Open the dialogue
#### Make report to inform the issue
#### Copy database information for a file
Select a file to copy its local database information. The command **Copy database information for the active file** performs the same inspection for the file open in the editor.
The report includes the Vault-relative path, document and chunk identifiers, local database revisions, conflicts, and local chunk availability. It does not query the remote or include file contents. Paths and identifiers can still be private metadata, so review the report before sharing it.
#### Write logs into the file
Setting key: writeLogToTheFile
@@ -721,17 +727,19 @@ Stop reflecting database changes to storage files.
### 3. Recovery and Repair
#### Recreate missing chunks for all files
#### Recreate chunks for current Vault files
This will recreate chunks for all files. If there were missing chunks, this may fix the errors.
Recreate chunks from files currently present in the Vault. This can repair missing chunks for those exact current contents after they have been confirmed as authoritative. It cannot reconstruct unavailable historical or conflict content.
#### Resolve All conflicted files by the newer one
Resolve all conflicted files by the newer one. Caution: This will overwrite the older one, and cannot resurrect the overwritten one.
After confirmation, resolve every conflict by modification time. This logically deletes every version except the newest one. It is a destructive policy choice and cannot recover content which is already unavailable.
#### Verify and repair all files
Compare the content of files between on local database and storage. If not matched, you will be asked which one you want to keep.
Compare each Vault file with every current live revision in the local database. Each winner and conflict revision is shown separately with its exact revision identifier, local chunk availability, and relationship to the current Vault file. Unavailable shared ancestors are reported separately because they prevent conservative three-way merging but are not live revisions which can be discarded.
`Retry reading revision` retries the configured chunk-retrieval path without changing the revision tree. `Discard unreadable revision` is offered only for an exact current live revision which remains unreadable; after confirmation, it creates a logical deletion for that revision. Prefer recovery from another replica or backup before discarding it.
#### Check and convert non-path-obfuscated files
@@ -1039,6 +1047,12 @@ Purge all download/upload cache.
Delete all data on the remote server.
### 6. Garbage Collection V3 (CouchDB only)
Garbage Collection V3 identifies chunk documents which are not reachable from any current file or live conflict branch, creates logical deletions for those chunks locally, propagates the deletions to CouchDB, and requests remote compaction.
Use it only when the Vault, local database, and remote are healthy, and every relevant device has synchronised. It can make an ordinary superseded file revision unreadable when no live state still needs its chunks. It does not repair corruption or replace a deliberate rebuild. See the [Garbage Collection V3 specification](specs_garbage_collection.md).
### 7. Reset
#### Delete local database to reset or uninstall Self-hosted LiveSync
+1 -1
View File
@@ -34,7 +34,7 @@ Before starting:
![P2P local database confirmation on the first device](../images/p2p-setup/guide-p2p-setup-first-rebuild-confirmation.png)
7. Keep optional features disabled until ordinary note synchronisation works.
8. Open `Self-hosted LiveSync: P2P Sync : Open P2P Status` from the command palette. After a P2P profile exists, the P2P ribbon icon provides the same destination. Select `Open connection` if signalling is disconnected.
8. After saving the P2P profile, open `Self-hosted LiveSync: P2P Sync : Open P2P Status` from the command palette. The P2P ribbon icon provides the same destination. Select `Open connection` if signalling is disconnected.
![First P2P device connected to the signalling relay](../images/p2p-setup/guide-p2p-setup-first-device-connected.png)
+20 -2
View File
@@ -46,6 +46,24 @@ The all-branch history check prevents a resolved conflict from being recreated m
The compatibility implementation currently selects the newer modification time for differing binary conflicts even when the general **Always overwrite with a newer file** option is disabled. This is existing behaviour, not a new 1.0 guarantee. Changing it to explicit selection only is a separate compatibility decision.
## Unreadable revisions and repair
A document revision can remain in the PouchDB tree while one or more chunks needed to reconstruct its content are unavailable. Missing content is not evidence that the revision is obsolete. LiveSync therefore leaves an unreadable winner or conflict revision in the tree instead of deleting it during automatic conflict processing.
**Hatch** → **Verify and repair all files** inspects the current winner, every current conflict revision, and the nearest shared ancestor for each conflict. It reports exact revision identifiers and local chunk availability separately:
- **Retry reading revision** attempts the configured chunk-retrieval path again. It does not change the revision tree.
- **Discard unreadable revision** is available only for a current winner or conflict revision which remains unreadable when the action is performed. It requires confirmation and creates a logical deletion for that exact revision.
- A shared ancestor is informational. An ancestor which is no longer a live revision cannot be discarded independently through this workflow. If its body is unavailable, conservative three-way merge remains disabled, although readable live revisions can still be selected manually.
Logical deletion does not recreate missing bytes, purge the document history, or prove that the deleted version was unimportant. Another replica or backup may still contain the missing chunks. Recover from that source before discarding a revision whenever possible.
**Recreate chunks for current Vault files** can recreate chunks only from files which are readable in the current Vault. It cannot reconstruct unique bytes from an unavailable historical or conflict revision.
Garbage Collection V3 treats every live conflict revision and its nearest available shared ancestor as reachable. Their locally available chunks are retained until the conflict is resolved. After resolution, chunks used only by the discarded branch or no-longer-needed merge ancestry can become eligible for collection. See the [Garbage Collection V3 specification](specs_garbage_collection.md).
A generation-one revision has no parent. When its body is unavailable, LiveSync cannot preserve a changed Vault file as a sibling branch without inventing ancestry. It leaves the operation unresolved. Recover the missing chunks from another replica or backup, or explicitly discard that live revision. If the current Vault file is the intended replacement, it can be stored after the unreadable revision has been logically deleted.
### Two devices independently create the same path
If two devices create the same full synchronised path before either device has
@@ -220,8 +238,8 @@ Do not:
## Verification
Commonlib's real-PouchDB and injected-boundary unit tests cover unequal branch lengths, exact shared ancestry, deterministic ordering of multiple live leaves, a sensible stage followed by reconstruction of a manual pair, content below a deleted losing leaf, recorded and reconstructed branch identity, ambiguous matches, conflict-time editing, logical deletion, case-only rename, cross-path rename, and safe unproven fallbacks.
Commonlib's real-PouchDB and injected-boundary unit tests cover unequal branch lengths, exact shared ancestry, deterministic ordering of multiple live leaves, a sensible stage followed by reconstruction of a manual pair, content below a deleted losing leaf, recorded and reconstructed branch identity, ambiguous matches, conflict-time editing, missing-body preservation when parent metadata is available, refusal to invent a parent for a generation-one revision, logical deletion, case-only rename, cross-path rename, and safe unproven fallbacks.
LiveSync's optional real-Obsidian two-Vault checks have two scopes. `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` resolves and edits a Markdown conflict, propagates it to a Vault which still displays the deleted losing content, and requires one live result to remain. `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` edits, deletes, case-renames, and cross-path-renames files while conflicts remain active; it verifies the parent revision of each resulting branch, replicates those exact trees, and confirms that the other live branches remain intact.
The focused `test:e2e:obsidian:conflict-dialog-policy` scenario creates three live versions in one real Obsidian Vault. It verifies the count warning, commits a concatenated child of the displayed winner, confirms that the untouched leaf remains as one conflict, postpones that remaining pair, restarts the isolated Obsidian profile, and confirms that only the live pair is reconstructed. It also verifies that an incoming resolution closes a stale dialogue, completes the waiting conflict operation, and clears the warning.
The focused `test:e2e:obsidian:conflict-dialog-policy` scenario creates three live versions in one real Obsidian Vault. It verifies the count warning, commits a concatenated child of the displayed winner, confirms that the untouched leaf remains as one conflict, postpones that remaining pair, restarts the isolated Obsidian profile, and confirms that only the live pair is reconstructed. It also verifies that an incoming resolution closes a stale dialogue, completes the waiting conflict operation, and clears the warning. The repair scenario removes a referenced local chunk, confirms that the exact unreadable live revision remains in the tree, and exercises explicit retry and discard controls without deleting another live revision.
+65
View File
@@ -0,0 +1,65 @@
# Garbage Collection V3
Garbage Collection V3 is a beta maintenance operation for CouchDB remotes. It removes chunk documents which are no longer required by the current local revision tree, propagates those logical deletions to CouchDB, and then requests remote compaction.
It is not a repair operation. Use it only when the Vault, the local LiveSync database, and the CouchDB remote are healthy, every relevant device has synchronised, and recoverable backups exist.
## Supported scope
Garbage Collection V3 is available in Edge Case mode for CouchDB. It is not offered for Object Storage or P2P:
- Object Storage has a different journal and object lifecycle.
- P2P has no central database to compact and cannot provide the accepted-device progress information required by this workflow.
The operation requires **Fetch chunks on demand** to be off so that its local reachability result is not confused with chunks which exist only on the remote.
## Workflow
After the user starts Garbage Collection V3, LiveSync:
1. completes a one-shot bidirectional CouchDB synchronisation;
2. reads the accepted-device list and current progress recorded on the remote;
3. requires parseable progress information, warns when an accepted device has no current information or device progress differs, then requires explicit confirmation;
4. computes the chunks reachable from the local PouchDB revision tree;
5. creates a logical deletion for each locally present chunk which is not reachable;
6. completes a push-only replication so that those deletions reach CouchDB;
7. requests CouchDB compaction and waits for it for up to two minutes; and
8. clears the local chunk caches.
If the initial synchronisation, device inspection, or confirmation fails, the workflow stops before collection. Missing or invalid device progress is treated as a failed inspection rather than offered as a confirmation override. If push-only replication fails, the local logical deletions have already been created, but remote compaction is not started; synchronise again before retrying or using another device. A compaction failure or completion timeout is reported separately and is not also reported as a successful completion.
## Reachability rules
A chunk remains reachable when it is referenced by any of the following:
- the current database winner for a file;
- any other live conflict revision for that file;
- an available revision on either side of a live conflict which is required to describe the divergence; or
- the nearest available revision shared by both live conflict branches.
Chunk identifiers are content-derived and shared between files. Reachability is therefore collected into one set across the database. A chunk used by two or more current files remains protected even when one file is updated or deleted.
An ordinary superseded linear revision does not protect its former chunks. Once no current file or live conflict branch references a chunk, it can be collected. After a conflict is resolved, chunks unique to the discarded branch and to no-longer-needed merge ancestry can also become eligible.
## Consequences
Garbage Collection deliberately trades historical recoverability for storage. A metadata revision may remain in the revision tree after a chunk which only that superseded revision used has been collected, so that historical body can become unreadable. Remote compaction can then discard old CouchDB revision bodies. Tombstones and retained metadata also consume storage, so the operation does not promise the smallest possible database.
Writing the same bytes again produces the same content-derived chunk identifier. If that chunk was collected previously, the normal chunk-writing path creates a new live revision for it, and ordinary replication can transfer it again. This does not recover an older file revision automatically; it only makes the newly written content available.
Garbage Collection does not reconstruct a chunk which is already missing, determine whether an unreadable revision is important, or repair a damaged local database. Use **Verify and repair all files**, another healthy replica, or a backup for those cases. Use **Overwrite Server Data with This Device's Files** only when a chosen Vault is authoritative and a deliberate remote rebuild is required.
## Verification
Commonlib tests use real in-memory PouchDB revision trees to verify:
- collection eligibility after a normal file update;
- protection of chunks shared by multiple current files;
- protection of all live conflict branches and their nearest available shared ancestor;
- eligibility of losing-branch and ancestor-only chunks after conflict resolution;
- propagation of chunk deletion to another PouchDB database; and
- recreation and propagation when the same content is written again.
Self-hosted LiveSync unit tests verify that Garbage Collection V3 uses Commonlib's revision-aware result, deletes only unreachable chunks, performs the initial bidirectional and final push-only replications in order, and requests remote compaction.
A disposable real-CouchDB integration test verifies logical deletion on the server, retention of shared and conflict chunks, compaction completion, replication after collection, and recreation of a content-addressed chunk. CouchDB's choice and timing of physical byte reclamation remain an external database boundary, so the test does not assert a particular reduction in database size.
+13 -4
View File
@@ -53,9 +53,16 @@ Check Obsidian's `Detect all file extensions`, LiveSync selectors, ignore files,
If the log reports missing chunks or a size mismatch:
1. restart Obsidian once to rule out an interrupted fetch;
2. on a device which has the correct file, run `Recreate missing chunks for all files`, then synchronise; and
3. if the mismatch remains, run `Verify and repair all files` from `Hatch` and review which copy is authoritative.
1. stop editing the affected file and keep a separate copy of any readable content;
2. restart Obsidian once to rule out an interrupted fetch;
3. synchronise a device or restore a backup which still has the correct content;
4. on that healthy device, run `Recreate chunks for current Vault files`, then synchronise;
5. run `Verify and repair all files` from `Hatch`; review the winner, every conflict revision, and any unavailable shared ancestor separately; and
6. use `Discard unreadable revision` only after confirming that the exact revision is no longer recoverable or wanted.
`Retry reading revision` does not change the revision tree. `Discard unreadable revision` creates a logical deletion for one current winner or conflict revision after rechecking it. It does not purge history or reconstruct missing content. An unavailable non-live ancestor cannot be deleted through this workflow; it disables conservative three-way merge but does not prevent explicit selection between readable live revisions.
`Recreate chunks for current Vault files` uses current Vault content. It cannot recreate unique bytes which exist only in an unreadable historical or conflict revision.
## A configuration mismatch dialogue blocks synchronisation
@@ -110,6 +117,8 @@ Enable Obsidian's `Detect all file extensions`, then check LiveSync selectors, i
Run `Generate full report for opening the issue with debug info` to copy the current settings summary and recent verbose log lines. Remove credentials, remote URLs, Vault names, file contents, and other private information before sharing it.
When a problem concerns one file, run **Copy database information for the active file**, or use **Hatch****Copy database information for a file** to select another file. The report describes this device's local database view, including the Vault-relative path, document and chunk identifiers, local database revisions, conflicts, and local chunk availability. It does not query the remote server or include file contents. Treat paths and identifiers as private metadata before sharing.
Use `Show log` for live inspection. Logs are intentionally kept in memory for a limited time to reduce accidental disclosure. Enable `Write logs into the file` only while reproducing a problem, then disable it and remove the file after review because persistent logging affects performance and may contain private data.
![Write logs into the file](../images/write_logs_into_the_file.png)
@@ -120,7 +129,7 @@ Browser security errors, particularly CORS failures, may reach the plug-in only
LiveSync stores file metadata, chunks, revision history, conflicts, deletions, and tombstones. Deleting or shortening a file therefore does not immediately remove every object which once represented it.
Garbage Collection can remove unreferenced chunks, but it is appropriate only when the Vault and local database are healthy and all relevant devices have synchronised. Tombstones and retained revisions are not free, so Garbage Collection does not guarantee a minimal database.
Garbage Collection V3 can remove unreferenced chunks from a healthy CouchDB setup, but it is appropriate only when the Vault and local database are healthy and all relevant devices have synchronised. Current files and live conflict branches protect their required chunks; an ordinary superseded revision does not. Tombstones and retained metadata are not free, so Garbage Collection does not guarantee a minimal database. Review the [Garbage Collection V3 specification](specs_garbage_collection.md) before using it.
`Overwrite Server Data with This Device's Files` is a separate rebuild operation and is the more certain way to reconstruct a central remote from a chosen authoritative Vault. It is also destructive and may discard changes which exist only on another device. Review [Recovery and flag files](recovery.md#garbage-collection-is-not-rebuild) before choosing between them.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "1.0.0-beta.3",
"version": "1.0.0-beta.4",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",
+9 -9
View File
@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "1.0.0-beta.3",
"version": "1.0.0-beta.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "1.0.0-beta.3",
"version": "1.0.0-beta.4",
"license": "MIT",
"workspaces": [
"src/apps/cli",
@@ -22,7 +22,7 @@
"@smithy/querystring-builder": "^4.2.9",
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/livesync-commonlib": "0.1.0-rc.11",
"@vrtmrz/livesync-commonlib": "0.1.0-rc.12",
"@vrtmrz/obsidian-plugin-kit": "0.1.2",
"diff-match-patch": "^1.0.5",
"fflate": "^0.8.2",
@@ -4764,9 +4764,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.0-rc.11",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.0-rc.11.tgz",
"integrity": "sha512-o811duZFajxDFI6cy7zwtXPg6lihx5e1MH6Xse1sx4HseYurbaDf9DtZJdAtfkjTxl5wOZRSnVMoMLn+I/xD+g==",
"version": "0.1.0-rc.12",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.0-rc.12.tgz",
"integrity": "sha512-pMiOL4x5pDKCYOwtH3nG+9Ra1sLjowItBUrYoHpvcrweV4NSN0OkAEMk1vxBQlKMmKtnrAdy0WuQvmsdLOWHqA==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
@@ -15913,7 +15913,7 @@
},
"src/apps/cli": {
"name": "self-hosted-livesync-cli",
"version": "1.0.0-beta.3-cli",
"version": "1.0.0-beta.4-cli",
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
@@ -15938,7 +15938,7 @@
},
"src/apps/webapp": {
"name": "livesync-webapp",
"version": "1.0.0-beta.3-webapp",
"version": "1.0.0-beta.4-webapp",
"dependencies": {
"octagonal-wheels": "^0.1.51"
},
@@ -15950,7 +15950,7 @@
}
},
"src/apps/webpeer": {
"version": "1.0.0-beta.3-webpeer",
"version": "1.0.0-beta.4-webpeer",
"dependencies": {
"octagonal-wheels": "^0.1.51"
},
+4 -2
View File
@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "1.0.0-beta.3",
"version": "1.0.0-beta.4",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -50,6 +50,7 @@
"test:e2e:obsidian:onboarding-invitation": "tsx test/e2e-obsidian/scripts/onboarding-invitation.ts",
"test:e2e:obsidian:dialog-mounts": "tsx test/e2e-obsidian/scripts/dialog-mounts.ts",
"test:e2e:obsidian:conflict-dialog-policy": "tsx test/e2e-obsidian/scripts/conflict-dialog-policy.ts",
"test:e2e:obsidian:revision-repair": "tsx test/e2e-obsidian/scripts/revision-repair.ts",
"test:e2e:obsidian:settings-ui": "tsx test/e2e-obsidian/scripts/settings-ui.ts",
"test:e2e:obsidian:review-harness": "tsx test/e2e-obsidian/scripts/review-harness.ts",
"test:e2e:obsidian:p2p-pane": "tsx test/e2e-obsidian/scripts/p2p-pane.ts",
@@ -63,6 +64,7 @@
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
"test:e2e:obsidian:setup-uri-workflow": "tsx test/e2e-obsidian/scripts/setup-uri-workflow.ts",
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
"test:e2e:obsidian:security-seed-reconnect": "tsx test/e2e-obsidian/scripts/security-seed-reconnect.ts",
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
"test:e2e:obsidian:setting-markdown-export": "tsx test/e2e-obsidian/scripts/setting-markdown-export.ts",
@@ -163,7 +165,7 @@
"@smithy/querystring-builder": "^4.2.9",
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/livesync-commonlib": "0.1.0-rc.11",
"@vrtmrz/livesync-commonlib": "0.1.0-rc.12",
"@vrtmrz/obsidian-plugin-kit": "0.1.2",
"diff-match-patch": "^1.0.5",
"fflate": "^0.8.2",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "self-hosted-livesync-cli",
"private": true,
"version": "1.0.0-beta.3-cli",
"version": "1.0.0-beta.4-cli",
"main": "dist/index.cjs",
"type": "module",
"scripts": {
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "livesync-webapp",
"private": true,
"version": "1.0.0-beta.3-webapp",
"version": "1.0.0-beta.4-webapp",
"type": "module",
"description": "Browser-based Self-hosted LiveSync using FileSystem API",
"scripts": {
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "webpeer",
"private": true,
"version": "1.0.0-beta.3-webpeer",
"version": "1.0.0-beta.4-webpeer",
"type": "module",
"scripts": {
"dev": "vite",
@@ -9,8 +9,7 @@
export const liveSyncProvisionalEnglishMessages = {
"This first setup has several short steps because it confirms encryption, the connection method, and which device provides the initial data. Once it is complete, additional devices can reuse a Setup URI.":
"This first setup has several short steps because it confirms encryption, the connection method, and which device provides the initial data. Once it is complete, additional devices can reuse a Setup URI.",
"Setup Complete: Preparing to Fetch from Another Device":
"Setup Complete: Preparing to Fetch from Another Device",
"Setup Complete: Preparing to Fetch from Another Device": "Setup Complete: Preparing to Fetch from Another Device",
"The P2P connection has been configured successfully. The initial synchronisation data must now be fetched from an online source device.":
"The P2P connection has been configured successfully. The initial synchronisation data must now be fetched from an online source device.",
"After restarting, select an online source device for the initial Fetch. The local LiveSync database on this device will be rebuilt from that source. Unsynchronised files in this Vault may conflict with the fetched data.":
@@ -64,6 +63,57 @@ export const liveSyncProvisionalEnglishMessages = {
"This file has unresolved conflicts.": "This file has unresolved conflicts.",
"This file has ${COUNT} unresolved versions. They will be reviewed one pair at a time.":
"This file has ${COUNT} unresolved versions. They will be reviewed one pair at a time.",
"Sync now": "Sync now",
"Apply pending changes now": "Apply pending changes now",
"Copy database information for the active file": "Copy database information for the active file",
"Copy database information for a file": "Copy database information for a file",
"Copy revision, conflict, and local chunk availability information, including document and chunk identifiers but not file contents.":
"Copy revision, conflict, and local chunk availability information, including document and chunk identifiers but not file contents.",
"Choose file": "Choose file",
"Choose a file to inspect": "Choose a file to inspect",
"Database information for ${FILE}": "Database information for ${FILE}",
"All revisions and chunk availability below are a snapshot of this device's local database; the remote is not queried. Review the Vault-relative path, document identifier, content-derived chunk identifiers, and metadata before sharing this report. File contents are omitted.":
"All revisions and chunk availability below are a snapshot of this device's local database; the remote is not queried. Review the Vault-relative path, document identifier, content-derived chunk identifiers, and metadata before sharing this report. File contents are omitted.",
"Vault file: modified ${TIME}, size ${SIZE}": "Vault file: modified ${TIME}, size ${SIZE}",
"Vault file: missing": "Vault file: missing",
"Local database document: missing": "Local database document: missing",
"${ROLE}: ${REVISION}": "${ROLE}: ${REVISION}",
"Winner revision": "Winner revision",
"Conflict revision": "Conflict revision",
"Unknown revision": "Unknown revision",
"Logical deletion": "Logical deletion",
"Readable on this device; recorded size ${RECORDED}, decoded size ${ACTUAL}":
"Readable on this device; recorded size ${RECORDED}, decoded size ${ACTUAL}",
"Unreadable on this device; ${COUNT} referenced chunks are missing or deleted":
"Unreadable on this device; ${COUNT} referenced chunks are missing or deleted",
"Matches the current Vault file": "Matches the current Vault file",
"Differs from the current Vault file": "Differs from the current Vault file",
"Retry reading revision": "Retry reading revision",
"Discard unreadable revision": "Discard unreadable revision",
"Discard database revision ${REVISION} of ${FILE}? This creates a logical deletion for that exact live revision. Missing content cannot be recovered by this action.":
"Discard database revision ${REVISION} of ${FILE}? This creates a logical deletion for that exact live revision. Missing content cannot be recovered by this action.",
"Revision metadata is unavailable on this device": "Revision metadata is unavailable on this device",
"Shared ancestor ${REVISION} is not readable on this device. Automatic three-way merging may be unavailable, but the live revisions remain available for explicit review.":
"Shared ancestor ${REVISION} is not readable on this device. Automatic three-way merging may be unavailable, but the live revisions remain available for explicit review.",
"No shared ancestor is available for this conflict. The live revisions remain available for explicit review.":
"No shared ancestor is available for this conflict. The live revisions remain available for explicit review.",
"Show revision history": "Show revision history",
"Use Vault file in local database": "Use Vault file in local database",
"Restore database winner to Vault": "Restore database winner to Vault",
"Copy database information": "Copy database information",
"Recreate chunks for current Vault files": "Recreate chunks for current Vault files",
"Recreate chunks from the files currently present in this Vault. This cannot reconstruct unavailable historical or conflict content.":
"Recreate chunks from the files currently present in this Vault. This cannot reconstruct unavailable historical or conflict content.",
"Recreate current chunks": "Recreate current chunks",
"Resolve every conflict by modification time? This logically deletes every version except the newest one and cannot recover content which is already unavailable.":
"Resolve every conflict by modification time? This logically deletes every version except the newest one and cannot recover content which is already unavailable.",
"Resolve all conflicts by the newest version": "Resolve all conflicts by the newest version",
"Verify and repair all files": "Verify and repair all files",
"Compare each Vault file with every live local-database revision. Unreadable conflict versions remain visible until you retry or explicitly discard an exact revision.":
"Compare each Vault file with every live local-database revision. Unreadable conflict versions remain visible until you retry or explicitly discard an exact revision.",
"Verify all": "Verify all",
"Connection settings": "Connection settings",
"Saved connections": "Saved connections",
} as const;
export type LiveSyncProvisionalMessageKey = keyof typeof liveSyncProvisionalEnglishMessages;
+2
View File
@@ -30,6 +30,8 @@ describe("LiveSync-owned translation catalogue", () => {
it("uses LiveSync-owned provisional English without extending Commonlib's message contract", () => {
expect($msg("This file has unresolved conflicts.")).toBe("This file has unresolved conflicts.");
expect($msg("More actions for ${DEVICE}", { DEVICE: "phone" })).toBe("More actions for phone");
expect($msg("Connection settings")).toBe("Connection settings");
expect($msg("Saved connections")).toBe("Saved connections");
expect(
$msg("This file has ${COUNT} unresolved versions. They will be reviewed one pair at a time.", {
COUNT: "3",
@@ -0,0 +1,96 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
addIcon: vi.fn(),
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
Notice: class Notice {},
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("./PluginDialogModal.ts", () => ({
PluginDialogModal: class PluginDialogModal {},
}));
vi.mock("@/features/HiddenFileCommon/JsonResolveModal.ts", () => ({
JsonResolveModal: class JsonResolveModal {},
}));
vi.mock("@/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts", () => ({
ConflictResolveModal: class ConflictResolveModal {},
}));
vi.mock("@/features/LiveSyncCommands.ts", () => ({
LiveSyncCommands: class LiveSyncCommands {
core!: { services: unknown };
get services() {
return this.core.services;
}
},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
EVEN: Symbol("even"),
disposeMemoObject: vi.fn(),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
memoIfNotExist: vi.fn(),
memoObject: vi.fn(),
retrieveMemoObject: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/events.ts", () => ({
EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG: "open-plugin-sync",
eventHub: {
onEvent: vi.fn(),
},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import { ConfigSync } from "./CmdConfigSync";
describe("ConfigSync commands", () => {
it("shows the Customisation Sync command only whilst the feature is enabled", () => {
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
const settings = {
usePluginSync: false,
};
const showPluginSyncModal = vi.fn();
const configSync = Object.create(ConfigSync.prototype) as ConfigSync;
Object.assign(configSync, {
core: {
settings,
services: {
API: {
addCommand: vi.fn((command) => commands.push(command)),
},
},
},
addRibbonIcon: vi.fn(() => ({
addClass: vi.fn(),
})),
showPluginSyncModal,
});
configSync.onload();
const command = commands.find(({ id }) => id === "livesync-plugin-dialog-ex");
expect(command?.checkCallback?.(true)).toBe(false);
settings.usePluginSync = true;
expect(command?.checkCallback?.(true)).toBe(true);
expect(command?.checkCallback?.(false)).toBe(true);
expect(showPluginSyncModal).toHaveBeenCalledOnce();
});
});
+8 -2
View File
@@ -454,8 +454,14 @@ export class ConfigSync extends LiveSyncCommands {
this.services.API.addCommand({
id: "livesync-plugin-dialog-ex",
name: "Show customization sync dialog",
callback: () => {
this.showPluginSyncModal();
checkCallback: (checking) => {
if (!this.isThisModuleEnabled()) {
return false;
}
if (!checking) {
this.showPluginSyncModal();
}
return true;
},
});
this.addRibbonIcon("custom-sync", $msg("cmdConfigSync.showCustomizationSync"), () => {
@@ -54,10 +54,7 @@ import { EVENT_SETTING_SAVED, eventHub } from "@/common/events.ts";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import type { LiveSyncCore } from "@/main.ts";
import { tryGetFilePath } from "@vrtmrz/livesync-commonlib/compat/common/utils.doc";
import {
configureHiddenFileSyncMode,
type ConfigureHiddenFileSyncResult,
} from "./configureHiddenFileSyncMode.ts";
import { configureHiddenFileSyncMode, type ConfigureHiddenFileSyncResult } from "./configureHiddenFileSyncMode.ts";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import { getObsidianCommunityPluginManager } from "@/common/obsidianCommunityPlugins.ts";
type SyncDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce";
@@ -109,37 +106,45 @@ export class HiddenFileSync extends LiveSyncCommands {
this.services.API.addCommand({
id: "livesync-sync-internal",
name: "(re)initialise hidden files between storage and database",
callback: () => {
if (this.isReady()) {
checkCallback: (checking) => {
if (!this.isManualCommandAvailable()) return false;
if (!checking) {
void this.initialiseInternalFileSync("safe", true);
}
return true;
},
});
this.services.API.addCommand({
id: "livesync-scaninternal-storage",
name: "Scan hidden file changes on the storage",
callback: () => {
if (this.isReady()) {
checkCallback: (checking) => {
if (!this.isManualCommandAvailable()) return false;
if (!checking) {
void this.scanAllStorageChanges(true);
}
return true;
},
});
this.services.API.addCommand({
id: "livesync-scaninternal-database",
name: "Scan hidden file changes on the local database",
callback: () => {
if (this.isReady()) {
checkCallback: (checking) => {
if (!this.isManualCommandAvailable()) return false;
if (!checking) {
void this.scanAllDatabaseChanges(true);
}
return true;
},
});
this.services.API.addCommand({
id: "livesync-internal-scan-offline-changes",
name: "Scan and apply all offline hidden-file changes",
callback: () => {
if (this.isReady()) {
checkCallback: (checking) => {
if (!this.isManualCommandAvailable()) return false;
if (!checking) {
void this.applyOfflineChanges(true);
}
return true;
},
});
eventHub.onEvent(EVENT_SETTING_SAVED, () => {
@@ -191,12 +196,16 @@ export class HiddenFileSync extends LiveSyncCommands {
}
isReady() {
if (!this._isMainReady) return false;
if (!this._isMainReady()) return false;
if (this._isMainSuspended()) return false;
if (!this.isThisModuleEnabled()) return false;
return true;
}
private isManualCommandAvailable() {
return this.settings.useAdvancedMode && this.isReady() && this._isDatabaseReady();
}
async performStartupScan(showNotice: boolean) {
await this.applyOfflineChanges(showNotice);
}
@@ -8,13 +8,16 @@ vi.mock("@/features/HiddenFileCommon/JsonResolveModal.ts", () => ({
vi.mock("@/features/LiveSyncCommands.ts", () => ({
LiveSyncCommands: class LiveSyncCommands {
plugin!: { app: unknown };
core!: { services: unknown };
core!: { services: unknown; settings: unknown };
get app() {
return this.plugin.app;
}
get services() {
return this.core.services;
}
get settings() {
return this.core.settings;
}
},
}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
@@ -25,6 +28,66 @@ import { HiddenFileSync } from "./CmdHiddenFileSync.ts";
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
describe("HiddenFileSync configuration-change notices", () => {
it("shows manual Hidden File Sync commands only when the feature, Advanced mode, and runtime are ready", () => {
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
const settings = {
syncInternalFiles: false,
useAdvancedMode: false,
};
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
core: {
settings,
services: {
API: {
addCommand: vi.fn((command) => commands.push(command)),
},
},
},
_isMainReady: vi.fn(() => true),
_isMainSuspended: vi.fn(() => false),
_isDatabaseReady: vi.fn(() => true),
});
hiddenFileSync.onload();
const commandIds = [
"livesync-sync-internal",
"livesync-scaninternal-storage",
"livesync-scaninternal-database",
"livesync-internal-scan-offline-changes",
];
for (const commandId of commandIds) {
const command = commands.find(({ id }) => id === commandId);
expect(command?.checkCallback?.(true)).toBe(false);
}
settings.syncInternalFiles = true;
settings.useAdvancedMode = true;
for (const commandId of commandIds) {
const command = commands.find(({ id }) => id === commandId);
expect(command?.checkCallback?.(true)).toBe(true);
}
});
it("does not report Hidden File Sync as ready before the main runtime is ready", () => {
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
core: {
settings: {
syncInternalFiles: true,
},
},
_isMainReady: vi.fn(() => false),
_isMainSuspended: vi.fn(() => false),
});
expect(hiddenFileSync.isReady()).toBe(false);
});
it("groups plug-in reloads and an Obsidian restart into one finished Notice", async () => {
const noticeGroups = {
setItem: vi.fn(),
@@ -0,0 +1,257 @@
import { describe, expect, it, vi } from "vitest";
import PouchDB from "pouchdb-core";
import HttpPouch from "pouchdb-adapter-http";
import MemoryAdapter from "pouchdb-adapter-memory";
import replication from "pouchdb-replication";
vi.mock("@/features/LiveSyncCommands", () => ({
LiveSyncCommands: class LiveSyncCommands {
core!: { settings: unknown };
get settings() {
return this.core.settings;
}
},
}));
vi.mock("@/common/events", () => ({
EVENT_ANALYSE_DB_USAGE: "analyse",
EVENT_REQUEST_PERFORM_GC_V3: "gc",
eventHub: {
onEvent: vi.fn(),
},
}));
import {
DEFAULT_SETTINGS,
REMOTE_COUCHDB,
type DocumentID,
type EntryDoc,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import { LocalDatabaseMaintenance } from "./CmdLocalDatabaseMainte";
PouchDB.plugin(HttpPouch).plugin(MemoryAdapter).plugin(replication);
type FixtureContent = {
type: "leaf" | "plain";
data?: string;
path?: string;
children?: string[];
ctime?: number;
mtime?: number;
size?: number;
eden?: Record<string, never>;
};
type FixtureDocument = PouchDB.Core.PutDocument<FixtureContent> & {
_id: DocumentID;
_revisions?: {
start: number;
ids: string[];
};
};
function chunk(id: string, data = id): FixtureDocument {
return {
_id: id as DocumentID,
type: "leaf",
data,
};
}
function revision(id: string, rev: string, history: string[], children: string[]): FixtureDocument {
return {
_id: id as DocumentID,
_rev: rev,
_revisions: {
start: Number(rev.split("-")[0]),
ids: history,
},
type: "plain",
path: id,
children,
ctime: 1,
mtime: 1,
size: children.length,
eden: {},
} as unknown as FixtureDocument;
}
function liveSyncDatabaseFor(database: PouchDB.Database<FixtureContent>): LiveSyncLocalDB {
const subject = Object.create(LiveSyncLocalDB.prototype) as LiveSyncLocalDB;
Object.assign(subject, {
localDatabase: database as unknown as PouchDB.Database<EntryDoc>,
});
return subject;
}
function requiredEnvironment(name: "hostname" | "username" | "password"): string {
const value = process.env[name];
if (!value) {
throw new Error(`Missing required integration-test environment variable: ${name}`);
}
return value;
}
describe("LocalDatabaseMaintenance Garbage Collection V3 with CouchDB", () => {
it("propagates collection safely, completes compaction, and permits content-addressed chunk recreation", async () => {
const databaseName = `livesync-gcv3-${crypto.randomUUID()}`;
const local = new PouchDB<FixtureContent>(`${databaseName}-source`, { adapter: "memory" });
const replica = new PouchDB<FixtureContent>(`${databaseName}-replica`, { adapter: "memory" });
const remote = new PouchDB<FixtureContent>(
`${requiredEnvironment("hostname").replace(/\/+$/u, "")}/${databaseName}`,
{
adapter: "http",
auth: {
username: requiredEnvironment("username"),
password: requiredEnvironment("password"),
},
}
);
try {
await remote.info();
await local.bulkDocs([
chunk("h:obsolete"),
chunk("h:current"),
chunk("h:shared"),
chunk("h:base"),
chunk("h:left"),
chunk("h:right"),
]);
const firstRevision = revision("first.md", "1-first", ["first"], ["h:obsolete"]);
delete firstRevision._rev;
delete firstRevision._revisions;
await local.put(firstRevision);
await local.put({
...(await local.get("first.md")),
children: ["h:current"],
});
for (const id of ["second.md", "third.md"]) {
const sharedRevision = revision(id, `1-${id}`, [id], ["h:shared"]);
delete sharedRevision._rev;
delete sharedRevision._revisions;
await local.put(sharedRevision);
}
await local.bulkDocs(
[
revision("conflicted.md", "1-base", ["base"], ["h:base"]),
revision("conflicted.md", "2-left", ["left", "base"], ["h:left"]),
revision("conflicted.md", "2-right", ["right", "base"], ["h:right"]),
],
{ new_edits: false }
);
const liveSyncDatabase = liveSyncDatabaseFor(local);
const replicationModes: string[] = [];
const replicator = {
openOneShotReplication: vi.fn(
async (
_settings: typeof DEFAULT_SETTINGS,
_showResult: boolean,
_ignoreCleanLock: boolean,
mode: string
) => {
replicationModes.push(mode);
if (mode === "sync") {
await local.sync(remote);
} else if (mode === "pushOnly") {
await local.replicate.to(remote);
} else {
throw new Error(`Unexpected replication mode: ${mode}`);
}
return true;
}
),
getConnectedDeviceList: vi.fn(() =>
Promise.resolve({
accepted_nodes: ["integration-device"],
node_info: {
"integration-device": {
progress: "10-local",
device_name: "Integration device",
app_version: "1.12.7",
plugin_version: "1.0.0-beta.0",
},
},
})
),
connectRemoteCouchDBWithSetting: vi.fn(() => Promise.resolve({ db: remote })),
};
const notice = vi.fn();
const maintenance = Object.create(LocalDatabaseMaintenance.prototype) as LocalDatabaseMaintenance;
Object.assign(maintenance, {
core: {
settings: {
...DEFAULT_SETTINGS,
remoteType: REMOTE_COUCHDB,
},
replicator,
confirm: {
askSelectStringDialogue: vi.fn(() => Promise.resolve("Proceed Garbage Collection")),
},
},
localDatabase: liveSyncDatabase,
_notice: notice,
});
vi.spyOn(maintenance, "ensureAvailable").mockResolvedValue(true);
const clearHash = vi.spyOn(maintenance, "clearHash").mockImplementation(() => undefined);
await maintenance.gcv3();
expect(replicationModes).toEqual(["sync", "pushOnly"]);
expect(clearHash).toHaveBeenCalledOnce();
expect(notice).toHaveBeenCalledWith("Compaction on remote database completed successfully.", "gc-compact");
expect(notice).not.toHaveBeenCalledWith("Compaction on remote database timed out.", "gc-compact");
expect(notice).not.toHaveBeenCalledWith("Compaction on remote database failed.", "gc-compact");
const obsoleteRow = (await remote.allDocs({ keys: ["h:obsolete"] })).rows[0];
expect(obsoleteRow).toMatchObject({
id: "h:obsolete",
value: {
deleted: true,
},
});
for (const retainedChunk of ["h:current", "h:shared", "h:base", "h:left", "h:right"]) {
await expect(remote.get(retainedChunk)).resolves.toMatchObject({
_id: retainedChunk,
type: "leaf",
});
}
await expect(remote.get("second.md")).resolves.toMatchObject({ children: ["h:shared"] });
await expect(remote.get("third.md")).resolves.toMatchObject({ children: ["h:shared"] });
await expect(remote.get("conflicted.md", { conflicts: true })).resolves.toMatchObject({
_conflicts: [expect.any(String)],
});
await remote.replicate.to(replica);
await expect(replica.get("h:obsolete")).rejects.toMatchObject({ status: 404 });
for (const retainedChunk of ["h:current", "h:shared", "h:base", "h:left", "h:right"]) {
await expect(replica.get(retainedChunk)).resolves.toMatchObject({
_id: retainedChunk,
type: "leaf",
});
}
await local.put(chunk("h:obsolete", "recreated"));
await local.replicate.to(remote);
await remote.replicate.to(replica);
await expect(remote.get("h:obsolete")).resolves.toMatchObject({
_id: "h:obsolete",
type: "leaf",
data: "recreated",
});
await expect(replica.get("h:obsolete")).resolves.toMatchObject({
_id: "h:obsolete",
type: "leaf",
data: "recreated",
});
} finally {
await Promise.all([local.destroy(), replica.destroy(), remote.destroy()]);
}
}, 30_000);
});
@@ -4,6 +4,7 @@ import {
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
REMOTE_COUCHDB,
type DocumentID,
type EntryDoc,
type EntryLeaf,
@@ -38,16 +39,27 @@ export class LocalDatabaseMaintenance extends LiveSyncCommands {
id: "analyse-database",
name: "Analyse Database Usage (advanced)",
icon: "database-search",
callback: async () => {
await this.analyseDatabase();
checkCallback: (checking) => {
if (!this.settings.useAdvancedMode || !this._isDatabaseReady()) return false;
if (!checking) {
void this.analyseDatabase();
}
return true;
},
});
this.plugin.addCommand({
id: "gc-v3",
name: "Garbage Collection V3 (advanced, beta)",
icon: "trash-2",
callback: async () => {
await this.gcv3();
checkCallback: (checking) => {
const isApplicableRemote = this.settings.remoteType === REMOTE_COUCHDB;
if (!this.settings.useEdgeCaseMode || !this._isDatabaseReady() || !isApplicableRemote) {
return false;
}
if (!checking) {
void this.gcv3();
}
return true;
},
});
eventHub.onEvent(EVENT_ANALYSE_DB_USAGE, () => this.analyseDatabase());
@@ -450,7 +462,7 @@ Note: **Make sure to synchronise all devices before deletion.**
const confirmMessage = `This function deletes unused chunks from the device. If there are differences between devices, some chunks may be missing when resolving conflicts.
Be sure to synchronise before executing.
However, if you have deleted them, you may be able to recover them by performing Hatch -> Recreate missing chunks for all files.
If chunks used by current Vault files are deleted, Hatch -> Recreate chunks for current Vault files can recreate them only from files currently present in the Vault. It cannot recover unreadable historical or conflict content.
Are you ready to delete unused chunks?`;
@@ -748,7 +760,7 @@ Success: ${successCount}, Errored: ${errored}`;
timeout -= 2000;
if (timeout <= 0) {
this._notice("Compaction on remote database timed out.", "gc-compact");
break;
return;
}
} else {
break;
@@ -871,9 +883,14 @@ It is preferable to update all devices if possible. If you have any devices that
}
//2. Check whether the progress values in NodeData are roughly the same (only the numerical part is needed).
const progressValues = Object.values(node_info)
.map((e) => e.progress.split("-")[0])
.map((e) => parseInt(e));
const progressValues = Object.values(node_info).map((entry) => {
const progress = typeof entry.progress === "string" ? entry.progress.split("-")[0] : "";
return /^\d+$/u.test(progress) ? Number(progress) : Number.NaN;
});
if (progressValues.length === 0 || progressValues.some((progress) => !Number.isSafeInteger(progress))) {
this._notice("No connected device information found. Cancelling Garbage Collection.");
return;
}
const maxProgress = Math.max(...progressValues);
const minProgress = Math.min(...progressValues);
const progressDifference = maxProgress - minProgress;
@@ -912,41 +929,22 @@ This may indicate that some devices have not completed synchronisation, which co
const gcStartTime = Date.now();
// Perform Garbage Collection (new implementation).
const localDatabase = this.localDatabase.localDatabase;
const usedChunks = new Set<DocumentID>();
const allChunks = new Map<DocumentID, string>();
const IDs = this.localDatabase.findEntryNames("", "", {});
let i = 0;
const doc_count = (await localDatabase.info()).doc_count;
for await (const id of IDs) {
const doc = await this.localDatabase.getRaw(id as DocumentID);
i++;
if (i % 100 == 0) {
this._notice(`Garbage Collection: Scanned ${i} / ~${doc_count} `, "gc-scanning");
}
if (!doc) continue;
if ("children" in doc) {
const children = (doc.children || []) as DocumentID[];
for (const chunkId of children) {
usedChunks.add(chunkId);
}
} else if (doc.type === EntryTypes.CHUNK) {
allChunks.set(doc._id, doc._rev);
}
}
// Use the revision-aware reachability scan. Reading only winning revisions
// would make chunks used exclusively by live conflict branches look unused.
const { used: usedChunks, existing: allChunks } = await this.localDatabase.allChunks();
this._notice(
`Garbage Collection: Scanning completed. Total chunks: ${allChunks.size}, Used chunks: ${usedChunks.size}`,
"gc-scanning"
);
const unusedChunks = [...allChunks.keys()].filter((e) => !usedChunks.has(e));
const unusedChunks = [...allChunks.entries()].filter(([chunkId]) => !usedChunks.has(chunkId));
this._notice(`Garbage Collection: Found ${unusedChunks.length} unused chunks to delete.`, "gc-scanning");
const deleteChunkDocs = unusedChunks.map(
(chunkId) =>
([chunkId, chunk]) =>
({
_id: chunkId,
_id: chunkId as DocumentID,
_deleted: true,
_rev: allChunks.get(chunkId),
_rev: chunk._rev,
}) as EntryLeaf
);
const response = await localDatabase.bulkDocs(deleteChunkDocs);
@@ -1,5 +1,43 @@
import { describe, expect, it, vi } from "vitest";
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("octagonal-wheels/number", () => ({
sizeToHumanReadable: vi.fn((value: number) => `${value} B`),
}));
vi.mock("octagonal-wheels/concurrency/lock_v2", () => ({
serialized: vi.fn((_key: string, task: () => unknown) => task()),
}));
vi.mock("octagonal-wheels/collection", () => ({
arrayToChunkedArray: vi.fn((values: unknown[]) => [values]),
}));
vi.mock("@vrtmrz/livesync-commonlib/compat/common/utils", async (importOriginal) => {
const actual = await importOriginal<typeof import("@vrtmrz/livesync-commonlib/compat/common/utils")>();
return {
...actual,
delay: vi.fn(async () => undefined),
};
});
vi.mock("@/features/LiveSyncCommands", () => ({
LiveSyncCommands: class LiveSyncCommands {
core!: { settings: unknown };
get settings() {
return this.core.settings;
}
},
}));
vi.mock("@/common/events", () => ({
EVENT_ANALYSE_DB_USAGE: "analyse",
EVENT_REQUEST_PERFORM_GC_V3: "gc",
eventHub: {
onEvent: vi.fn(),
},
}));
import {
DEFAULT_SETTINGS,
REMOTE_COUCHDB,
REMOTE_MINIO,
REMOTE_P2P,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LocalDatabaseMaintenance } from "./CmdLocalDatabaseMainte";
import { ensureLocalDatabaseMaintenancePrerequisites } from "./maintenancePrerequisites";
function createPrerequisites(settingsOverride: Partial<typeof DEFAULT_SETTINGS> = {}) {
@@ -22,6 +60,52 @@ function createPrerequisites(settingsOverride: Partial<typeof DEFAULT_SETTINGS>
}
describe("LocalDatabaseMaintenance prerequisites", () => {
it("shows database analysis in Advanced mode and Garbage Collection only in applicable Edge Case mode", () => {
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
const settings: {
useAdvancedMode: boolean;
useEdgeCaseMode: boolean;
remoteType: string;
} = {
useAdvancedMode: false,
useEdgeCaseMode: false,
remoteType: REMOTE_COUCHDB,
};
const maintenance = Object.create(LocalDatabaseMaintenance.prototype) as LocalDatabaseMaintenance;
Object.assign(maintenance, {
plugin: {
addCommand: vi.fn((command) => commands.push(command)),
},
core: {
settings,
},
_isDatabaseReady: vi.fn(() => true),
});
maintenance.onload();
const analyse = commands.find(({ id }) => id === "analyse-database");
const garbageCollect = commands.find(({ id }) => id === "gc-v3");
expect(analyse?.checkCallback?.(true)).toBe(false);
expect(garbageCollect?.checkCallback?.(true)).toBe(false);
settings.useAdvancedMode = true;
expect(analyse?.checkCallback?.(true)).toBe(true);
expect(garbageCollect?.checkCallback?.(true)).toBe(false);
settings.useEdgeCaseMode = true;
expect(garbageCollect?.checkCallback?.(true)).toBe(true);
settings.remoteType = REMOTE_P2P;
expect(garbageCollect?.checkCallback?.(true)).toBe(false);
settings.remoteType = REMOTE_MINIO;
expect(garbageCollect?.checkCallback?.(true)).toBe(false);
});
it("asks to disable on-demand chunk fetching before maintenance actions", async () => {
const { settings, askSelectStringDialogue, applyPartial } = createPrerequisites();
@@ -107,4 +191,231 @@ describe("LocalDatabaseMaintenance prerequisites", () => {
expect(askSelectStringDialogue).not.toHaveBeenCalled();
expect(applyPartial).not.toHaveBeenCalled();
});
it("describes the current chunk-recreation action without promising historical recovery", async () => {
const maintenance = Object.create(LocalDatabaseMaintenance.prototype) as LocalDatabaseMaintenance;
const askSelectStringDialogue = vi.fn().mockResolvedValue("Cancel");
Object.assign(maintenance, {
core: {
confirm: {
askSelectStringDialogue,
},
},
_log: vi.fn(),
});
vi.spyOn(maintenance, "ensureAvailable").mockResolvedValue(true);
vi.spyOn(maintenance, "trackChanges").mockResolvedValue(undefined);
await maintenance.performGC();
const message = vi.mocked(askSelectStringDialogue).mock.calls[0]?.[0] as string;
expect(message).toContain("Hatch -> Recreate chunks for current Vault files");
expect(message).toContain("only from files currently present in the Vault");
expect(message).not.toContain("Recreate missing chunks for all files");
});
});
describe("LocalDatabaseMaintenance Garbage Collection V3", () => {
it("does not report remote compaction as successful after its completion wait times out", async () => {
const maintenance = Object.create(LocalDatabaseMaintenance.prototype) as LocalDatabaseMaintenance;
const notice = vi.fn();
const remoteDatabase = {
compact: vi.fn(async () => ({ ok: true })),
info: vi.fn(async () => ({ compact_running: true })),
};
Object.assign(maintenance, {
core: {
replicator: {
connectRemoteCouchDBWithSetting: vi.fn(async () => ({ db: remoteDatabase })),
},
settings: {
...DEFAULT_SETTINGS,
remoteType: REMOTE_COUCHDB,
},
},
_notice: notice,
});
await maintenance.compactDatabase();
expect(notice).toHaveBeenCalledWith("Compaction on remote database timed out.", "gc-compact");
expect(notice).not.toHaveBeenCalledWith(
"Compaction on remote database completed successfully.",
"gc-compact"
);
});
it.each([
["no device progress entries", {}],
[
"an unparseable device progress entry",
{
"device-a": {
progress: "",
device_name: "Device A",
app_version: "1.12.7",
plugin_version: "1.0.0-beta.0",
},
},
],
[
"a missing device progress entry",
{
"device-a": {
device_name: "Device A",
app_version: "1.12.7",
plugin_version: "1.0.0-beta.0",
},
},
],
] as const)("cancels before collection when the milestone has %s", async (_case, nodeInfo) => {
const maintenance = Object.create(LocalDatabaseMaintenance.prototype) as LocalDatabaseMaintenance;
const pushModes: string[] = [];
const allChunks = vi.fn(async () => ({
used: new Set<string>(),
existing: new Map(),
}));
const replicator = {
openOneShotReplication: vi.fn(async (...args: unknown[]) => {
pushModes.push(String(args[3]));
return true;
}),
getConnectedDeviceList: vi.fn(async () => ({
accepted_nodes: Object.keys(nodeInfo),
node_info: nodeInfo,
})),
};
const notice = vi.fn();
Object.assign(maintenance, {
core: {
settings: {
...DEFAULT_SETTINGS,
remoteType: REMOTE_COUCHDB,
},
replicator,
confirm: {
askSelectStringDialogue: vi.fn(async () => "Proceed Garbage Collection"),
},
},
localDatabase: {
allChunks,
localDatabase: {
bulkDocs: vi.fn(async () => []),
},
},
_notice: notice,
});
vi.spyOn(maintenance, "ensureAvailable").mockResolvedValue(true);
vi.spyOn(maintenance, "compactDatabase").mockResolvedValue(undefined);
vi.spyOn(maintenance, "clearHash").mockImplementation(() => undefined);
await maintenance.gcv3();
expect(allChunks).not.toHaveBeenCalled();
expect(pushModes).toEqual(["sync"]);
expect(notice).toHaveBeenCalledWith(
"No connected device information found. Cancelling Garbage Collection."
);
});
it("keeps chunks referenced by a live conflict revision and deletes only unreachable chunks", async () => {
const maintenance = Object.create(LocalDatabaseMaintenance.prototype) as LocalDatabaseMaintenance;
const pushModes: string[] = [];
const deletedChunks: Array<{ _id: string; _rev?: string; _deleted?: boolean }> = [];
const allChunks = vi.fn(async () => ({
used: new Set(["h:winner", "h:conflict"]),
existing: new Map([
["h:winner", { _id: "h:winner", _rev: "1-winner", type: "leaf", data: "winner" }],
["h:conflict", { _id: "h:conflict", _rev: "1-conflict", type: "leaf", data: "conflict" }],
["h:obsolete", { _id: "h:obsolete", _rev: "1-obsolete", type: "leaf", data: "obsolete" }],
]),
}));
const rawDocuments = new Map<string, object>([
[
"note.md",
{
_id: "note.md",
_rev: "2-winner",
_conflicts: ["2-conflict"],
type: "plain",
children: ["h:winner"],
},
],
["h:winner", { _id: "h:winner", _rev: "1-winner", type: "leaf", data: "winner" }],
["h:conflict", { _id: "h:conflict", _rev: "1-conflict", type: "leaf", data: "conflict" }],
["h:obsolete", { _id: "h:obsolete", _rev: "1-obsolete", type: "leaf", data: "obsolete" }],
]);
const findEntryNames = vi.fn(async function* () {
yield* rawDocuments.keys();
});
const getRaw = vi.fn(async (id: string) => rawDocuments.get(id));
const localDatabase = {
allChunks,
localDatabase: {
info: vi.fn(async () => ({ doc_count: rawDocuments.size })),
bulkDocs: vi.fn(async (docs: Array<{ _id: string; _rev?: string; _deleted?: boolean }>) => {
deletedChunks.push(...docs);
return docs.map(({ _id }) => ({ ok: true, id: _id, rev: "2-deleted" }));
}),
},
findEntryNames,
getRaw,
};
const replicator = {
openOneShotReplication: vi.fn(
async (
_settings: typeof DEFAULT_SETTINGS,
_showResult: boolean,
_ignoreCleanLock: boolean,
mode: string
) => {
pushModes.push(mode);
return true;
}
),
getConnectedDeviceList: vi.fn(async () => ({
accepted_nodes: ["device-a"],
node_info: {
"device-a": {
progress: "10-local",
device_name: "Device A",
app_version: "1.12.7",
plugin_version: "1.0.0-beta.0",
},
},
})),
};
Object.assign(maintenance, {
core: {
settings: {
...DEFAULT_SETTINGS,
remoteType: REMOTE_COUCHDB,
},
replicator,
confirm: {
askSelectStringDialogue: vi.fn(async () => "Proceed Garbage Collection"),
},
},
localDatabase,
_notice: vi.fn(),
});
vi.spyOn(maintenance, "ensureAvailable").mockResolvedValue(true);
vi.spyOn(maintenance, "compactDatabase").mockResolvedValue(undefined);
vi.spyOn(maintenance, "clearHash").mockImplementation(() => undefined);
await maintenance.gcv3();
expect(allChunks).toHaveBeenCalledOnce();
expect(findEntryNames).not.toHaveBeenCalled();
expect(getRaw).not.toHaveBeenCalled();
expect(deletedChunks).toEqual([
{
_id: "h:obsolete",
_rev: "1-obsolete",
_deleted: true,
},
]);
expect(pushModes).toEqual(["sync", "pushOnly"]);
expect(maintenance.compactDatabase).toHaveBeenCalledOnce();
});
});
@@ -86,8 +86,11 @@ export class ModuleConflictResolver extends AbstractModule {
return MISSING_OR_ERROR;
}
if (rightLeaf == false) {
// Conflicted item could not load, delete this.
return await this.services.conflict.resolveByDeletingRevision(path, rightRev, "MISSING OLD REV");
// A locally unreadable conflict leaf may still be recoverable from another
// replica or backup. Keep it visible for explicit repair instead of treating
// missing chunks as evidence that the branch is obsolete.
this._log(`could not read conflicted revision ${rightRev}:${path}`, LOG_LEVEL_NOTICE);
return MISSING_OR_ERROR;
}
const isSame = leftLeaf.data == rightLeaf.data && leftLeaf.deleted == rightLeaf.deleted;
@@ -4,6 +4,7 @@ import {
DEFAULT_SETTINGS,
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
MISSING_OR_ERROR,
type FilePathWithPrefix,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
@@ -189,6 +190,28 @@ describe("ModuleConflictResolver independent same-path creation", () => {
});
describe("ModuleConflictResolver sensible merge hand-off", () => {
it("keeps an unreadable non-winner revision unresolved", async () => {
const path = "missing-conflict-body.md" as FilePathWithPrefix;
const { module, resolveByDeletingRevision, tryAutoMerge } = createModule();
tryAutoMerge.mockResolvedValue({
leftRev: "3-current",
rightRev: "2-unreadable",
leftLeaf: {
rev: "3-current",
data: "Readable current body\n",
ctime: 1,
mtime: 3,
deleted: false,
},
rightLeaf: false,
});
const result = await module.checkConflictAndPerformAutoMerge(path);
expect(result).toBe(MISSING_OR_ERROR);
expect(resolveByDeletingRevision).not.toHaveBeenCalled();
});
it("stores the merged body and removes the resolved conflict leaf", async () => {
const path = "sensible.md" as FilePathWithPrefix;
const { module, resolveByDeletingRevision, tryAutoMerge } = createModule();
+23 -10
View File
@@ -2,24 +2,29 @@ import type { LiveSyncCore } from "@/main";
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
import { fireAndForget } from "octagonal-wheels/promises";
import { AbstractModule } from "@/modules/AbstractModule";
import { $msg } from "@/common/translation";
import { copyFileDatabaseInfo } from "@/serviceFeatures/fileDatabaseInfo";
// Separated Module for basic menu commands, which are not related to obsidian specific features. It is expected to be used in other platforms with minimal changes.
// However, it is odd that it has here at all; it really ought to be in each respective feature. It will likely be moved eventually. Until now, addCommand pointed to Obsidian's version.
export class ModuleBasicMenu extends AbstractModule {
_everyOnloadStart(): Promise<boolean> {
this.addCommand({
id: "livesync-replicate",
name: "Replicate now",
name: $msg("Sync now"),
callback: async () => {
await this.services.replication.replicate();
},
});
this.addCommand({
id: "livesync-dump",
name: "Dump information of this doc ",
callback: () => {
name: $msg("Copy database information for the active file"),
checkCallback: (checking) => {
const file = this.services.vault.getActiveFilePath();
if (!file) return;
fireAndForget(() => this.localDatabase.getDBEntry(file, {}, true, false));
if (!file) return false;
if (!checking) {
fireAndForget(() => copyFileDatabaseInfo(this.core, file));
}
return true;
},
});
this.addCommand({
@@ -56,14 +61,18 @@ export class ModuleBasicMenu extends AbstractModule {
this.addCommand({
id: "livesync-scan-files",
name: "Scan storage and database again",
callback: async () => {
await this.services.vault.scanVault(true);
checkCallback: (checking) => {
if (!this.settings.useAdvancedMode) return false;
if (!checking) {
fireAndForget(() => this.services.vault.scanVault(true));
}
return true;
},
});
this.addCommand({
id: "livesync-runbatch",
name: "Run pended batch processes",
name: $msg("Apply pending changes now"),
callback: async () => {
await this.services.fileProcessing.commitPendingFileEvents();
},
@@ -73,8 +82,12 @@ export class ModuleBasicMenu extends AbstractModule {
this.addCommand({
id: "livesync-abortsync",
name: "Abort synchronization immediately",
callback: () => {
this.core.replicator.terminateSync();
checkCallback: (checking) => {
if (!this.settings.useAdvancedMode) return false;
if (!checking) {
this.core.replicator.terminateSync();
}
return true;
},
});
return Promise.resolve(true);
@@ -0,0 +1,170 @@
import { describe, expect, it, vi } from "vitest";
import type { Command } from "@/deps";
import { ModuleBasicMenu } from "./ModuleBasicMenu";
type RegisteredCommand = Command & {
checkCallback?: (checking: boolean) => boolean | void;
};
function createFixture() {
const commands: RegisteredCommand[] = [];
const settings = {
liveSync: false,
useAdvancedMode: false,
enableDebugTools: false,
};
const services = {
API: {
addLog: vi.fn(),
addCommand: vi.fn((command: RegisteredCommand) => {
commands.push(command);
return command;
}),
registerWindow: vi.fn(),
addRibbonIcon: vi.fn(),
registerProtocolHandler: vi.fn(),
},
replication: {
replicate: vi.fn(async () => undefined),
},
vault: {
getActiveFilePath: vi.fn((): string | null => "note.md"),
scanVault: vi.fn(async () => undefined),
},
control: {
applySettings: vi.fn(async () => undefined),
},
setting: {
saveSettingData: vi.fn(async () => undefined),
},
appLifecycle: {
isSuspended: vi.fn(() => false),
setSuspended: vi.fn(),
},
fileProcessing: {
commitPendingFileEvents: vi.fn(async () => true),
},
UI: {
promptCopyToClipboard: vi.fn(async (_title: string, _value: string) => true),
},
path: {
path2id: vi.fn(async () => "f:note"),
},
};
const core = {
settings,
_services: services,
services,
localDatabase: {
getDBEntry: vi.fn(async () => false),
localDatabase: {
get: vi.fn(async () => ({
_id: "f:note",
_rev: "2-current",
_conflicts: [],
path: "note.md",
ctime: 100,
mtime: 200,
size: 12,
type: "plain",
children: ["h:private-chunk-id"],
eden: {},
})),
},
getDBEntryMeta: vi.fn(async () => ({
_id: "f:note",
_rev: "2-current",
_conflicts: [],
path: "note.md",
ctime: 100,
mtime: 200,
size: 12,
type: "plain",
datatype: "plain",
data: "",
children: ["h:private-chunk-id"],
eden: {},
})),
allDocsRaw: vi.fn(async () => ({
rows: [{ id: "h:private-chunk-id", key: "h:private-chunk-id", value: { rev: "1-chunk" } }],
})),
},
storageAccess: {
isExistsIncludeHidden: vi.fn(async () => true),
statHidden: vi.fn(async () => ({ ctime: 100, mtime: 200, size: 12, type: "file" })),
},
replicator: {
terminateSync: vi.fn(),
},
};
const module = new ModuleBasicMenu(core as never);
return {
commands,
core,
module,
services,
settings,
getCommand(id: string) {
const command = commands.find((candidate) => candidate.id === id);
expect(command, `command ${id}`).toBeDefined();
return command!;
},
};
}
describe("ModuleBasicMenu command palette", () => {
it("uses clear user-facing names without changing the established command IDs", async () => {
const fixture = createFixture();
await fixture.module._everyOnloadStart();
expect(fixture.getCommand("livesync-replicate").name).toBe("Sync now");
expect(fixture.getCommand("livesync-runbatch").name).toBe("Apply pending changes now");
});
it("keeps maintenance commands out of the normal palette", async () => {
const fixture = createFixture();
await fixture.module._everyOnloadStart();
expect(fixture.getCommand("livesync-scan-files").checkCallback?.(true)).toBe(false);
expect(fixture.getCommand("livesync-abortsync").checkCallback?.(true)).toBe(false);
fixture.settings.useAdvancedMode = true;
expect(fixture.getCommand("livesync-scan-files").checkCallback?.(true)).toBe(true);
expect(fixture.getCommand("livesync-abortsync").checkCallback?.(true)).toBe(true);
});
it("keeps active-file database information available and opens it in a copy dialogue", async () => {
const fixture = createFixture();
await fixture.module._everyOnloadStart();
const command = fixture.getCommand("livesync-dump");
expect(command.name).toBe("Copy database information for the active file");
expect(command.checkCallback?.(true)).toBe(true);
command.checkCallback?.(false);
await vi.waitFor(() => {
expect(fixture.services.UI.promptCopyToClipboard).toHaveBeenCalledOnce();
});
const [title, report] = fixture.services.UI.promptCopyToClipboard.mock.calls[0];
expect(title).toBe("Database information for note.md");
expect(report).toContain("note.md");
expect(report).toContain("2-current");
expect(report).toContain("h:private-chunk-id");
expect(report).toContain("1-chunk");
expect(fixture.core.localDatabase.getDBEntry).not.toHaveBeenCalled();
});
it("hides the active-file database report when no file is active", async () => {
const fixture = createFixture();
fixture.services.vault.getActiveFilePath.mockReturnValue(null);
await fixture.module._everyOnloadStart();
expect(fixture.getCommand("livesync-dump").checkCallback?.(true)).toBe(false);
});
});
+341 -158
View File
@@ -3,14 +3,13 @@ import {
type DocumentID,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type LoadedEntry,
type MetaEntry,
type FilePath,
type EntryDoc,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { createBlob, getFileRegExp, isDocContentSame, readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { Logger } from "@vrtmrz/livesync-commonlib/compat/common/logger";
import { addPrefix, shouldBeIgnored, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { shouldBeIgnored } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { $msg } from "@/common/translation";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
@@ -21,12 +20,24 @@ import {
EVENT_REQUEST_RUN_FIX_INCOMPLETE,
eventHub,
} from "@/common/events.ts";
import { ICHeader, ICXHeader, PSCHeader } from "@/common/types.ts";
import { ICHeader } from "@/common/types.ts";
import { HiddenFileSync } from "@/features/HiddenFileSync/CmdHiddenFileSync.ts";
import { EVENT_REQUEST_SHOW_HISTORY } from "@/common/obsidianEvents.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { isNotFoundError } from "@vrtmrz/livesync-commonlib/compat/common/utils.doc";
import {
chooseAndCopyFileDatabaseInfo,
collectFileDatabaseInfoPaths,
copyFileDatabaseInfo,
retryReadFileDatabaseRevision,
} from "@/serviceFeatures/fileDatabaseInfo.ts";
import {
discardUnreadableLiveRevision,
inspectFileRepair,
type FileRepairInspection,
type FileRepairRevision,
} from "@/serviceFeatures/fileRepair.ts";
export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
// const hatchWarn = this.createEl(paneEl, "div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
// hatchWarn.addClass("op-warn-info");
@@ -67,6 +78,18 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
await this.app.commands.executeCommandById("obsidian-livesync:dump-debug-info");
})
);
new Setting(paneEl)
.setName($msg("Copy database information for a file"))
.setDesc(
$msg(
"Copy revision, conflict, and local chunk availability information, including document and chunk identifiers but not file contents."
)
)
.addButton((button) =>
button.setButtonText($msg("Choose file")).onClick(async () => {
await chooseAndCopyFileDatabaseInfo(this.core);
})
);
new Setting(paneEl)
.setName($msg("Analyse database usage"))
.setDesc(
@@ -99,132 +122,300 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
});
void addPanel(paneEl, "Recovery and Repair").then((paneEl) => {
const addResult = async (path: string, file: FilePathWithPrefix | false, fileOnDB: LoadedEntry | false) => {
const storageFileStat = file ? await this.core.storageAccess.statHidden(file) : null;
resultArea.appendChild(
this.createEl(resultArea, "div", {}, (el) => {
el.appendChild(this.createEl(el, "h6", { text: path }));
el.appendChild(
this.createEl(el, "div", {}, (infoGroupEl) => {
infoGroupEl.appendChild(
this.createEl(infoGroupEl, "div", {
text: `Storage : Modified: ${!storageFileStat ? `Missing:` : `${new Date(storageFileStat.mtime).toLocaleString()}, Size:${storageFileStat.size}`}`,
})
);
infoGroupEl.appendChild(
this.createEl(infoGroupEl, "div", {
text: `Database: Modified: ${!fileOnDB ? `Missing:` : `${new Date(fileOnDB.mtime).toLocaleString()}, Size:${fileOnDB.size} (actual size:${readAsBlob(fileOnDB).size})`}`,
})
);
})
const resultArea = paneEl.createDiv({ text: "", cls: "sls-repair-results" });
const addActionButton = (
parent: HTMLElement,
text: string,
action: (button: HTMLButtonElement) => Promise<void> | void,
warning = false
) => {
this.createEl(parent, "button", { text }, (button) => {
if (warning) {
button.addClass("mod-warning");
}
button.onClickEvent(async () => {
button.disabled = true;
try {
await action(button);
} finally {
if (button.isConnected) {
button.disabled = false;
}
}
});
});
};
const storeStorageInDatabase = async (path: string): Promise<boolean> => {
if (path.startsWith(".")) {
const addOn = this.core.getAddOn<HiddenFileSync>(HiddenFileSync.name);
if (!addOn) {
return false;
}
const file = (await addOn.scanInternalFiles()).find((entry) => entry.path === path);
if (!file) {
Logger(`Failed to find the file in the internal files: ${path}`, LOG_LEVEL_NOTICE);
return false;
}
return Boolean(await addOn.storeInternalFileToDatabase(file, true));
}
return Boolean(await this.core.fileHandler.storeFileToDB(path as FilePath, true));
};
const applyWinnerToStorage = async (
path: string,
revision: FileRepairRevision
): Promise<boolean> => {
if (revision.loadedEntry === false) {
return false;
}
if (revision.loadedEntry.path.startsWith(ICHeader)) {
const addOn = this.core.getAddOn<HiddenFileSync>(HiddenFileSync.name);
return addOn
? Boolean(await addOn.extractInternalFileFromDatabase(path as FilePath, true))
: false;
}
return Boolean(await this.core.fileHandler.dbToStorage(revision.loadedEntry as MetaEntry, null, true));
};
const addRepairResult = (inspection: FileRepairInspection) => {
const { information, revisions } = inspection;
const path = information.path;
const card = this.createEl(resultArea, "div", { cls: "sls-repair-result" });
const refresh = async () => {
card.remove();
const refreshed = await inspectFileRepair(this.core, path);
if (refreshed.requiresAttention) {
addRepairResult(refreshed);
} else {
Logger(`Verification no longer reports a problem for ${path}`, LOG_LEVEL_NOTICE);
}
};
this.createEl(card, "h6", { text: path });
if (information.storage.exists) {
this.createEl(card, "div", {
text: $msg("Vault file: modified ${TIME}, size ${SIZE}", {
TIME: new Date(information.storage.mtime ?? 0).toLocaleString(),
SIZE: `${information.storage.size ?? 0}`,
}),
});
} else {
this.createEl(card, "div", { text: $msg("Vault file: missing") });
}
if (!information.database.exists) {
this.createEl(card, "div", { text: $msg("Local database document: missing") });
}
const addRevision = (revision: FileRepairRevision) => {
const { metadata } = revision;
const revisionEl = this.createEl(card, "div", { cls: "sls-repair-revision" });
this.createEl(revisionEl, "div", {
text: $msg("${ROLE}: ${REVISION}", {
ROLE: revision.role === "winner" ? $msg("Winner revision") : $msg("Conflict revision"),
REVISION: metadata.revision ?? $msg("Unknown revision"),
}),
cls: "sls-repair-revision-title",
});
if (metadata.deleted) {
this.createEl(revisionEl, "div", { text: $msg("Logical deletion") });
} else if (revision.contentReadable) {
this.createEl(revisionEl, "div", {
text: $msg("Readable on this device; recorded size ${RECORDED}, decoded size ${ACTUAL}", {
RECORDED: `${metadata.recordedSize}`,
ACTUAL: `${revision.loadedEntry === false ? 0 : readAsBlob(revision.loadedEntry).size}`,
}),
});
} else {
const missing = metadata.chunks.filter(
({ embedded, localDatabaseState }) =>
!embedded && localDatabaseState !== "available"
);
if (fileOnDB && file) {
el.appendChild(
this.createEl(el, "button", { text: "Show history" }, (buttonEl) => {
buttonEl.onClickEvent(() => {
eventHub.emitEvent(EVENT_REQUEST_SHOW_HISTORY, {
file: file,
fileOnDB: fileOnDB,
});
});
})
);
this.createEl(revisionEl, "div", {
text: $msg("Unreadable on this device; ${COUNT} referenced chunks are missing or deleted", {
COUNT: `${missing.length}`,
}),
cls: "mod-warning",
});
if (missing.length > 0) {
this.createEl(revisionEl, "code", {
text: missing
.slice(0, 3)
.map(({ id }) => id)
.join(", ") + (missing.length > 3 ? ", …" : ""),
});
}
if (file) {
el.appendChild(
this.createEl(el, "button", { text: "Storage -> Database" }, (buttonEl) => {
buttonEl.onClickEvent(async () => {
if (file.startsWith(".")) {
const addOn = this.core.getAddOn<HiddenFileSync>(HiddenFileSync.name);
if (addOn) {
const file = (await addOn.scanInternalFiles()).find((e) => e.path == path);
if (!file) {
Logger(
`Failed to find the file in the internal files: ${path}`,
LOG_LEVEL_NOTICE
);
return;
}
if (!(await addOn.storeInternalFileToDatabase(file, true))) {
Logger(
`Failed to store the file to the database (Hidden file): ${file.path}`,
LOG_LEVEL_NOTICE
);
return;
}
}
} else {
if (!(await this.core.fileHandler.storeFileToDB(file, true))) {
Logger(
`Failed to store the file to the database: ${file}`,
LOG_LEVEL_NOTICE
);
return;
}
if (revision.contentMatchesStorage === true) {
this.createEl(revisionEl, "div", { text: $msg("Matches the current Vault file") });
} else if (revision.contentMatchesStorage === false) {
this.createEl(revisionEl, "div", { text: $msg("Differs from the current Vault file") });
}
if (!metadata.deleted && !revision.contentReadable && metadata.revision) {
const actions = this.createEl(revisionEl, "div", { cls: "sls-repair-actions" });
addActionButton(actions, $msg("Retry reading revision"), async () => {
const loaded = await retryReadFileDatabaseRevision(this.core, path, metadata.revision!);
Logger(
loaded
? `Revision ${metadata.revision} of ${path} is readable after retry`
: `Revision ${metadata.revision} of ${path} remains unreadable`,
LOG_LEVEL_NOTICE
);
await refresh();
});
addActionButton(
actions,
$msg("Discard unreadable revision"),
async () => {
const confirmed =
(await this.core.confirm.askYesNoDialog(
$msg(
"Discard database revision ${REVISION} of ${FILE}? This creates a logical deletion for that exact live revision. Missing content cannot be recovered by this action.",
{
REVISION: metadata.revision!,
FILE: path,
}
),
{
title: $msg("Discard unreadable revision"),
defaultOption: "No",
}
el.remove();
});
})
);
}
if (fileOnDB) {
el.appendChild(
this.createEl(el, "button", { text: "Database -> Storage" }, (buttonEl) => {
buttonEl.onClickEvent(async () => {
if (fileOnDB.path.startsWith(ICHeader)) {
const addOn = this.core.getAddOn<HiddenFileSync>(HiddenFileSync.name);
if (addOn) {
if (
!(await addOn.extractInternalFileFromDatabase(path as FilePath, true))
) {
Logger(
`Failed to store the file to the database (Hidden file): ${file}`,
LOG_LEVEL_NOTICE
);
return;
}
}
} else {
if (
!(await this.core.fileHandler.dbToStorage(
fileOnDB as MetaEntry,
null,
true
))
) {
Logger(
`Failed to store the file to the storage: ${fileOnDB.path}`,
LOG_LEVEL_NOTICE
);
return;
}
)) === "yes";
if (!confirmed) {
return;
}
const result = await discardUnreadableLiveRevision(
this.core,
path,
metadata.revision!
);
Logger(
`Discard unreadable revision ${metadata.revision} of ${path}: ${result}`,
result === "discarded" ? LOG_LEVEL_NOTICE : LOG_LEVEL_VERBOSE
);
await refresh();
},
true
);
}
};
revisions.forEach(addRevision);
for (const revision of information.database.unavailableConflictRevisions) {
const revisionEl = this.createEl(card, "div", { cls: "sls-repair-revision" });
this.createEl(revisionEl, "div", {
text: $msg("${ROLE}: ${REVISION}", {
ROLE: $msg("Conflict revision"),
REVISION: revision,
}),
cls: "sls-repair-revision-title",
});
this.createEl(revisionEl, "div", {
text: $msg("Revision metadata is unavailable on this device"),
cls: "mod-warning",
});
const actions = this.createEl(revisionEl, "div", { cls: "sls-repair-actions" });
addActionButton(actions, $msg("Retry reading revision"), async () => {
await retryReadFileDatabaseRevision(this.core, path, revision);
await refresh();
});
addActionButton(
actions,
$msg("Discard unreadable revision"),
async () => {
const confirmed =
(await this.core.confirm.askYesNoDialog(
$msg(
"Discard database revision ${REVISION} of ${FILE}? This creates a logical deletion for that exact live revision. Missing content cannot be recovered by this action.",
{
REVISION: revision,
FILE: path,
}
el.remove();
});
})
);
),
{
title: $msg("Discard unreadable revision"),
defaultOption: "No",
}
)) === "yes";
if (!confirmed) {
return;
}
await discardUnreadableLiveRevision(this.core, path, revision);
await refresh();
},
true
);
}
for (const base of information.database.mergeBases) {
if (base.contentAvailableLocally) {
continue;
}
this.createEl(card, "div", {
text: base.revision
? $msg(
"Shared ancestor ${REVISION} is not readable on this device. Automatic three-way merging may be unavailable, but the live revisions remain available for explicit review.",
{
REVISION: base.revision,
}
)
: $msg(
"No shared ancestor is available for this conflict. The live revisions remain available for explicit review."
),
cls: "sls-repair-ancestor-warning",
});
}
const winner = revisions.find(({ role }) => role === "winner");
const actions = this.createEl(card, "div", { cls: "sls-repair-actions" });
if (winner?.loadedEntry && information.storage.exists) {
const winnerEntry = winner.loadedEntry;
addActionButton(actions, $msg("Show revision history"), () => {
eventHub.emitEvent(EVENT_REQUEST_SHOW_HISTORY, {
file: path as FilePathWithPrefix,
fileOnDB: winnerEntry,
});
});
}
if (
information.storage.exists &&
information.database.conflictCount === 0 &&
(!winner || winner.contentReadable)
) {
addActionButton(actions, $msg("Use Vault file in local database"), async () => {
if (!(await storeStorageInDatabase(path))) {
Logger(`Failed to store the Vault file in the local database: ${path}`, LOG_LEVEL_NOTICE);
return;
}
return el;
})
);
await refresh();
});
}
if (
!information.storage.exists &&
information.database.conflictCount === 0 &&
winner?.loadedEntry
) {
addActionButton(actions, $msg("Restore database winner to Vault"), async () => {
if (!(await applyWinnerToStorage(path, winner))) {
Logger(`Failed to restore the database winner to the Vault: ${path}`, LOG_LEVEL_NOTICE);
return;
}
await refresh();
});
}
addActionButton(actions, $msg("Copy database information"), async () => {
await copyFileDatabaseInfo(this.core, path);
});
};
const checkBetweenStorageAndDatabase = async (file: FilePathWithPrefix, fileOnDB: LoadedEntry) => {
const dataContent = readAsBlob(fileOnDB);
const content = createBlob(await this.core.storageAccess.readHiddenFileBinary(file));
if (await isDocContentSame(content, dataContent)) {
Logger(`Compare: SAME: ${file}`);
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file}`, LOG_LEVEL_NOTICE);
void addResult(file, file, fileOnDB);
}
};
new Setting(paneEl)
.setName("Recreate missing chunks for all files")
.setDesc("This will recreate chunks for all files. If there were missing chunks, this may fix the errors.")
.setName($msg("Recreate chunks for current Vault files"))
.setDesc(
$msg(
"Recreate chunks from the files currently present in this Vault. This cannot reconstruct unavailable historical or conflict content."
)
)
.addButton((button) =>
button
.setButtonText("Recreate all")
.setButtonText($msg("Recreate current chunks"))
.setCta()
.onClick(async () => {
await this.core.fileHandler.createAllChunks(true);
@@ -240,40 +431,40 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
.setButtonText("Resolve All")
.setCta()
.onClick(async () => {
const confirmed =
(await this.core.confirm.askYesNoDialog(
$msg(
"Resolve every conflict by modification time? This logically deletes every version except the newest one and cannot recover content which is already unavailable."
),
{
title: $msg("Resolve all conflicts by the newest version"),
defaultOption: "No",
}
)) === "yes";
if (!confirmed) {
return;
}
await this.services.conflict.resolveAllConflictedFilesByNewerOnes();
})
);
new Setting(paneEl)
.setName("Verify and repair all files")
.setName($msg("Verify and repair all files"))
.setDesc(
"Compare the content of files between on local database and storage. If not matched, you will be asked which one you want to keep."
$msg(
"Compare each Vault file with every live local-database revision. Unreadable conflict versions remain visible until you retry or explicitly discard an exact revision."
)
)
.addButton((button) =>
button
.setButtonText("Verify all")
.setButtonText($msg("Verify all"))
.setDisabled(false)
.setCta()
.onClick(async () => {
resultArea.replaceChildren();
Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify");
const ignorePatterns = getFileRegExp(this.core.settings, "syncInternalFilesIgnorePatterns");
const targetPatterns = getFileRegExp(this.core.settings, "syncInternalFilesTargetPatterns");
this.core.localDatabase.clearCaches();
Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify");
const files = this.core.settings.syncInternalFiles
? await this.core.storageAccess.getFilesIncludeHidden("/", targetPatterns, ignorePatterns)
: await this.core.storageAccess.getFileNames();
const documents = [] as FilePath[];
const adn = this.core.localDatabase.findAllDocs();
for await (const i of adn) {
const path = this.services.path.getPath(i);
if (path.startsWith(ICXHeader)) continue;
if (path.startsWith(PSCHeader)) continue;
if (!this.core.settings.syncInternalFiles && path.startsWith(ICHeader)) continue;
documents.push(stripAllPrefixes(path));
}
const allPaths = [...new Set([...documents, ...files])];
const allPaths = await collectFileDatabaseInfoPaths(this.core);
let i = 0;
const incProc = () => {
i++;
@@ -295,28 +486,21 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
: false;
const fileOnStorage = stat != null ? stat : false;
if (!(await this.services.vault.isTargetFile(path))) return incProc();
const releaser = await semaphore.acquire(1);
if (fileOnStorage && this.services.vault.isFileSizeTooLarge(fileOnStorage.size))
return incProc();
const releaser = await semaphore.acquire(1);
try {
const isHiddenFile = path.startsWith(".");
const dbPath = isHiddenFile ? addPrefix(path, ICHeader) : path;
const fileOnDB = await this.core.localDatabase.getDBEntry(dbPath);
if (fileOnDB && this.services.vault.isFileSizeTooLarge(fileOnDB.size))
const inspection = await inspectFileRepair(this.core, path);
const winner = inspection.revisions.find(({ role }) => role === "winner");
if (
winner &&
this.services.vault.isFileSizeTooLarge(winner.metadata.recordedSize)
)
return incProc();
if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
void addResult(path, path, false);
return incProc();
}
if (fileOnDB && !fileOnStorage) {
Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
void addResult(path, false, fileOnDB);
return incProc();
}
if (fileOnStorage && fileOnDB) {
await checkBetweenStorageAndDatabase(path, fileOnDB);
if (inspection.requiresAttention) {
addRepairResult(inspection);
} else {
Logger(`Compare: SAME: ${path}`);
}
} catch (ex) {
Logger(`Error while processing ${path}`, LOG_LEVEL_NOTICE);
@@ -335,7 +519,6 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
// Logger(`${i}/${files.length}\n`, LOG_LEVEL_NOTICE, "verify-processed");
})
);
const resultArea = paneEl.createDiv({ text: "" });
new Setting(paneEl)
.setName("Check and convert non-path-obfuscated files")
.setDesc("")
@@ -187,7 +187,7 @@ export function paneMaintenance(
)
.addOnUpdate(this.onlyOnMinIO);
});
void addPanel(paneEl, "Garbage Collection V3 (Beta)", (e) => e, this.onlyOnP2POrCouchDB).then((paneEl) => {
void addPanel(paneEl, "Garbage Collection V3 (Beta)", (e) => e, this.onlyOnCouchDB).then((paneEl) => {
new Setting(paneEl)
.setName("Perform Garbage Collection")
.setDesc("Perform Garbage Collection to remove unused chunks and reduce database size.")
@@ -133,8 +133,8 @@ export function paneRemoteConfig(
}
{
// TODO: very WIP. need to refactor the UI.
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleRemoteServer"), () => {}).then((paneEl) => {
const actions = new Setting(paneEl).setName("Remote Databases");
void addPanel(paneEl, $msg("Connection settings"), () => {}).then((paneEl) => {
const actions = new Setting(paneEl).setName($msg("Saved connections"));
// actions.addButton((button) =>
// button
// .setButtonText("Change Remote and Setup")
@@ -12,7 +12,7 @@ import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts
import type { PageFunctions } from "./SettingPane.ts";
import { visibleOnly } from "./SettingPane.ts";
import { request } from "@/deps.ts";
import { SetupManager, UserMode } from "@/modules/features/SetupManager.ts";
import { SetupManager } from "@/modules/features/SetupManager.ts";
import { LiveSyncError } from "@vrtmrz/livesync-commonlib/compat/common/LSError";
import {
createCoreSettingsAfterFullReset,
@@ -40,8 +40,7 @@ export function paneSetup(
.addButton((text) => {
text.setButtonText($msg("Rerun Wizard")).onClick(async () => {
const setupManager = this.core.getModule(SetupManager);
await setupManager.onOnboard(UserMode.ExistingUser);
// await this.plugin.moduleSetupObsidian.onBoardingWizard(true);
await setupManager.startOnBoarding();
});
});
@@ -68,6 +68,8 @@
display: flex;
flex-direction: column;
padding-bottom: var(--keyboard-height, 0px);
user-select: text;
-webkit-user-select: text;
}
.dialog-host :global(button) {
+454
View File
@@ -0,0 +1,454 @@
import { $msg } from "@/common/translation";
import type {
FilePath,
FilePathWithPrefix,
LoadedEntry,
ObsidianLiveSyncSettings,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { getFileRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { isNotFoundError } from "@vrtmrz/livesync-commonlib/compat/common/utils.doc";
import { ICHeader, ICXHeader, PSCHeader } from "@vrtmrz/livesync-commonlib/compat/common/models/fileaccess.const";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService, IUIService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
type DatabaseMeta = LoadedEntry & {
_rawStorageType: string | null;
_legacyBodyPresent: boolean;
_revs_info?: Array<{
rev: string;
status: string;
}>;
};
export type FileDatabaseInfoCore = {
localDatabase: Pick<
LiveSyncLocalDB,
"allDocsRaw" | "findAllDocs" | "getDBEntryFromMeta" | "getDBEntry" | "localDatabase"
>;
services: {
path: Pick<IPathService, "path2id">;
UI: IUIService;
};
settings: ObsidianLiveSyncSettings;
storageAccess: Pick<
StorageAccess,
"getFileNames" | "getFilesIncludeHidden" | "isExistsIncludeHidden" | "statHidden"
>;
};
export type RevisionDatabaseInfo = {
documentId: string;
revision: string | null;
current: boolean;
deleted: boolean;
storageType: string;
storageLayout: "chunked" | "legacy-inline";
ctime: number;
mtime: number;
recordedSize: number;
revisionHistory: Array<{
revision: string;
status: string;
}>;
chunkReferences: number;
uniqueChunkReferences: number;
embeddedChunkReferences: number;
locallyStoredChunkReferences: number;
contentAvailableLocally: boolean;
chunks: Array<{
id: string;
referenceCount: number;
embedded: boolean;
storedInLocalDatabase: boolean;
localDatabaseState: "available" | "deleted" | "missing";
localDatabaseRevision: string | null;
}>;
};
export type FileDatabaseMergeBaseInfo = {
winnerRevision: string;
conflictRevision: string;
revision: string | null;
metadataAvailableLocally: boolean;
contentAvailableLocally: boolean;
missingChunkIds: string[];
unavailableSharedRevisions: string[];
};
export type FileDatabaseInfo = {
path: string;
databasePath: FilePathWithPrefix | FilePath;
storage: {
exists: boolean;
ctime?: number;
mtime?: number;
size?: number;
};
database: {
source: "local database on this device";
remoteQueried: false;
exists: boolean;
currentRevision: string | null;
conflictCount: number;
conflictRevisions: string[];
unavailableConflictRevisions: string[];
revisions: RevisionDatabaseInfo[];
mergeBases: FileDatabaseMergeBaseInfo[];
};
};
const REPORT_WARNING =
"All revisions and chunk availability below are a snapshot of this device's local database; the remote is not queried. Review the Vault-relative path, document identifier, content-derived chunk identifiers, and metadata before sharing this report. File contents are omitted.";
function toDatabasePath(path: string): FilePathWithPrefix | FilePath {
if (path.startsWith(".")) {
return addPrefix(path as FilePath, ICHeader);
}
return path as FilePath;
}
type RawDatabaseDocument = {
_id: string;
_rev?: string;
_conflicts?: string[];
_deleted?: boolean;
_revs_info?: Array<{
rev: string;
status: string;
}>;
children?: string[];
ctime?: number;
deleted?: boolean;
data?: string | string[];
eden?: Record<string, unknown>;
mtime?: number;
size?: number;
type?: string;
};
async function getLocalDatabaseMeta(
core: FileDatabaseInfoCore,
path: FilePathWithPrefix | FilePath,
options: PouchDB.Core.GetOptions
): Promise<DatabaseMeta | false> {
const documentId = await core.services.path.path2id(path);
let raw: RawDatabaseDocument;
try {
raw = await core.localDatabase.localDatabase.get(documentId, options);
} catch (error) {
if (isNotFoundError(error)) {
return false;
}
throw error;
}
if (raw.type === "leaf") {
return false;
}
if (raw.type && raw.type !== "notes" && raw.type !== "newnote" && raw.type !== "plain") {
return false;
}
const rawStorageType = raw.type ?? null;
const legacy = rawStorageType === null || rawStorageType === "notes";
const type = legacy ? "notes" : rawStorageType;
const legacyBodyPresent =
legacy && (typeof raw.data === "string" || (Array.isArray(raw.data) && raw.data.every((item) => typeof item === "string")));
return {
_id: raw._id,
_rev: raw._rev,
_conflicts: raw._conflicts,
_revs_info: raw._revs_info,
path,
data: legacyBodyPresent ? raw.data : "",
ctime: raw.ctime ?? 0,
mtime: raw.mtime ?? 0,
size: raw.size ?? 0,
children: type === "newnote" || type === "plain" ? (raw.children ?? []) : [],
datatype: type === "newnote" ? "newnote" : "plain",
deleted: raw.deleted ?? raw._deleted,
type,
eden: raw.eden ?? {},
_rawStorageType: rawStorageType,
_legacyBodyPresent: legacyBodyPresent,
} as DatabaseMeta;
}
async function collectRevisionDatabaseInfo(
core: FileDatabaseInfoCore,
meta: DatabaseMeta,
current: boolean
): Promise<RevisionDatabaseInfo> {
const legacy = meta._rawStorageType === null || meta._rawStorageType === "notes";
const children = legacy ? [] : "children" in meta ? meta.children : [];
const uniqueChildren = [...new Set(children)];
const referenceCounts = new Map<string, number>();
for (const child of children) {
referenceCounts.set(child, (referenceCounts.get(child) ?? 0) + 1);
}
const embeddedChildren = new Set(
Object.keys("eden" in meta && meta.eden ? meta.eden : {}).filter((id) => uniqueChildren.includes(id))
);
const localRows =
uniqueChildren.length === 0
? []
: (
await core.localDatabase.allDocsRaw({
keys: uniqueChildren,
include_docs: false,
})
).rows;
const localChunkStates = new Map(
localRows
.filter((row) => "value" in row)
.map(
(row) =>
[
row.key,
{
state: row.value.deleted ? ("deleted" as const) : ("available" as const),
revision: row.value.rev,
},
] as const
)
);
return {
documentId: meta._id,
revision: meta._rev ?? null,
current,
deleted: Boolean(meta.deleted ?? meta._deleted),
storageType: meta._rawStorageType ?? "absent",
storageLayout: legacy ? "legacy-inline" : "chunked",
ctime: meta.ctime,
mtime: meta.mtime,
recordedSize: meta.size,
revisionHistory: (meta._revs_info ?? []).map(({ rev, status }) => ({
revision: rev,
status,
})),
chunkReferences: children.length,
uniqueChunkReferences: uniqueChildren.length,
embeddedChunkReferences: children.filter((id) => embeddedChildren.has(id)).length,
locallyStoredChunkReferences: children.filter((id) => localChunkStates.get(id)?.state === "available").length,
contentAvailableLocally: legacy
? meta._legacyBodyPresent
: uniqueChildren.every(
(id) => embeddedChildren.has(id) || localChunkStates.get(id)?.state === "available"
),
chunks: uniqueChildren.map((id) => {
const localState = localChunkStates.get(id);
return {
id,
referenceCount: referenceCounts.get(id) ?? 0,
embedded: embeddedChildren.has(id),
storedInLocalDatabase: localState?.state === "available",
localDatabaseState: localState?.state ?? "missing",
localDatabaseRevision: localState?.revision ?? null,
};
}),
};
}
function revisionHistory(meta: DatabaseMeta): Array<{ revision: string; status: string }> {
const history = (meta._revs_info ?? []).map(({ rev, status }) => ({
revision: rev,
status,
}));
if (meta._rev && !history.some(({ revision }) => revision === meta._rev)) {
history.unshift({
revision: meta._rev,
status: "available",
});
}
return history;
}
function missingChunkIds(info: RevisionDatabaseInfo): string[] {
return info.chunks
.filter(({ embedded, localDatabaseState }) => !embedded && localDatabaseState !== "available")
.map(({ id }) => id);
}
export async function inspectFileDatabaseInfo(core: FileDatabaseInfoCore, path: string): Promise<FileDatabaseInfo> {
const storageExists = await core.storageAccess.isExistsIncludeHidden(path);
const storageStat = storageExists ? await core.storageAccess.statHidden(path) : null;
const databasePath = toDatabasePath(path);
const currentMeta = await getLocalDatabaseMeta(core, databasePath, {
conflicts: true,
revs: true,
revs_info: true,
});
const revisions: RevisionDatabaseInfo[] = [];
const conflictRevisions = currentMeta === false ? [] : (currentMeta._conflicts ?? []);
const unavailableConflictRevisions: string[] = [];
const mergeBases: FileDatabaseMergeBaseInfo[] = [];
const metadataByRevision = new Map<string, DatabaseMeta | false>();
if (currentMeta !== false && currentMeta._rev) {
metadataByRevision.set(currentMeta._rev, currentMeta);
}
const getRevisionMeta = async (revision: string): Promise<DatabaseMeta | false> => {
const cached = metadataByRevision.get(revision);
if (cached !== undefined) {
return cached;
}
const meta = await getLocalDatabaseMeta(core, databasePath, {
rev: revision,
revs: true,
revs_info: true,
});
metadataByRevision.set(revision, meta);
return meta;
};
if (currentMeta) {
revisions.push(await collectRevisionDatabaseInfo(core, currentMeta, true));
for (const revision of conflictRevisions) {
const conflictMeta = await getRevisionMeta(revision);
if (conflictMeta) {
revisions.push(await collectRevisionDatabaseInfo(core, conflictMeta, false));
const winnerHistory = revisionHistory(currentMeta);
const conflictHistory = revisionHistory(conflictMeta);
const conflictHistoryByRevision = new Map(
conflictHistory.map(({ revision: historyRevision, status }) => [historyRevision, status])
);
const sharedHistory = winnerHistory.filter(({ revision: historyRevision }) =>
conflictHistoryByRevision.has(historyRevision)
);
const sharedRevision = sharedHistory[0]?.revision ?? null;
const unavailableSharedRevisions = sharedHistory
.filter(
({ revision: historyRevision, status }) =>
status !== "available" ||
conflictHistoryByRevision.get(historyRevision) !== "available"
)
.map(({ revision: historyRevision }) => historyRevision);
const sharedMeta = sharedRevision ? await getRevisionMeta(sharedRevision) : false;
const sharedInfo = sharedMeta
? await collectRevisionDatabaseInfo(core, sharedMeta, false)
: undefined;
mergeBases.push({
winnerRevision: currentMeta._rev ?? "",
conflictRevision: revision,
revision: sharedRevision,
metadataAvailableLocally: Boolean(sharedMeta),
contentAvailableLocally: sharedInfo?.contentAvailableLocally ?? false,
missingChunkIds: sharedInfo ? missingChunkIds(sharedInfo) : [],
unavailableSharedRevisions,
});
} else {
unavailableConflictRevisions.push(revision);
}
}
}
const report: FileDatabaseInfo = {
path,
databasePath,
storage: storageStat
? {
exists: true,
ctime: storageStat.ctime,
mtime: storageStat.mtime,
size: storageStat.size,
}
: {
exists: false,
},
database: {
source: "local database on this device",
remoteQueried: false,
exists: currentMeta !== false,
currentRevision: currentMeta ? (currentMeta._rev ?? null) : null,
conflictCount: conflictRevisions.length,
conflictRevisions,
unavailableConflictRevisions,
revisions,
mergeBases,
},
};
return report;
}
export async function readFileDatabaseRevisionLocally(
core: FileDatabaseInfoCore,
path: string,
revision: string
): Promise<LoadedEntry | false> {
const databasePath = toDatabasePath(path);
const meta = await getLocalDatabaseMeta(core, databasePath, {
rev: revision,
revs: true,
revs_info: true,
});
if (!meta) {
return false;
}
const info = await collectRevisionDatabaseInfo(core, meta, false);
if (info.deleted || !info.contentAvailableLocally) {
return false;
}
return await core.localDatabase.getDBEntryFromMeta(meta, false, false);
}
export async function retryReadFileDatabaseRevision(
core: FileDatabaseInfoCore,
path: string,
revision: string
): Promise<LoadedEntry | false> {
return await core.localDatabase.getDBEntry(toDatabasePath(path), { rev: revision }, false, true, true);
}
export async function buildFileDatabaseInfoReport(core: FileDatabaseInfoCore, path: string): Promise<string> {
const report = await inspectFileDatabaseInfo(core, path);
return `${$msg(REPORT_WARNING)}
\`\`\`json
${JSON.stringify(report, null, 2)}
\`\`\``;
}
export async function copyFileDatabaseInfo(core: FileDatabaseInfoCore, path: string): Promise<boolean> {
const report = await buildFileDatabaseInfoReport(core, path);
return await core.services.UI.promptCopyToClipboard(
$msg("Database information for ${FILE}", { FILE: path }),
report
);
}
export async function collectFileDatabaseInfoPaths(core: FileDatabaseInfoCore): Promise<string[]> {
const ignorePatterns = getFileRegExp(core.settings, "syncInternalFilesIgnorePatterns");
const targetPatterns = getFileRegExp(core.settings, "syncInternalFilesTargetPatterns");
const storagePaths = core.settings.syncInternalFiles
? await core.storageAccess.getFilesIncludeHidden("/", targetPatterns, ignorePatterns)
: await core.storageAccess.getFileNames();
const databasePaths: string[] = [];
for await (const entry of core.localDatabase.findAllDocs()) {
const prefixedPath = entry.path;
if (prefixedPath.startsWith(ICXHeader) || prefixedPath.startsWith(PSCHeader)) {
continue;
}
if (!core.settings.syncInternalFiles && prefixedPath.startsWith(ICHeader)) {
continue;
}
databasePaths.push(stripAllPrefixes(prefixedPath));
}
return [...new Set([...storagePaths, ...databasePaths])].sort((left, right) =>
left < right ? -1 : left > right ? 1 : 0
);
}
export async function chooseAndCopyFileDatabaseInfo(core: FileDatabaseInfoCore): Promise<boolean> {
const paths = await collectFileDatabaseInfoPaths(core);
const selected = await core.services.UI.confirm.askSelectString($msg("Choose a file to inspect"), paths);
if (!selected) {
return false;
}
return await copyFileDatabaseInfo(core, selected);
}
@@ -0,0 +1,413 @@
import { describe, expect, it, vi } from "vitest";
import {
buildFileDatabaseInfoReport,
chooseAndCopyFileDatabaseInfo,
collectFileDatabaseInfoPaths,
inspectFileDatabaseInfo,
readFileDatabaseRevisionLocally,
retryReadFileDatabaseRevision,
} from "./fileDatabaseInfo";
async function* documents(paths: string[]) {
for (const path of paths) {
yield {
_id: `f:${path}`,
path,
};
}
}
function createCore() {
const current = {
_id: "f:note",
_rev: "3-current",
_conflicts: ["2-conflict"],
_revs_info: [
{ rev: "3-current", status: "available" },
{ rev: "2-parent", status: "missing" },
],
path: "note.md",
ctime: 100,
mtime: 300,
size: 42,
type: "plain",
datatype: "plain",
data: "secret current body",
children: ["h:private-current", "h:private-current", "h:private-embedded", "h:private-deleted"],
eden: {
"h:private-embedded": {
data: "secret embedded body",
epoch: 1,
},
},
};
const conflict = {
...current,
_rev: "2-conflict",
_conflicts: undefined,
_revs_info: [{ rev: "2-conflict", status: "available" }],
mtime: 200,
data: "secret conflict body",
children: ["h:private-missing"],
eden: {},
};
const promptCopyToClipboard = vi.fn(async (_title: string, _value: string) => true);
const askSelectString = vi.fn(async () => "db-only.md");
const core = {
settings: {
syncInternalFiles: false,
syncInternalFilesIgnorePatterns: "",
syncInternalFilesTargetPatterns: "",
},
storageAccess: {
isExistsIncludeHidden: vi.fn(async () => true),
statHidden: vi.fn(async () => ({
ctime: 90,
mtime: 310,
size: 45,
type: "file",
})),
getFileNames: vi.fn(async () => ["z.md", "a.md"]),
getFilesIncludeHidden: vi.fn(async () => [".obsidian/app.json", "a.md"]),
},
localDatabase: {
getDBEntryFromMeta: vi.fn(async (meta: typeof current) => ({
...meta,
data: ["loaded body"],
})),
getDBEntry: vi.fn(async () => current),
localDatabase: {
get: vi.fn(async (_id: string, options?: { rev?: string }) =>
options?.rev === "2-conflict" ? conflict : current
),
},
allDocsRaw: vi.fn(async ({ keys }: { keys: string[] }) => ({
rows: [
...(keys.includes("h:private-current")
? [
{
id: "h:private-current",
key: "h:private-current",
value: { rev: "1-chunk" },
},
]
: []),
...(keys.includes("h:private-deleted")
? [
{
id: "h:private-deleted",
key: "h:private-deleted",
value: { rev: "4-deleted-chunk", deleted: true },
},
]
: []),
],
})),
findAllDocs: vi.fn(() => documents(["db-only.md", "i:.obsidian/app.json", "ix:ignore", "ps:setting"])),
},
services: {
path: {
path2id: vi.fn(async () => "f:note"),
},
UI: {
promptCopyToClipboard,
confirm: {
askSelectString,
},
},
},
};
return {
askSelectString,
conflict,
core,
current,
promptCopyToClipboard,
};
}
describe("file database information", () => {
it("reports document and chunk revisions without exposing file contents", async () => {
const { core } = createCore();
const report = await buildFileDatabaseInfoReport(core as never, "note.md");
expect(report).toContain('"path": "note.md"');
expect(report).toContain('"documentId": "f:note"');
expect(report).toContain('"revision": "3-current"');
expect(report).toContain('"revision": "2-conflict"');
expect(report).toContain('"storageType": "plain"');
expect(report).toContain('"storageLayout": "chunked"');
expect(report).toContain('"contentAvailableLocally": false');
expect(report).toContain('"id": "h:private-current"');
expect(report).toContain('"localDatabaseRevision": "1-chunk"');
expect(report).toContain('"referenceCount": 2');
expect(report).toContain('"id": "h:private-embedded"');
expect(report).toContain('"embedded": true');
expect(report).toContain('"id": "h:private-deleted"');
expect(report).toContain('"localDatabaseState": "deleted"');
expect(report).toContain('"localDatabaseRevision": "4-deleted-chunk"');
expect(report).toContain('"id": "h:private-missing"');
expect(report).toContain('"localDatabaseState": "missing"');
expect(report).toContain('"localDatabaseRevision": null');
expect(report).not.toContain("secret current body");
expect(report).not.toContain("secret conflict body");
expect(report).not.toContain("secret embedded body");
});
it.each([
{
name: "notes",
document: {
type: "notes",
data: "secret legacy body",
},
storageType: "notes",
},
{
name: "an absent type",
document: {
type: undefined,
data: ["secret", " legacy body"],
},
storageType: "absent",
},
])("reports $name as legacy inline storage without exposing its body", async ({ document, storageType }) => {
const { core, current } = createCore();
core.localDatabase.localDatabase.get.mockResolvedValue({
...current,
...document,
_conflicts: [],
children: ["h:must-not-be-treated-as-a-chunk"],
} as never);
const info = await inspectFileDatabaseInfo(core as never, "note.md");
const report = await buildFileDatabaseInfoReport(core as never, "note.md");
expect(info.database.revisions).toEqual([
expect.objectContaining({
storageType,
storageLayout: "legacy-inline",
chunkReferences: 0,
contentAvailableLocally: true,
}),
]);
expect(report).not.toContain("secret legacy body");
expect(report).not.toContain("h:must-not-be-treated-as-a-chunk");
});
it("reports the exact shared ancestor and its missing chunks for each conflict", async () => {
const { conflict, core, current } = createCore();
const parent = {
...current,
_rev: "2-parent",
_conflicts: undefined,
_revs_info: [
{ rev: "2-parent", status: "available" },
{ rev: "1-root", status: "missing" },
],
children: ["h:missing-parent"],
eden: {},
};
core.localDatabase.localDatabase.get.mockImplementation(async (_id: string, options?: { rev?: string }) => {
if (options?.rev === "2-conflict") {
return {
...conflict,
_revs_info: [
{ rev: "2-conflict", status: "available" },
{ rev: "2-parent", status: "available" },
{ rev: "1-root", status: "missing" },
],
};
}
if (options?.rev === "2-parent") {
return parent;
}
return {
...current,
_revs_info: [
{ rev: "3-current", status: "available" },
{ rev: "2-parent", status: "available" },
{ rev: "1-root", status: "missing" },
],
};
});
const info = await inspectFileDatabaseInfo(core as never, "note.md");
expect(info.database.mergeBases).toEqual([
{
winnerRevision: "3-current",
conflictRevision: "2-conflict",
revision: "2-parent",
metadataAvailableLocally: true,
contentAvailableLocally: false,
missingChunkIds: ["h:missing-parent"],
unavailableSharedRevisions: ["1-root"],
},
]);
});
it("does not decode a revision whose chunks are not all available locally", async () => {
const { core } = createCore();
await expect(readFileDatabaseRevisionLocally(core as never, "note.md", "3-current")).resolves.toBe(false);
expect(core.localDatabase.getDBEntryFromMeta).not.toHaveBeenCalled();
});
it("decodes an exact revision after confirming that every chunk is available locally", async () => {
const { core, current } = createCore();
core.localDatabase.localDatabase.get.mockResolvedValue({
...current,
children: ["h:available"],
eden: {},
} as never);
core.localDatabase.allDocsRaw.mockResolvedValue({
rows: [
{
id: "h:available",
key: "h:available",
value: { rev: "1-available" },
},
],
});
await expect(readFileDatabaseRevisionLocally(core as never, "note.md", "3-current")).resolves.toEqual(
expect.objectContaining({
data: ["loaded body"],
})
);
expect(core.localDatabase.getDBEntryFromMeta).toHaveBeenCalledWith(
expect.objectContaining({
_rev: "3-current",
}),
false,
false
);
});
it("retries an exact revision through the configured chunk retrieval path", async () => {
const { core } = createCore();
await retryReadFileDatabaseRevision(core as never, "note.md", "2-conflict");
expect(core.localDatabase.getDBEntry).toHaveBeenCalledWith(
"note.md",
{ rev: "2-conflict" },
false,
true,
true
);
});
it("reports the exact revision as locally available after retry recovers its missing chunk", async () => {
const { conflict, core } = createCore();
let recovered = false;
core.localDatabase.getDBEntry.mockImplementation(async () => {
recovered = true;
return conflict as never;
});
core.localDatabase.allDocsRaw.mockImplementation(async ({ keys }: { keys: string[] }) => ({
rows:
recovered && keys.includes("h:private-missing")
? [
{
id: "h:private-missing",
key: "h:private-missing",
value: { rev: "1-recovered" },
},
]
: [],
}));
await expect(
retryReadFileDatabaseRevision(core as never, "note.md", "2-conflict")
).resolves.not.toBe(false);
const information = await inspectFileDatabaseInfo(core as never, "note.md");
expect(
information.database.revisions.find(({ revision }) => revision === "2-conflict")
).toEqual(
expect.objectContaining({
contentAvailableLocally: true,
chunks: [
expect.objectContaining({
id: "h:private-missing",
localDatabaseState: "available",
localDatabaseRevision: "1-recovered",
}),
],
})
);
});
it("keeps the exact revision identifiers when conflict metadata is unavailable", async () => {
const { conflict, core, current } = createCore();
core.localDatabase.localDatabase.get.mockImplementation(async (_id: string, options?: { rev?: string }) => {
if (options?.rev === "2-unavailable") {
throw Object.assign(new Error("missing"), { status: 404 });
}
if (options?.rev === "2-conflict") {
return conflict;
}
return { ...current, _conflicts: ["2-conflict", "2-unavailable"] };
});
const report = await buildFileDatabaseInfoReport(core as never, "note.md");
expect(report).toContain('"conflictRevisions"');
expect(report).toContain('"2-conflict"');
expect(report).toContain('"2-unavailable"');
expect(report).toContain('"unavailableConflictRevisions"');
});
it("reads an existing local document even when current synchronisation filters exclude its path", async () => {
const { core, current } = createCore();
core.services.path.path2id.mockResolvedValue("f:ignored");
core.localDatabase.localDatabase.get.mockResolvedValue({
...current,
_id: "f:ignored",
_rev: "5-ignored",
_conflicts: [],
_revs_info: [],
path: "ignored.md",
ctime: 10,
mtime: 20,
size: 30,
children: [],
});
const report = await buildFileDatabaseInfoReport(core as never, "ignored.md");
expect(report).toContain('"exists": true');
expect(report).toContain('"documentId": "f:ignored"');
expect(report).toContain('"revision": "5-ignored"');
});
it("offers the union of storage and database paths and excludes inactive internal namespaces", async () => {
const { core } = createCore();
await expect(collectFileDatabaseInfoPaths(core as never)).resolves.toEqual(["a.md", "db-only.md", "z.md"]);
core.settings.syncInternalFiles = true;
await expect(collectFileDatabaseInfoPaths(core as never)).resolves.toEqual([
".obsidian/app.json",
"a.md",
"db-only.md",
]);
});
it("copies the selected file report through the existing copy dialogue", async () => {
const { askSelectString, core, promptCopyToClipboard } = createCore();
await expect(chooseAndCopyFileDatabaseInfo(core as never)).resolves.toBe(true);
expect(askSelectString).toHaveBeenCalledWith("Choose a file to inspect", ["a.md", "db-only.md", "z.md"]);
expect(promptCopyToClipboard).toHaveBeenCalledWith(
"Database information for db-only.md",
expect.stringContaining('"path": "db-only.md"')
);
});
});
+109
View File
@@ -0,0 +1,109 @@
import type { LoadedEntry } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { createBlob, isDocContentSame, readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { IFileHandler } from "@vrtmrz/livesync-commonlib/compat/interfaces/FileHandler";
import {
inspectFileDatabaseInfo,
readFileDatabaseRevisionLocally,
type FileDatabaseInfo,
type FileDatabaseInfoCore,
type RevisionDatabaseInfo,
} from "./fileDatabaseInfo";
export type FileRepairCore = FileDatabaseInfoCore & {
fileHandler: Pick<IFileHandler, "deleteRevisionFromDB">;
storageAccess: FileDatabaseInfoCore["storageAccess"] & Pick<StorageAccess, "readHiddenFileBinary">;
};
export type FileRepairRevision = {
role: "winner" | "conflict";
metadata: RevisionDatabaseInfo;
contentReadable: boolean;
contentMatchesStorage: boolean | null;
loadedEntry: LoadedEntry | false;
};
export type FileRepairInspection = {
information: FileDatabaseInfo;
revisions: FileRepairRevision[];
requiresAttention: boolean;
};
export type DiscardUnreadableRevisionResult =
| "discarded"
| "failed"
| "no-longer-live"
| "revision-is-readable";
export async function inspectFileRepair(core: FileRepairCore, path: string): Promise<FileRepairInspection> {
const information = await inspectFileDatabaseInfo(core, path);
const storageContent = information.storage.exists
? createBlob(await core.storageAccess.readHiddenFileBinary(path))
: undefined;
const revisions: FileRepairRevision[] = [];
for (const metadata of information.database.revisions) {
const loadedEntry =
metadata.deleted || !metadata.contentAvailableLocally
? false
: await readFileDatabaseRevisionLocally(core, path, metadata.revision ?? "");
const contentReadable = metadata.deleted || loadedEntry !== false;
const contentMatchesStorage =
storageContent && loadedEntry !== false
? await isDocContentSame(storageContent, readAsBlob(loadedEntry))
: null;
revisions.push({
role: metadata.current ? "winner" : "conflict",
metadata,
contentReadable,
contentMatchesStorage,
loadedEntry,
});
}
const winner = revisions.find(({ role }) => role === "winner");
const databaseAndStorageDiffer =
information.storage.exists !== information.database.exists ||
(information.storage.exists &&
winner !== undefined &&
(winner.metadata.deleted || winner.contentMatchesStorage === false)) ||
(!information.storage.exists && winner !== undefined && !winner.metadata.deleted);
const unreadableLiveRevision =
information.database.unavailableConflictRevisions.length > 0 ||
revisions.some(({ contentReadable }) => !contentReadable);
const requiresAttention =
databaseAndStorageDiffer ||
information.database.conflictCount > 0 ||
unreadableLiveRevision ||
(information.database.exists && winner === undefined);
return {
information,
revisions,
requiresAttention,
};
}
export async function discardUnreadableLiveRevision(
core: FileRepairCore,
path: string,
revision: string
): Promise<DiscardUnreadableRevisionResult> {
const latest = await inspectFileDatabaseInfo(core, path);
const liveRevisions = [
latest.database.currentRevision,
...latest.database.conflictRevisions,
].filter((candidate): candidate is string => candidate !== null);
if (!liveRevisions.includes(revision)) {
return "no-longer-live";
}
const metadata = latest.database.revisions.find((candidate) => candidate.revision === revision);
const metadataUnavailable = latest.database.unavailableConflictRevisions.includes(revision);
if (!metadataUnavailable && (metadata?.deleted || metadata?.contentAvailableLocally)) {
return "revision-is-readable";
}
const deleted = await core.fileHandler.deleteRevisionFromDB(latest.databasePath, revision);
return deleted ? "discarded" : "failed";
}
+172
View File
@@ -0,0 +1,172 @@
import { describe, expect, it, vi } from "vitest";
import {
discardUnreadableLiveRevision,
inspectFileRepair,
} from "./fileRepair";
function createCore() {
const current = {
_id: "f:note",
_rev: "3-current",
_conflicts: ["2-conflict"],
_revs_info: [{ rev: "3-current", status: "available" }],
path: "note.md",
ctime: 1,
mtime: 3,
size: 7,
type: "plain",
children: ["h:current"],
eden: {},
};
const conflict = {
...current,
_rev: "2-conflict",
_conflicts: undefined,
_revs_info: [{ rev: "2-conflict", status: "available" }],
mtime: 2,
children: ["h:missing-conflict"],
};
const deleteRevisionFromDB = vi.fn(async () => true);
const core = {
settings: {
syncInternalFiles: false,
syncInternalFilesIgnorePatterns: "",
syncInternalFilesTargetPatterns: "",
},
storageAccess: {
isExistsIncludeHidden: vi.fn(async () => true),
statHidden: vi.fn(async () => ({
ctime: 1,
mtime: 3,
size: 7,
type: "file",
})),
readHiddenFileBinary: vi.fn(async () => new TextEncoder().encode("current").buffer),
getFileNames: vi.fn(async () => ["note.md"]),
getFilesIncludeHidden: vi.fn(async () => ["note.md"]),
},
localDatabase: {
localDatabase: {
get: vi.fn(async (_id: string, options?: { rev?: string }) =>
options?.rev === "2-conflict" ? conflict : current
),
},
allDocsRaw: vi.fn(async ({ keys }: { keys: string[] }) => ({
rows: keys.includes("h:current")
? [
{
id: "h:current",
key: "h:current",
value: { rev: "1-current" },
},
]
: [],
})),
getDBEntryFromMeta: vi.fn(async (meta: typeof current) => ({
...meta,
data: [meta._rev === "3-current" ? "current" : "conflict"],
})),
getDBEntry: vi.fn(async () => false),
findAllDocs: vi.fn(async function* () {
yield current;
}),
},
fileHandler: {
deleteRevisionFromDB,
},
services: {
path: {
path2id: vi.fn(async () => "f:note"),
},
UI: {
confirm: {},
},
},
};
return {
conflict,
core,
current,
deleteRevisionFromDB,
};
}
describe("file repair inspection", () => {
it("shows the winner and every conflict revision independently", async () => {
const { core } = createCore();
const inspection = await inspectFileRepair(core as never, "note.md");
expect(inspection.revisions).toEqual([
expect.objectContaining({
role: "winner",
contentReadable: true,
contentMatchesStorage: true,
metadata: expect.objectContaining({
revision: "3-current",
}),
}),
expect.objectContaining({
role: "conflict",
contentReadable: false,
contentMatchesStorage: null,
metadata: expect.objectContaining({
revision: "2-conflict",
}),
}),
]);
expect(inspection.requiresAttention).toBe(true);
});
it("rechecks liveness and readability before discarding an exact revision", async () => {
const { core, deleteRevisionFromDB } = createCore();
await expect(
discardUnreadableLiveRevision(core as never, "note.md", "2-conflict")
).resolves.toBe("discarded");
await expect(
discardUnreadableLiveRevision(core as never, "note.md", "3-current")
).resolves.toBe("revision-is-readable");
expect(deleteRevisionFromDB).toHaveBeenCalledOnce();
expect(deleteRevisionFromDB).toHaveBeenCalledWith("note.md", "2-conflict");
});
it("allows an exact unreadable generation-one winner to be discarded explicitly", async () => {
const { core, current, deleteRevisionFromDB } = createCore();
current._rev = "1-root";
current._conflicts = [];
current.children = ["h:missing-root"];
core.localDatabase.allDocsRaw.mockResolvedValue({ rows: [] });
const inspection = await inspectFileRepair(core as never, "note.md");
expect(inspection.revisions).toEqual([
expect.objectContaining({
role: "winner",
contentReadable: false,
metadata: expect.objectContaining({
revision: "1-root",
}),
}),
]);
await expect(
discardUnreadableLiveRevision(core as never, "note.md", "1-root")
).resolves.toBe("discarded");
expect(deleteRevisionFromDB).toHaveBeenCalledWith("note.md", "1-root");
});
it("refuses to discard a revision which stopped being a live leaf", async () => {
const { core, current, deleteRevisionFromDB } = createCore();
core.localDatabase.localDatabase.get.mockResolvedValue({
...current,
_conflicts: [],
});
await expect(
discardUnreadableLiveRevision(core as never, "note.md", "2-conflict")
).resolves.toBe("no-longer-live");
expect(deleteRevisionFromDB).not.toHaveBeenCalled();
});
});
+69 -14
View File
@@ -1,4 +1,4 @@
import { getLanguage, requireApiVersion } from "@/deps";
import { getLanguage, Notice, requireApiVersion } from "@/deps";
import { createServiceFeature } from "@vrtmrz/livesync-commonlib/compat/interfaces/ServiceModule";
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "@/common/rosetta";
import { $msg, __onMissingTranslation, setLang } from "@/common/translation";
@@ -15,7 +15,40 @@ function tryGetLanguage(onError: (error: unknown) => void) {
return "en";
}
export const enableI18nFeature = createServiceFeature(async ({ services: { setting, API } }) => {
class ObsidianLanguageAppliedNotice {
private reminder: Notice | undefined;
show(openDetails: () => void): void {
this.clear();
let reminderAnchor: HTMLAnchorElement | undefined;
const appliedMessage =
$msg("dialog.yourLanguageAvailable")
.split(/\r?\n\s*\r?\n/u, 1)[0]
?.trim() ?? $msg("Display Language");
const fragment = createFragment((documentFragment) => {
documentFragment.createSpan({
text: `${appliedMessage} `,
});
documentFragment.createEl("a", { text: $msg("Open the dialog") }, (anchor) => {
reminderAnchor = anchor;
anchor.addEventListener("click", (event) => {
event.preventDefault();
this.clear();
openDetails();
});
});
});
this.reminder = new Notice(fragment, 0);
reminderAnchor?.closest<HTMLElement>(".notice")?.classList.add("livesync-language-applied-notice");
}
clear(): void {
this.reminder?.hide();
this.reminder = undefined;
}
}
export const enableI18nFeature = createServiceFeature(async ({ services: { setting, API, appLifecycle } }) => {
// Clear missing translation handler to avoid unnecessary warnings.
__onMissingTranslation(() => {});
let isChanged = false;
@@ -36,26 +69,48 @@ export const enableI18nFeature = createServiceFeature(async ({ services: { setti
// settings.displayLanguage = obsidianLanguage as I18N_LANGS;
await setting.applyPartial({ displayLanguage: obsidianLanguage as I18N_LANGS });
isChanged = true;
setLang(settings.displayLanguage);
setLang(obsidianLanguage as I18N_LANGS);
} else if (settings.displayLanguage == "") {
// settings.displayLanguage = "def";
await setting.applyPartial({ displayLanguage: "def" });
setLang(settings.displayLanguage);
setLang("def");
await setting.saveSettingData();
}
}
if (isChanged) {
const revert = $msg("dialog.yourLanguageAvailable.btnRevertToDefault");
if (
(await API.confirm.askSelectStringDialogue($msg(`dialog.yourLanguageAvailable`), ["OK", revert], {
defaultAction: "OK",
title: $msg(`dialog.yourLanguageAvailable.Title`),
})) == revert
) {
await setting.applyPartial({ displayLanguage: "def" });
setLang(settings.displayLanguage);
}
await setting.saveSettingData();
const reminder = new ObsidianLanguageAppliedNotice();
appLifecycle.onUnload.addHandler(() => {
reminder.clear();
return Promise.resolve(true);
});
reminder.show(() => {
void (async () => {
try {
const revert = $msg("dialog.yourLanguageAvailable.btnRevertToDefault");
if (
(await API.confirm.askSelectStringDialogue(
$msg(`dialog.yourLanguageAvailable`),
["OK", revert],
{
defaultAction: "OK",
title: $msg("Display Language"),
}
)) == revert
) {
await setting.applyPartial({ displayLanguage: "def" });
setLang("def");
await setting.saveSettingData();
}
} catch (error) {
API.addLog(
`Failed to open translation details: ${String(error)}`,
LOG_LEVEL_VERBOSE,
"i18n-language"
);
}
})();
});
}
return true;
});
@@ -0,0 +1,110 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const noticeState = vi.hoisted(() => ({
instances: [] as Array<{ hide: ReturnType<typeof vi.fn>; duration: number }>,
spanTexts: [] as string[],
}));
vi.mock("@/deps", () => ({
getLanguage: () => "ja",
requireApiVersion: () => true,
Notice: class {
hide = vi.fn();
constructor(_fragment: unknown, duration: number) {
noticeState.instances.push({ hide: this.hide, duration });
}
},
}));
vi.mock("@/common/translation", () => ({
$msg: (key: string) =>
({
"dialog.yourLanguageAvailable": "Translation has been applied.\n\nMore details.",
"dialog.yourLanguageAvailable.btnRevertToDefault": "Keep Default",
"dialog.yourLanguageAvailable.Title": "Translation is available!",
"Display Language": "Display language",
"Open the dialog": "Open the dialogue",
})[key] ?? key,
__onMissingTranslation: vi.fn(),
setLang: vi.fn(),
}));
import { enableI18nFeature } from "./enablei18n.ts";
describe("automatic display language", () => {
let clickDetails: ((event: { preventDefault(): void }) => void) | undefined;
beforeEach(() => {
noticeState.instances.length = 0;
noticeState.spanTexts.length = 0;
clickDetails = undefined;
vi.stubGlobal("createFragment", (build: (fragment: unknown) => void) => {
const anchor = {
addEventListener: (_event: string, listener: (event: { preventDefault(): void }) => void) => {
clickDetails = listener;
},
closest: () => ({ classList: { add: vi.fn() } }),
};
const fragment = {
createSpan: ({ text }: { text: string }) => noticeState.spanTexts.push(text),
createEl: (_tag: string, _options: unknown, configure: (element: typeof anchor) => void) => {
configure(anchor);
return anchor;
},
};
build(fragment);
return fragment;
});
});
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
it("lets start-up continue and opens translation details only from a persistent Notice", async () => {
const settings = { displayLanguage: "" };
const applyPartial = vi.fn(async (partial: Partial<typeof settings>) => Object.assign(settings, partial));
const saveSettingData = vi.fn().mockResolvedValue(undefined);
const askSelectStringDialogue = vi.fn().mockResolvedValue("Keep Default");
const unloadHandlers: Array<() => Promise<boolean>> = [];
const host = {
services: {
setting: {
currentSettings: () => settings,
applyPartial,
saveSettingData,
},
API: {
addLog: vi.fn(),
confirm: { askSelectStringDialogue },
},
appLifecycle: {
onUnload: {
addHandler: (handler: () => Promise<boolean>) => unloadHandlers.push(handler),
},
},
},
};
await expect(enableI18nFeature(host as never)).resolves.toBe(true);
expect(settings.displayLanguage).toBe("ja");
expect(saveSettingData).toHaveBeenCalledOnce();
expect(askSelectStringDialogue).not.toHaveBeenCalled();
expect(noticeState.instances).toHaveLength(1);
expect(noticeState.instances[0]?.duration).toBe(0);
expect(noticeState.spanTexts).toEqual(["Translation has been applied. "]);
expect(clickDetails).toBeTypeOf("function");
clickDetails?.({ preventDefault: vi.fn() });
await vi.waitFor(() => expect(askSelectStringDialogue).toHaveBeenCalledOnce());
expect(askSelectStringDialogue.mock.calls[0]?.[2]).toMatchObject({ title: "Display language" });
await vi.waitFor(() => expect(settings.displayLanguage).toBe("def"));
expect(saveSettingData).toHaveBeenCalledTimes(2);
await expect(unloadHandlers[0]?.()).resolves.toBe(true);
expect(noticeState.instances[0]?.hide).toHaveBeenCalled();
});
});
+5 -1
View File
@@ -65,7 +65,11 @@ export function useSetupQRCodeFeature(host: NecessaryServices<"API" | "UI" | "se
host.services.API.addCommand({
id: "livesync-setting-qr",
name: "Show settings as a QR code",
callback: () => fireAndForget(encodeSetupSettingsAsQR(host)),
checkCallback: (checking) => {
if (!host.services.setting.currentSettings().isConfigured) return false;
if (!checking) fireAndForget(encodeSetupSettingsAsQR(host));
return true;
},
});
host.services.context.events.onEvent(EVENT_REQUEST_SHOW_SETUP_QR, () =>
fireAndForget(() => encodeSetupSettingsAsQR(host))
@@ -114,4 +114,44 @@ describe("setupObsidian/qrCode", () => {
);
expect(onEventSpy).toHaveBeenCalledWith(EVENT_REQUEST_SHOW_SETUP_QR, expect.any(Function));
});
it("keeps the QR command out of the palette until setup is complete", async () => {
const addHandler = vi.fn();
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
const settings = { isConfigured: false };
const host = {
services: {
context: createServiceContext(),
API: {
addCommand: vi.fn((command) => commands.push(command)),
},
appLifecycle: {
onLoaded: {
addHandler,
},
},
setting: {
currentSettings: vi.fn(() => settings),
},
UI: {
confirm: {
confirmWithMessage: vi.fn(),
},
},
},
} as any;
useSetupQRCodeFeature(host);
const loadedHandler = addHandler.mock.calls[0][0] as () => Promise<boolean>;
await loadedHandler();
const command = commands.find((candidate) => candidate.id === "livesync-setting-qr")!;
expect(command.checkCallback?.(true)).toBe(false);
settings.isConfigured = true;
expect(command.checkCallback?.(true)).toBe(true);
});
});
@@ -47,11 +47,6 @@ export function useSetupManagerHandlersFeature(
setupManager: SetupManager
) {
host.services.appLifecycle.onLoaded.addHandler(() => {
host.services.API.addCommand({
id: "livesync-open-onboarding",
name: "Open onboarding wizard",
callback: () => fireAndForget(() => openOnboarding(setupManager)),
});
host.services.API.addCommand({
id: "livesync-opensetupuri",
name: "Use the copied setup URI (Formerly Open setup URI)",
@@ -109,7 +109,7 @@ describe("setupObsidian/setupManagerHandlers", () => {
expect(preventDefault).toHaveBeenCalledOnce();
});
it("useSetupManagerHandlersFeature should register onLoaded handler that wires command and events", async () => {
it("keeps onboarding out of the command palette while wiring the setup URI command and events", async () => {
const addHandler = vi.fn();
const addCommand = vi.fn();
const events = { onEvent: vi.fn() };
@@ -147,10 +147,9 @@ describe("setupObsidian/setupManagerHandlers", () => {
const loadedHandler = addHandler.mock.calls[0][0] as () => Promise<boolean>;
await loadedHandler();
expect(addCommand).toHaveBeenCalledWith(
expect(addCommand).not.toHaveBeenCalledWith(
expect.objectContaining({
id: "livesync-open-onboarding",
name: "Open onboarding wizard",
})
);
expect(addCommand).toHaveBeenCalledWith(
+17 -3
View File
@@ -50,19 +50,33 @@ export function useSetupURIFeature(host: NecessaryServices<"API" | "UI" | "setti
host.services.API.addCommand({
id: "livesync-copysetupuri",
name: "Copy settings as a new setup URI",
callback: () => fireAndForget(copySetupURI(host, log)),
checkCallback: (checking) => {
if (!host.services.setting.currentSettings().isConfigured) return false;
if (!checking) fireAndForget(copySetupURI(host, log));
return true;
},
});
host.services.API.addCommand({
id: "livesync-copysetupuri-short",
name: "Copy settings as a new setup URI (With customization sync)",
callback: () => fireAndForget(copySetupURI(host, log, false)),
checkCallback: (checking) => {
const settings = host.services.setting.currentSettings();
if (!settings.isConfigured || !settings.usePluginSync) return false;
if (!checking) fireAndForget(copySetupURI(host, log, false));
return true;
},
});
host.services.API.addCommand({
id: "livesync-copysetupurifull",
name: "Copy settings as a new setup URI (Full)",
callback: () => fireAndForget(copySetupURIFull(host, log)),
checkCallback: (checking) => {
const settings = host.services.setting.currentSettings();
if (!settings.isConfigured || !settings.useAdvancedMode) return false;
if (!checking) fireAndForget(copySetupURIFull(host, log));
return true;
},
});
host.services.context.events.onEvent(EVENT_REQUEST_COPY_SETUP_URI, () =>
@@ -156,4 +156,57 @@ describe("setupObsidian/setupUri", () => {
expect(addCommand).toHaveBeenCalledWith(expect.objectContaining({ id: "livesync-copysetupurifull" }));
expect(onEventSpy).toHaveBeenCalledWith(EVENT_REQUEST_COPY_SETUP_URI, expect.any(Function));
});
it("shows Setup URI variants only when their configuration level is relevant", async () => {
const addHandler = vi.fn();
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
const settings = {
isConfigured: false,
usePluginSync: false,
useAdvancedMode: false,
};
const host = {
services: {
context: createServiceContext(),
API: {
addCommand: vi.fn((command) => commands.push(command)),
addLog: vi.fn(),
},
appLifecycle: {
onLoaded: {
addHandler,
},
},
setting: {
currentSettings: vi.fn(() => settings),
},
UI: {
confirm: {
askString: vi.fn(() => "pass"),
},
promptCopyToClipboard: vi.fn(() => true),
},
},
} as any;
useSetupURIFeature(host);
const loadedHandler = addHandler.mock.calls[0][0] as () => Promise<boolean>;
await loadedHandler();
const command = (id: string) => commands.find((candidate) => candidate.id === id)!;
expect(command("livesync-copysetupuri").checkCallback?.(true)).toBe(false);
settings.isConfigured = true;
expect(command("livesync-copysetupuri").checkCallback?.(true)).toBe(true);
expect(command("livesync-copysetupuri-short").checkCallback?.(true)).toBe(false);
expect(command("livesync-copysetupurifull").checkCallback?.(true)).toBe(false);
settings.usePluginSync = true;
settings.useAdvancedMode = true;
expect(command("livesync-copysetupuri-short").checkCallback?.(true)).toBe(true);
expect(command("livesync-copysetupurifull").checkCallback?.(true)).toBe(true);
});
});
+35 -14
View File
@@ -67,7 +67,12 @@ export function useP2PReplicatorUI(
showWindow: (type: string) => Promise<void>;
showWindowOnRight?: (type: string) => Promise<void>;
registerWindow: (type: string, factory: (leaf: WorkspaceLeaf) => unknown) => void;
addCommand: (command: { id: string; name: string; callback: () => void }) => unknown;
addCommand: (command: {
id: string;
name: string;
callback?: () => void;
checkCallback?: (checking: boolean) => boolean | void;
}) => unknown;
addRibbonIcon: (
icon: string,
title: string,
@@ -146,8 +151,12 @@ export function useP2PReplicatorUI(
api.addCommand({
id: "open-p2p-server-status",
name: "P2P Sync : Open P2P Status",
callback: () => {
void openStatusPane();
checkCallback: (checking) => {
if (!hasP2PConfiguration(host.services.setting.currentSettings())) return false;
if (!checking) {
void openStatusPane();
}
return true;
},
});
host.services.API.addCommand({
@@ -155,11 +164,15 @@ export function useP2PReplicatorUI(
name: "Replicate P2P to default peer",
checkCallback: (isChecking: boolean) => {
const settings = host.services.setting.currentSettings();
if (isChecking) {
if (settings.remoteType == REMOTE_P2P) return false;
return replicator.replicator?.server?.isServing ?? false;
const isAvailable =
hasP2PConfiguration(settings) &&
settings.remoteType !== REMOTE_P2P &&
(replicator.replicator?.server?.isServing ?? false);
if (!isAvailable) return false;
if (!isChecking) {
runOpenReplication();
}
runOpenReplication();
return true;
},
});
host.services.API.addCommand({
@@ -167,11 +180,15 @@ export function useP2PReplicatorUI(
name: "Replicate now by P2P",
checkCallback: (isChecking: boolean) => {
const settings = host.services.setting.currentSettings();
if (isChecking) {
if (settings.remoteType == REMOTE_P2P) return false;
return replicator.replicator?.server?.isServing ?? false;
const isAvailable =
hasP2PConfiguration(settings) &&
settings.remoteType !== REMOTE_P2P &&
(replicator.replicator?.server?.isServing ?? false);
if (!isAvailable) return false;
if (!isChecking) {
runOpenReplication();
}
runOpenReplication();
return true;
},
});
@@ -179,10 +196,14 @@ export function useP2PReplicatorUI(
id: "p2p-sync-targets",
name: "P2P: Sync with targets",
checkCallback: (isChecking: boolean) => {
if (isChecking) {
return replicator.replicator?.server?.isServing ?? false;
const isAvailable =
hasP2PConfiguration(host.services.setting.currentSettings()) &&
(replicator.replicator?.server?.isServing ?? false);
if (!isAvailable) return false;
if (!isChecking) {
void replicator.replicator?.replicateFromCommand(true);
}
void replicator.replicator?.replicateFromCommand(true);
return true;
},
});
@@ -85,7 +85,12 @@ describe("useP2PReplicatorUI commands", () => {
onSettingLoaded: { addHandler: vi.fn() },
onLayoutReady: { addHandler: vi.fn() },
},
setting: { currentSettings: vi.fn(() => ({ remoteType: "COUCHDB" })) },
setting: {
currentSettings: vi.fn(() => ({
remoteType: "COUCHDB",
P2P_Enabled: true,
})),
},
replicator: { runFiniteReplicationActivity },
},
} as any;
@@ -143,7 +148,11 @@ describe("useP2PReplicatorUI commands", () => {
});
it("retains only the current P2P status command and routes existing open requests to it", async () => {
const commands: Array<{ id: string; callback?: () => void }> = [];
const commands: Array<{
id: string;
callback?: () => void;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
let initialise: (() => Promise<unknown>) | undefined;
const showWindow = vi.fn(async () => undefined);
const showWindowOnRight = vi.fn(async () => undefined);
@@ -183,12 +192,90 @@ describe("useP2PReplicatorUI commands", () => {
expect(commands.map((command) => command.id)).not.toContain("open-p2p-replicator");
expect(commands.map((command) => command.id)).toContain("open-p2p-server-status");
expect(commands.find((command) => command.id === "open-p2p-server-status")?.checkCallback?.(true)).toBe(false);
eventHub.emitEvent(EVENT_REQUEST_OPEN_P2P);
await vi.waitFor(() => expect(showWindowOnRight).toHaveBeenCalledWith("p2p-status"));
expect(showWindow).not.toHaveBeenCalledWith("p2p");
});
it("shows P2P commands only when a P2P configuration exists and their runtime prerequisites are met", async () => {
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
let initialise: (() => Promise<unknown>) | undefined;
let settings: Record<string, unknown> = {
remoteType: "COUCHDB",
remoteConfigurations: {},
};
const host = {
services: {
context: createServiceContext(),
API: {
showWindow: vi.fn(async () => undefined),
showWindowOnRight: vi.fn(async () => undefined),
registerWindow: vi.fn(),
addCommand: vi.fn((command) => commands.push(command)),
addRibbonIcon: vi.fn(),
},
appLifecycle: {
onInitialise: {
addHandler: vi.fn((handler) => {
initialise = handler;
}),
},
onSettingLoaded: { addHandler: vi.fn() },
onLayoutReady: { addHandler: vi.fn() },
},
setting: {
currentSettings: vi.fn(() => settings),
onSettingSaved: { addHandler: vi.fn() },
},
replicator: { runFiniteReplicationActivity: vi.fn() },
},
} as any;
const p2p = {
replicator: {
server: { isServing: true },
openReplication: vi.fn(),
replicateFromCommand: vi.fn(),
},
} as any;
useP2PReplicatorUI(host, {} as any, p2p);
await initialise?.();
for (const commandId of [
"open-p2p-server-status",
"replicate-now-by-p2p-default-peer",
"replicate-now-by-p2p",
"p2p-sync-targets",
]) {
expect(commands.find(({ id }) => id === commandId)?.checkCallback?.(true)).toBe(false);
}
settings = {
...settings,
remoteConfigurations: {
peer: {
id: "peer",
name: "Peer",
uri: "sls+p2p://room?passphrase=secret",
isEncrypted: false,
},
},
};
for (const commandId of [
"open-p2p-server-status",
"replicate-now-by-p2p-default-peer",
"replicate-now-by-p2p",
"p2p-sync-targets",
]) {
expect(commands.find(({ id }) => id === commandId)?.checkCallback?.(true)).toBe(true);
}
});
it("does not open the P2P status pane automatically when the workspace becomes ready", async () => {
let layoutReady: (() => Promise<unknown>) | undefined;
const showWindow = vi.fn(async () => undefined);
+52
View File
@@ -12,6 +12,11 @@
background-color: var(--text-muted);
}
.vpk-action-dialog__message {
user-select: text;
-webkit-user-select: text;
}
.conflict-dev-name {
display: inline-block;
min-width: 5em;
@@ -355,6 +360,10 @@ body {
justify-content: center;
}
body:not(.is-mobile):has(.sls-setting) .notice:has(.sls-onboarding-invitation-action) {
margin-right: 96px;
}
.sls-review-harness {
box-sizing: border-box;
max-width: 100%;
@@ -591,6 +600,49 @@ body.is-mobile .livesync-compatibility-review-notice {
word-break: break-all;
}
.sls-repair-results {
display: grid;
gap: var(--size-4-3);
}
.sls-repair-result {
padding: var(--size-4-3);
border: 1px solid var(--background-modifier-border);
border-radius: var(--radius-m);
background: var(--background-secondary);
}
.sls-repair-revision {
margin-top: var(--size-4-2);
padding: var(--size-4-2);
border-left: 3px solid var(--background-modifier-border);
background: var(--background-primary-alt);
}
.sls-repair-revision-title {
font-weight: var(--font-semibold);
overflow-wrap: anywhere;
}
.sls-repair-revision code {
display: block;
margin-top: var(--size-4-1);
overflow-wrap: anywhere;
white-space: normal;
}
.sls-repair-ancestor-warning {
margin-top: var(--size-4-2);
color: var(--text-warning);
}
.sls-repair-actions {
display: flex;
flex-wrap: wrap;
gap: var(--size-4-2);
margin-top: var(--size-4-2);
}
/* Diff navigation */
.diff-options-row {
display: flex;
+34 -4
View File
@@ -30,6 +30,24 @@ On macOS, `@vrtmrz/obsidian-test-session` keeps the generated Vault and profile
Multi-session workflows must keep each started Obsidian session tracked until its stop operation completes. If a scenario throws, teardown stops every active session before disposing its temporary Vault and profile, so a failed CLI or synchronisation operation cannot leave Obsidian using directories which have already been removed.
## Observing and diagnosing a scenario
Use externally visible behaviour as the pass condition: Vault files, remote-service state, revision data, or visible Obsidian UI. A log line can explain a failure, but should not replace an assertion about the resulting behaviour.
The maintained runner provides several complementary observation paths:
- `evalObsidianJson()` and `obsidian-cli eval` can read a small, explicitly selected piece of LiveSync or Obsidian state.
- `withObsidianPage()` can inspect the active renderer, invoke a registered command, or interact with visible UI through CDP. `captureObsidianPage()`, `captureObsidianDialogue()`, and `captureObsidianElement()` retain screenshots; the capture helpers also write a full-page `.failure.png` before rethrowing a UI assertion failure.
- `session.app.output()` returns the standard output and standard error captured from the isolated Obsidian process. This is especially useful when the renderer or CLI becomes unreachable.
- **Show log** (`obsidian-livesync:view-log`) exposes the recent LiveSync log, while **Copy full report to clipboard** (`obsidian-livesync:dump-debug-info`) opens the generated diagnostic report. `dialog-mounts.ts` verifies both surfaces, and focused scenarios may inspect the log pane and `appLifecycle.getUnresolvedMessages()` for a bounded set of expected errors.
- Renderer `console` messages and uncaught page errors are not retained automatically. A focused investigation can attach `page.on("console", ...)` and `page.on("pageerror", ...)` while it owns a `withObsidianPage()` callback. That observer ends when the callback closes its CDP connection, so use it around the action under investigation rather than treating it as a session-wide audit trail.
If a scenario times out or appears to do nothing, capture the visible page before teardown, then record a bounded state snapshot and the relevant tail of the LiveSync log, unresolved messages, and process output. If an unexplained Notice appears, retain a screenshot while it is still visible before opening or dismissing it, then use the log or full report to identify its source. A Notice alone is not enough evidence for its cause.
Set `showVerboseLog: true` only in isolated plug-in data when a focused investigation needs it. Keep captured output short and redact it before retaining or sharing it: logs and reports can contain Vault paths, document names, endpoints, credentials, Setup URIs, passphrases, or Security Seed material. Do not collect verbose logs from an ordinary user Vault.
Collect evidence before cleanup, and keep process, Vault, profile, and remote-fixture cleanup in `finally`. After `app.emulateMobile(true)`, use the active CDP renderer for fixture operations because Obsidian may remove desktop-only CLI commands. Visually inspect screenshots before copying selected images into user documentation; a passing locator assertion does not establish that a dialogue is readable or unobstructed.
## Local Setup
Set `OBSIDIAN_BINARY` when Obsidian is not installed in a standard location. Set `OBSIDIAN_CLI` as well when its companion executable is outside the built-in discovery paths.
@@ -55,6 +73,7 @@ After changing plug-in source, use the focused wrapper rather than invoking a sc
```bash
npm run test:e2e:obsidian:focused -- settings-ui
npm run test:e2e:obsidian:focused -- two-vault-sync
npm run test:e2e:obsidian:focused -- security-seed-reconnect
```
The wrapper accepts only maintained real-Obsidian scenario names; run it with `--help` for the current list. It deliberately does not manage CouchDB, Object Storage, or the P2P signalling relay. Start the required fixture first, or use the complete service-managed suite.
@@ -81,7 +100,7 @@ The underlying `test:e2e:obsidian:<scenario>` scripts remain available for an im
`test:contract:context:cli` builds the Node CLI and runs its existing Deno setup, put, read, list, information, remove, conflict-resolution, and revision workflow. `test:contract:context:obsidian` builds the plug-in and runs the real-Obsidian smoke test, including the Context inspection. These runtime scripts are local validation entry points and are not added to the default CI gate by this change.
`test:e2e:obsidian:onboarding-invitation` starts an unconfigured temporary Vault with no plug-in data and verifies that startup selects Commonlib's new-Vault recommendations, offers the setup wizard without opening it, and does not scan Vault files automatically. It checks the invitation action and introduction in mobile test mode, then uses the permanent command to reopen the wizard on the desktop. This scenario owns the unconfigured-startup boundary only; configured compatibility review remains covered by `settings-ui`, and the setup workflows remain covered by their dedicated scenarios.
`test:e2e:obsidian:onboarding-invitation` starts an unconfigured temporary Vault with no plug-in data and verifies that startup selects Commonlib's new-Vault recommendations, offers the setup wizard without opening it, and does not scan Vault files automatically. It checks the invitation action and introduction in mobile test mode, then reopens the wizard from **Self-hosted LiveSync settings****Setup** on the desktop. This scenario owns the unconfigured-startup boundary only; configured compatibility review remains covered by `settings-ui`, and the setup workflows remain covered by their dedicated scenarios.
`test:e2e:obsidian:dialog-mounts` starts a temporary real Obsidian session and exercises remote selection and CouchDB settings through `SetupManager`, plus Setup URI entry through the registered command. It verifies the compatibility pause and remote-size review, the distinction between a central data-storage server and P2P signalling, the explicit tested and untested CouchDB save actions, the internal-API warning, the Setup URI controls, automatic adjustment when differences are limited to compatible chunk settings, and both manual configuration-mismatch routes. The same session opens the live log and generated full report, reaches the `Hatch` recovery controls, writes and removes its own persistent log, and runs the missing-chunk recreation and file-verification actions against the empty disposable Vault. It captures representative desktop and mobile dialogues, checks the mobile layout and vertically stacked actions, closes each route through its normal controls, and verifies that each mounted operation settles without an error. It does not apply a remote configuration, contact a remote service, or claim to repair a deliberately damaged database.
@@ -93,7 +112,7 @@ The mobile pass uses Obsidian's `app.emulateMobile(true)`, a 390 by 844 CSS-pixe
`test:e2e:obsidian:p2p-pane` starts one unconfigured CouchDB-only session and one configured P2P session. It proves that the command remains registered while the retired command, automatic pane, and unconfigured ribbon entry are absent. For the configured profile, it verifies that the ribbon and current status command reach the pane without opening it at start-up, checks its connection control and horizontal layout, and captures unobstructed desktop and mobile screenshots. It deliberately uses no relay or peer: replacement of the active replicator is covered by focused unit tests, the Deno and Compose CLI P2P lifecycle suite covers the headless transport, and `p2p-setup-uri-workflow` owns the visible transfer path between two real Obsidian sessions.
`test:e2e:obsidian:local-suite` builds the plug-in and, unless `LIVESYNC_CLI_COMMAND` selects an external CLI, the local LiveSync CLI. It then runs discovery, smoke, the onboarding invitation, Svelte dialogue mounting, settings UI, the Review Harness, the P2P status pane, Vault reflection, CouchDB upload and manual setup, CLI-to-Obsidian synchronisation, Object Storage upload and Setup URI round-trip, P2P Setup URI round-trip, startup scan, provisioned CouchDB Setup URI, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export in sequence. Start the local CouchDB, MinIO, and P2P relay fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run.
`test:e2e:obsidian:local-suite` builds the plug-in and, unless `LIVESYNC_CLI_COMMAND` selects an external CLI, the local LiveSync CLI. It then runs discovery, smoke, the onboarding invitation, Svelte dialogue mounting, revision repair, settings UI, the Review Harness, the P2P status pane, Vault reflection, CouchDB upload and manual setup, CLI-to-Obsidian synchronisation, Object Storage upload and Setup URI round-trip, P2P Setup URI round-trip, startup scan, provisioned CouchDB Setup URI, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export in sequence. Start the local CouchDB, MinIO, and P2P relay fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run.
`test:e2e:obsidian:couchdb-upload` reuses the CouchDB variables from `.test.env` or the process environment. It expects a reachable CouchDB service, creates a unique database, starts from configured plug-in data without the device-local compatibility marker, and verifies the copied-or-restored Vault explanation in the actual compatibility dialogue. It captures the summary and details, resumes explicitly, confirms that the marker was recorded, creates a note in real Obsidian, commits the note into the local database, runs one-shot synchronisation, and verifies that the remote database contains both the metadata document and its chunk documents.
@@ -101,7 +120,7 @@ The same workflow checks the two remote-activity status boundaries. It first hol
`test:e2e:obsidian:couchdb-manual-setup-workflow` follows the visible onboarding path for the first device when no Setup URI is available. It enters end-to-end encryption and CouchDB details, runs the read-only `Check server requirements` step, requires the prepared fixture to pass without applying a server fix, and lets the onboarding connection test create the named database. After Rebuild completes on the first device, it creates an ordinary note, asks that working device to generate a Setup URI for a second device, completes Fetch there, and verifies a bidirectional note round-trip. The workflow captures each decision point and the expanded server-check result; password controls remain visually masked.
If this status workflow fails while Obsidian is running, it writes a full-page screenshot and a JSON snapshot of the status text and counters under `/tmp/obsidian-livesync-e2e`. The dialogue-mount workflow leaves desktop and mobile screenshots for both representative Svelte routes, and the Hidden File Sync workflow captures the successfully displayed JSON Resolve dialogue before selecting an option. The suite therefore records representative evidence without capturing every interaction. Set `E2E_OBSIDIAN_DIAGNOSTICS_DIR` to use another directory.
If this status workflow fails while Obsidian is running, it writes a full-page screenshot and a JSON snapshot of the status text and counters under `/tmp/obsidian-livesync-e2e`. The dialogue-mount workflow leaves desktop and mobile screenshots for both representative Svelte routes, the Hidden File Sync workflow captures the successfully displayed JSON Resolve dialogue before selecting an option, and the Security Seed reconnect workflow captures each significant application state. The suite therefore records representative evidence without capturing every interaction. Set `E2E_OBSIDIAN_DIAGNOSTICS_DIR` to use another directory.
The two-Vault workflow performs the missing-marker review once for each isolated Vault. Later process launches reuse the same profile-backed acknowledgement, rather than seeding a replacement or repeatedly applying a decision for the first device. The Hidden File Sync scenario is narrower: it starts from an explicitly acknowledged marker because it tests consumer-owned hidden-file behaviour, JSON resolution, target filtering, and grouped mobile Notices rather than duplicating the compatibility workflow. After `app.emulateMobile(true)`, its fixture operations use the active DevTools renderer because Obsidian can remove desktop-only CLI commands in mobile mode.
@@ -134,8 +153,16 @@ LIVESYNC_CLI_COMMAND="docker run --rm --network host --user $(id -u):$(id -g) --
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, ordinary rename, a case-only file name change within the same directory, deletion, and a separate encrypted round-trip with Path Obfuscation enabled. Its target-filter scenario confirms that one Vault receives and checkpoints a remote document without reflecting it, restarts with the same profile and filter, and then reflects the stored document after the filter is broadened through the settings service. Directory case changes deliberately remain outside this scenario because they require directory-aware rename handling. The optional Markdown conflict check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`. It creates divergent revisions in two separate Vaults, performs a conservative merge on one Vault, edits that result again, and requires the other Vault to replace its known deleted losing revision without recreating the conflict. The separate `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` check keeps four conflicts active while one Vault edits, deletes, performs a case-only rename, and performs a cross-path rename. It asserts that each operation extends the revision displayed on that device, replicates the exact resulting revision tree, and preserves the other live branch. During focused development, `E2E_OBSIDIAN_ONLY_CONFLICT_OPERATIONS=true` runs that self-contained scope without the ordinary, target-filter, or encrypted scenarios. Both conflict checks remain outside the default local suite.
`test:e2e:obsidian:security-seed-reconnect` is a focused CouchDB release-acceptance workflow. Device A first recognises an initial remote Security Seed, stops automatic replication while remaining open, and creates an unsent note. The runner replaces only the Security Seed in the managed remote synchronisation-parameter fixture. Device A must retain its deliberately stale cached value until the next one-shot synchronisation, refresh it before sending, and upload an HKDF-encrypted payload which uses the replacement value. A fresh device B must decrypt that note and send an encrypted note back; the original device A then receives the return journey with its Vault and isolated profile preserved. Desktop Obsidian may enforce a single application instance, so the two device sessions run sequentially after the same-process stale-cache assertion has completed.
The workflow creates a random dedicated database, records only SHA-256 Seed fingerprints, and never writes a Seed, passphrase, or CouchDB credentials to its result. It also requires the remote Seed and all other synchronisation parameters to remain unchanged after the replacement revision, rejects HKDF and Seed errors from either session, writes `security-seed-reconnect-result.json`, and verifies that every Obsidian process, temporary Vault, isolated profile, and database has been removed. The result file and stage screenshots are retained in `E2E_OBSIDIAN_DIAGNOSTICS_DIR`; the screenshots show ordinary Vault content, not settings or secrets. The strict cleanup workflow rejects `E2E_OBSIDIAN_KEEP_VAULT` and `E2E_OBSIDIAN_KEEP_COUCHDB`.
This proves in real Obsidian the plug-in behaviour shared by supported platforms, including the encrypted bidirectional round-trip and protection against a stale client restoring the old remote Seed. It does not verify iPadOS-specific background or reconnect lifecycle behaviour, and it does not count as Android device evidence. The workflow remains outside `test:e2e:obsidian:local-suite` because it is a focused release-acceptance check.
`test:e2e:obsidian:conflict-dialog-policy` creates three real local revision leaves without a remote service and opens the pairwise merge dialogue in Obsidian. It verifies the three-version count, requires the four decision buttons to be stacked vertically, concatenates the displayed pair as a child of the displayed winner, confirms that the untouched leaf remains as one conflict, postpones that remaining pair, restarts the same isolated Vault and profile, and confirms that only the two live versions are reconstructed. It also verifies that an ordinary repeated conflict check does not reopen a postponed dialogue, that **Resolve if conflicted.** explicitly reopens it, and that the active editor retains the appropriate unresolved-conflict warning. The scenario then invokes the same Commonlib consumer boundary used for an incoming replicated document and checks that a postponed warning disappears, an open stale dialogue closes, and the conflict-processing queue completes even when the dialogue closes immediately. This isolates the Obsidian UI contract from transport and second-device setup. The fixture owns one temporary Vault and profile, and the session runner stops Obsidian before removing them.
`test:e2e:obsidian:revision-repair` creates two live revisions in a temporary real Obsidian Vault, removes a chunk used only by the non-winning revision, and proves that automatic conflict checking does not discard the unreadable branch. **Verify and repair all files** must show the winner and conflict separately, identify the exact unreadable revision and missing chunk, and leave the revision tree unchanged when reading is retried. The scenario then verifies both the cancellation path and the explicit confirmation path for discarding that exact unreadable live revision, requires the winner to remain unchanged, and captures the repair card. It uses no remote service; a retry is therefore expected to remain unreadable unless the chunk is already available locally.
`test:e2e:obsidian:hidden-file-snippet-sync` runs a two-vault hidden file round-trip. It verifies creation and deletion of a real `.obsidian/snippets/*.css` file, automatic JSON conflict merging for a hidden file with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target patterns where one vault ignores a hidden file that the other vault synchronises. Initial enablement must open one user-visible progress Notice before the enabled setting is saved, then retain that Notice while its nested rebuild and scan phases continue in the ordinary log. The configured fixture starts with a current CouchDB remote profile, so migration from legacy remote settings remains the responsibility of the upgrade scenarios and cannot add unrelated Notices to this check. It also covers [issue #555](https://github.com/vrtmrz/obsidian-livesync/issues/555) by requiring several plug-in and settings changes to share one separate action Notice whose controls remain usable in mobile layouts; a manually dismissed group must not repeat its acknowledged rows when a later change arrives.
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
@@ -187,6 +214,7 @@ Useful environment variables:
- `E2E_OBSIDIAN_SKIP_EXTRACT=true`: download the AppImage without extracting it.
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS`: smoke timeout in milliseconds.
- `E2E_OBSIDIAN_DIALOG_TIMEOUT_MS`: timeout for a representative Svelte dialogue to mount, expose its principal controls, and close; default is 10 seconds.
- `E2E_OBSIDIAN_REVISION_REPAIR_TIMEOUT_MS`: timeout for each visible revision-repair control and result; default is 15 seconds.
- `E2E_OBSIDIAN_SETTINGS_TIMEOUT_MS`: timeout for the settings pane and its deletion controls to become visible; default is 10 seconds.
- `E2E_OBSIDIAN_REVIEW_HARNESS_TIMEOUT_MS`: timeout for Review Harness view and action boundaries; default is 15 seconds.
- `E2E_OBSIDIAN_P2P_PANE_TIMEOUT_MS`: timeout for the P2P status pane and its principal connection control; default is 10 seconds.
@@ -201,12 +229,14 @@ Useful environment variables:
- `LIVESYNC_CLI_COMMAND`: optional LiveSync CLI executable and prefix arguments used by the CLI-to-Obsidian compatibility check. The default is the locally built CLI.
- `E2E_LIVESYNC_SOURCE_ARTIFACT_ROOT`: optional cache directory containing the exact pinned 0.25.83 plug-in artefacts. Cached files are always checksum-verified.
- `E2E_LIVESYNC_TARGET_ARTIFACT_ROOT`: directory containing the built 1.0 target `main.js`, `manifest.json`, and `styles.css`; default is the repository root.
- `E2E_OBSIDIAN_ARTIFACT_ROOT`: directory containing the plug-in artefact installed by a direct scenario invocation; default is the repository root.
- `E2E_OBSIDIAN_ARTIFACT_REVISION`: exact source commit recorded by the Security Seed reconnect result when `E2E_OBSIDIAN_ARTIFACT_ROOT` is a downloaded artefact rather than a Git worktree.
- `E2E_OBSIDIAN_FILE_TIMEOUT_MS`: timeout for waiting until a note created through Obsidian's vault API is reflected to disk.
- `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready.
- `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database.
- `E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS`: timeout for waiting until CouchDB contains uploaded E2E documents.
- `E2E_OBSIDIAN_REMOTE_ACTIVITY_TIMEOUT_MS`: timeout for an observed remote activity to enter or leave its status boundary; default is 30 seconds.
- `E2E_OBSIDIAN_DIAGNOSTICS_DIR`: directory for screenshots and status snapshots captured after a remote-activity failure; default is `/tmp/obsidian-livesync-e2e`.
- `E2E_OBSIDIAN_DIAGNOSTICS_DIR`: directory for screenshots and status snapshots, including the Security Seed reconnect stages; default is `/tmp/obsidian-livesync-e2e`.
- `E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS`: timeout for waiting until Object Storage contains uploaded E2E objects.
- `E2E_OBSIDIAN_KEEP_COUCHDB=true`: keep the temporary CouchDB database for inspection.
- `E2E_OBSIDIAN_KEEP_OBJECT_STORAGE=true`: keep the temporary Object Storage prefix for inspection.
+46 -2
View File
@@ -42,6 +42,12 @@ export type CouchDbDatabaseInfo = {
update_seq: number | string;
};
export type CouchDbPutResponse = {
ok: boolean;
id: string;
rev: string;
};
function parseEnvFile(content: string): Record<string, string> {
const entries = content
.split(/\r?\n/u)
@@ -79,6 +85,14 @@ function databaseUrl(config: Pick<CouchDbConfig, "uri">, dbName: string, suffix
return `${config.uri.replace(/\/+$/u, "")}/${encodeURIComponent(dbName)}${suffix}`;
}
function documentSuffix(documentId: string): string {
const localPrefix = "_local/";
if (documentId.startsWith(localPrefix)) {
return `/_local/${encodeURIComponent(documentId.slice(localPrefix.length))}`;
}
return `/${encodeURIComponent(documentId)}`;
}
async function couchDbRequest(
config: Pick<CouchDbConfig, "uri" | "username" | "password">,
path: string,
@@ -146,8 +160,8 @@ export async function putCouchDbDocument(
config: CouchDbConfig,
dbName: string,
document: CouchDbDocument
): Promise<void> {
const response = await fetch(databaseUrl(config, dbName, `/${encodeURIComponent(document._id)}`), {
): Promise<CouchDbPutResponse> {
const response = await fetch(databaseUrl(config, dbName, documentSuffix(document._id)), {
method: "PUT",
headers: {
authorization: authHeader(config),
@@ -160,6 +174,23 @@ export async function putCouchDbDocument(
`Failed to write CouchDB document ${document._id}. HTTP ${response.status}: ${await response.text()}`
);
}
return (await response.json()) as CouchDbPutResponse;
}
export async function fetchCouchDbDocument(
config: CouchDbConfig,
dbName: string,
documentId: string
): Promise<CouchDbDocument> {
const response = await fetch(databaseUrl(config, dbName, documentSuffix(documentId)), {
headers: { authorization: authHeader(config) },
});
if (!response.ok) {
throw new Error(
`Failed to read CouchDB document ${documentId}. HTTP ${response.status}: ${await response.text()}`
);
}
return (await response.json()) as CouchDbDocument;
}
export async function deleteCouchDbDatabase(config: CouchDbConfig, dbName: string): Promise<void> {
@@ -174,6 +205,19 @@ export async function deleteCouchDbDatabase(config: CouchDbConfig, dbName: strin
}
}
export async function couchDbDatabaseExists(config: CouchDbConfig, dbName: string): Promise<boolean> {
const response = await fetch(databaseUrl(config, dbName), {
headers: { authorization: authHeader(config) },
});
if (response.status === 404) {
return false;
}
if (!response.ok) {
throw new Error(`Failed to inspect CouchDB ${dbName}. HTTP ${response.status}: ${await response.text()}`);
}
return true;
}
export async function fetchAllCouchDbDocs(config: CouchDbConfig, dbName: string): Promise<CouchDbAllDocsResponse> {
const response = await fetch(databaseUrl(config, dbName, "/_all_docs?include_docs=true"), {
headers: { authorization: authHeader(config) },
@@ -10,6 +10,7 @@ vi.mock("./cli.ts", () => ({ evalObsidianJson }));
import {
assertE2eCompatibilityMarker,
createE2eCouchDbPluginData,
waitForLiveSyncCoreReady,
type CompatibilityMarkerState,
} from "./liveSyncWorkflow.ts";
@@ -54,3 +55,25 @@ describe("configured CouchDB fixture", () => {
expect(pluginData.activeConfigurationId).toBe(Object.keys(remoteConfigurations ?? {})[0]);
});
});
describe("Real Obsidian core readiness", () => {
it("retries while the plug-in core is temporarily unavailable during reload", async () => {
evalObsidianJson.mockReset();
evalObsidianJson
.mockRejectedValueOnce(new Error("Cannot read properties of undefined (reading 'core')"))
.mockResolvedValueOnce({
databaseReady: true,
appReady: true,
configured: true,
remoteType: "",
settingVersion: 10,
suspended: false,
});
await expect(waitForLiveSyncCoreReady("obsidian-cli", {}, 1000)).resolves.toMatchObject({
databaseReady: true,
appReady: true,
});
expect(evalObsidianJson).toHaveBeenCalledTimes(2);
});
});
+38 -19
View File
@@ -353,31 +353,50 @@ export async function waitForLiveSyncCoreReady(
): Promise<CoreReadiness> {
const deadline = Date.now() + timeoutMs;
let lastReadiness: CoreReadiness | undefined;
let lastError: unknown;
while (Date.now() < deadline) {
lastReadiness = await evalObsidianJson<CoreReadiness>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const settings=core.services.setting.currentSettings();",
"return JSON.stringify({",
"databaseReady:core.services.database.isDatabaseReady(),",
"appReady:core.services.appLifecycle.isReady(),",
"configured:settings?.isConfigured===true,",
"remoteType:settings?.remoteType??'',",
"settingVersion:settings?.settingVersion,",
"suspended:core.services.appLifecycle.isSuspended(),",
"});",
"})()",
].join(""),
env
);
try {
lastReadiness = await evalObsidianJson<CoreReadiness>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync']?.core;",
"if(!core) return JSON.stringify({databaseReady:false,appReady:false});",
"const settings=core.services.setting.currentSettings();",
"return JSON.stringify({",
"databaseReady:core.services.database.isDatabaseReady(),",
"appReady:core.services.appLifecycle.isReady(),",
"configured:settings?.isConfigured===true,",
"remoteType:settings?.remoteType??'',",
"settingVersion:settings?.settingVersion,",
"suspended:core.services.appLifecycle.isSuspended(),",
"});",
"})()",
].join(""),
env
);
lastError = undefined;
} catch (error) {
// Obsidian reloads the renderer while enabling the plug-in. During
// that short window the CLI can reach the Vault before the plug-in
// catalogue has exposed its core. This is a readiness state, not a
// failed scenario, so retain the error for the eventual timeout.
lastError = error;
await new Promise((resolve) => setTimeout(resolve, 500));
continue;
}
if (lastReadiness.databaseReady && lastReadiness.appReady) {
return lastReadiness;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Self-hosted LiveSync core readiness: ${JSON.stringify(lastReadiness)}`);
const errorSuffix =
lastError === undefined
? ""
: ` Last error: ${lastError instanceof Error ? lastError.message : String(lastError)}`;
throw new Error(
`Timed out waiting for Self-hosted LiveSync core readiness: ${JSON.stringify(lastReadiness)}${errorSuffix}`
);
}
/**
@@ -0,0 +1,63 @@
import { describe, expect, it } from "vitest";
import {
SECURITY_SEED_DOCUMENT_ID,
changedSynchronisationParameterFields,
fingerprintSecuritySeed,
replaceSecuritySeed,
requireSecuritySeedDocument,
snapshotSecuritySeedDocument,
} from "./securitySeed.ts";
const seedA = Buffer.alloc(32, 1).toString("base64");
const seedB = Buffer.alloc(32, 2).toString("base64");
describe("Security Seed E2E evidence", () => {
it("reports stable, non-secret fingerprints", () => {
expect(fingerprintSecuritySeed(seedA)).toMatch(/^sha256:[0-9a-f]{16}$/u);
expect(fingerprintSecuritySeed(seedA)).toBe(fingerprintSecuritySeed(seedA));
expect(fingerprintSecuritySeed(seedA)).not.toBe(fingerprintSecuritySeed(seedB));
});
it("redacts the Seed from the machine-readable document snapshot", () => {
const document = requireSecuritySeedDocument({
_id: SECURITY_SEED_DOCUMENT_ID,
_rev: "0-1",
type: "syncinfo",
protocolVersion: 2,
pbkdf2salt: seedA,
});
const snapshot = snapshotSecuritySeedDocument(document);
expect(snapshot).toEqual({
id: SECURITY_SEED_DOCUMENT_ID,
revision: "0-1",
fingerprint: fingerprintSecuritySeed(seedA),
fields: {
type: "syncinfo",
protocolVersion: 2,
},
});
expect(JSON.stringify(snapshot)).not.toContain(seedA);
});
it("replaces only the Seed and identifies later synchronisation-parameter changes", () => {
const before = requireSecuritySeedDocument({
_id: SECURITY_SEED_DOCUMENT_ID,
_rev: "0-1",
type: "syncinfo",
protocolVersion: 2,
pbkdf2salt: seedA,
});
const replaced = replaceSecuritySeed(before, seedB);
const laterRevision = {
...replaced,
_rev: "0-3",
};
expect(before.pbkdf2salt).toBe(seedA);
expect(replaced.pbkdf2salt).toBe(seedB);
expect(changedSynchronisationParameterFields(before, replaced)).toEqual(["pbkdf2salt"]);
expect(changedSynchronisationParameterFields(replaced, laterRevision)).toEqual([]);
});
});
+77
View File
@@ -0,0 +1,77 @@
import { createHash, randomBytes } from "node:crypto";
import type { CouchDbDocument } from "./couchdb.ts";
export const SECURITY_SEED_DOCUMENT_ID = "_local/obsidian_livesync_sync_parameters";
export type SecuritySeedDocument = CouchDbDocument & {
_id: typeof SECURITY_SEED_DOCUMENT_ID;
_rev: string;
pbkdf2salt: string;
};
export type SecuritySeedDocumentSnapshot = {
id: string;
revision: string;
fingerprint: string;
fields: Record<string, unknown>;
};
function decodeSecuritySeed(seed: string): Buffer {
const bytes = Buffer.from(seed, "base64");
if (seed.length === 0 || bytes.length === 0) {
throw new Error("The Security Seed is empty or is not valid base64.");
}
return bytes;
}
export function createSecuritySeed(): string {
return randomBytes(32).toString("base64");
}
export function fingerprintSecuritySeed(seed: string): string {
const bytes = Uint8Array.from(decodeSecuritySeed(seed));
return `sha256:${createHash("sha256").update(bytes).digest("hex").slice(0, 16)}`;
}
export function requireSecuritySeedDocument(document: CouchDbDocument): SecuritySeedDocument {
if (document._id !== SECURITY_SEED_DOCUMENT_ID) {
throw new Error(`Unexpected synchronisation-parameter document: ${document._id}`);
}
if (typeof document._rev !== "string" || document._rev.length === 0) {
throw new Error("The synchronisation-parameter document does not have a revision.");
}
if (typeof document.pbkdf2salt !== "string") {
throw new Error("The synchronisation-parameter document does not have a Security Seed.");
}
decodeSecuritySeed(document.pbkdf2salt);
return document as SecuritySeedDocument;
}
export function replaceSecuritySeed(document: SecuritySeedDocument, replacementSeed: string): SecuritySeedDocument {
decodeSecuritySeed(replacementSeed);
return {
...document,
pbkdf2salt: replacementSeed,
};
}
export function snapshotSecuritySeedDocument(document: SecuritySeedDocument): SecuritySeedDocumentSnapshot {
const { _id, _rev, pbkdf2salt, ...fields } = document;
return {
id: _id,
revision: _rev,
fingerprint: fingerprintSecuritySeed(pbkdf2salt),
fields,
};
}
export function changedSynchronisationParameterFields(
before: SecuritySeedDocument,
after: SecuritySeedDocument
): string[] {
const ignoredFields = new Set(["_rev"]);
return [...new Set([...Object.keys(before), ...Object.keys(after)])]
.filter((key) => !ignoredFields.has(key))
.filter((key) => JSON.stringify(before[key]) !== JSON.stringify(after[key]))
.sort();
}
+65 -15
View File
@@ -194,6 +194,28 @@ async function verifyRemoteSizeNoticeAndDialogue(): Promise<{
.filter({ hasText: "Synchronisation paused for compatibility review" }),
});
await compatibilityReview.waitFor({ state: "visible", timeout: uiTimeoutMs });
const message = compatibilityReview.locator(".vpk-action-dialog__message");
const textSelection = await message.evaluate((element) => {
const selection = window.getSelection();
const range = document.createRange();
range.selectNodeContents(element);
selection?.removeAllRanges();
selection?.addRange(range);
const selectedText = selection?.toString() ?? "";
selection?.removeAllRanges();
return {
selectedText,
userSelect: getComputedStyle(element).userSelect,
};
});
if (
textSelection.userSelect !== "text" ||
!textSelection.selectedText.includes("Remote synchronisation is paused on this device")
) {
throw new Error(
`Expected the action dialogue message to be selectable, received user-select=${textSelection.userSelect} and selected text '${textSelection.selectedText}'.`
);
}
const actions = compatibilityReview.locator(".vpk-action-dialog__actions--vertical");
await actions.waitFor({ state: "visible", timeout: uiTimeoutMs });
const flexDirection = await actions.evaluate((element) => getComputedStyle(element).flexDirection);
@@ -286,12 +308,32 @@ async function verifyRemoteSelectionDialogue(mode: DialogueMode): Promise<string
for (const label of ["CouchDB", "S3-compatible Object Storage", "Peer-to-Peer (P2P)"]) {
await dialogue.getByText(label, { exact: true }).waitFor({ state: "visible", timeout: uiTimeoutMs });
}
await dialogue
.getByText(
"No central data-storage server is required, but a signalling relay is required for peer discovery.",
{ exact: false }
)
.waitFor({ state: "visible", timeout: uiTimeoutMs });
const p2pDescription = dialogue.getByText(
"No central data-storage server is required, but a signalling relay is required for peer discovery.",
{ exact: false }
);
await p2pDescription.waitFor({ state: "visible", timeout: uiTimeoutMs });
const textSelection = await p2pDescription.evaluate((element) => {
const selection = window.getSelection();
const range = document.createRange();
range.selectNodeContents(element);
selection?.removeAllRanges();
selection?.addRange(range);
const selectedText = selection?.toString() ?? "";
selection?.removeAllRanges();
return {
selectedText,
userSelect: getComputedStyle(element).userSelect,
};
});
if (
textSelection.userSelect !== "text" ||
!textSelection.selectedText.includes("signalling relay is required for peer discovery")
) {
throw new Error(
`Expected Svelte dialogue prose to be selectable, received user-select=${textSelection.userSelect} and selected text '${textSelection.selectedText}'.`
);
}
await modal
.getByRole("button", { name: "No, please take me back" })
.waitFor({ state: "visible", timeout: uiTimeoutMs });
@@ -722,7 +764,7 @@ async function verifyHatchSurfacesAndSafeActions(): Promise<string> {
await liveSyncSettings.locator('.sls-setting-menu-btn[title="Hatch"]').click({ timeout: uiTimeoutMs });
for (const label of [
"Write logs into the file",
"Recreate missing chunks for all files",
"Recreate chunks for current Vault files",
"Verify and repair all files",
]) {
await liveSyncSettings.locator(".setting-item-name", { hasText: label }).waitFor({
@@ -730,7 +772,7 @@ async function verifyHatchSurfacesAndSafeActions(): Promise<string> {
timeout: uiTimeoutMs,
});
}
await liveSyncSettings.getByRole("button", { name: "Recreate all", exact: true }).waitFor({
await liveSyncSettings.getByRole("button", { name: "Recreate current chunks", exact: true }).waitFor({
state: "visible",
timeout: uiTimeoutMs,
});
@@ -739,7 +781,7 @@ async function verifyHatchSurfacesAndSafeActions(): Promise<string> {
timeout: uiTimeoutMs,
});
await liveSyncSettings
.locator(".setting-item-name", { hasText: "Recreate missing chunks for all files" })
.locator(".setting-item-name", { hasText: "Recreate chunks for current Vault files" })
.scrollIntoViewIfNeeded();
return liveSyncSettings;
}
@@ -831,9 +873,13 @@ async function verifyHatchSurfacesAndSafeActions(): Promise<string> {
}
};
}, repairRunStateKey);
await liveSyncSettings.getByRole("button", { name: "Recreate all", exact: true }).click({
timeout: uiTimeoutMs,
});
await page
.locator(".sls-setting:visible")
.last()
.getByRole("button", { name: "Recreate current chunks", exact: true })
.click({
timeout: uiTimeoutMs,
});
await page.waitForFunction(
(stateKey) =>
(globalThis as unknown as Record<string, DialogueRunState | undefined>)[stateKey]?.done === true,
@@ -848,9 +894,13 @@ async function verifyHatchSurfacesAndSafeActions(): Promise<string> {
throw new Error(`Recreate missing chunks failed: ${repairState.error}`);
}
await liveSyncSettings.getByRole("button", { name: "Verify all", exact: true }).click({
timeout: uiTimeoutMs,
});
await page
.locator(".sls-setting:visible")
.last()
.getByRole("button", { name: "Verify all", exact: true })
.click({
timeout: uiTimeoutMs,
});
await page
.locator(".notice")
.filter({ hasText: /^done$/u })
+1
View File
@@ -15,6 +15,7 @@ const testSteps: Step[] = [
{ name: "smoke", args: ["run", "test:e2e:obsidian:smoke"] },
{ name: "onboarding invitation", args: ["run", "test:e2e:obsidian:onboarding-invitation"] },
{ name: "Svelte dialogue mounts", args: ["run", "test:e2e:obsidian:dialog-mounts"] },
{ name: "revision repair", args: ["run", "test:e2e:obsidian:revision-repair"] },
{ name: "settings UI", args: ["run", "test:e2e:obsidian:settings-ui"] },
{ name: "Review Harness", args: ["run", "test:e2e:obsidian:review-harness"] },
{ name: "P2P status pane", args: ["run", "test:e2e:obsidian:p2p-pane"] },
@@ -26,7 +26,10 @@ type UnconfiguredStartupEvidence = {
};
type ObsidianTestApp = {
commands?: { executeCommandById(commandId: string): boolean };
setting?: {
open(): void;
openTabById(tabId: string): void;
};
};
type ObsidianTestGlobal = typeof globalThis & { app?: ObsidianTestApp };
@@ -151,14 +154,38 @@ async function captureAndCloseIntro(filename: string, mobile: boolean): Promise<
return screenshot;
}
async function openPermanentCommand(): Promise<void> {
const opened = await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
return await page.evaluate(
(commandId) => (globalThis as ObsidianTestGlobal).app?.commands?.executeCommandById(commandId) === true,
"obsidian-livesync:livesync-open-onboarding"
);
async function openOnboardingFromSettings(): Promise<void> {
await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
await page.evaluate(() => {
const setting = (globalThis as ObsidianTestGlobal).app?.setting;
if (setting === undefined) throw new Error("Obsidian settings are unavailable");
setting.open();
setting.openTabById("obsidian-livesync");
});
const liveSyncSettings = page.locator(".sls-setting");
await liveSyncSettings.waitFor({ state: "visible", timeout: uiTimeoutMs });
await liveSyncSettings.locator('.sls-setting-menu-btn[title="Setup"]').click({ timeout: uiTimeoutMs });
const onboardingSetting = liveSyncSettings.locator(".setting-item").filter({
has: page.locator(".setting-item-name").filter({ hasText: "Rerun Onboarding Wizard" }),
});
await onboardingSetting.waitFor({ state: "visible", timeout: uiTimeoutMs });
await onboardingSetting
.getByRole("button", { name: "Rerun Wizard", exact: true })
.click({ timeout: uiTimeoutMs });
await onboardingDialogue(page).waitFor({ state: "visible", timeout: uiTimeoutMs });
});
}
async function closeSettings(): Promise<void> {
await withObsidianPage(obsidianRemoteDebuggingPort(), async (page) => {
const settingsContainer = page.locator(".modal-container").filter({
has: page.locator(".sls-setting"),
});
await settingsContainer.locator(".modal-close-button").click({ timeout: uiTimeoutMs });
await settingsContainer.waitFor({ state: "hidden", timeout: uiTimeoutMs });
});
if (!opened) throw new Error("The permanent onboarding command was not registered.");
}
async function main(): Promise<void> {
@@ -190,8 +217,9 @@ async function main(): Promise<void> {
console.log(`Fresh Vault startup evidence: ${JSON.stringify(evidence)}`);
const desktopInvitation = await captureDesktopInvitation();
await openPermanentCommand();
const commandIntro = await captureAndCloseIntro("onboarding-intro-command-desktop.png", false);
await openOnboardingFromSettings();
const settingsIntro = await captureAndCloseIntro("onboarding-intro-settings-desktop.png", false);
await closeSettings();
const mobileInvitation = await captureAndSelectMobileInvitation();
const mobileIntro = await captureAndCloseIntro("onboarding-intro-mobile.png", true);
@@ -200,7 +228,7 @@ async function main(): Promise<void> {
desktopInvitation,
mobileInvitation,
mobileIntro,
commandIntro,
settingsIntro,
].join(", ")}`
);
} finally {
@@ -0,0 +1,334 @@
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { evalObsidianJson } from "../runner/cli.ts";
import {
createE2eObsidianDeviceLocalState,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { captureObsidianElement, withObsidianPage } from "../runner/ui.ts";
import { createTemporaryVault } from "../runner/vault.ts";
const path = "revision-repair.md";
const baseContent = "Revision repair\n\nShared base.\n";
const branchContents = [
`Revision repair\n\nLeft branch.\n${"L".repeat(4096)}\n`,
`Revision repair\n\nRight branch.\n${"R".repeat(4096)}\n`,
] as const;
const uiTimeoutMs = Number(process.env.E2E_OBSIDIAN_REVISION_REPAIR_TIMEOUT_MS ?? 15000);
type BrokenRevisionFixture = {
winnerRevision: string;
conflictRevision: string;
missingChunkId: string;
};
type RevisionTree = {
winnerRevision: string;
conflictRevisions: string[];
};
type ObsidianSettingsController = {
open(): void;
openTabById(tabId: string): void;
};
type ObsidianTestGlobal = typeof globalThis & {
app?: {
setting?: ObsidianSettingsController;
};
};
async function createAndOpenBaseFile(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(baseContent)};`,
"let file=app.vault.getAbstractFileByPath(path);",
"if(!file) file=await app.vault.create(path,content);",
"await app.workspace.getLeaf(false).openFile(file);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function createBrokenConflict(
cliBinary: string,
env: NodeJS.ProcessEnv,
baseRevision: string
): Promise<BrokenRevisionFixture> {
return await evalObsidianJson<BrokenRevisionFixture>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const baseRevision=${JSON.stringify(baseRevision)};`,
`const contents=${JSON.stringify(branchContents)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const id=await core.services.path.path2id(path);",
"for(const [index,content] of contents.entries()){",
" const blob=new Blob([content],{type:'text/plain'});",
" const now=Date.now()+index;",
" const result=await core.localDatabase.putDBEntry({",
" _id:id,path,data:blob,ctime:now,mtime:now,",
" size:(await blob.arrayBuffer()).byteLength,children:[],",
" datatype:'plain',type:'plain',eden:{},",
" },false,baseRevision);",
" if(!result?.ok) throw new Error(`Could not create repair conflict: ${path}`);",
"}",
"const tree=await core.localDatabase.localDatabase.get(id,{conflicts:true});",
"const conflictRevision=tree._conflicts?.[0];",
"if(!tree._rev||!conflictRevision){",
" throw new Error(`Repair fixture did not produce two live revisions: ${path}`);",
"}",
"const conflict=await core.localDatabase.localDatabase.get(id,{rev:conflictRevision});",
"const embedded=new Set(Object.keys(conflict.eden??{}));",
"const missingChunkId=(conflict.children??[]).find((child)=>!embedded.has(child));",
"if(!missingChunkId){",
" throw new Error(`Repair fixture did not create an independent chunk: ${conflictRevision}`);",
"}",
"const chunk=await core.localDatabase.localDatabase.get(missingChunkId);",
"await core.localDatabase.localDatabase.remove(chunk);",
"core.localDatabase.clearCaches();",
"const unreadable=await core.localDatabase.getDBEntry(path,{rev:conflictRevision},false,true,true);",
"if(unreadable!==false){",
" throw new Error(`The selected revision remained readable after its chunk was removed: ${conflictRevision}`);",
"}",
"return JSON.stringify({",
" winnerRevision:tree._rev,",
" conflictRevision,",
" missingChunkId,",
"});",
"})()",
].join(""),
env
);
}
async function readRevisionTree(cliBinary: string, env: NodeJS.ProcessEnv): Promise<RevisionTree> {
return await evalObsidianJson<RevisionTree>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const id=await core.services.path.path2id(path);",
"const tree=await core.localDatabase.localDatabase.get(id,{conflicts:true});",
"return JSON.stringify({",
" winnerRevision:tree._rev,",
" conflictRevisions:tree._conflicts??[],",
"});",
"})()",
].join(""),
env
);
}
async function requestConflictCheck(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"core.localDatabase.clearCaches();",
"await core.services.conflict.queueCheckFor(path);",
"await core.services.conflict.ensureAllProcessed();",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const cliBinary = cli.binary;
const vault = await createTemporaryVault("obsidian-livesync-revision-repair-");
let session: ObsidianLiveSyncSession | undefined;
try {
session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
pluginData: {
doctorProcessedVersion: "1.0.0",
isConfigured: true,
liveSync: false,
remoteType: "",
couchDB_URI: "",
couchDB_DBNAME: "",
couchDB_USER: "",
couchDB_PASSWORD: "",
remoteConfigurations: {},
activeConfigurationId: "",
notifyThresholdOfRemoteStorageSize: -1,
periodicReplication: false,
syncAfterMerge: false,
syncOnEditorSave: false,
syncOnFileOpen: false,
syncOnSave: false,
syncOnStart: false,
disableMarkdownAutoMerge: true,
showMergeDialogOnlyOnActive: true,
useEden: false,
},
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
await createAndOpenBaseFile(cliBinary, session.cliEnv);
const base = await waitForLocalDatabaseEntry(cliBinary, session.cliEnv, path);
const fixture = await createBrokenConflict(cliBinary, session.cliEnv, base.rev);
await requestConflictCheck(cliBinary, session.cliEnv);
const afterAutomaticCheck = await readRevisionTree(cliBinary, session.cliEnv);
if (
afterAutomaticCheck.winnerRevision !== fixture.winnerRevision ||
!afterAutomaticCheck.conflictRevisions.includes(fixture.conflictRevision)
) {
throw new Error(
`Automatic conflict checking discarded the unreadable revision: ${JSON.stringify({
fixture,
afterAutomaticCheck,
})}`
);
}
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
await page.evaluate(() => {
const setting = (globalThis as ObsidianTestGlobal).app?.setting;
if (setting === undefined) throw new Error("Obsidian settings are unavailable");
setting.open();
setting.openTabById("obsidian-livesync");
});
const settings = page.locator(".sls-setting");
await settings.waitFor({ state: "visible", timeout: uiTimeoutMs });
await settings.locator('.sls-setting-menu-btn[title="Hatch"]').click({ timeout: uiTimeoutMs });
const verifySetting = settings.locator(".setting-item").filter({
has: page.getByText("Verify and repair all files", { exact: true }),
});
await verifySetting.getByRole("button", { name: "Verify all", exact: true }).click({
timeout: uiTimeoutMs,
});
const card = settings.locator(".sls-repair-result").filter({ hasText: path });
await card.waitFor({ state: "visible", timeout: uiTimeoutMs });
const brokenRevision = card
.locator(".sls-repair-revision")
.filter({ hasText: fixture.conflictRevision });
await brokenRevision
.getByText(/Unreadable on this device/u)
.waitFor({ state: "visible", timeout: uiTimeoutMs });
await brokenRevision.getByText(fixture.missingChunkId, { exact: false }).waitFor({
state: "visible",
timeout: uiTimeoutMs,
});
if ((await card.locator(".sls-repair-revision").count()) !== 2) {
throw new Error("Verify and Repair did not render the winner and conflict revision separately.");
}
await brokenRevision.getByRole("button", { name: "Retry reading revision", exact: true }).click({
timeout: uiTimeoutMs,
});
await settings
.locator(".sls-repair-result")
.filter({ hasText: path })
.locator(".sls-repair-revision")
.filter({ hasText: fixture.conflictRevision })
.getByText(/Unreadable on this device/u)
.waitFor({ state: "visible", timeout: uiTimeoutMs });
});
const afterRetry = await readRevisionTree(cliBinary, session.cliEnv);
if (!afterRetry.conflictRevisions.includes(fixture.conflictRevision)) {
throw new Error(`Retry changed the revision tree: ${JSON.stringify(afterRetry)}`);
}
const screenshot = await captureObsidianElement(
session.remoteDebuggingPort,
"revision-repair-unreadable-conflict.png",
(page) => page.locator(".sls-repair-result").filter({ hasText: path })
);
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
const settings = page.locator(".sls-setting");
const brokenRevision = () =>
settings
.locator(".sls-repair-result")
.filter({ hasText: path })
.locator(".sls-repair-revision")
.filter({ hasText: fixture.conflictRevision });
await brokenRevision()
.getByRole("button", { name: "Discard unreadable revision", exact: true })
.click({ timeout: uiTimeoutMs });
const confirmation = page.locator(".modal-container").filter({
has: page.locator(".modal-title").filter({ hasText: "Discard unreadable revision" }),
});
await confirmation.waitFor({ state: "visible", timeout: uiTimeoutMs });
await confirmation.getByRole("button", { name: "No", exact: true }).click({ timeout: uiTimeoutMs });
await confirmation.waitFor({ state: "hidden", timeout: uiTimeoutMs });
});
const afterCancellation = await readRevisionTree(cliBinary, session.cliEnv);
if (!afterCancellation.conflictRevisions.includes(fixture.conflictRevision)) {
throw new Error(`Cancelling discard changed the revision tree: ${JSON.stringify(afterCancellation)}`);
}
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
const settings = page.locator(".sls-setting");
const brokenRevision = settings
.locator(".sls-repair-result")
.filter({ hasText: path })
.locator(".sls-repair-revision")
.filter({ hasText: fixture.conflictRevision });
await brokenRevision
.getByRole("button", { name: "Discard unreadable revision", exact: true })
.click({ timeout: uiTimeoutMs });
const confirmation = page.locator(".modal-container").filter({
has: page.locator(".modal-title").filter({ hasText: "Discard unreadable revision" }),
});
await confirmation.waitFor({ state: "visible", timeout: uiTimeoutMs });
await confirmation.getByRole("button", { name: "Yes", exact: true }).click({ timeout: uiTimeoutMs });
await settings
.locator(".sls-repair-revision")
.filter({ hasText: fixture.conflictRevision })
.waitFor({ state: "hidden", timeout: uiTimeoutMs });
});
const afterDiscard = await readRevisionTree(cliBinary, session.cliEnv);
if (
afterDiscard.winnerRevision !== fixture.winnerRevision ||
afterDiscard.conflictRevisions.length !== 0
) {
throw new Error(
`Explicit discard did not remove only the selected unreadable revision: ${JSON.stringify({
fixture,
afterDiscard,
})}`
);
}
console.log(
"Real Obsidian kept an unreadable conflict revision through automatic checking and retry, rendered every live revision separately, required confirmation, and discarded only the selected revision."
);
console.log(`Repair screenshot: ${screenshot}`);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+2
View File
@@ -8,6 +8,7 @@ const focusedScenarios = new Set([
"smoke",
"onboarding-invitation",
"dialog-mounts",
"revision-repair",
"settings-ui",
"review-harness",
"p2p-pane",
@@ -21,6 +22,7 @@ const focusedScenarios = new Set([
"startup-scan",
"setup-uri-workflow",
"two-vault-sync",
"security-seed-reconnect",
"hidden-file-snippet-sync",
"customisation-sync",
"setting-markdown-export",
@@ -0,0 +1,745 @@
import { execFileSync } from "node:child_process";
import { createHash, randomUUID } from "node:crypto";
import { access, mkdir, readFile, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
couchDbDatabaseExists,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchAllCouchDbDocs,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
putCouchDbDocument,
waitForCouchDbDocs,
type CouchDbConfig,
type CouchDbDocument,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
createE2eCouchDbPluginData,
createE2eObsidianDeviceLocalState,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import {
SECURITY_SEED_DOCUMENT_ID,
changedSynchronisationParameterFields,
createSecuritySeed,
fingerprintSecuritySeed,
replaceSecuritySeed,
requireSecuritySeedDocument,
snapshotSecuritySeedDocument,
type SecuritySeedDocument,
type SecuritySeedDocumentSnapshot,
} from "../runner/securitySeed.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { captureObsidianPage } from "../runner/ui.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ??= "15000";
const outboundPath = "E2E/security-seed/device-a.md";
const returnPath = "E2E/security-seed/device-b.md";
const hkdfErrorMessages = [
"Encryption with HKDF failed",
"Decryption with HKDF failed",
"Failed to initialise the encryption key",
"Failed to obtain PBKDF2 salt",
] as const;
type RunnerContext = {
binary: string;
cliBinary: string;
artifactRoot: string;
couchDb: CouchDbConfig;
dbName: string;
activeSessions: Set<ObsidianLiveSyncSession>;
allSessions: ObsidianLiveSyncSession[];
screenshots: string[];
};
type DeviceLabel = "device-a" | "device-a-return" | "device-b";
type SourceEvidence = {
exactCommit: string;
revisionSource: "git-worktree" | "provided-artifact";
workingTreeClean: boolean | null;
pluginVersion: string;
pluginArtifactSha256: string;
};
type ReplicationSettingsState = {
liveSync: boolean;
syncOnStart: boolean;
syncOnSave: boolean;
periodicReplication: boolean;
syncOnFileOpen: boolean;
syncOnEditorSave: boolean;
};
type SessionHealth = {
matchingErrorMessages: string[];
};
type ScenarioEvidence = {
source: SourceEvidence;
securitySeed: {
initial: SecuritySeedDocumentSnapshot;
replacement: SecuritySeedDocumentSnapshot;
final: SecuritySeedDocumentSnapshot;
cachedBeforeReplacement: string;
cachedAfterRemoteReplacement: string;
cachedAfterReplication: string;
replacementChangedFields: string[];
finalChangedFields: string[];
};
synchronisation: {
deviceAToDeviceB: boolean;
deviceBToDeviceA: boolean;
deviceAEncryptedPayload: boolean;
deviceBEncryptedPayload: boolean;
};
health: {
deviceA: SessionHealth;
deviceB: SessionHealth;
};
screenshots: string[];
};
type TeardownEvidence = {
sessionsStopped: boolean;
vaultRemoved: boolean;
profileRemoved: boolean;
databaseRemoved: boolean;
remainingTrackedSessions: number;
};
class MultipleErrors extends Error {
readonly errors: unknown[];
constructor(message: string, errors: unknown[]) {
super(message);
this.name = "MultipleErrors";
this.errors = errors;
}
}
function assertEqual(actual: unknown, expected: unknown, message: string): void {
if (actual !== expected) {
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
}
}
function inspectGitRevision(
artifactRoot: string
): Pick<SourceEvidence, "exactCommit" | "revisionSource" | "workingTreeClean"> {
try {
const exactCommit = execFileSync("git", ["-C", artifactRoot, "rev-parse", "HEAD"], {
encoding: "utf-8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
const status = execFileSync("git", ["-C", artifactRoot, "status", "--porcelain"], {
encoding: "utf-8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
return {
exactCommit,
revisionSource: "git-worktree",
workingTreeClean: status.length === 0,
};
} catch {
const exactCommit = process.env.E2E_OBSIDIAN_ARTIFACT_REVISION?.trim();
if (!exactCommit) {
throw new Error(
"E2E_OBSIDIAN_ARTIFACT_REVISION is required when the plug-in artefact is not in a Git worktree."
);
}
return {
exactCommit,
revisionSource: "provided-artifact",
workingTreeClean: null,
};
}
}
async function inspectSourceEvidence(artifactRoot: string): Promise<SourceEvidence> {
const manifest = JSON.parse(await readFile(join(artifactRoot, "manifest.json"), "utf-8")) as {
version?: unknown;
};
if (typeof manifest.version !== "string" || manifest.version.length === 0) {
throw new Error("The plug-in manifest does not have a version.");
}
const mainJs = await readFile(join(artifactRoot, "main.js"));
return {
...inspectGitRevision(artifactRoot),
pluginVersion: manifest.version,
pluginArtifactSha256: createHash("sha256").update(Uint8Array.from(mainJs)).digest("hex"),
};
}
function e2eeSettings(passphrase: string): Record<string, unknown> {
return {
encrypt: true,
passphrase,
usePathObfuscation: true,
E2EEAlgorithm: "v2",
};
}
async function captureStage(context: RunnerContext, session: ObsidianLiveSyncSession, filename: string): Promise<void> {
const screenshot = await captureObsidianPage(session.remoteDebuggingPort, filename, async () => undefined);
context.screenshots.push(screenshot);
console.log(`Security Seed E2E screenshot: ${screenshot}`);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
passphrase: string,
deviceLabel: DeviceLabel
): Promise<ObsidianLiveSyncSession> {
const couchDbSettings = {
uri: context.couchDb.uri,
username: context.couchDb.username,
password: context.couchDb.password,
dbName: context.dbName,
};
const overrides = e2eeSettings(passphrase);
const session = await startObsidianLiveSyncSession({
binary: context.binary,
cliBinary: context.cliBinary,
artifactRoot: context.artifactRoot,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
pluginData: createE2eCouchDbPluginData(couchDbSettings, overrides),
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
context.activeSessions.add(session);
context.allSessions.push(session);
try {
await captureStage(context, session, `security-seed-${deviceLabel}-startup.png`);
await waitForLiveSyncCoreReady(context.cliBinary, session.cliEnv);
await prepareRemote(context.cliBinary, session.cliEnv);
await captureStage(context, session, `security-seed-${deviceLabel}-configured.png`);
return session;
} catch (error) {
await captureStage(context, session, `security-seed-${deviceLabel}-setup-failure.png`).catch(() => undefined);
await stopTrackedSession(context, session);
throw error;
}
}
async function stopTrackedSession(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
if (!context.activeSessions.has(session)) {
return;
}
await session.app.stop();
context.activeSessions.delete(session);
}
async function stopTrackedSessions(context: RunnerContext): Promise<void> {
const errors: unknown[] = [];
for (const session of [...context.activeSessions]) {
try {
await stopTrackedSession(context, session);
} catch (error) {
errors.push(error);
}
}
if (errors.length > 0) {
throw new MultipleErrors("Could not stop every Real Obsidian session.", errors);
}
}
async function pauseAutomaticReplication(cliBinary: string, env: NodeJS.ProcessEnv): Promise<ReplicationSettingsState> {
const state = await evalObsidianJson<ReplicationSettingsState>(
cliBinary,
[
"(()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"core.services.replicator.getActiveReplicator()?.closeReplication();",
"const settings=core.services.setting.currentSettings();",
"return JSON.stringify({",
"liveSync:Boolean(settings.liveSync),",
"syncOnStart:Boolean(settings.syncOnStart),",
"syncOnSave:Boolean(settings.syncOnSave),",
"periodicReplication:Boolean(settings.periodicReplication),",
"syncOnFileOpen:Boolean(settings.syncOnFileOpen),",
"syncOnEditorSave:Boolean(settings.syncOnEditorSave),",
"});",
"})()",
].join(""),
env
);
for (const [name, enabled] of Object.entries(state)) {
if (enabled) {
throw new Error(`Automatic replication remained enabled through ${name}.`);
}
}
return state;
}
async function cachedSecuritySeedFingerprint(cliBinary: string, env: NodeJS.ProcessEnv): Promise<string> {
const result = await evalObsidianJson<{ fingerprint: string }>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const settings=core.services.setting.currentSettings();",
"const replicator=core.services.replicator.getActiveReplicator();",
"const seed=await replicator.getReplicationPBKDF2Salt(settings,false);",
"const digest=await crypto.subtle.digest('SHA-256',seed);",
"const fingerprint='sha256:'+Array.from(new Uint8Array(digest))",
".map((value)=>value.toString(16).padStart(2,'0')).join('').slice(0,16);",
"return JSON.stringify({fingerprint});",
"})()",
].join(""),
env
);
return result.fingerprint;
}
async function writeNoteViaObsidian(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
content: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(content)};`,
"const folder=path.split('/').slice(0,-1).join('/');",
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.modify(existing,content);",
"else await app.vault.create(path,content);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function openNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const file=app.vault.getAbstractFileByPath(path);",
"if(!file) throw new Error(`Could not find note to open: ${path}`);",
"await app.workspace.getLeaf(false).openFile(file);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function pathExists(path: string): Promise<boolean> {
try {
await access(path);
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
async function waitForPathContent(
vaultPath: string,
path: string,
expected: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 15000)
): Promise<void> {
const fullPath = join(vaultPath, path);
const deadline = Date.now() + timeoutMs;
let lastContent = "";
while (Date.now() < deadline) {
if (await pathExists(fullPath)) {
lastContent = await readFile(fullPath, "utf-8");
if (lastContent === expected) {
return;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}. Last content:\n${lastContent}`);
}
function remoteContainsEntry(documents: CouchDbDocument[], entry: LocalDatabaseEntry): boolean {
const ids = new Set(documents.map((document) => document._id));
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
}
async function assertEntryNotRemote(context: RunnerContext, entry: LocalDatabaseEntry): Promise<void> {
const response = await fetchAllCouchDbDocs(context.couchDb, context.dbName);
const documents = response.rows.flatMap((row) => (row.doc ? [row.doc] : []));
if (remoteContainsEntry(documents, entry)) {
throw new Error("The pending device-A document reached CouchDB before the Security Seed replacement.");
}
}
async function waitForEncryptedRemoteEntry(context: RunnerContext, entry: LocalDatabaseEntry): Promise<boolean> {
const documents = await waitForCouchDbDocs(context.couchDb, context.dbName, (docs) =>
remoteContainsEntry(docs, entry)
);
const byId = new Map(documents.map((document) => [document._id, document]));
const encrypted = entry.children.every((childId) => {
const data = byId.get(childId)?.data;
return typeof data === "string" && data.startsWith("%=");
});
if (!encrypted) {
throw new Error("A replicated chunk did not use the expected HKDF-encrypted payload format.");
}
return true;
}
async function inspectSessionHealth(cliBinary: string, env: NodeJS.ProcessEnv): Promise<SessionHealth> {
return await evalObsidianJson<SessionHealth>(
cliBinary,
[
"(async()=>{",
`const patterns=${JSON.stringify(hkdfErrorMessages)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"await core.services.API.showWindow('log-log');",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let text='';",
"for(let i=0;i<20;i++){",
"text=Array.from(document.querySelectorAll('.logpane .log pre'))",
".map((element)=>element.textContent??'').join('\\n');",
"if(text.length>0) break;",
"await sleep(50);",
"}",
"const unresolved=JSON.stringify((await core.services.appLifecycle.getUnresolvedMessages()).flat());",
"const matchingErrorMessages=patterns.filter((pattern)=>text.includes(pattern)||unresolved.includes(pattern));",
"for(const leaf of app.workspace.getLeavesOfType('log-log')) leaf.detach();",
"return JSON.stringify({matchingErrorMessages});",
"})()",
].join(""),
env
);
}
async function fetchSecuritySeedDocument(context: RunnerContext): Promise<SecuritySeedDocument> {
return requireSecuritySeedDocument(
await fetchCouchDbDocument(context.couchDb, context.dbName, SECURITY_SEED_DOCUMENT_ID)
);
}
async function replaceRemoteSecuritySeed(
context: RunnerContext,
before: SecuritySeedDocument,
replacementSeed: string
): Promise<SecuritySeedDocument> {
const replacement = replaceSecuritySeed(before, replacementSeed);
const putResult = await putCouchDbDocument(context.couchDb, context.dbName, replacement);
const after = await fetchSecuritySeedDocument(context);
assertEqual(after._rev, putResult.rev, "The replacement Security Seed revision was not stored.");
assertEqual(
fingerprintSecuritySeed(after.pbkdf2salt),
fingerprintSecuritySeed(replacementSeed),
"The replacement Security Seed was not stored."
);
const changedFields = changedSynchronisationParameterFields(before, after);
assertEqual(
JSON.stringify(changedFields),
JSON.stringify(["pbkdf2salt"]),
"Replacing the remote Security Seed changed another synchronisation parameter."
);
return after;
}
async function runScenario(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<ScenarioEvidence> {
const passphrase = `security-seed-e2e-${randomUUID()}`;
const source = await inspectSourceEvidence(context.artifactRoot);
let sessionA = await startConfiguredSession(context, vaultA, passphrase, "device-a");
await pushLocalChanges(context.cliBinary, sessionA.cliEnv);
await captureStage(context, sessionA, "security-seed-device-a-initial-sync.png");
const initialDocument = await fetchSecuritySeedDocument(context);
const initial = snapshotSecuritySeedDocument(initialDocument);
const cachedBeforeReplacement = await cachedSecuritySeedFingerprint(context.cliBinary, sessionA.cliEnv);
assertEqual(
cachedBeforeReplacement,
initial.fingerprint,
"Device A did not cache the initial remote Security Seed."
);
await pauseAutomaticReplication(context.cliBinary, sessionA.cliEnv);
const outboundContent = `Encrypted from device A: ${randomUUID()}\n`;
await writeNoteViaObsidian(context.cliBinary, sessionA.cliEnv, outboundPath, outboundContent);
const outboundEntry = await waitForLocalDatabaseEntry(context.cliBinary, sessionA.cliEnv, outboundPath);
await assertEntryNotRemote(context, outboundEntry);
const replacementSeed = createSecuritySeed();
const replacementDocument = await replaceRemoteSecuritySeed(context, initialDocument, replacementSeed);
const replacement = snapshotSecuritySeedDocument(replacementDocument);
await openNoteViaObsidian(context.cliBinary, sessionA.cliEnv, outboundPath);
await captureStage(context, sessionA, "security-seed-device-a-replacement-pending.png");
const cachedAfterRemoteReplacement = await cachedSecuritySeedFingerprint(context.cliBinary, sessionA.cliEnv);
assertEqual(
cachedAfterRemoteReplacement,
initial.fingerprint,
"Device A did not retain the deliberately stale Security Seed before replication."
);
assertEqual(
replacement.fingerprint,
fingerprintSecuritySeed(replacementSeed),
"The runner did not install the intended replacement Security Seed."
);
await pushLocalChanges(context.cliBinary, sessionA.cliEnv);
const cachedAfterReplication = await cachedSecuritySeedFingerprint(context.cliBinary, sessionA.cliEnv);
assertEqual(
cachedAfterReplication,
replacement.fingerprint,
"Device A did not refresh the Security Seed before replication."
);
const deviceAEncryptedPayload = await waitForEncryptedRemoteEntry(context, outboundEntry);
await captureStage(context, sessionA, "security-seed-device-a-refreshed-sync.png");
const deviceAHealthBeforeRestart = await inspectSessionHealth(context.cliBinary, sessionA.cliEnv);
await stopTrackedSession(context, sessionA);
const sessionB = await startConfiguredSession(context, vaultB, passphrase, "device-b");
await pushLocalChanges(context.cliBinary, sessionB.cliEnv);
await waitForPathContent(vaultB.path, outboundPath, outboundContent);
await openNoteViaObsidian(context.cliBinary, sessionB.cliEnv, outboundPath);
await captureStage(context, sessionB, "security-seed-device-b-received.png");
await pauseAutomaticReplication(context.cliBinary, sessionB.cliEnv);
const returnContent = `Encrypted from device B: ${randomUUID()}\n`;
await writeNoteViaObsidian(context.cliBinary, sessionB.cliEnv, returnPath, returnContent);
const returnEntry = await waitForLocalDatabaseEntry(context.cliBinary, sessionB.cliEnv, returnPath);
await pushLocalChanges(context.cliBinary, sessionB.cliEnv);
const deviceBEncryptedPayload = await waitForEncryptedRemoteEntry(context, returnEntry);
const deviceBHealth = await inspectSessionHealth(context.cliBinary, sessionB.cliEnv);
await stopTrackedSession(context, sessionB);
sessionA = await startConfiguredSession(context, vaultA, passphrase, "device-a-return");
await pushLocalChanges(context.cliBinary, sessionA.cliEnv);
await waitForPathContent(vaultA.path, returnPath, returnContent);
await openNoteViaObsidian(context.cliBinary, sessionA.cliEnv, returnPath);
await captureStage(context, sessionA, "security-seed-device-a-return-received.png");
const finalDocument = await fetchSecuritySeedDocument(context);
const final = snapshotSecuritySeedDocument(finalDocument);
assertEqual(
final.fingerprint,
replacement.fingerprint,
"A client rolled the remote Security Seed back after reconnecting."
);
const finalChangedFields = changedSynchronisationParameterFields(replacementDocument, finalDocument);
if (finalChangedFields.length > 0) {
throw new Error(
`A client rewrote unexpected synchronisation-parameter fields: ${finalChangedFields.join(", ")}`
);
}
const deviceAHealthAfterRestart = await inspectSessionHealth(context.cliBinary, sessionA.cliEnv);
const deviceAHealth = {
matchingErrorMessages: [
...new Set([
...deviceAHealthBeforeRestart.matchingErrorMessages,
...deviceAHealthAfterRestart.matchingErrorMessages,
]),
],
};
if (deviceAHealth.matchingErrorMessages.length > 0 || deviceBHealth.matchingErrorMessages.length > 0) {
throw new Error(
`HKDF or Security Seed errors were logged: ${JSON.stringify({
deviceA: deviceAHealth.matchingErrorMessages,
deviceB: deviceBHealth.matchingErrorMessages,
})}`
);
}
return {
source,
securitySeed: {
initial,
replacement,
final,
cachedBeforeReplacement,
cachedAfterRemoteReplacement,
cachedAfterReplication,
replacementChangedFields: changedSynchronisationParameterFields(initialDocument, replacementDocument),
finalChangedFields,
},
synchronisation: {
deviceAToDeviceB: true,
deviceBToDeviceA: true,
deviceAEncryptedPayload,
deviceBEncryptedPayload,
},
health: {
deviceA: deviceAHealth,
deviceB: deviceBHealth,
},
screenshots: [...context.screenshots],
};
}
async function cleanupResources(
context: RunnerContext,
vaults: TemporaryVault[],
databaseCreated: boolean
): Promise<TeardownEvidence> {
const errors: unknown[] = [];
try {
await stopTrackedSessions(context);
} catch (error) {
errors.push(error);
}
for (const vault of vaults) {
try {
await vault.dispose();
} catch (error) {
errors.push(error);
}
}
if (databaseCreated) {
try {
await deleteCouchDbDatabase(context.couchDb, context.dbName);
} catch (error) {
errors.push(error);
}
}
const sessionsStopped = context.allSessions.every(
(session) => session.app.process.exitCode !== null || session.app.process.signalCode !== null
);
const vaultRemoved = (await Promise.all(vaults.map(async (vault) => !(await pathExists(vault.path))))).every(
Boolean
);
const profileRemoved = (await Promise.all(vaults.map(async (vault) => !(await pathExists(vault.statePath))))).every(
Boolean
);
let databaseRemoved = !databaseCreated;
if (databaseCreated) {
try {
databaseRemoved = !(await couchDbDatabaseExists(context.couchDb, context.dbName));
} catch (error) {
errors.push(error);
}
}
const evidence = {
sessionsStopped,
vaultRemoved,
profileRemoved,
databaseRemoved,
remainingTrackedSessions: context.activeSessions.size,
};
if (!sessionsStopped || !vaultRemoved || !profileRemoved || !databaseRemoved || context.activeSessions.size > 0) {
errors.push(new Error(`Security Seed E2E teardown was incomplete: ${JSON.stringify(evidence)}`));
}
if (errors.length > 0) {
throw Object.assign(new MultipleErrors("Security Seed E2E teardown failed.", errors), {
evidence,
});
}
return evidence;
}
async function writeResult(result: unknown): Promise<string> {
const outputDirectory = process.env.E2E_OBSIDIAN_DIAGNOSTICS_DIR ?? "/tmp/obsidian-livesync-e2e";
const resultPath = join(outputDirectory, "security-seed-reconnect-result.json");
await mkdir(outputDirectory, { recursive: true });
await writeFile(resultPath, `${JSON.stringify(result, null, 2)}\n`, "utf-8");
return resultPath;
}
async function main(): Promise<void> {
if (process.env.E2E_OBSIDIAN_KEEP_VAULT === "true" || process.env.E2E_OBSIDIAN_KEEP_COUCHDB === "true") {
throw new Error("The Security Seed reconnect scenario requires strict Vault, profile, and database cleanup.");
}
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const artifactRoot = resolve(process.env.E2E_OBSIDIAN_ARTIFACT_ROOT ?? process.cwd());
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "security-seed-reconnect");
const context: RunnerContext = {
binary,
cliBinary: cli.binary,
artifactRoot,
couchDb,
dbName,
activeSessions: new Set(),
allSessions: [],
screenshots: [],
};
const vaults: TemporaryVault[] = [];
let databaseCreated = false;
let evidence: ScenarioEvidence | undefined;
let scenarioError: unknown;
let teardown: TeardownEvidence | undefined;
let teardownError: unknown;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
databaseCreated = true;
vaults.push(await createTemporaryVault("obsidian-livesync-security-seed-a-"));
vaults.push(await createTemporaryVault("obsidian-livesync-security-seed-b-"));
evidence = await runScenario(context, vaults[0], vaults[1]);
} catch (error) {
scenarioError = error;
} finally {
try {
teardown = await cleanupResources(context, vaults, databaseCreated);
} catch (error) {
teardownError = error;
}
}
if (scenarioError !== undefined || teardownError !== undefined) {
const errors = [scenarioError, teardownError].filter((error) => error !== undefined);
if (errors.length === 1) {
throw errors[0];
}
throw new MultipleErrors("Security Seed reconnect scenario and teardown both failed.", errors);
}
if (!evidence || !teardown) {
throw new Error("Security Seed reconnect evidence was not produced.");
}
const result = {
scenario: "security-seed-reconnect",
...evidence,
teardown,
limitations: {
platformCommonRealObsidian: true,
iPadOsBackgroundReconnect: false,
androidDeviceLifecycle: false,
},
};
const resultPath = await writeResult(result);
console.log(`Security Seed E2E result: ${resultPath}`);
console.log(JSON.stringify(result, null, 2));
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exitCode = 1;
});
+11
View File
@@ -214,6 +214,17 @@ async function verifyEffectiveSettings(): Promise<void> {
throw new Error("The Change Log still contains a compatibility or release-note acknowledgement control.");
}
await liveSyncSettings.locator('.sls-setting-menu-btn[title="Remote Configuration"]').click();
const connectionPanel = liveSyncSettings
.locator("h4.sls-setting-panel-title")
.filter({ hasText: "Connection settings" })
.locator("..");
await connectionPanel.waitFor({ state: "visible", timeout: uiTimeoutMs });
await connectionPanel.getByText("Saved connections", { exact: true }).waitFor({
state: "visible",
timeout: uiTimeoutMs,
});
await liveSyncSettings.locator('.sls-setting-menu-btn[title="Sync Settings"]').click();
const deletionPanel = liveSyncSettings
.locator("h4.sls-setting-panel-title")
+22
View File
@@ -12,6 +12,28 @@ Earlier releases remain available in the 0.25 release history and the legacy rel
## Unreleased
## 1.0.0-beta.4
25th July, 2026
### Improved
- **Verify and repair all files** now reports the database winner, every conflict revision, missing chunks, and unavailable shared ancestors separately. It can retry an exact revision without changing the tree, while discarding an unreadable live revision requires explicit confirmation.
- Command-palette actions now use clearer names and appear only when their feature and current context make them usable. Renamed commands keep their identifiers, so hotkeys already assigned to them continue to work. The onboarding wizard can be reopened from **Self-hosted LiveSync settings****Setup**.
- Text in setup and review dialogues can now be selected for copying or translation.
- When LiveSync adopts an available interface translation on first start-up, it now continues initialisation and leaves a persistent Notice from which the translation details can be opened, instead of waiting for an unsolicited dialogue.
### Fixed
- An unreadable conflict revision is no longer deleted automatically merely because its chunks are unavailable on the current device.
- Garbage Collection V3 now protects chunks required by every live conflict branch and the available revision ancestry needed to review and merge conflicts, instead of considering only the database winner. The action is offered only for CouchDB because P2P has no central database to compact and does not provide the device inventory required by the workflow. Collection now stops when device progress cannot be verified, and a compaction timeout is no longer followed by a contradictory success message.
### Testing
- Added regressions for revision repair, command availability, selectable dialogues, conflict-aware chunk reachability, device-progress safeguards, and compaction timeouts.
- Added a real CouchDB integration test for logical chunk deletion, shared and conflict chunk retention, compaction completion, downstream replication, and content-addressed chunk recreation.
- Added a real Obsidian encrypted reconnect scenario which replaces the remote Security Seed while one client retains the previous value, verifies that synchronisation refreshes it without restoring the old value, and proves a bidirectional encrypted round-trip.
## 1.0.0-beta.3
24th July, 2026
+2 -1
View File
@@ -9,5 +9,6 @@
"1.0.0-beta.0": "1.7.2",
"1.0.0-beta.1": "1.7.2",
"1.0.0-beta.2": "1.7.2",
"1.0.0-beta.3": "1.7.2"
"1.0.0-beta.3": "1.7.2",
"1.0.0-beta.4": "1.7.2"
}