mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-08-12 22:55:46 +00:00
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 vorotamoroz
|
c933674a0d | ||
|
vorotamorozandGitHub
|
eda78dee36 |
@@ -141,6 +141,10 @@ This field stores an array of Chunk Document IDs.
|
||||
|
||||
\_id is generated based on the path of the Obsidian note.
|
||||
|
||||
The validation, quarantine, and explicit repair contract for normal-file
|
||||
Metadata whose stored path does not derive its actual ID is defined in
|
||||
[Normal-file Metadata Document ID Validation and Repair](design_docs/metadata_document_id_validation_and_repair.md).
|
||||
|
||||
- If the path starts with `_`, it is converted to `/_` for convenience.
|
||||
- If Case Sensitive is disabled, it is converted to lowercase.
|
||||
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
# Normal-file Metadata Document ID Validation and Repair
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Problem and scope
|
||||
|
||||
A normal-file Metadata document is addressed by an ID derived from its recorded
|
||||
Vault-relative path. Historical data can contain a readable Metadata document
|
||||
whose actual local database ID no longer matches that derivation. An ordinary
|
||||
path-based read then looks up a different ID. It may reach a separate,
|
||||
consistently addressed Metadata document, or it may find no document at all;
|
||||
it cannot reach the malformed document which the Offline Scanner enumerated.
|
||||
|
||||
The mismatch can repeatedly produce failed reflection, and an offline-deletion
|
||||
decision can be made before that failure. The scanner must therefore recognise
|
||||
the mismatch before any file reflection, database deletion, expired-history
|
||||
cleanup, or last-seen update.
|
||||
|
||||
This design covers ordinary Vault files. Hidden File Sync, Customisation Sync,
|
||||
and the obsolete plug-in storage namespace retain their feature-specific
|
||||
processing. A disagreement between the document-ID namespace and recorded-path
|
||||
namespace is reported, but is not repaired by this workflow.
|
||||
|
||||
## Evidence and cause boundary
|
||||
|
||||
The reported data included readable paths which could be enumerated from
|
||||
Metadata but could not be fetched again through the path-derived lookup. The
|
||||
Vault also had a history of case changes in folder names. This is consistent
|
||||
with an ID/path mismatch, but it does not prove whether a historical rename,
|
||||
interrupted migration, or earlier path-setting change created it.
|
||||
|
||||
The repair workflow must not infer that the current path is authoritative merely
|
||||
because it is readable. It is available only when the local evidence is
|
||||
unambiguous and current.
|
||||
|
||||
## Identity invariant
|
||||
|
||||
For normal-file Metadata:
|
||||
|
||||
actualDocumentId === path2id(declaredPath)
|
||||
|
||||
The active path service owns the derivation. In particular,
|
||||
handleFilenameCaseSensitive, usePathObfuscation, and the path-obfuscation
|
||||
passphrase can change the expected ID. The E2EE Security Seed and Chunk settings
|
||||
do not directly participate in this ID.
|
||||
|
||||
Inspection and repair use the current local path service. They do not query the
|
||||
remote or decide whether this device's settings should become authoritative.
|
||||
Commonlib recalculates the expected ID during its pre-mutation inspection, so a
|
||||
local ID-derivation setting change makes an earlier approval stale. An
|
||||
intentional whole-database change to ID-derivation settings requires the
|
||||
established rebuild workflow, not this one-entry repair.
|
||||
|
||||
## Offline Scanner decision
|
||||
|
||||
The Offline Scanner validates each decoded Metadata document while its actual ID
|
||||
is still available. It does this before target-file policy and path-keyed pair
|
||||
construction.
|
||||
|
||||
- Consistent normal-file Metadata continues through the existing scan.
|
||||
- Consistent special-namespace Metadata remains owned by its feature.
|
||||
- An ID/path or namespace mismatch is left unchanged and does not enter pair
|
||||
processing.
|
||||
- If consistent, selected Metadata represents the same case-normalised path,
|
||||
that Metadata and its storage file continue through the established
|
||||
path-based scan. A stale enumerated document must not suppress this flow.
|
||||
- If no consistent, selected Metadata represents that logical path, its storage
|
||||
entry is also withheld. No storage write, database deletion, or last-seen
|
||||
update is performed for that withheld path.
|
||||
- Expired logical deletion history is not hard-tombstoned while its identity is
|
||||
inconsistent.
|
||||
|
||||
The ordinary scan still returns its established Boolean execution result. A
|
||||
recognised mismatch is a quarantined input, not a new public scan result and not
|
||||
a Fast Setup or CLI policy. Detailed inspection is deliberately separate.
|
||||
|
||||
## Inspection decision
|
||||
|
||||
inspectMetadataDocumentIdentities is read-only and enumerates the local database
|
||||
by actual document ID. This is necessary because a path-first Inspector cannot
|
||||
discover the malformed source.
|
||||
|
||||
The existing **Inspect conflicts and file/database differences** interface shows
|
||||
one card for each mismatch. It excludes that card's path from ordinary
|
||||
path-based repair only when no consistently addressed Metadata document can be
|
||||
resolved for the same logical path. A stale entry does not hide the normal
|
||||
inspection of a resolvable entry. Multiple affected files are presented
|
||||
separately; there is no batch repair.
|
||||
|
||||
A one-entry repair is offered only when all of these checks pass:
|
||||
|
||||
- the mismatch is within the normal-file namespace;
|
||||
- the source is the current live revision and has no conflicts;
|
||||
- the recorded path is valid and selected by current synchronisation policy;
|
||||
- one case-normalised path maps to one source under the active filename setting;
|
||||
- only one malformed source expects the target ID; and
|
||||
- the target ID is absent, or contains an exact structural copy left by an
|
||||
earlier attempt.
|
||||
|
||||
An exact structural copy has the same path, timestamps, size, type, Chunk
|
||||
references, Eden data, and logical-deletion state. Inspection does not fetch
|
||||
Chunk content or query the remote. Missing content remains the responsibility
|
||||
of the existing file and Chunk repair tools.
|
||||
|
||||
## Repair decision
|
||||
|
||||
The user explicitly confirms one actual ID, expected ID, and source revision.
|
||||
Commonlib then:
|
||||
|
||||
1. acquires the existing ordered document locks for the source and target IDs;
|
||||
2. reruns the complete inspection and rejects stale or unsafe input;
|
||||
3. reads the exact approved source revision;
|
||||
4. removes the path from the Offline Scanner's durable last-seen map;
|
||||
5. writes the expected target ID when it is absent;
|
||||
6. reads the target back and verifies the exact structural copy;
|
||||
7. hard-tombstones the source using the approved source revision; and
|
||||
8. returns control to LiveSync, which requests an ordinary Vault scan.
|
||||
|
||||
The repair result and the follow-up scan result remain separate. If the scan is
|
||||
suspended, returns false, or raises an error after the source has been removed,
|
||||
LiveSync reports that the identity repair completed and directs the operator to
|
||||
run the ordinary scan separately. It does not describe the completed mutation
|
||||
as a failed or rolled-back repair.
|
||||
|
||||
The target is always verified before the source is removed. If target creation
|
||||
fails, the source remains. If source removal fails, the exact target remains and
|
||||
the same one-entry action can finish the operation after a new inspection. This
|
||||
retry property is an implementation safety guarantee, not a separate public
|
||||
repair mode.
|
||||
|
||||
The target receives new CouchDB revision ancestry because ancestry cannot move
|
||||
between document IDs. Users are told to back up the device, pause editing and
|
||||
synchronisation on other devices, allow the change to replicate, and inspect
|
||||
again.
|
||||
|
||||
## Case handling
|
||||
|
||||
The active handleFilenameCaseSensitive setting defines whether path claims are
|
||||
folded before ambiguity is assessed. When case-insensitive handling is active,
|
||||
a consistently addressed entry may continue through the existing path-based
|
||||
flow even if a stale case variant is also reported. The stale entry is not
|
||||
automatically selected or removed unless the one-entry repair preconditions
|
||||
hold. When case-sensitive handling is active, intentional variants remain
|
||||
distinct.
|
||||
|
||||
This workflow does not rename Vault files or folders, infer a preferred folder
|
||||
name from one device, or coordinate a repair across devices. The repair changes
|
||||
one local database and relies on ordinary replication afterwards. Other devices
|
||||
must remain paused until that result has replicated and a new inspection is
|
||||
clean.
|
||||
|
||||
For widespread cross-device naming differences, the operator must choose an
|
||||
authoritative Vault, stop every participating device, correct its storage names
|
||||
outside Obsidian, rebuild the central remote from that Vault, and reset the
|
||||
other devices from the verified remote. During Fast Setup on an empty Vault,
|
||||
there are no storage names to correct: the scanner reflects every consistently
|
||||
addressable Metadata entry and reports only the references which remain
|
||||
unresolved.
|
||||
|
||||
## Non-goals
|
||||
|
||||
This change does not:
|
||||
|
||||
- repair several entries automatically or in a batch;
|
||||
- choose between competing case variants;
|
||||
- rename storage files or folders;
|
||||
- coordinate a distributed repair across devices;
|
||||
- migrate an entire database after path-obfuscation or case-setting changes;
|
||||
- repair special-namespace Metadata;
|
||||
- reconstruct unavailable Chunk content;
|
||||
- query or modify the remote directly; or
|
||||
- change Fast Setup, daemon, or CLI completion policy.
|
||||
|
||||
## Verification
|
||||
|
||||
Focused Commonlib tests cover early quarantine, continued processing of a
|
||||
resolvable same-path entry, expired logical-deletion retention, namespace
|
||||
routing, read-only actual-ID inspection, repair preconditions, target-first
|
||||
ordering, stale approval, exact-target retry, source preservation on failure,
|
||||
and last-seen clearing. LiveSync tests cover selective presentation-path
|
||||
withholding, separate confirmation, cancellation, and the ordinary scan request
|
||||
after a completed repair.
|
||||
@@ -54,6 +54,25 @@ The `Hatch` recovery controls are ordered by escalation. Running **Recreate chun
|
||||
|
||||
An absent Vault file and a logical-deletion winner already agree and do not require a repair card unless another live branch remains. If the scan reports many unrelated files, or the local database itself is incomplete or corrupt, stop the per-file workflow and use [Reset synchronisation on this device](#reset-synchronisation-on-this-device) from a trusted remote. If the central remote must instead be reconstructed from an authoritative Vault, use [Overwrite server data with this device's files](#overwrite-server-data-with-this-devices-files).
|
||||
|
||||
Metadata document-ID mismatches use a separate action in the same Inspector. Follow [Repair a Metadata document ID mismatch](#repair-a-metadata-document-id-mismatch) rather than applying a file revision by path.
|
||||
|
||||
## Repair a Metadata document ID mismatch
|
||||
|
||||
Use this workflow when **Inspect conflicts and file/database differences** reports `Metadata entry requires review and was left unchanged`. The Inspector found local Metadata whose stored document ID no longer represents its recorded path. It leaves the entry unchanged, while any consistently addressed Metadata for the same logical path remains available to ordinary inspection and Vault reflection. This inspection does not query the remote.
|
||||
|
||||
1. Back up this device. If other devices share the database, stop editing and pause synchronisation on them.
|
||||
2. Confirm that the current file-name case and path obfuscation settings are intended for this database. If either setting was deliberately changed for the whole database, stop this workflow and use Rebuild instead.
|
||||
3. Open **Self-hosted LiveSync settings** → **Hatch** → **Inspect conflicts and file/database differences**, then select **Begin inspection**.
|
||||
4. Find the affected Metadata card and review its recorded path, stored document ID, expected document ID, and source revision.
|
||||
5. Continue only when the card says `Repair is available for this entry.` Open its wrench menu and select **Repair this Metadata document ID**. If the action is unavailable, do not force an ID: the entry is ambiguous, conflicted, deleted, outside the normal-file namespace, or otherwise unsafe for one-entry repair.
|
||||
6. Review the warning and select **Repair Metadata ID**. LiveSync rechecks the source revision and expected ID, writes and verifies the target, then removes the obsolete ID.
|
||||
7. Wait for the ordinary Vault scan to complete. If LiveSync reports that the repair completed but the scan did not run, keep synchronisation paused, resolve the reported scan condition, then run the **Scan storage and database again** command.
|
||||
8. Allow this device to upload the repair. Resume the other devices one at a time, then run the inspection again and confirm that the Metadata card no longer appears and the Vault file has the intended content.
|
||||
|
||||
This action changes one local database entry. It does not rename Vault files or folders, repair several entries at once, coordinate other devices, or preserve CouchDB revision ancestry across the two document IDs.
|
||||
|
||||
If many entries reflect folder-name differences across devices, stop every device, choose the authoritative Vault, close Obsidian, correct the actual storage names with operating-system tools, then rebuild the central remote from that Vault and reset the other devices. During Fast Setup on an empty Vault, there are no storage names to correct: allow consistently addressable Metadata to be reflected, then inspect any remaining unresolved references.
|
||||
|
||||
## Reset synchronisation on this device
|
||||
|
||||
Use this when the remote copy is trusted but this device's local LiveSync database is incomplete, corrupt, or no longer aligned with it.
|
||||
|
||||
@@ -749,6 +749,8 @@ Compare each Vault file with every current live revision in the local database.
|
||||
|
||||
Select **Begin inspection** to run the inspection. Each reported file and live revision has a wrench menu for read-only comparison, applying an exact database revision to the Vault, recording an exact byte match, preserving the Vault file as a child of a selected branch, retrying chunk retrieval, or explicitly discarding a branch. Destructive actions require confirmation. Follow [Recover a conflicted or mismatched file](recovery.md#recover-a-conflicted-or-mismatched-file) before changing revision history.
|
||||
|
||||
The same inspection also reports local Metadata whose stored document ID does not agree with its recorded path. A stale entry does not suppress ordinary inspection when consistently addressed Metadata can still be resolved for that logical path; otherwise, the unresolved path is excluded from ordinary file-repair actions. When one live, unconflicted entry has an unambiguous target, its wrench menu offers a separately confirmed, one-entry repair. The target is derived from the current local file-name case and path obfuscation settings, then written and verified before the obsolete ID is removed. Ambiguous, conflicted, deleted, excluded, or otherwise unsafe entries remain read-only. This action does not rename Vault files or folders. Follow [Repair a Metadata document ID mismatch](recovery.md#repair-a-metadata-document-id-mismatch) for the complete backup, repair, propagation, and verification procedure. For widespread naming differences across devices, use that guide to choose an authoritative Vault, correct its storage names while Obsidian is closed, rebuild the central remote, and reset the other devices.
|
||||
|
||||
#### Resolve All conflicted files by the newer one
|
||||
|
||||
After confirmation, resolve every conflict by modification time. This logically deletes every version except the newest one. It is a destructive policy choice and cannot recover content which is already unavailable.
|
||||
|
||||
@@ -66,6 +66,14 @@ The repair card uses compact diagnostic rows which remain readable in a narrow m
|
||||
|
||||
`Recreate chunks for current Vault files` uses current Vault content. It cannot recreate unique bytes which exist only in an unreadable historical or conflict revision.
|
||||
|
||||
## A Metadata entry requires review
|
||||
|
||||
When **Inspect conflicts and file/database differences** reports `Metadata entry requires review and was left unchanged`, the local database contains Metadata whose stored document ID does not agree with the ID derived from its recorded path. LiveSync withholds that entry from ordinary file reflection and deletion rather than guessing which identity is intended. The inspection is local and does not query the remote.
|
||||
|
||||
Do not change file-name case handling or path obfuscation merely to make the displayed IDs agree. Follow [Repair a Metadata document ID mismatch](recovery.md#repair-a-metadata-document-id-mismatch) when the card offers **Repair this Metadata document ID**. If no repair action is offered, the entry is ambiguous, conflicted, deleted, outside the normal-file namespace, or otherwise unsafe for one-entry repair. Preserve the evidence and use the wider recovery guidance instead of forcing a target ID.
|
||||
|
||||
If many entries reflect deliberate folder-name or ID-derivation differences across devices, choose an authoritative Vault and use the established Rebuild workflow. A one-entry repair is not a distributed rename or database migration.
|
||||
|
||||
## A configuration mismatch dialogue blocks synchronisation
|
||||
|
||||
Some settings must match across devices. LiveSync pauses synchronisation when the local and remote values differ rather than propagating an unexpected change silently.
|
||||
|
||||
Generated
+4
-4
@@ -23,7 +23,7 @@
|
||||
"@smithy/types": "^4.14.3",
|
||||
"@smithy/util-retry": "^4.4.5",
|
||||
"@vrtmrz/browser-ui-kit": "0.1.0",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.11",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.12",
|
||||
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
|
||||
"@vrtmrz/ui-interactions": "0.1.2",
|
||||
"diff-match-patch": "^1.0.5",
|
||||
@@ -4775,9 +4775,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vrtmrz/livesync-commonlib": {
|
||||
"version": "0.1.11",
|
||||
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.11.tgz",
|
||||
"integrity": "sha512-7hwQSTND9GFdJqIz6UhWN/W6BC0uehtKXjm2q6He2vk4s2xLMnTKwmTqJH5iiQQFVvmMROJUA3eqUm5K2W2/tw==",
|
||||
"version": "0.1.12",
|
||||
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.12.tgz",
|
||||
"integrity": "sha512-SYqUsMcz9241qEUqmnIwVKOPXJB4/FXjXQh8nHPmX004KkV5PUnAm4ClhNwlQzvYfpTI5IHYKej+JIndiQYXmw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.808.0",
|
||||
|
||||
+1
-1
@@ -177,7 +177,7 @@
|
||||
"@smithy/types": "^4.14.3",
|
||||
"@smithy/util-retry": "^4.4.5",
|
||||
"@vrtmrz/browser-ui-kit": "0.1.0",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.11",
|
||||
"@vrtmrz/livesync-commonlib": "0.1.12",
|
||||
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
|
||||
"@vrtmrz/ui-interactions": "0.1.2",
|
||||
"diff-match-patch": "^1.0.5",
|
||||
|
||||
@@ -150,9 +150,38 @@ export const liveSyncProvisionalEnglishMessages = {
|
||||
"Resolve every conflict by modification time? This logically deletes every version except the newest one and cannot recover content which is already unavailable.",
|
||||
"Resolve all conflicts by the newest version": "Resolve all conflicts by the newest version",
|
||||
"Inspect conflicts and file/database differences": "Inspect conflicts and file/database differences",
|
||||
"Scan every Vault file and live local-database revision for conflicts, missing chunks, and differences. Each result provides actions for the exact revision.":
|
||||
"Scan every Vault file and live local-database revision for conflicts, missing chunks, and differences. Each result provides actions for the exact revision.",
|
||||
"Scan Vault files and local-database Metadata for conflicts, missing chunks, identity mismatches, and differences. Each result provides actions for one exact entry or revision.":
|
||||
"Scan Vault files and local-database Metadata for conflicts, missing chunks, identity mismatches, and differences. Each result provides actions for one exact entry or revision.",
|
||||
"Begin inspection": "Begin inspection",
|
||||
"Metadata entry requires review and was left unchanged": "Metadata entry requires review and was left unchanged",
|
||||
"The stored document ID does not match the ID derived from its recorded path.":
|
||||
"The stored document ID does not match the ID derived from its recorded path.",
|
||||
"The stored document ID and recorded path are handled by different synchronisation features.":
|
||||
"The stored document ID and recorded path are handled by different synchronisation features.",
|
||||
"Stored document ID: ${ID}": "Stored document ID: ${ID}",
|
||||
"Expected document ID: ${ID}": "Expected document ID: ${ID}",
|
||||
"Source revision: ${REVISION}": "Source revision: ${REVISION}",
|
||||
"One-step repair is unavailable because this entry is ambiguous, no longer current, or unsafe to change.":
|
||||
"One-step repair is unavailable because this entry is ambiguous, no longer current, or unsafe to change.",
|
||||
"An exact target is already present; repair can remove the obsolete ID.":
|
||||
"An exact target is already present; repair can remove the obsolete ID.",
|
||||
"Repair is available for this entry.": "Repair is available for this entry.",
|
||||
"Repair this Metadata document ID": "Repair this Metadata document ID",
|
||||
"Repair Metadata ID": "Repair Metadata ID",
|
||||
"Keep unchanged": "Keep unchanged",
|
||||
"Repair Metadata document ID": "Repair Metadata document ID",
|
||||
"This moves one local Metadata entry to the ID derived from its recorded path.\n\n**File:** `${FILE}` \n**Source:** `${SOURCE}@${REVISION}` \n**Target:** `${TARGET}`\n\nThe target is verified before the source is removed. Its CouchDB revision ancestry cannot be preserved.\n\n> [!warning] Before repairing\n> - Back up this device.\n> - If file-name case or path obfuscation was intentionally changed for the whole database, use Rebuild instead.\n> - If other devices share this database, pause them, allow this device to upload the repair, then resume them one at a time.":
|
||||
"This moves one local Metadata entry to the ID derived from its recorded path.\n\n**File:** `${FILE}` \n**Source:** `${SOURCE}@${REVISION}` \n**Target:** `${TARGET}`\n\nThe target is verified before the source is removed. Its CouchDB revision ancestry cannot be preserved.\n\n> [!warning] Before repairing\n> - Back up this device.\n> - If file-name case or path obfuscation was intentionally changed for the whole database, use Rebuild instead.\n> - If other devices share this database, pause them, allow this device to upload the repair, then resume them one at a time.",
|
||||
"Metadata document ID repair and the ordinary Vault scan completed. Run this inspection again after synchronisation.":
|
||||
"Metadata document ID repair and the ordinary Vault scan completed. Run this inspection again after synchronisation.",
|
||||
"Metadata document ID repair completed, but the ordinary Vault scan did not run. Keep synchronisation paused, resolve the scan condition, then run 'Scan storage and database again'.":
|
||||
"Metadata document ID repair completed, but the ordinary Vault scan did not run. Keep synchronisation paused, resolve the scan condition, then run 'Scan storage and database again'.",
|
||||
"The inspected state changed. No repair was performed; run inspection again.":
|
||||
"The inspected state changed. No repair was performed; run inspection again.",
|
||||
"Repair stopped after creating the target. The source was retained. Run inspection again before retrying.":
|
||||
"Repair stopped after creating the target. The source was retained. Run inspection again before retrying.",
|
||||
"Repair failed before the source was removed. Run inspection again before retrying.":
|
||||
"Repair failed before the source was removed. Run inspection again before retrying.",
|
||||
"Connection settings": "Connection settings",
|
||||
"Saved connections": "Saved connections",
|
||||
} as const;
|
||||
|
||||
@@ -7,7 +7,7 @@ import {
|
||||
type EntryDoc,
|
||||
type diff_result,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { createBlob, readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import { createBlob, escapeMarkdownValue, readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import { Logger } from "@vrtmrz/livesync-commonlib/compat/common/logger";
|
||||
import { shouldBeIgnored } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
import { Menu, diff_match_patch, setIcon } from "@/deps.ts";
|
||||
@@ -44,6 +44,21 @@ import {
|
||||
getFileRepairRevisionComparison,
|
||||
} from "@/serviceFeatures/fileRepairPresentation.ts";
|
||||
import { ConflictResolveModal } from "@/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts";
|
||||
import {
|
||||
inspectMetadataDocumentIdentities,
|
||||
MetadataDocumentRepairResults,
|
||||
OfflineScanUnresolvedReasons,
|
||||
repairMetadataDocumentIdentity,
|
||||
type MetadataDocumentIdentityIssue,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
import {
|
||||
metadataIdentityPathKey,
|
||||
selectUnresolvedMetadataIdentityEntries,
|
||||
} from "@/serviceFeatures/metadataIdentityInspection.ts";
|
||||
import {
|
||||
executeMetadataIdentityRepair,
|
||||
MetadataIdentityRepairExecutions,
|
||||
} from "@/serviceFeatures/metadataIdentityRepair.ts";
|
||||
export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
|
||||
// const hatchWarn = this.createEl(paneEl, "div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
|
||||
// hatchWarn.addClass("op-warn-info");
|
||||
@@ -178,6 +193,150 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
|
||||
});
|
||||
});
|
||||
};
|
||||
const addMetadataIdentityResult = (entry: MetadataDocumentIdentityIssue) => {
|
||||
const { diagnostic } = entry.inspection;
|
||||
const card = this.createEl(resultArea, "div", { cls: "sls-repair-result" });
|
||||
this.createEl(card, "h6", { text: diagnostic.declaredPath });
|
||||
this.createEl(card, "div", {
|
||||
text: $msg("Metadata entry requires review and was left unchanged"),
|
||||
cls: "sls-repair-status-warning",
|
||||
});
|
||||
this.createEl(card, "div", {
|
||||
text:
|
||||
diagnostic.reason === OfflineScanUnresolvedReasons.DOCUMENT_ID_MISMATCH
|
||||
? $msg("The stored document ID does not match the ID derived from its recorded path.")
|
||||
: $msg(
|
||||
"The stored document ID and recorded path are handled by different synchronisation features."
|
||||
),
|
||||
cls: "sls-repair-metric",
|
||||
});
|
||||
this.createEl(card, "div", {
|
||||
text: $msg("Stored document ID: ${ID}", { ID: diagnostic.actualDocumentId }),
|
||||
cls: "sls-repair-metric",
|
||||
});
|
||||
if (diagnostic.expectedDocumentId !== undefined) {
|
||||
this.createEl(card, "div", {
|
||||
text: $msg("Expected document ID: ${ID}", { ID: diagnostic.expectedDocumentId }),
|
||||
cls: "sls-repair-metric",
|
||||
});
|
||||
}
|
||||
this.createEl(card, "div", {
|
||||
text: $msg("Source revision: ${REVISION}", {
|
||||
REVISION: entry.sourceRevision ?? $msg("Unknown revision"),
|
||||
}),
|
||||
cls: "sls-repair-metric",
|
||||
});
|
||||
if (entry.logicallyDeleted) {
|
||||
this.createEl(card, "div", {
|
||||
text: $msg("🗑️ Logical deletion"),
|
||||
cls: "sls-repair-metric mod-warning",
|
||||
});
|
||||
}
|
||||
if (entry.conflictRevisions.length > 0) {
|
||||
this.createEl(card, "div", {
|
||||
text: $msg("⚠️ Conflicts: ${COUNT}", { COUNT: `${entry.conflictRevisions.length}` }),
|
||||
cls: "sls-repair-metric mod-warning",
|
||||
});
|
||||
}
|
||||
|
||||
if (
|
||||
!entry.repairAvailable ||
|
||||
diagnostic.expectedDocumentId === undefined ||
|
||||
entry.sourceRevision === null
|
||||
) {
|
||||
this.createEl(card, "div", {
|
||||
text: $msg(
|
||||
"One-step repair is unavailable because this entry is ambiguous, no longer current, or unsafe to change."
|
||||
),
|
||||
cls: "sls-repair-metric mod-warning",
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
this.createEl(card, "div", {
|
||||
text: entry.targetAlreadyPresent
|
||||
? $msg("An exact target is already present; repair can remove the obsolete ID.")
|
||||
: $msg("Repair is available for this entry."),
|
||||
cls: "sls-repair-status-ok",
|
||||
});
|
||||
const request = {
|
||||
actualDocumentId: diagnostic.actualDocumentId,
|
||||
expectedDocumentId: diagnostic.expectedDocumentId,
|
||||
sourceRevision: entry.sourceRevision,
|
||||
};
|
||||
const repairAction = $msg("Repair Metadata ID");
|
||||
const keepAction = $msg("Keep unchanged");
|
||||
addActionMenu(card, $msg("More actions for ${FILE}", { FILE: diagnostic.declaredPath }), [
|
||||
{
|
||||
title: $msg("Repair this Metadata document ID"),
|
||||
warning: true,
|
||||
run: async () => {
|
||||
const execution = await executeMetadataIdentityRepair(request, {
|
||||
confirm: async () =>
|
||||
(await this.core.confirm.confirmWithMessage(
|
||||
$msg("Repair Metadata document ID"),
|
||||
$msg(
|
||||
"This moves one local Metadata entry to the ID derived from its recorded path.\n\n**File:** `${FILE}` \n**Source:** `${SOURCE}@${REVISION}` \n**Target:** `${TARGET}`\n\nThe target is verified before the source is removed. Its CouchDB revision ancestry cannot be preserved.\n\n> [!warning] Before repairing\n> - Back up this device.\n> - If file-name case or path obfuscation was intentionally changed for the whole database, use Rebuild instead.\n> - If other devices share this database, pause them, allow this device to upload the repair, then resume them one at a time.",
|
||||
{
|
||||
FILE: escapeMarkdownValue(diagnostic.declaredPath),
|
||||
SOURCE: escapeMarkdownValue(diagnostic.actualDocumentId),
|
||||
REVISION: escapeMarkdownValue(entry.sourceRevision!),
|
||||
TARGET: escapeMarkdownValue(diagnostic.expectedDocumentId!),
|
||||
}
|
||||
),
|
||||
[repairAction, keepAction],
|
||||
keepAction,
|
||||
undefined,
|
||||
"vertical"
|
||||
)) === repairAction,
|
||||
repair: async (repairRequest) =>
|
||||
await repairMetadataDocumentIdentity(this.core, repairRequest),
|
||||
requestOrdinaryScan: async () => await this.services.vault.scanVault(true, false),
|
||||
});
|
||||
|
||||
if (execution.status === MetadataIdentityRepairExecutions.CANCELLED) return;
|
||||
|
||||
const result = execution.result;
|
||||
if (result.message) Logger(result.message, LOG_LEVEL_VERBOSE);
|
||||
if (result.status === MetadataDocumentRepairResults.COMPLETED) {
|
||||
if (execution.scanError !== undefined) {
|
||||
Logger(execution.scanError, LOG_LEVEL_VERBOSE);
|
||||
}
|
||||
resultArea.replaceChildren();
|
||||
this.createEl(resultArea, "div", {
|
||||
text: execution.scanCompleted
|
||||
? $msg(
|
||||
"Metadata document ID repair and the ordinary Vault scan completed. Run this inspection again after synchronisation."
|
||||
)
|
||||
: $msg(
|
||||
"Metadata document ID repair completed, but the ordinary Vault scan did not run. Keep synchronisation paused, resolve the scan condition, then run 'Scan storage and database again'."
|
||||
),
|
||||
cls: execution.scanCompleted
|
||||
? "sls-repair-status-ok"
|
||||
: "sls-repair-metric mod-warning",
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const resultMessage =
|
||||
result.status === MetadataDocumentRepairResults.STALE ||
|
||||
result.status === MetadataDocumentRepairResults.BLOCKED
|
||||
? $msg("The inspected state changed. No repair was performed; run inspection again.")
|
||||
: result.targetCreated
|
||||
? $msg(
|
||||
"Repair stopped after creating the target. The source was retained. Run inspection again before retrying."
|
||||
)
|
||||
: $msg(
|
||||
"Repair failed before the source was removed. Run inspection again before retrying."
|
||||
);
|
||||
this.createEl(card, "div", {
|
||||
text: resultMessage,
|
||||
cls: "sls-repair-metric mod-warning",
|
||||
});
|
||||
},
|
||||
},
|
||||
]);
|
||||
};
|
||||
const findHiddenFile = async (path: string) => {
|
||||
const addOn = this.core.getAddOn<HiddenFileSync>(HiddenFileSync.name);
|
||||
if (!addOn) {
|
||||
@@ -827,7 +986,7 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
|
||||
.setName($msg("Inspect conflicts and file/database differences"))
|
||||
.setDesc(
|
||||
$msg(
|
||||
"Scan every Vault file and live local-database revision for conflicts, missing chunks, and differences. Each result provides actions for the exact revision."
|
||||
"Scan Vault files and local-database Metadata for conflicts, missing chunks, identity mismatches, and differences. Each result provides actions for one exact entry or revision."
|
||||
)
|
||||
)
|
||||
.addButton((button) =>
|
||||
@@ -839,6 +998,13 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
|
||||
resultArea.replaceChildren();
|
||||
Logger("Start inspecting file/database state", LOG_LEVEL_NOTICE, "verify");
|
||||
this.core.localDatabase.clearCaches();
|
||||
const identityEntries = await inspectMetadataDocumentIdentities(this.core);
|
||||
const handleFilenameCaseSensitive = this.core.settings.handleFilenameCaseSensitive;
|
||||
const unresolvedIdentity = selectUnresolvedMetadataIdentityEntries(
|
||||
identityEntries,
|
||||
handleFilenameCaseSensitive
|
||||
);
|
||||
unresolvedIdentity.entries.forEach(addMetadataIdentityResult);
|
||||
const allPaths = await collectFileDatabaseInfoPaths(this.core);
|
||||
let i = 0;
|
||||
const incProc = () => {
|
||||
@@ -853,6 +1019,13 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
|
||||
const semaphore = Semaphore(10);
|
||||
const processes = allPaths.map(async (path) => {
|
||||
try {
|
||||
if (
|
||||
unresolvedIdentity.unresolvedPathKeys.has(
|
||||
metadataIdentityPathKey(path, handleFilenameCaseSensitive)
|
||||
)
|
||||
) {
|
||||
return incProc();
|
||||
}
|
||||
if (shouldBeIgnored(path)) {
|
||||
return incProc();
|
||||
}
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
import type { MetadataDocumentIdentityIssue } from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
|
||||
export function metadataIdentityPathKey(path: string, handleFilenameCaseSensitive: boolean): string {
|
||||
const vaultPath = stripAllPrefixes(path as FilePathWithPrefix);
|
||||
return handleFilenameCaseSensitive ? vaultPath : vaultPath.toLowerCase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Select unresolved identity evidence for read-only presentation and create
|
||||
* the path keys which must be withheld from ordinary path-based repair.
|
||||
*/
|
||||
export function selectUnresolvedMetadataIdentityEntries(
|
||||
entries: readonly MetadataDocumentIdentityIssue[],
|
||||
handleFilenameCaseSensitive: boolean
|
||||
): {
|
||||
entries: MetadataDocumentIdentityIssue[];
|
||||
unresolvedPathKeys: ReadonlySet<string>;
|
||||
} {
|
||||
return {
|
||||
entries: [...entries],
|
||||
unresolvedPathKeys: new Set(
|
||||
entries
|
||||
.filter(({ ordinaryPathAvailable }) => !ordinaryPathAvailable)
|
||||
.map(({ inspection }) =>
|
||||
metadataIdentityPathKey(inspection.diagnostic.declaredPath, handleFilenameCaseSensitive)
|
||||
)
|
||||
),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { MetadataDocumentIdentityIssue } from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
import { metadataIdentityPathKey, selectUnresolvedMetadataIdentityEntries } from "./metadataIdentityInspection";
|
||||
|
||||
function createEntries(): MetadataDocumentIdentityIssue[] {
|
||||
return [
|
||||
{
|
||||
inspection: {
|
||||
status: "unresolved",
|
||||
diagnostic: {
|
||||
reason: "document-id-mismatch",
|
||||
actualDocumentId: "f:stale",
|
||||
declaredPath: "Folder/Renamed.md",
|
||||
expectedDocumentId: "f:renamed",
|
||||
actualNamespace: "normal",
|
||||
declaredPathNamespace: "normal",
|
||||
},
|
||||
},
|
||||
sourceRevision: "3-stale",
|
||||
logicallyDeleted: false,
|
||||
conflictRevisions: [],
|
||||
repairAvailable: false,
|
||||
targetAlreadyPresent: false,
|
||||
ordinaryPathAvailable: false,
|
||||
},
|
||||
{
|
||||
inspection: {
|
||||
status: "unresolved",
|
||||
diagnostic: {
|
||||
reason: "namespace-mismatch",
|
||||
actualDocumentId: "f:stale-internal-path",
|
||||
declaredPath: "i:.Obsidian/App.json",
|
||||
actualNamespace: "normal",
|
||||
declaredPathNamespace: "internal",
|
||||
},
|
||||
},
|
||||
sourceRevision: "2-stale",
|
||||
logicallyDeleted: false,
|
||||
conflictRevisions: [],
|
||||
repairAvailable: false,
|
||||
targetAlreadyPresent: false,
|
||||
ordinaryPathAvailable: false,
|
||||
},
|
||||
] as unknown as MetadataDocumentIdentityIssue[];
|
||||
}
|
||||
|
||||
describe("Metadata identity inspection presentation", () => {
|
||||
it("derives case-insensitive Vault path keys for unresolved evidence", () => {
|
||||
const result = selectUnresolvedMetadataIdentityEntries(createEntries(), false);
|
||||
|
||||
expect(result.entries.map(({ sourceRevision }) => sourceRevision)).toEqual(["3-stale", "2-stale"]);
|
||||
expect([...result.unresolvedPathKeys]).toEqual(["folder/renamed.md", ".obsidian/app.json"]);
|
||||
expect(metadataIdentityPathKey("folder/RENAMED.md", false)).toBe("folder/renamed.md");
|
||||
expect(result.unresolvedPathKeys.has(metadataIdentityPathKey("folder/RENAMED.md", false))).toBe(true);
|
||||
});
|
||||
|
||||
it("retains case distinctions when filename handling is case-sensitive", () => {
|
||||
const result = selectUnresolvedMetadataIdentityEntries(createEntries(), true);
|
||||
|
||||
expect(result.unresolvedPathKeys.has("Folder/Renamed.md")).toBe(true);
|
||||
expect(result.unresolvedPathKeys.has("folder/renamed.md")).toBe(false);
|
||||
});
|
||||
|
||||
it("does not suppress ordinary inspection when the path has resolvable Metadata", () => {
|
||||
const entries = createEntries();
|
||||
entries[0] = {
|
||||
...entries[0],
|
||||
ordinaryPathAvailable: true,
|
||||
} as MetadataDocumentIdentityIssue;
|
||||
|
||||
const result = selectUnresolvedMetadataIdentityEntries(entries, false);
|
||||
|
||||
expect(result.entries).toHaveLength(2);
|
||||
expect(result.unresolvedPathKeys.has("folder/renamed.md")).toBe(false);
|
||||
expect(result.unresolvedPathKeys.has(".obsidian/app.json")).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,71 @@
|
||||
import type {
|
||||
MetadataDocumentRepairRequest,
|
||||
MetadataDocumentRepairResult,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
import { MetadataDocumentRepairResults } from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
|
||||
export const MetadataIdentityRepairExecutions = {
|
||||
CANCELLED: "cancelled",
|
||||
REPAIR_RESULT: "repair-result",
|
||||
} as const;
|
||||
|
||||
export type MetadataIdentityRepairExecution =
|
||||
| { status: typeof MetadataIdentityRepairExecutions.CANCELLED }
|
||||
| {
|
||||
status: typeof MetadataIdentityRepairExecutions.REPAIR_RESULT;
|
||||
result: MetadataDocumentRepairResult;
|
||||
scanCompleted: boolean;
|
||||
scanError?: unknown;
|
||||
};
|
||||
|
||||
export interface MetadataIdentityRepairDependencies {
|
||||
confirm: () => Promise<boolean>;
|
||||
repair: (request: MetadataDocumentRepairRequest) => Promise<MetadataDocumentRepairResult>;
|
||||
requestOrdinaryScan: () => Promise<boolean>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Coordinate one explicitly confirmed Metadata identity repair.
|
||||
*
|
||||
* This consumer boundary deliberately keeps inspection approval, Commonlib
|
||||
* mutation, and the subsequent ordinary Vault scan as separate operations.
|
||||
* Cancellation cannot reach the mutation, and only a completed repair hands
|
||||
* reconciliation back to the Offline Scanner. Commonlib re-inspects the
|
||||
* source and expected ID under the current local path settings immediately
|
||||
* before mutation, so remote replication state is not part of this boundary.
|
||||
*/
|
||||
export async function executeMetadataIdentityRepair(
|
||||
request: MetadataDocumentRepairRequest,
|
||||
dependencies: MetadataIdentityRepairDependencies
|
||||
): Promise<MetadataIdentityRepairExecution> {
|
||||
if (!(await dependencies.confirm())) {
|
||||
return { status: MetadataIdentityRepairExecutions.CANCELLED };
|
||||
}
|
||||
const result = await dependencies.repair(request);
|
||||
if (result.status !== MetadataDocumentRepairResults.COMPLETED) {
|
||||
return {
|
||||
status: MetadataIdentityRepairExecutions.REPAIR_RESULT,
|
||||
result,
|
||||
scanCompleted: false,
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
const scanCompleted = await dependencies.requestOrdinaryScan();
|
||||
return {
|
||||
status: MetadataIdentityRepairExecutions.REPAIR_RESULT,
|
||||
result,
|
||||
scanCompleted,
|
||||
};
|
||||
} catch (scanError) {
|
||||
// The Metadata identity mutation has already completed. Preserve that
|
||||
// result separately so a follow-up scan failure cannot be mistaken for
|
||||
// a failed or rolled-back repair.
|
||||
return {
|
||||
status: MetadataIdentityRepairExecutions.REPAIR_RESULT,
|
||||
result,
|
||||
scanCompleted: false,
|
||||
scanError,
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
import type { DocumentID } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import type {
|
||||
MetadataDocumentRepairRequest,
|
||||
MetadataDocumentRepairResult,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
import { executeMetadataIdentityRepair } from "./metadataIdentityRepair";
|
||||
|
||||
const request: MetadataDocumentRepairRequest = {
|
||||
actualDocumentId: "f:stale" as DocumentID,
|
||||
expectedDocumentId: "f:expected" as DocumentID,
|
||||
sourceRevision: "4-source",
|
||||
};
|
||||
|
||||
function createDependencies() {
|
||||
const events: string[] = [];
|
||||
return {
|
||||
events,
|
||||
confirm: vi.fn(async () => true),
|
||||
repair: vi.fn(async (): Promise<MetadataDocumentRepairResult> => {
|
||||
events.push("repair");
|
||||
return {
|
||||
status: "completed" as const,
|
||||
...request,
|
||||
targetCreated: true,
|
||||
};
|
||||
}),
|
||||
requestOrdinaryScan: vi.fn(async () => {
|
||||
events.push("scan");
|
||||
return true;
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
describe("executeMetadataIdentityRepair", () => {
|
||||
it("performs no mutation when the separate confirmation is cancelled", async () => {
|
||||
const dependencies = createDependencies();
|
||||
dependencies.confirm.mockResolvedValue(false);
|
||||
|
||||
await expect(executeMetadataIdentityRepair(request, dependencies)).resolves.toEqual({
|
||||
status: "cancelled",
|
||||
});
|
||||
expect(dependencies.repair).not.toHaveBeenCalled();
|
||||
expect(dependencies.requestOrdinaryScan).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("requests an ordinary scan only after Commonlib completes the exact repair", async () => {
|
||||
const dependencies = createDependencies();
|
||||
|
||||
await expect(executeMetadataIdentityRepair(request, dependencies)).resolves.toMatchObject({
|
||||
status: "repair-result",
|
||||
result: { status: "completed" },
|
||||
scanCompleted: true,
|
||||
});
|
||||
expect(dependencies.repair).toHaveBeenCalledWith(request);
|
||||
expect(dependencies.requestOrdinaryScan).toHaveBeenCalledOnce();
|
||||
expect(dependencies.repair.mock.invocationCallOrder[0]).toBeLessThan(
|
||||
dependencies.requestOrdinaryScan.mock.invocationCallOrder[0]
|
||||
);
|
||||
expect(dependencies.events).toEqual(["repair", "scan"]);
|
||||
});
|
||||
|
||||
it("keeps a completed repair distinct when the ordinary scan cannot start", async () => {
|
||||
const dependencies = createDependencies();
|
||||
dependencies.requestOrdinaryScan.mockResolvedValue(false);
|
||||
|
||||
await expect(executeMetadataIdentityRepair(request, dependencies)).resolves.toMatchObject({
|
||||
status: "repair-result",
|
||||
result: { status: "completed" },
|
||||
scanCompleted: false,
|
||||
});
|
||||
expect(dependencies.requestOrdinaryScan).toHaveBeenCalledOnce();
|
||||
expect(dependencies.events).toEqual(["repair"]);
|
||||
});
|
||||
|
||||
it("keeps a completed repair distinct when requesting the ordinary scan throws", async () => {
|
||||
const dependencies = createDependencies();
|
||||
const error = new Error("scan unavailable");
|
||||
dependencies.requestOrdinaryScan.mockRejectedValue(error);
|
||||
|
||||
await expect(executeMetadataIdentityRepair(request, dependencies)).resolves.toMatchObject({
|
||||
status: "repair-result",
|
||||
result: { status: "completed" },
|
||||
scanCompleted: false,
|
||||
scanError: error,
|
||||
});
|
||||
expect(dependencies.events).toEqual(["repair"]);
|
||||
});
|
||||
|
||||
it("does not scan after a stale, blocked, or failed repair result", async () => {
|
||||
for (const status of ["stale", "blocked", "failed"] as const) {
|
||||
const dependencies = createDependencies();
|
||||
dependencies.repair.mockResolvedValue({
|
||||
status,
|
||||
...request,
|
||||
targetCreated: false,
|
||||
});
|
||||
|
||||
await expect(executeMetadataIdentityRepair(request, dependencies)).resolves.toMatchObject({
|
||||
status: "repair-result",
|
||||
result: { status },
|
||||
scanCompleted: false,
|
||||
});
|
||||
expect(dependencies.requestOrdinaryScan).not.toHaveBeenCalled();
|
||||
}
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user