diff --git a/docs/settings.md b/docs/settings.md index 60e52b99..649fedec 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -8,12 +8,12 @@ There are many settings in Self-hosted LiveSync. This document describes each se The following status applies to optional and compatibility features in the 1.0 line: -| Status | Features | 1.0 policy | -| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Supported, opt-in | Peer-to-Peer Synchronisation, Hidden File Sync, and Customisation Sync | Maintained and covered by focused real-runtime tests. Enable them only where their separate setup and operational constraints are acceptable. | -| Maintained, advanced | Data Compression | Available as an explicit storage and bandwidth trade-off. It remains disabled by default because the measured processing and memory costs outweigh the mixed-dataset saving. | -| Beta or experimental | JWT authentication, ignore files, automatic newer-file conflict resolution, and Garbage Collection V3 | Retained for explicit testing and specialised use. They remain disabled by default and are not part of the minimum supported setup. | -| Compatibility only | V1 dynamic iteration counts, the old IndexedDB adapter, non-current hash algorithms, Eden chunks, and the stored `doNotUseFixedRevisionForChunks` key | Existing settings and data remain readable. New Vaults use the current defaults, and compatibility controls are shown only where a migration or recovery path still needs them. | +| Status | Features | 1.0 policy | +| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Supported, opt-in | Peer-to-Peer Synchronisation, Hidden File Sync, and Customisation Sync | Maintained and covered by focused real-runtime tests. Enable them only where their separate setup and operational constraints are acceptable. | +| Maintained, advanced | Data Compression | Available as an explicit storage and bandwidth trade-off. It remains disabled by default because the measured processing and memory costs outweigh the mixed-dataset saving. | +| Beta or experimental | JWT authentication, ignore files, automatic newer-file conflict resolution, and Garbage Collection V3 | Retained for explicit testing and specialised use. They remain disabled by default and are not part of the minimum supported setup. | +| Compatibility only | V1 dynamic iteration counts, the old IndexedDB adapter, non-current hash algorithms, Eden chunks, and the stored `doNotUseFixedRevisionForChunks` key | Existing settings and data remain readable. New Vaults use the current defaults, and compatibility controls are shown only where a migration or recovery path still needs them. | | Icon | Description | | :--: | ------------------------------------------------------------------ | @@ -34,7 +34,7 @@ The following status applies to optional and compatibility features in the 1.0 l This pane always shows the current release history. It does not track whether a particular plug-in version has been read and does not open automatically after an ordinary update. -Internal database or settings compatibility reviews use a separate safety dialogue, not this pane. The dialogue explains why remote synchronisation has been paused and preserves the automatic synchronisation choices which were configured before the update. A configured Vault which was copied, restored, or opened in a new Obsidian profile can require this review because its device-local acknowledgement is not part of the Vault data. An empty local database is not accepted as evidence that it is safe to continue. Closing the dialogue keeps synchronisation paused. When the detected state can be handled by the running version, the explicit resume action records the current internal database version and restores the configured behaviour. A persistent Notice and the `Review why synchronisation is paused` command reopen the review. An older installation cannot dismiss a pause caused by a newer database or settings version. +Internal database or settings compatibility reviews use a separate safety dialogue, not this pane. The dialogue explains why remote synchronisation has been paused and preserves the automatic synchronisation choices which were configured before the update. A configured Vault which was copied, restored, or opened in a new Obsidian profile can require this review because its device-local acknowledgement is not part of the Vault data. An empty local database is not accepted as evidence that it is safe to continue. An existing unconfigured Vault remains in onboarding without this synchronisation warning; its missing acknowledgement is not filled in automatically, so it is evaluated if the Vault is configured later. Closing the dialogue keeps synchronisation paused. When the detected state can be handled by the running version, the explicit resume action records the current internal database version and restores the configured behaviour. A persistent Notice and the `Review why synchronisation is paused` command reopen the review. An older installation cannot dismiss a pause caused by a newer database or settings version. ## 1. Setup @@ -637,6 +637,8 @@ If this is set, changes to local files which are matched by the ignore files wil Setting key: ignoreFiles Comma separated `.gitignore, .dockerignore` +When a saved setting changes whether a normal file can be reflected, LiveSync re-evaluates the normal-file metadata already held in the local database. This covers selector expressions, ignore-file settings, maximum-size and modification-time limits, and file-name case handling. A remote revision which was received and checkpointed while excluded can therefore be reflected after the criteria are broadened, without rewinding the remote checkpoint. Narrowing the criteria does not delete files which have already been reflected. + ### 2. Hidden Files (Advanced) #### Ignore patterns @@ -898,7 +900,7 @@ If this enabled, All files are handled as case-Sensitive (Previous behaviour). When this setting is disabled, changing only the letter case of a file name within the same directory is synchronised as a rename. Changing the letter case of a directory name is not supported by this handling. -New Vaults use case-insensitive handling for cross-platform compatibility. Existing settings with an explicit value preserve that choice. If an existing Vault has no saved value, remote synchronisation remains paused until case-sensitive handling is explicitly retained. Changing such a Vault to case-insensitive handling requires checking for paths which differ only by letter case and rebuilding the database through the normal compatibility setting workflow. +New Vaults use case-insensitive handling for cross-platform compatibility. Existing settings with an explicit value preserve that choice. Earlier releases also followed the case-insensitive branch when this value was absent, so 1.0 saves a missing legacy value as `false` without requiring a compatibility review or database rebuild. ### 4. Compatibility (Internal API Usage) diff --git a/test/e2e-obsidian/README.md b/test/e2e-obsidian/README.md index c741edc3..3c722766 100644 --- a/test/e2e-obsidian/README.md +++ b/test/e2e-obsidian/README.md @@ -70,6 +70,7 @@ npm run test:e2e:obsidian:runner npm run test:e2e:obsidian:install-appimage npm run test:e2e:obsidian:discover npm run test:e2e:obsidian:cli-help -- vaults verbose +npm run test:e2e:obsidian:upgrade-from-stable -- --transport all npm run test:e2e:obsidian:local-suite npm run test:e2e:obsidian:local-suite:services ``` @@ -100,7 +101,7 @@ The same workflow checks the two remote-activity status boundaries. It first hol If this status workflow fails while Obsidian is running, it writes a full-page screenshot and a JSON snapshot of the status text and counters under `/tmp/obsidian-livesync-e2e`. The dialogue-mount workflow leaves desktop and mobile screenshots for both representative Svelte routes, and the Hidden File Sync workflow captures the successfully displayed JSON Resolve dialogue before selecting an option. The suite therefore records representative evidence without capturing every interaction. Set `E2E_OBSIDIAN_DIAGNOSTICS_DIR` to use another directory. -The two-Vault workflow performs the missing-marker review once for each isolated Vault. Later process launches seed the acknowledgement which the same real profile would retain, rather than repeatedly applying a first-device decision. The Hidden File Sync scenario is narrower: it starts from an explicitly acknowledged marker because it tests consumer-owned hidden-file behaviour, JSON resolution, target filtering, and grouped mobile Notices rather than duplicating the compatibility workflow. After `app.emulateMobile(true)`, its fixture operations use the active DevTools renderer because Obsidian can remove desktop-only CLI commands in mobile mode. +The two-Vault workflow performs the missing-marker review once for each isolated Vault. Later process launches reuse the same profile-backed acknowledgement, rather than seeding a replacement or repeatedly applying a first-device decision. The Hidden File Sync scenario is narrower: it starts from an explicitly acknowledged marker because it tests consumer-owned hidden-file behaviour, JSON resolution, target filtering, and grouped mobile Notices rather than duplicating the compatibility workflow. After `app.emulateMobile(true)`, its fixture operations use the active DevTools renderer because Obsidian can remove desktop-only CLI commands in mobile mode. `test:e2e:obsidian:cli-to-obsidian-sync` is the cross-runtime compatibility check for the official LiveSync CLI and the real Obsidian plug-in. Build the plug-in first, and build the local CLI too when no external CLI command is selected. The script uses E2EE, Path Obfuscation, and the current preferred chunk settings to create and synchronise a note through the CLI, starts real Obsidian with an isolated Vault and profile, synchronises the same CouchDB database, and verifies that the plug-in materialises identical note content. This covers the boundary that CLI-only and plug-in-only round trips do not exercise. @@ -129,7 +130,7 @@ LIVESYNC_CLI_COMMAND="docker run --rm --network host --user $(id -u):$(id -g) -- `test:e2e:obsidian:setup-uri-workflow` runs the repository's public Commonlib-backed CouchDB provisioning and Setup URI tools against the local CouchDB fixture. It configures the first data-less real Obsidian Vault through the visible onboarding wizard and uses Rebuild. After that device is working, it generates a new Setup URI through the registered command; the second real Obsidian Vault uses that URI for Fetch instead of reusing the provisioning-time bootstrap URI. The workflow verifies ordinary notes from the first device to the second and back again, independently enables Hidden File Sync on each device, and verifies a snippet. The retained Setup URI screenshots show only encrypted URIs and visually masked Setup URI passphrases; plaintext credentials are not captured. Files prefixed with `guide-` capture the relevant dialogue, settings panel, or workspace leaf without transient Notices. Public documentation copies selected images only after visual inspection; the E2E run does not overwrite repository documentation assets. -`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, ordinary rename, a case-only file name change within the same directory, deletion, per-device target filters where one vault ignores a note that the other vault synchronises, and a separate encrypted round-trip with Path Obfuscation enabled. Directory case changes deliberately remain outside this scenario because they require directory-aware rename handling. The optional Markdown conflict automatic merge check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`, but it is not part of the default local suite. +`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, ordinary rename, a case-only file name change within the same directory, deletion, and a separate encrypted round-trip with Path Obfuscation enabled. Its target-filter scenario confirms that one Vault receives and checkpoints a remote document without reflecting it, restarts with the same profile and filter, and then reflects the stored document after the filter is broadened through the settings service. Directory case changes deliberately remain outside this scenario because they require directory-aware rename handling. The optional Markdown conflict automatic merge check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`, but it is not part of the default local suite. `test:e2e:obsidian:hidden-file-snippet-sync` runs a two-vault hidden file round-trip. It verifies creation and deletion of a real `.obsidian/snippets/*.css` file, automatic JSON conflict merging for a hidden file with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target patterns where one vault ignores a hidden file that the other vault synchronises. It also covers [issue #555](https://github.com/vrtmrz/obsidian-livesync/issues/555) by requiring several plug-in and settings changes to share one mobile-safe Notice with actionable touch targets; a manually dismissed group must not repeat its acknowledged rows when a later change arrives. @@ -137,6 +138,22 @@ LIVESYNC_CLI_COMMAND="docker run --rm --network host --user $(id -u):$(id -g) -- `test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`. +`test:e2e:obsidian:upgrade-from-stable` is the release-acceptance upgrade workflow. It installs the exact published 0.25.83 artefacts into an isolated Vault, verifies their pinned SHA-256 values, and then replaces only the plug-in artefacts with the current target while retaining the same Vault and isolated Obsidian profile. The first run downloads the old release into the ignored `_testdata/releases` cache; every later run verifies the cached bytes before use. + +The workflow first exercises a non-empty legacy settings document which has no `isConfigured` or file-name case value. It verifies that 0.25.83 treats a default-equivalent document as unconfigured. That release can persist the inferred boolean during a later, unrelated settings-save event, so the runner accepts either an absent value or the inferred `false` on disk, then restores the same minimal pre-flag document deliberately before installing 1.0. The target independently proves its direct migration: the Vault remains unconfigured instead of receiving new-Vault recommendations, case-insensitive handling becomes explicit, no compatibility pause or acknowledgement marker is created while onboarding remains pending, and a second 1.0 start is idempotent. The absent marker is deliberately deferred rather than accepted; a later configured start must evaluate it. This fixture rewrite is limited to the missing-flag boundary; the configured transport upgrades use only state created and saved by 0.25.83 itself. + +For CouchDB and Object Storage, the workflow then configures 0.25.83 from its own defaults, saves the selected remote, and restarts that release with the same profile before creating history. This both verifies that the old settings persist and lets the old release initialise its replicator from the same saved state as an ordinary existing Vault. The runner waits for that release's asynchronously initialised persistent node identity, creates, edits, renames, and deletes notes, and synchronises each transition before installing the target. Every launch of the upgraded device uses the same isolated Obsidian profile. The session layer closes the renderer before its process-tree fallback, so Chromium persists the legacy compatibility marker naturally; the target must read and migrate that actual profile state to its current namespaced key. The final target restart likewise consumes the marker persisted by the preceding target session. The runner does not reconstruct that device's Vault data, plug-in settings, local database files, device-local state, or remote state. Before the target performs any synchronisation, it must retain the same Vault profile, local database, node identity, remote profile, local checkpoint, and remote milestone. The local node-info document is the identity source of truth; a transient replicator field is used only to confirm that the old asynchronous initialisation has completed. Its first synchronisation must be a no-op: CouchDB document revisions and `update_seq` must remain unchanged, while Object Storage must neither upload nor download journal bodies. The upgraded device then sends a new delta. A separate fresh 1.0 verifier starts from an explicit current-version settings and compatibility fixture, receives the complete surviving history, and returns another delta; it is not part of the legacy-profile migration assertion. The upgraded Vault receives that return journey and retains it across restart. + +Run the focused wrapper after source changes so that the target plug-in is rebuilt first: + +```bash +npm run test:e2e:obsidian:focused -- upgrade-from-stable --transport all --manage-services +``` + +Use `--transport couchdb` or `--transport object-storage` for a focused rerun. `--manage-services` starts and stops the required local fixture or fixtures; add `--keep-services` only when they should remain available for inspection. Set `E2E_LIVESYNC_TARGET_ARTIFACT_ROOT` to validate another already-built target directory, or `E2E_LIVESYNC_SOURCE_ARTIFACT_ROOT` to use an explicit cache directory whose files still match the pinned release hashes. + +This workflow is deliberately excluded from `local-suite`. It downloads a published historical artefact, reuses one profile across multiple application versions, and is an expensive release-acceptance gate rather than a routine current-version scenario. P2P is also excluded because cross-version P2P interoperability is a separate physical validation boundary. + Start the local fixtures first when they are not already running: ```bash @@ -175,6 +192,8 @@ Useful environment variables: - `E2E_OBSIDIAN_CLI_TIMEOUT_MS`: timeout for each `obsidian-cli` invocation. - `E2E_LIVESYNC_CLI_TIMEOUT_MS`: timeout for each official LiveSync CLI invocation in the CLI-to-Obsidian compatibility check; default is 60 seconds. - `LIVESYNC_CLI_COMMAND`: optional LiveSync CLI executable and prefix arguments used by the CLI-to-Obsidian compatibility check. The default is the locally built CLI. +- `E2E_LIVESYNC_SOURCE_ARTIFACT_ROOT`: optional cache directory containing the exact pinned 0.25.83 plug-in artefacts. Cached files are always checksum-verified. +- `E2E_LIVESYNC_TARGET_ARTIFACT_ROOT`: directory containing the built 1.0 target `main.js`, `manifest.json`, and `styles.css`; default is the repository root. - `E2E_OBSIDIAN_FILE_TIMEOUT_MS`: timeout for waiting until a note created through Obsidian's vault API is reflected to disk. - `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready. - `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database. diff --git a/updates.md b/updates.md index 0bb4983e..d863b88e 100644 --- a/updates.md +++ b/updates.md @@ -16,7 +16,6 @@ Earlier releases remain available in the [0.25 release history](docs/releases/0. - This is an opt-in 1.0 integration preview for BRAT and real-Vault validation. It does not replace the latest stable release. Use it with a current backup, and update every participating device before resuming synchronisation. - An upgraded, copied, or restored Vault may pause replication for an explicit compatibility review. The review preserves the existing automatic synchronisation choices and resumes them only after the decision has been saved successfully. -- If an existing Vault has no saved file-name case policy, choose either the legacy case-sensitive behaviour or cross-platform case-insensitive handling. Check for case-only path conflicts and rebuild the local database before adopting the case-insensitive policy. ### Improved @@ -30,10 +29,10 @@ Earlier releases remain available in the [0.25 release history](docs/releases/0. - Self-hosted CouchDB provisioning and Setup URI generation now consume the immutable Commonlib registry package. New database provisioning records the package-owned database version, and generated CouchDB, Object Storage, and random-room P2P URIs use current new-Vault defaults and selected remote profiles instead of legacy embedded settings. - Removed the ineffective 'Use the trash bin' toggle from the settings interface. Remote deletions continue to follow Obsidian's deletion preference, while the legacy setting key remains accepted for compatibility. - Kept content-derived chunk revisions permanently enabled, as they have been since 0.25.6, and removed the obsolete stored key from recommendations, database-maintenance prerequisites, and review tooling. -- Aligned new-Vault, full-reset, and CLI-generated settings with the 1.0 recommendations. New Vaults use cross-platform case-insensitive file-name handling, while existing Vaults require the explicit compatibility decision described above. +- Aligned new-Vault, full-reset, and CLI-generated settings with the 1.0 recommendations. New Vaults use cross-platform case-insensitive file-name handling. Existing Vaults preserve an explicit choice, while a missing legacy value is saved as case-insensitive to retain its earlier effective behaviour. - Improved P2P restart and settings-reapplication handling by serialising transport start and stop, keeping event handlers bound to the active replicator, and using one package-owned Trystero transport generation. - Kept the release history available in the settings while removing automatic unread-version tracking and redirection. Release versions are no longer treated as a data-compatibility signal. -- Internal database and settings compatibility reviews now use a dedicated explanation and an explicit, case-specific resume action. They block replication without rewriting the user's automatic synchronisation choices, and older installations cannot dismiss state created by a newer version. +- Internal database and settings compatibility reviews now use a dedicated explanation and an explicit resume action. They block replication without rewriting the user's automatic synchronisation choices, and older installations cannot dismiss state created by a newer version. - Stacked confirmation and compatibility-review actions vertically, and kept persistent review reminders clear of mobile close controls. Long setup dialogues now keep their action area inside the mobile safe area while their content remains scrollable. A configured Vault with no device-local acknowledgement now explains that it may have been copied, restored, or opened in a new profile instead of trying to infer safety from an empty local database. - Combined Hidden File Sync plug-in reload and Obsidian restart notifications into one mobile-safe message with clearly labelled actions, avoiding the stacked notifications reported in [issue #555](https://github.com/vrtmrz/obsidian-livesync/issues/555). A group dismissed by the user no longer reappears with its previously acknowledged rows when a later change arrives. - Replaced remote-size decision prompts shown during startup with long-lived, clickable notices. The detailed choices now open only when requested and no longer select an answer automatically after a timeout. @@ -41,6 +40,9 @@ Earlier releases remain available in the [0.25 release history](docs/releases/0. ### Fixed +- Existing Vaults with no stored file-name case value now persist the case-insensitive behaviour used by earlier releases, without presenting a spurious compatibility review or requiring a database rebuild. +- Legacy settings without an explicit configured-state flag retain the 0.25.83 inference: default-equivalent settings remain unconfigured, non-default settings remain configured, and either result is persisted during migration. An unconfigured Vault stays in onboarding without a misleading compatibility pause; its missing acknowledgement remains pending for any later configured start. +- Changing normal-file reflection criteria now re-evaluates documents which were received and checkpointed while excluded, so broadening a selector, ignore rule, size limit, modification-time limit, or file-name case policy can reflect those documents without requiring another remote revision. - P2P first-device setup no longer attempts central-remote reset or upload operations which do not exist for a peer-to-peer destination. Its initialisation confirmations now describe the local database preparation instead of warning about overwriting a central server. - Importing a P2P Setup URI on an additional device now asks for the source peer once and completes one finite Fetch before reflecting the Vault, instead of opening the peer-selection dialogue twice. - Config Doctor now limits the V3 chunk-size recommendation to self-hosted CouchDB and no longer offers the CouchDB-specific change for Object Storage or P2P profiles.