From 53281c5fe5f0b1d4da789e622cc2aac2b1994420 Mon Sep 17 00:00:00 2001 From: vorotamoroz Date: Thu, 12 Nov 2026 19:00:57 +0000 Subject: [PATCH] docs: finalise 1.0.50 release notes and publication records --- docs/README.md | 4 +- .../design_docs/cli_watcher_path_filtering.md | 7 +- docs/design_docs/file_path_index.md | 12 +- docs/releases/1.0.md | 148 ++++++++++++++++++ updates.md | 147 ----------------- 5 files changed, 160 insertions(+), 158 deletions(-) diff --git a/docs/README.md b/docs/README.md index e9ea6d80..9cad0616 100644 --- a/docs/README.md +++ b/docs/README.md @@ -41,7 +41,7 @@ Use these references for the area being changed, together with its relevant ADR. | P2P connection preparation | [TURN connection settings](design_docs/renewable_turn_credentials.md) | | Journal storage internals | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) | | CLI database selection and interrupted reconstruction | [CLI database continuity](design_docs/cli_database_continuity.md) | -| CLI filesystem monitoring | [CLI watcher path filtering](design_docs/cli_watcher_path_filtering.md): accepted implementation; locally validated, not yet released | +| CLI filesystem monitoring | [CLI watcher path filtering](design_docs/cli_watcher_path_filtering.md): accepted implementation included in 1.0.50; reporter confirmation remains pending | The installed `@vrtmrz/livesync-commonlib` package is authoritative for shared runtime contracts. Use the exact dependency selected by [package.json](../package.json) and its lockfile when checking behaviour. The [Commonlib documentation](https://github.com/vrtmrz/livesync-commonlib/tree/main/docs) is maintained in that repository; this index does not replace its package contracts. @@ -66,7 +66,7 @@ These records are useful for design work. Their proposed representation or behav | Document | Recorded state | |---|---| -| [File path index and watched CLI file names](design_docs/file_path_index.md) | Unreleased implementation design for #1346; shared name index and collision queries; Commonlib package adoption pending | +| [File path index and watched CLI file names](design_docs/file_path_index.md) | Implementation design for #1346, included in 1.0.50; shared name index and collision queries; Commonlib 0.5.0 adopted | | [Tiered Chunk Storage with Live Compaction](design_docs/tired_chunk_pack.md) | In design | | [Tiered Chunk Storage for Bucket Sync](design_docs/tired_chunk_pack_bucket.md) | In design | | [Design intent for Metadata and Chunks](design_docs/intention_of_chunks.md) | Incomplete explanation; use the data-structure reference for current representations | diff --git a/docs/design_docs/cli_watcher_path_filtering.md b/docs/design_docs/cli_watcher_path_filtering.md index 93f77c3a..6ad6fcee 100644 --- a/docs/design_docs/cli_watcher_path_filtering.md +++ b/docs/design_docs/cli_watcher_path_filtering.md @@ -7,8 +7,9 @@ status: accepted # CLI watcher path filtering -Implementation note: the correction is implemented and locally validated, but -has not been released. The reviewed baseline remains Self-hosted LiveSync +Implementation note: the correction is included in Self-hosted LiveSync +`1.0.50` and has been validated locally and in CI. The reviewed baseline remains +Self-hosted LiveSync `78ebc2d2` and Commonlib `0.4.3`. The original failure was reproduced against Self-hosted LiveSync `71893f2e` (`1.0.47`) with Commonlib `0.4.1` and Chokidar `6.0.0`. Reporter confirmation remains pending. @@ -317,7 +318,7 @@ view databases remain excluded with the option. The two skipped unit cases are platform-dependent: a Windows different-drive case and an existing host-filesystem alias case. Windows runtime behaviour and -reporter confirmation remain unverified. The correction remains unreleased. +reporter confirmation remain unverified. The correction is included in `1.0.50`. ### Shared file-name source integration diff --git a/docs/design_docs/file_path_index.md b/docs/design_docs/file_path_index.md index 925b08cf..62b426d2 100644 --- a/docs/design_docs/file_path_index.md +++ b/docs/design_docs/file_path_index.md @@ -2,20 +2,20 @@ date: 2026-11-12 commonlib-version: "0.4.3" self-hosted-livesync-version: "1.0.49" -status: implemented-unreleased +status: implemented --- # File path index and watched CLI file names ## Status and purpose -This is an implementation design for [#1346](https://github.com/vrtmrz/obsidian-livesync/issues/1346), reviewed against Self-hosted LiveSync `78ebc2d298c39df0ce2be3a4e62059cd74be86a0`, Commonlib `0.4.3`, and the CLI's resolved `chokidar` `6.0.0`. An unreleased Self-hosted LiveSync implementation follows this design. The version fields identify the reviewed baseline, not a release containing this change. +This is an implementation design for [#1346](https://github.com/vrtmrz/obsidian-livesync/issues/1346), reviewed against Self-hosted LiveSync `78ebc2d298c39df0ce2be3a4e62059cd74be86a0`, Commonlib `0.4.3`, and the CLI's resolved `chokidar` `6.0.0`. This implementation is included in Self-hosted LiveSync `1.0.50`. The version fields identify the reviewed baseline, not a release containing this change. -The Commonlib portion is published in [0.5.0](https://github.com/vrtmrz/livesync-commonlib/releases/tag/0.5.0), and the consumer pins that exact registry version. Clean-install checks, unit tests, and plug-in and application builds have passed against that artefact. The consumer change remains unreleased. +The Commonlib portion is published in [0.5.0](https://github.com/vrtmrz/livesync-commonlib/releases/tag/0.5.0), and the consumer pins that exact registry version. Clean-install checks, unit tests, and plug-in and application builds have passed against that artefact. The CLI should maintain its file-name inventory while running as a daemon. A shared in-memory `FilePathIndex` should group equivalent paths and accept incremental membership changes. Ordinary content edits should neither enumerate the whole Vault nor rebuild its identity map. -Commonlib currently caches the collision map by `totalStorageFileEventCount`, path-normalisation version, and effective case policy. A changed event count makes `targetFilter` obtain and sort all names again. `ServiceFileAccessBase.getFileNames()` obtains those names through `getFiles()`, which performs a recursive directory scan and metadata reads in the CLI. Obsidian supplies its in-memory Vault listing instead. The issue is the cost of maintaining the lookup, while collision rejection remains required. +In the reviewed baseline, Commonlib caches the collision map by `totalStorageFileEventCount`, path-normalisation version, and effective case policy. A changed event count makes `targetFilter` obtain and sort all names again. `ServiceFileAccessBase.getFileNames()` obtains those names through `getFiles()`, which performs a recursive directory scan and metadata reads in the CLI. Obsidian supplies its in-memory Vault listing instead. The issue is the cost of maintaining the lookup, while collision rejection remains required. A baseline experiment with 40 files and five directories, including the root, found one full enumeration and 45 adapter `stat` or `statResolved` calls after a content-only event. Repeated checks at the same event count reused the map. All four Legacy/NFC and case-policy combinations took this path. These are adapter-operation counts, not OS syscall counts or an end-to-end daemon benchmark. The issue's reported syscall reduction combines the reporter's local collision-check change with [#1347](https://github.com/vrtmrz/obsidian-livesync/pull/1347); it does not measure either change separately. @@ -49,7 +49,7 @@ The index expresses LiveSync path identity and belongs in Commonlib under the [p ## Shared index -The proposed public operations are: +The shared public operations are: ```ts interface FilePathDelta { @@ -68,7 +68,7 @@ declare class FilePathIndex { } ``` -These signatures describe the unreleased API; they are not available from the reviewed published baseline. The class performs no I/O, subscribes to no events, and owns no promises or settings service. An empty instance is a valid empty *data structure*; its host owner determines whether it is ready to answer storage queries. +These signatures describe the API introduced in Commonlib `0.5.0`; they are not available from the reviewed `0.4.3` baseline. The class performs no I/O, subscribes to no events, and owns no promises or settings service. An empty instance is a valid empty *data structure*; its host owner determines whether it is ready to answer storage queries. Retain a `Set` of original paths. Lazily construct identity buckets for an explicit comparison policy. There are at most four supported policies: Legacy/NFC crossed with case-sensitive/case-insensitive handling. Cache only those four possibilities, and update every materialised policy map when membership changes. Reject unsupported policy values before modifying or querying state. diff --git a/docs/releases/1.0.md b/docs/releases/1.0.md index 68ad4d28..eb62376e 100644 --- a/docs/releases/1.0.md +++ b/docs/releases/1.0.md @@ -2,6 +2,154 @@ This document contains earlier published releases from the 1.0 line of the [current Self-hosted LiveSync release history](../../updates.md). Beta and release-candidate builds published before 1.0.0 are recorded in the [1.0 preview history](1.0-previews.md). Earlier release lines continue in the [0.25 history](0.25.md) and the [legacy history](legacy.md). + +## 1.0.45 + +5th November, 2026 + +Some of my recent replies have been sporting the Codex icon. Rest assured: I simply ask Codex to polish my English and discreetly intercept the occasional ill-judged remark before I post. + +### New Feature + +#### Synchronisation + +- We can recognise equivalent composed and decomposed Unicode file names as the same file when synchronising through Object Storage or P2P. (#1309) + - Select **Unicode NFC path identity** under **General Settings** → **File identity**. Supported by the CLI and Obsidian; physical file names and Chunk IDs remain unchanged. + - To migrate a Legacy database, update every participating client, stop synchronisation and editing, and preserve a complete, backed-up authoritative Vault, unique edits, and required history. Resolve colliding names before Rebuild on the source and Fetch on the remaining devices. With P2P, Rebuild prepares only the source database; keep that device online for Fetch. + - New CLI settings and Obsidian manual setup use NFC. Existing settings and older Setup URIs retain Legacy identity; initial CouchDB or Object Storage setup follows the remote's declared identity. + - Object Storage remotes with existing Journals but no compatibility declaration also require Rebuild, even in Legacy mode. + +### Fixed + +#### Object Storage + +- Journal receive failures now stop replication instead of silently passing over records which could not be decoded, processed, or saved. (#1318) + - After correcting the cause, retry replication with the same local database to receive the failed Journal again. + - If a Journal cannot be recovered and missing content or history is acceptable, use **Maintenance** → **Syncing** → **Skip stopped Journal and resume**. The confirmation identifies its exact key and failure stage. Only that Journal is skipped; the next failure still stops receipt. + - Journals already passed over by an earlier version require **Reset journal received history** to be replayed. This re-reads Journals still available remotely, except those sent by this device, and cannot recreate missing remote Journals. + +- A remote-only Rebuild resends the local database even when the Object Storage remote is already empty and this device retains its earlier transfer progress. + +#### Synchronisation + +- CouchDB synchronisation and Fetch no longer fail with decryption errors caused by stale encrypted responses in the Internal API cache after a Rebuild or plug-in restart. (#1234) + - Reads use a cache query parameter based on the current start-up and Security Seed. + - **Use headers only for HTTP cache control (compatibility)** under **Power users** → **CouchDB Connection Tweak** is disabled by default. Enable it only if the server or proxy rejects the additional query parameter; affected native caches may then reuse stale encrypted responses after Rebuild. + +#### Conflict resolution + +- Disabling or reloading the plug-in no longer causes a secondary error when pending work writes to the log. Late logs remain available in Obsidian's console. + - Open conflict dialogues for ordinary files close immediately, and queued checks report cancellation. + +#### Settings + +- The English description of **Do not split chunks in the background** now correctly states that enabling the setting moves Chunk splitting to the UI thread. (#1313) + +#### Translations + +- The notice shown when a translation is available now displays the **Display Language** setting label correctly in English, French, Hebrew, Russian, and Simplified Chinese. Hebrew interface messages also display their referenced labels correctly. (#1298) + +#### WebApp + +- The experimental WebApp rejects unsupported NFC path identity before opening its database or processing Vault files, preserving the selected files and settings. + - WebApp remains Legacy-only. Changing the setting does not migrate an NFC database. + +### Improved + +#### Translations + +- More of the interface is available in Japanese, including dialogue buttons, Vault history, SyncGuard, inspection reports, and Fetch restart confirmations. Japanese setup and settings guides have also been refreshed. (#1317) + +### CLI + +#### New Feature + +- We can resume Object Storage receipt after deliberately skipping one unrecoverable Journal with `journal-skip `. + - Retry `sync` with `--debug` to inspect the stopped Journal key and failure stage. The command retries replication first and skips only the supplied Journal if it remains stopped. + - Use this only when missing content or history is acceptable. The next Journal failure still stops receipt. + +#### Fixed + +- `fetch` and `rebuild` no longer replay pending file updates from the previous database at start-up. The old pending updates are discarded only after the local database reset succeeds. +- `fetch` and `daemon` can write received files to the Vault when `handleFilenameCaseSensitive` is enabled. The CLI detects whether the Vault filesystem distinguishes letter case. (#1319) +- NFC file lookups and directory scans report permission and I/O failures. Unreadable files are no longer treated as absent, and incomplete scans no longer count as successful. + +### Miscellaneous + +- Updated dependencies. + +### Acknowledgements + +Thank you for your contributions! + +- @cwatanab, for improving Japanese translations and documentation, correcting translated labels, and correcting the Chunk splitting setting description (#1298, #1313, #1317). + +## 1.0.44 + +1st November, 2026 + +Recently, I have been concentrating on maintenance and improvements to individual features. I began to wonder whether I had been focusing a little too closely on the details and could give more attention to the overall picture. That led me to a somewhat broader round of changes, including reworking parts of the architecture. Along the way, I was able to bring several implementations of the same concepts together, introduce shared abstractions, and tidy up the codebase a little. + +Please let me know if any problems arise from these changes. + +### Fixed + +#### Synchronisation + +- Files containing a raw unit separator (`U+001F`) are no longer truncated when received through S3-compatible Object Storage. (#1303) + - Update every participating client to prevent further truncation. Updating alone does not repair copies already damaged. + - Before recovery, preserve verified intact copies, unique unsynchronised edits, and any required history, then follow the [recovery procedure](https://github.com/vrtmrz/obsidian-livesync/blob/1.0.44/docs/recovery.md#recover-files-truncated-during-object-storage-reception). + - Rebuild and Fetch leave Hidden File Sync and Customisation Sync disabled. Initialise them deliberately after ordinary file recovery is complete. + +- We can continue CouchDB synchronisation after rebuilding an encrypted remote when **Use Internal API** is enabled. (#1234) + - Cached responses encrypted with the previous Security Seed no longer interfere with subsequent reads. + +#### Conflict resolution + +- A queued conflict check no longer gets stuck waiting for a response to an outdated comparison of the same file. + +#### Hidden File Sync and Customisation Sync + +- Newer local hidden files are no longer omitted when initialising Hidden File Sync with `Merge`. +- Refreshing the Customisation Sync catalogue no longer brings back outdated items from an earlier scan. + +### Improved + +#### Hidden File Sync + +- We can review the latest hidden-file revisions when changes arrive during a manual JSON comparison. + - The earlier comparison closes, and the latest revisions are checked again. If another device has resolved the conflict, the comparison closes without asking for another selection. + +### Sunset + +#### Recovery + +- Recovery from old cleanup locks is now unified around the standard recovery dialogue, including on devices using the legacy IndexedDB adapter. + - Use Fetch to reconstruct the local database from the remote, unlock the device when both databases are already aligned, or dismiss the dialogue. + - These choices replace the legacy in-place `Cleanup` operation. Garbage Collection V3 remains available. + - Before Fetch, preserve unsynchronised work and any required local history, as it replaces the local database. + +### CLI + +#### Fixed + +- We can continue using the reconstructed database after CLI `fetch` or `rebuild`, avoiding the empty-database restart and file deletion reported in #1304. + - Subsequent commands retain the selected database even without `--write-settings`. Interrupted reconstruction blocks ordinary synchronisation until recovery. + - Older runtime state with no recorded owner requires explicit reconstruction from a verified source. Back up unique edits before choosing Fetch or Rebuild. + +- The CLI mirror starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. + +### Miscellaneous + +- Installation and update downloads are smaller. +- Database initialisation handles encrypted change notifications more reliably. (Commonlib #191) + +### Acknowledgements + +Thank you for your contributions! + +- @AdrianNicu, for improving decryption handling during database initialisation (Commonlib #191). + ## 1.0.43 30th October, 2026 diff --git a/updates.md b/updates.md index 47b22b51..adc622e5 100644 --- a/updates.md +++ b/updates.md @@ -139,150 +139,3 @@ If we use P2P synchronisation, let us make sure all our devices are on the same ### Documentation - Clarified backend selection and hosting requirements, including Object Storage request costs. Added a guide to P2P diagnostic observations, logs, and transfer progress. - -## 1.0.45 - -5th November, 2026 - -Some of my recent replies have been sporting the Codex icon. Rest assured: I simply ask Codex to polish my English and discreetly intercept the occasional ill-judged remark before I post. - -### New Feature - -#### Synchronisation - -- We can recognise equivalent composed and decomposed Unicode file names as the same file when synchronising through Object Storage or P2P. (#1309) - - Select **Unicode NFC path identity** under **General Settings** → **File identity**. Supported by the CLI and Obsidian; physical file names and Chunk IDs remain unchanged. - - To migrate a Legacy database, update every participating client, stop synchronisation and editing, and preserve a complete, backed-up authoritative Vault, unique edits, and required history. Resolve colliding names before Rebuild on the source and Fetch on the remaining devices. With P2P, Rebuild prepares only the source database; keep that device online for Fetch. - - New CLI settings and Obsidian manual setup use NFC. Existing settings and older Setup URIs retain Legacy identity; initial CouchDB or Object Storage setup follows the remote's declared identity. - - Object Storage remotes with existing Journals but no compatibility declaration also require Rebuild, even in Legacy mode. - -### Fixed - -#### Object Storage - -- Journal receive failures now stop replication instead of silently passing over records which could not be decoded, processed, or saved. (#1318) - - After correcting the cause, retry replication with the same local database to receive the failed Journal again. - - If a Journal cannot be recovered and missing content or history is acceptable, use **Maintenance** → **Syncing** → **Skip stopped Journal and resume**. The confirmation identifies its exact key and failure stage. Only that Journal is skipped; the next failure still stops receipt. - - Journals already passed over by an earlier version require **Reset journal received history** to be replayed. This re-reads Journals still available remotely, except those sent by this device, and cannot recreate missing remote Journals. - -- A remote-only Rebuild resends the local database even when the Object Storage remote is already empty and this device retains its earlier transfer progress. - -#### Synchronisation - -- CouchDB synchronisation and Fetch no longer fail with decryption errors caused by stale encrypted responses in the Internal API cache after a Rebuild or plug-in restart. (#1234) - - Reads use a cache query parameter based on the current start-up and Security Seed. - - **Use headers only for HTTP cache control (compatibility)** under **Power users** → **CouchDB Connection Tweak** is disabled by default. Enable it only if the server or proxy rejects the additional query parameter; affected native caches may then reuse stale encrypted responses after Rebuild. - -#### Conflict resolution - -- Disabling or reloading the plug-in no longer causes a secondary error when pending work writes to the log. Late logs remain available in Obsidian's console. - - Open conflict dialogues for ordinary files close immediately, and queued checks report cancellation. - -#### Settings - -- The English description of **Do not split chunks in the background** now correctly states that enabling the setting moves Chunk splitting to the UI thread. (#1313) - -#### Translations - -- The notice shown when a translation is available now displays the **Display Language** setting label correctly in English, French, Hebrew, Russian, and Simplified Chinese. Hebrew interface messages also display their referenced labels correctly. (#1298) - -#### WebApp - -- The experimental WebApp rejects unsupported NFC path identity before opening its database or processing Vault files, preserving the selected files and settings. - - WebApp remains Legacy-only. Changing the setting does not migrate an NFC database. - -### Improved - -#### Translations - -- More of the interface is available in Japanese, including dialogue buttons, Vault history, SyncGuard, inspection reports, and Fetch restart confirmations. Japanese setup and settings guides have also been refreshed. (#1317) - -### CLI - -#### New Feature - -- We can resume Object Storage receipt after deliberately skipping one unrecoverable Journal with `journal-skip `. - - Retry `sync` with `--debug` to inspect the stopped Journal key and failure stage. The command retries replication first and skips only the supplied Journal if it remains stopped. - - Use this only when missing content or history is acceptable. The next Journal failure still stops receipt. - -#### Fixed - -- `fetch` and `rebuild` no longer replay pending file updates from the previous database at start-up. The old pending updates are discarded only after the local database reset succeeds. -- `fetch` and `daemon` can write received files to the Vault when `handleFilenameCaseSensitive` is enabled. The CLI detects whether the Vault filesystem distinguishes letter case. (#1319) -- NFC file lookups and directory scans report permission and I/O failures. Unreadable files are no longer treated as absent, and incomplete scans no longer count as successful. - -### Miscellaneous - -- Updated dependencies. - -### Acknowledgements - -Thank you for your contributions! - -- @cwatanab, for improving Japanese translations and documentation, correcting translated labels, and correcting the Chunk splitting setting description (#1298, #1313, #1317). - -## 1.0.44 - -1st November, 2026 - -Recently, I have been concentrating on maintenance and improvements to individual features. I began to wonder whether I had been focusing a little too closely on the details and could give more attention to the overall picture. That led me to a somewhat broader round of changes, including reworking parts of the architecture. Along the way, I was able to bring several implementations of the same concepts together, introduce shared abstractions, and tidy up the codebase a little. - -Please let me know if any problems arise from these changes. - -### Fixed - -#### Synchronisation - -- Files containing a raw unit separator (`U+001F`) are no longer truncated when received through S3-compatible Object Storage. (#1303) - - Update every participating client to prevent further truncation. Updating alone does not repair copies already damaged. - - Before recovery, preserve verified intact copies, unique unsynchronised edits, and any required history, then follow the [recovery procedure](docs/recovery.md#recover-files-truncated-during-object-storage-reception). - - Rebuild and Fetch leave Hidden File Sync and Customisation Sync disabled. Initialise them deliberately after ordinary file recovery is complete. - -- We can continue CouchDB synchronisation after rebuilding an encrypted remote when **Use Internal API** is enabled. (#1234) - - Cached responses encrypted with the previous Security Seed no longer interfere with subsequent reads. - -#### Conflict resolution - -- A queued conflict check no longer gets stuck waiting for a response to an outdated comparison of the same file. - -#### Hidden File Sync and Customisation Sync - -- Newer local hidden files are no longer omitted when initialising Hidden File Sync with `Merge`. -- Refreshing the Customisation Sync catalogue no longer brings back outdated items from an earlier scan. - -### Improved - -#### Hidden File Sync - -- We can review the latest hidden-file revisions when changes arrive during a manual JSON comparison. - - The earlier comparison closes, and the latest revisions are checked again. If another device has resolved the conflict, the comparison closes without asking for another selection. - -### Sunset - -#### Recovery - -- Recovery from old cleanup locks is now unified around the standard recovery dialogue, including on devices using the legacy IndexedDB adapter. - - Use Fetch to reconstruct the local database from the remote, unlock the device when both databases are already aligned, or dismiss the dialogue. - - These choices replace the legacy in-place `Cleanup` operation. Garbage Collection V3 remains available. - - Before Fetch, preserve unsynchronised work and any required local history, as it replaces the local database. - -### CLI - -#### Fixed - -- We can continue using the reconstructed database after CLI `fetch` or `rebuild`, avoiding the empty-database restart and file deletion reported in #1304. - - Subsequent commands retain the selected database even without `--write-settings`. Interrupted reconstruction blocks ordinary synchronisation until recovery. - - Older runtime state with no recorded owner requires explicit reconstruction from a verified source. Back up unique edits before choosing Fetch or Rebuild. - -- The CLI mirror starts correctly on Windows when the Vault path is omitted and the database directory is also used as the Vault. - -### Miscellaneous - -- Installation and update downloads are smaller. -- Database initialisation handles encrypted change notifications more reliably. (Commonlib #191) - -### Acknowledgements - -Thank you for your contributions! - -- @AdrianNicu, for improving decryption handling during database initialisation (Commonlib #191).