- Fixed flaky timing issues in P2P synchronisation.
- Fixed more binary file handling issues in CLI.
### Tests
- Rewrite P2P end-to-end tests to use the CLI as host.
Now, Self-hosted LiveSync has finally begun to be split into the Self-hosted LiveSync plugin for Obsidian, and a properly abstracted version of it.
This may not offer much benefit to Obsidian plugin users, or might even cause a slight inconvenience, but I believe it will certainly help improve testability and make the ecosystem better.
However, I do not see the point in putting something with little benefit into beta, so I am handling this on the alpha branch. I would actually preferred to create an R&D branch, but I was not keen on the ampersand, and I feel it will eventually become a proper beta anyway.
### Refactored
- Separated `ObsidianLiveSyncPlugin` into `ObsidianLiveSyncPlugin` and `LiveSyncBaseCore`.
- Now `LiveSyncCore` indicates the type specified version of `LiveSyncBaseCore`.
- Referencing `plugin.xxx` has been rewritten to referencing the corresponding service or `core.xxx`.
### Internal API changes
- Storage Access APIs are now yielding Promises. This is to allow more limited storage platforms to be supported.
### R&D
- Browser-version of Self-hosted LiveSync is now in development. This is not intended for public use now, but I will eventually make it available for testing.
- We can see the code in `src/apps/webapp` for the browser version.
- `ModuleTargetFilter`, which was responsible for checking if a file is a target file, has been ported to a serviceFeature.
- And also tests have been added. The middleware-style-power.
- `ModuleObsidianAPI` has been removed and implemented in `APIService` and `RemoteService`.
- Now `APIService` is responsible for the network-online-status, not `databaseService.managers.networkManager`.
- Hidden file synchronisation now works!
- Now Hidden file synchronisation respects `.ignore` files.
- Replicator initialisation during rebuilding now works correctly.
### Refactored
- Some methods naming have been changed for better clarity, i.e., `_isTargetFileByLocalDB` is now `_isTargetAcceptedByLocalDB`.
### Fixed
- Unlocking the remote database after rebuilding has been fixed.
### Refactored
- Now `StorageEventManagerBase` is separated from `StorageEventManagerObsidian` following their concerns.
- Now `FileAccessBase` is separated from `FileAccessObsidian` following their concerns.
- No longer `No available splitter for settings!!` errors occur after fetching old remote settings while rebuilding local database.
### Improved
- Boot sequence warning is now kept in the in-editor notification area. (#748)
### New feature
- We can now set the maximum modified time for reflect events in the settings. (for #754)
### Refactored
- Module to service refactoring has been started for better maintainability:
- UI module has been moved to UI service.
### Behaviour change
- Default chunk splitter version has been changed to `Rabin-Karp` for new installations.
- Rewrite the service's binding/handler assignment systems
- Removed loopholes that allowed traversal between services to clarify dependencies.
- Consolidated the hidden state-related state, the handler, and the addition of bindings to the handler into a single object.
- Currently, functions that can have handlers added implement either addHandler or setHandler directly on the function itself.
I understand there are differing opinions on this, but for now, this is how it stands.
- Services now possess a Context. Please ensure each platform has a class that inherits from ServiceContext.
- To permit services to be dynamically bound, the services themselves are now defined by interfaces.
- Add some note for corrupting notice
- Array function standardised.
Fixed:
- Remove obsoleted accessing to Obsidian's global.
- Avoiding errors in exceptional circumstances.
- Removal of several outdated and inefficient QueueProcessor implementations
- Log output is now a bit efficient.
- The plug-in automatically fetches the missing chunks even if `Fetch chunks on demand` is disabled.
- This change is to avoid loss of data when receiving a bulk of revisions.
- This can be prevented by enabling `Use Only Local Chunks` in the settings.
- Storage application now saved during each event and restored on startup.
- Synchronisation result application is also now saved during each event and restored on startup.
- These may avoid some unexpected loss of data when the editor crashes.
### Fixed
- Now the plug-in waits for the application of pended batch changes before the synchronisation starts.
- This may avoid some unexpected loss or unexpected conflicts.
Plug-in sends custom headers correctly when RequestAPI is used.
- No longer causing unexpected chunk creation during `Reset synchronisation on This Device` with bucket sync.
### Refactored
- Synchronisation result application process has been refactored.
- Storage application process has been refactored.
- Please report if you find any unexpected behaviour after this update. A bit of large refactoring.
- We can analyse the local database with the `Analyse database usage` command.
- We can reset the notification threshold and check the remote usage at once with the `Reset notification threshold and check the remote database usage` command.
### Fixed
- Now the plug-in resets the remote size notification threshold after rebuild.
- Now fetching configuration from server can handle the empty remote correctly (reported on #756) by common-lib.
- No longer asking to switch adapters during rebuilding.
- Now we can switch the database adapter between IndexedDB and IDB without rebuilding (#747).
- No longer checking for the adapter by `Doctor`.
### Changes
- The default adapter is reverted to IDB to avoid memory leaks (#747).
### Fixed (?)
- Reverted QR code library to v1.4.4 (To make sure #752).
#### JWT Authentication
- Now we can use JWT Authentication ES512 correctly (#742).
- Several misdirections in the Setting dialogues have been fixed (i.e., seconds and minutes confusion...).
- The key area in the Setting dialogue has been enlarged and accepts newlines correctly.
- Caching of JWT tokens now works correctly
- Tokens are now cached and reused until they expire.
- They will be kept until 10% of the expiration duration is remaining or 10 seconds, whichever is longer (but at a maximum of 1 minute).
- JWT settings are now correctly displayed on the Setting dialogue.
#### Other fixes
- Receiving non-latest revisions no longer causes unexpected overwrites.
- On receiving revisions that made conflicting changes, we are still able to handle them.
### Improved
- No longer duplicated message notifications are shown when a connection to the remote server fails.
- Instead, a single notification is shown, and it will be kept on the notification area inside the editor until the situation is resolved.
- The notification area is no longer imposing, distracting, and overwhelming.
- With a pale background, but bordered and with icons.
### Fixed (This should be backported to 0.25.22 if the beta phase is prolonged)
- No longer larger files will not create a chunks during preparing `Reset Synchronisation on This Device`.
### Behaviour changes
- Setup wizard is now more `goal-oriented`. Brand-new screens are introduced.
- `Fetch everything` and `Rebuild everything` is now `Reset Synchronisation on This Device` and `Overwrite Server Data with This Device's Files`.
- Remote configuration and E2EE settings are now separated to each modal dialogue.
- Peer-to-Peer settings is also separated into its own modal dialogue.
- Setup-URI, and Report for the Issue are now not copied to clipboard automatically. Instead, there are copy dialogue and buttons to copy them explicitly.
- No longer optional features are introduced during the setup or `Reset Synchronisation on This Device`, `Overwrite Server Data with This Device's Files`.
- We cannot preform `Fetch everything` and `Rebuild everything` (Removed, so the old name) without restarting Obsidian now.
### Miscellaneous
- Setup QR Code generation is separated into a src/lib/src/API/processSetting.ts file. Please use it as a subrepository if you want to generate QR codes in your own application.
- Setup-URI is also separated into a src/lib/src/API/processSetting.ts
- Some direct access to web-APIs are now wrapped into the services layer.
### Dependency updates
- Many dependencies are updated. Please see `package.json`.
- As upgrading TypeScript, Fixed many UInt8Array<ArrayBuffer> and Uint8Array type mismatches.
- Fixed a bug that caused wrong event bindings and flag inversion (#727)
- This caused following issues:
- In some cases, settings changes were not applied or saved correctly.
- Automatic synchronisation did not begin correctly.
### Improved
- Too large diffs are not shown in the file comparison view, due to performance reasons.
- Fixed wrong event type bindings (which caused some events not to be handled correctly).
- Fixed detected a timing issue in StorageEventManager
- When multiple events for the same file are fired in quick succession, metadata has been kept older information. This induces unexpected wrong notifications and write prevention.
- Automatic translation detection on the first launch now works correctly (#630).
- No errors are shown during synchronisations in offline (if not explicitly requested) (#699).
- Missing some checking during automatic-synchronisation now works correctly.
- We can perform Garbage Collection (Beta2) without rebuilding the entire database, and also fetch the database.
### Fixed
- Resetting the bucket now properly clears all uploaded files.
### Refactored
- Some files have been moved to better reflect their purpose and improve maintainability.
- The extensive LiveSyncLocalDB has been split into separate files for each role.
### New feature
- Insecure chunk detection has been implemented.
### Fixed
- Unexpected `Failed to obtain PBKDF2 salt` or similar errors during bucket-synchronisation no longer occur.
- Unexpected long delays for chunk-missing documents when using bucket-synchronisation have been resolved.
- Fetched remote chunks are now properly stored in the local database if `Fetch chunks on demand` is enabled.
- The 'fetch' dialogue's message has been refined.
- No longer overwriting any corrupted documents to the storage on boot-sequence.
### Refactored
- Type errors have been corrected.
### Fixed
- Off-loaded chunking have been fixed to ensure proper functionality (#693).
- Chunk document ID assignment has been fixed.
- Replication prevention message during version up detection has been improved (#686).
- `Keep A` and `Keep B` on Conflict resolving dialogue has been renamed to `Use Base` and `Use Conflicted` (#691).
### Improved
- Metadata and content-size unmatched documents are now detected and reported, prevented to be applied to the storage.
### New Features
- `Scan for Broken files` has been implemented on `Hatch` -> `TroubleShooting`.
### Refactored
- Off-loaded processes have been refactored for the better maintainability.
- Removed unused code.
- Storage scanning no longer occurs when `Suspend file watching` is enabled (including boot-sequence).
### Improved
- Saving notes and files now consumes less memory.
- Chunk caching is now more efficient.
- Both of them (may) are effective for #692, #680, and some more.
### Changed
- `Incubate Chunks in Document` (also known as `Eden`) is now fully sunset.
- The `Compute revisions for chunks` setting has also been removed.
- As mentioned, `Memory cache size (by total characters)` has been removed.
### Refactored
- A significant refactoring of the core codebase is underway (please refer the release-note).
- The encryption algorithm now uses HKDF with a master key.
- `Fetch everything from the remote` now works correctly.
- Extra log messages during QR code decoding have been removed.
### Changed
- Some settings have been moved to the `Patches` pane:
### Behavioural and API Changes
- `DirectFileManipulatorV2` now requires new settings (as you may already know, E2EEAlgorithm).
- The database version has been increased to `12` from `10`.
### Fixed
- Batch Update is no longer available in LiveSync mode to avoid unexpected behaviour. (#653)
- Now compatible with Cloudflare R2 again for bucket synchronisation.
- Prevention of broken behaviour due to database connection failures added (#649).
### Improved
- We can use prefix for path for the Bucket synchronisation.
- The "Use Request API to avoid `inevitable` CORS problem" option is now promoted to the normal setting, not a niche patch.
### Fixed
- Now switching replicators applied immediately, without the need to restart Obsidian.
### Tidied up
- Some dependencies have been updated to the latest version.
### Fixed
- No longer conflicted files are handled in the boot-up process. No more unexpected overwriting.
- It ignores `Always overwrite with a newer file`, and always be prevented for the safety. Please pick it manually or open the file.
- Some log messages on conflict resolution has been corrected.
- Automatic merge notifications, displayed on the grounds of `same`, have been degraded to logs.
### Improved
- Now we can fetch the remote database with keeping local files completely intact.
- In new option, all files are stored into the local database before the fetching, and will be merged automatically or detected as conflicts.
- The dialogue presenting options when performing `Fetch` are now more informative.
### Refactored
- Some class methods have been fixed its arguments to be more consistent.
- Types have been defined for some conditional results.
### Fixed
- Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).
### Improved
- Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.
### Improved internally
- Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
### Improved
#### Peer-to-Peer
- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.
#### General
- New migration method has been implemented, that called `Doctor`.
- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.
### Behaviour and default changed
- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
### Refactored
- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
Improved
- New Translation: `es` (Spanish) by @zeedif (Thank you so much)!
- Now all of messages can be selectable and copyable, also on the iPhone, iPad, and Android devices. Now we can copy or share the messages easily.
New Feature
- Peer-to-Peer Synchronisation has been implemented!
Fixed
- No longer memory or resource leaks when the plug-in is disabled.
- Now deleted chunks are correctly detected on conflict resolution, and we are guided to resurrect them.
- Hanging issue during the initial synchronisation has been fixed.
- Some unnecessary logs have been removed.
- Now all modal dialogues are correctly closed when the plug-in is disabled.
Refactor
- Several interfaces have been moved to the separated library.
- Translations have been moved to each language file, and during the build, they are merged into one file.
- Non-mobile friendly code has been removed and replaced with the safer code.
- Started writing Platform impedance-matching-layer.
- Svelte has been updated to v5.
- Some function have got more robust type definitions.
- Terser optimisation has slightly improved.
- During the build, analysis meta-file of the bundled codes will be generated.
- Fixed the issue which the filename is shown as `undefined`.
- Fixed the issue where files transferred at short intervals were not reflected.
- Updated dependency (including new translation)
