Fixed and implemented

- Readme tidied.
- More faster e2ee.
- URI and databasename check improved.
This commit is contained in:
vorotamoroz
2022-06-23 18:26:43 +09:00
parent 3e76292aa7
commit 124a49b80f
8 changed files with 37 additions and 47 deletions

View File

@@ -1,32 +1,25 @@
# Self-hosted LiveSync # Self-hosted LiveSync
Sorry for late! [Japanese docs](./README_ja.md) is also coming up. [Japanese docs](./README_ja.md).
**Renamed from: obsidian-livesync** Self-hosted LiveSync is a community implemented synchronization plugin.
It uses Self-hosted or you procured CouchDB as the server. Available on every obsidian installed devices.
Using a self-hosted database, live-sync to multi-devices bidirectionally. Note: It has no compatibilities with official "Sync".
Runs in Mac, Android, Windows, and iOS. Perhaps available on Linux too.
Community implementation, not compatible with official "Sync".
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif) ![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
**It's getting almost stable now, But Please make sure to back your vault up!** If you install or upgrade LiveSync, please back your vault up.
Limitations: ~~Folder deletion handling is not completed.~~ **It would work now.** ## Features
## This plugin enables... - Visual conflict resolver included.
- Synchronize with other devices bidirectionally near-real-time
- Runs in Windows, Mac, iPad, iPhone, Android, Chromebook - You can use CouchDB or its compatibles like IBM Cloudant.
- Synchronize to Self-hosted Database - End-to-End encryption.
- Replicate to/from other devices bidirectionally near-real-time - Plugin synchronization(Beta)
- Resolving synchronizing conflicts in the Obsidian.
- You can use CouchDB or its compatibles like IBM Cloudant. CouchDB is OSS, and IBM Cloudant has the terms and certificates about security. Your notes are yours.
- Off-line sync is also available.
- End-to-End encryption is available (beta).
- Receive WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) (End-to-End encryption will not be applicable.) - Receive WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) (End-to-End encryption will not be applicable.)
It must be useful for the Researcher, Engineer, Developer who has to keep NDA or something like agreement. It must be useful for the Researcher, Engineer, Developer who has to keep NDA or something like agreement. Especially, in some companies, people have to store all data to their fully controlled host, even End-to-End encryption applied.
Especially, in some companies, people have to store all data to their fully controlled host, even End-to-End encryption applied.
## IMPORTANT NOTICE ## IMPORTANT NOTICE
@@ -38,7 +31,6 @@ Especially, in some companies, people have to store all data to their fully cont
## How to use ## How to use
### Get your database ready. ### Get your database ready.
First, get your database ready. IBM Cloudant is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below: First, get your database ready. IBM Cloudant is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below:
@@ -61,7 +53,7 @@ First, get your database ready. IBM Cloudant is preferred for testing. Or you ca
1. Enabling `Show staus inside editor` bring you information. While edit mode, you can see the status on the top-right of the editor. (Recommended) 1. Enabling `Show staus inside editor` bring you information. While edit mode, you can see the status on the top-right of the editor. (Recommended)
2. Enabling `Use history` let you see the diffs between your edit and synchronization. (Recommended) 2. Enabling `Use history` let you see the diffs between your edit and synchronization. (Recommended)
5. Back to the editor. I hope that initial scan is in the progress or done. 5. Back to the editor. I hope that initial scan is in the progress or done.
6. When status became stabilized (All ⏳ and 🧩 disappeared), you are ready to synchronize with the server. 6. When status became stabilized (All ⏳ and 🧩 have been disappeared), you are ready to synchronize with the server.
7. Press the replicate icon on the Ribbon or run `Replicate now` from the Command pallet. You'll send all your data to the server. 7. Press the replicate icon on the Ribbon or run `Replicate now` from the Command pallet. You'll send all your data to the server.
8. Open the command palette, `Copy setup URI`, and set the passphrase to encrypt the information. Then your configuration will be copied to the clipboard. Please share copied URI with your other devices. 8. Open the command palette, `Copy setup URI`, and set the passphrase to encrypt the information. Then your configuration will be copied to the clipboard. Please share copied URI with your other devices.
**IMPORTANT NOTICE: BE CAREFUL TO TREAT THIS URI. THE URI CONTAINS YOUR CREDENTIALS EVEN THOUGH NOBODY COULD READ WITHOUT THE PASSPHRASE.** **IMPORTANT NOTICE: BE CAREFUL TO TREAT THIS URI. THE URI CONTAINS YOUR CREDENTIALS EVEN THOUGH NOBODY COULD READ WITHOUT THE PASSPHRASE.**
@@ -71,20 +63,19 @@ First, get your database ready. IBM Cloudant is preferred for testing. Or you ca
Strongly recommend using the vault in which all files are completely synchronized including timestamps. Otherwise, some files will be corrupted if failed to resolve conflicts. To simplify, I recommend using a new empty vault. Strongly recommend using the vault in which all files are completely synchronized including timestamps. Otherwise, some files will be corrupted if failed to resolve conflicts. To simplify, I recommend using a new empty vault.
1. Install the plug-in. 1. Install the plug-in.
2. Open the link that you copied from the other device. 2. Open the link that you had been copied to the other device.
1. If you are hard to open the link (i.e., in android), you can use `Open setup URI` from the command palette and paste the URI into LiveSync manually.
3. The plug-in asks you that are you sure to apply the configurations. Please answer `Yes` and the following instruction below: 3. The plug-in asks you that are you sure to apply the configurations. Please answer `Yes` and the following instruction below:
1. Answer `Yes` to `Keep local DB?`. 1. Answer `Yes` to `Keep local DB?`.
*Note: If you started with existed vault, you have to answer `No`. And `No` to `Rebuild the database?`.* *Note: If you started with existed vault, you have to answer `No`. And `No` to `Rebuild the database?`.*
2. Answer `Yes` to `Keep remote DB?`. 2. Answer `Yes` to `Keep remote DB?`.
3. Answer `Yes` to `Unlock and replicate?`. 3. Answer `Yes` to `Replicate once?`.
Yes, you have to answer `Yes` to everything. Yes, you have to answer `Yes` to everything.
Then, all your settings are copied from the first device. Then, all your settings are copied from the first device.
4. Your notes will arrive soon. 4. Your notes will arrive soon.
## Something looks corrupted... ## Something looks corrupted...
Please open the link again and answer as below: Please open the link again and Answer as below:
- If your local database looks corrupted - If your local database looks corrupted
(in other words, when your Obsidian getting weird even standalone.) (in other words, when your Obsidian getting weird even standalone.)
- Answer `No` to `Keep local DB?` - Answer `No` to `Keep local DB?`
@@ -94,18 +85,12 @@ Please open the link again and answer as below:
If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized including a timestamp. So you can use the existed vault). If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized including a timestamp. So you can use the existed vault).
## Test Server ## Test Server
Setting up an instance of Cloudant or local CouchDB is a little complicated, so I made the [Tasting server of self-hosted-livesync](https://olstaste.vrtmrz.net/) up. Try free! Setting up an instance of Cloudant or local CouchDB is a little complicated, so I made the [Tasting server of self-hosted-livesync](https://olstaste.vrtmrz.net/) up. Try free!
Note: Please read "Limitations" carefully. Do not send your private vault. Note: Please read "Limitations" carefully. Do not send your private vault.
## WebClipper is also available. ## Information in StatusBar
Available from on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are work in progress.)
# Information in StatusBar
Synchronization status is shown in statusbar. Synchronization status is shown in statusbar.
@@ -116,16 +101,14 @@ Synchronization status is shown in statusbar.
- ⚠ Error occurred. - ⚠ Error occurred.
- ↑ Uploaded pieces - ↑ Uploaded pieces
- ↓ Downloaded pieces - ↓ Downloaded pieces
-Count of the pending process -Number of the pending processes
- 🧩 Count of the files that waiting for their chunks. - 🧩 Number of the files that waiting for their chunks.
If you have deleted or renamed files, please wait until this disappears. If you have deleted or renamed files, please wait until disappeared.
## Supplements ## Hints
(This section will be rewrited soon)
- When the file has been deleted, the deletion of the file is replicated to other devices.
- When the folder became empty by replication, The folder will be deleted in the default setting. But you can change this behaivour. Check the [Settings](docs/settings.md). - When the folder became empty by replication, The folder will be deleted in the default setting. But you can change this behaivour. Check the [Settings](docs/settings.md).
- LiveSync drains many batteries in mobile devices. - LiveSync mode drains many batteries in mobile devices. Periodic sync and some automatic sync is recommended.
- Mobile Obsidian can not connect to the non-secure(HTTP) or local CA-signed servers, even though the certificate is stored in the device store. - Mobile Obsidian can not connect to the non-secure(HTTP) or local CA-signed servers, even though the certificate is stored in the device store.
- There are no 'exclude_folders' like configurations. - There are no 'exclude_folders' like configurations.
- When synchronized, files are compared by their modified times and overwritten by the newer ones once. Then plugin checks the conflicts and if a merge is needed, the dialog will open. - When synchronized, files are compared by their modified times and overwritten by the newer ones once. Then plugin checks the conflicts and if a merge is needed, the dialog will open.
@@ -135,7 +118,11 @@ If you have deleted or renamed files, please wait until this disappears.
- Q: Database is growing, how can I shrink it up? - Q: Database is growing, how can I shrink it up?
A: each of the docs is saved with their old 100 revisions to detect and resolve confliction. Picture yourself that one device has been off the line for a while, and joined again. The device has to check his note and remote saved note. If exists in revision histories of remote notes even though the device's note is a little different from the latest one, it could be merged safely. Even if that is not in revision histories, we only have to check differences after the revision that both devices commonly have. This is like The git's conflict resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem. A: each of the docs is saved with their old 100 revisions to detect and resolve confliction. Picture yourself that one device has been off the line for a while, and joined again. The device has to check his note and remote saved note. If exists in revision histories of remote notes even though the device's note is a little different from the latest one, it could be merged safely. Even if that is not in revision histories, we only have to check differences after the revision that both devices commonly have. This is like The git's conflict resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- And more technical Information are in the [Technical Information](docs/tech_info.md) - And more technical Information are in the [Technical Information](docs/tech_info.md)
- If you want to synchronize files without obsidian, you can use [filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available.
Available from on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are work in progress.)
# License ## License
The source code is licensed MIT. The source code is licensed MIT.

View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.11.6", "version": "0.11.7",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.11.6", "version": "0.11.7",
"lockfileVersion": 2, "lockfileVersion": 2,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.11.6", "version": "0.11.7",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",

View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.11.6", "version": "0.11.7",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",

View File

@@ -4,7 +4,7 @@ import { path2id, id2path } from "./utils";
import { NewNotice, runWithLock } from "./lib/src/utils"; import { NewNotice, runWithLock } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { checkSyncInfo, connectRemoteCouchDBWithSetting } from "./utils_couchdb"; import { checkSyncInfo, connectRemoteCouchDBWithSetting } from "./utils_couchdb";
import { testCrypt } from "./lib/src/e2ee"; import { testCrypt } from "./lib/src/e2ee_v2";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
export class ObsidianLiveSyncSettingTab extends PluginSettingTab { export class ObsidianLiveSyncSettingTab extends PluginSettingTab {

Submodule src/lib updated: 6451afd112...4decf16d62

View File

@@ -29,7 +29,8 @@ import { DocumentHistoryModal } from "./DocumentHistoryModal";
//@ts-ignore //@ts-ignore
import PluginPane from "./PluginPane.svelte"; import PluginPane from "./PluginPane.svelte";
import { id2path, path2id } from "./utils"; import { id2path, path2id } from "./utils";
import { decrypt, encrypt } from "./lib/src/e2ee"; import { decrypt, encrypt } from "./lib/src/e2ee_v2";
const isDebug = false; const isDebug = false;
setNoticeClass(Notice); setNoticeClass(Notice);
class PluginDialogModal extends Modal { class PluginDialogModal extends Modal {

View File

@@ -40,6 +40,8 @@ export const connectRemoteCouchDBWithSetting = (settings: RemoteDBSettings, isMo
const connectRemoteCouchDB = async (uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> => { const connectRemoteCouchDB = async (uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> => {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid"; if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name cound not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name cound not contain spaces.";
let authHeader = ""; let authHeader = "";
if (auth.username && auth.password) { if (auth.username && auth.password) {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${auth.username}:${auth.password}`)); const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${auth.username}:${auth.password}`));