Compare commits

..

7 Commits

Author SHA1 Message Date
vorotamoroz
00dfae24d7 bump 2025-04-10 14:28:03 +01:00
vorotamoroz
d8a41fe45d ### New Feature
- Now, we can send custom headers to the server.
- Authentication with JWT in CouchDB is now supported.

### Improved

- The QR Code for set-up can be shown also from the setting dialogue now.
- Conflict checking for preventing unexpected overwriting on the boot-up process has been quite faster.

### Fixed

- Some bugs on Dev and Testing modules have been fixed.
2025-04-10 14:24:33 +01:00
vorotamoroz
30467d1c25 Add link to P2P pseudo client. 2025-04-04 12:30:21 +01:00
vorotamoroz
f8351f1d45 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-04-04 11:52:12 +01:00
vorotamoroz
5924af98ab Update Library (Probably no effect) 2025-04-04 11:52:05 +01:00
vorotamoroz
2769b61da4 Update setup_own_server.md
Add note; #609
2025-04-04 18:24:13 +09:00
vorotamoroz
bb4409221d Update README.md 2025-04-03 20:57:15 +09:00
19 changed files with 490 additions and 107 deletions

View File

@@ -18,10 +18,11 @@ Note: This plugin cannot synchronise with the official "Obsidian Sync".
- Supporting End-to-end encryption.
- Synchronisation of settings, snippets, themes, and plug-ins, via [Customization sync(Beta)](#customization-sync) or [Hidden File Sync](#hiddenfilesync)
- WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
- WebRTC peer-to-peer synchronisation without the need any `host` is now possible. (Experimental)
- WebRTC peer-to-peer synchronisation without the need for any `host` is now possible. (Experimental)
- This feature is still in the experimental stage. Please be careful when using it.
- Instead of using public servers, you can use [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) the pseudo client for receiving and sending between devices.
- There is an [explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync), although it is in Japanese. I will write the English version soon.
- A pre-built instance is served at [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (in the vrtmrz blog site). This is of course also peer-to-peer. Feel free to use it.
- There is an [English explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync-en.html), and [Japanese explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync).
This plug-in might be useful for researchers, engineers, and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind of knowing that their notes are fully private.

View File

@@ -31,7 +31,7 @@ export hostname=localhost:5984
export username=goojdasjdas #Please change as you like.
export password=kpkdasdosakpdsa #Please change as you like
# Prepare directories which saving data and configurations.
# Prepare directories which save data and configurations.
mkdir couchdb-data
mkdir couchdb-etc
```
@@ -45,19 +45,19 @@ $ docker run --name couchdb-for-ols --rm -it -e COUCHDB_USER=${username} -e COUC
If your container has been exited, please check the permission of couchdb-data, and couchdb-etc.
Once CouchDB run, these directories will be owned by uid:`5984`. Please chown it for you again.
2. Enable it in background
2. Enable it in the background
```
$ docker run --name couchdb-for-ols -d --restart always -e COUCHDB_USER=${username} -e COUCHDB_PASSWORD=${password} -v ${PWD}/couchdb-data:/opt/couchdb/data -v ${PWD}/couchdb-etc:/opt/couchdb/etc/local.d -p 5984:5984 couchdb
```
### B. Install CouchDB directly
Please refer the [official document](https://docs.couchdb.org/en/stable/install/index.html). However, we do not have to configure it fully. Just administrator needs to be configured.
Please refer to the [official document](https://docs.couchdb.org/en/stable/install/index.html). However, we do not have to configure it fully. Just the administrator needs to be configured.
## 2. Run couchdb-init.sh for initialise
```
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
```
If it results like following:
If it results like the following:
```
-- Configuring CouchDB by REST APIs... -->
{"ok":true}
@@ -80,7 +80,7 @@ Your CouchDB has been initialised successfully. If you want this manually, pleas
- You can skip this instruction if you using only in intranet and only with desktop devices.
- For mobile devices, Obsidian requires a valid SSL certificate. Usually, it needs exposing the internet.
Whatever solutions we can use. For the simplicity, following sample uses Cloudflare Zero Trust for testing.
Whatever solutions we can use. For simplicity, the following sample uses Cloudflare Zero Trust for testing.
```
cloudflared tunnel --url http://localhost:5984
@@ -99,12 +99,12 @@ You will then get the following output:
:
:
```
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into background once please.
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into the background once, please.
## 4. Client Setup
> [!TIP]
> Now manually configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set.
> Now manual configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set.
### 1. Generate the setup URI on a desktop device or server
```bash
@@ -116,6 +116,13 @@ export password=abc123
deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts
```
> [!TIP]
> What is the `passphrase`? Is it different from `uri_passphrase`?
> Yes, the `passphrase` we have exported now is for an End-to-End Encryption passphrase.
> And, `uri_passphrase` that used in the `generate_setupuri.ts` is a different one; for decrypting Set-up URI at using that.
> Why: I (vorotamoroz) think that the passphrase of the Setup-URI should be different from the E2EE passphrase to prevent exposure caused by operational errors or the possibility of evil in our environment. On top of that, I believe that it is desirable for the Setup-URI to be random. Setup-URI is inevitably long, so it goes through the clipboard. I think that its passphrase should not go through the same path, so it should essentially be typed manually.
> Hence, if we keep empty for uri_passphrase, generate_setupuri.ts generates an adjective-noun-randomnumber passphrase so that we can remember it without going through the clipboard.
You will then get the following output:
```bash

View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.24.22",
"version": "0.24.23",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

18
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "0.24.22",
"version": "0.24.23",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "0.24.22",
"version": "0.24.23",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.645.0",
@@ -18,7 +18,7 @@
"fflate": "^0.8.2",
"idb": "^8.0.2",
"minimatch": "^10.0.1",
"octagonal-wheels": "^0.1.24",
"octagonal-wheels": "^0.1.25",
"qrcode-generator": "^1.4.4",
"svelte-check": "^4.1.4",
"trystero": "^0.20.1",
@@ -8514,9 +8514,9 @@
}
},
"node_modules/octagonal-wheels": {
"version": "0.1.24",
"resolved": "https://registry.npmjs.org/octagonal-wheels/-/octagonal-wheels-0.1.24.tgz",
"integrity": "sha512-ywzcq3FyW/xM37RhXkgwkERzgO6hG7uQHfpqHKcvPaT0H54e0/WoWEV65A2ttGp7Kxrq+UgXxVM1UT+KcSbPkA==",
"version": "0.1.25",
"resolved": "https://registry.npmjs.org/octagonal-wheels/-/octagonal-wheels-0.1.25.tgz",
"integrity": "sha512-WXe+AKgDlYU9FZ2/CbRXeTvpL0xknrL1NuG27+E6Vzg4RmeyWh8hM4lItGCiSqvAGbm8atn50WtaMSY7y5qfGg==",
"license": "MIT",
"dependencies": {
"idb": "^8.0.2"
@@ -17577,9 +17577,9 @@
}
},
"octagonal-wheels": {
"version": "0.1.24",
"resolved": "https://registry.npmjs.org/octagonal-wheels/-/octagonal-wheels-0.1.24.tgz",
"integrity": "sha512-ywzcq3FyW/xM37RhXkgwkERzgO6hG7uQHfpqHKcvPaT0H54e0/WoWEV65A2ttGp7Kxrq+UgXxVM1UT+KcSbPkA==",
"version": "0.1.25",
"resolved": "https://registry.npmjs.org/octagonal-wheels/-/octagonal-wheels-0.1.25.tgz",
"integrity": "sha512-WXe+AKgDlYU9FZ2/CbRXeTvpL0xknrL1NuG27+E6Vzg4RmeyWh8hM4lItGCiSqvAGbm8atn50WtaMSY7y5qfGg==",
"requires": {
"idb": "^8.0.2"
}

View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.24.22",
"version": "0.24.23",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -77,7 +77,7 @@
"fflate": "^0.8.2",
"idb": "^8.0.2",
"minimatch": "^10.0.1",
"octagonal-wheels": "^0.1.24",
"octagonal-wheels": "^0.1.25",
"qrcode-generator": "^1.4.4",
"svelte-check": "^4.1.4",
"trystero": "^0.20.1",

View File

@@ -10,6 +10,7 @@ export const EVENT_REQUEST_OPEN_SETTINGS = "request-open-settings";
export const EVENT_REQUEST_OPEN_SETTING_WIZARD = "request-open-setting-wizard";
export const EVENT_REQUEST_OPEN_SETUP_URI = "request-open-setup-uri";
export const EVENT_REQUEST_COPY_SETUP_URI = "request-copy-setup-uri";
export const EVENT_REQUEST_SHOW_SETUP_QR = "request-show-setup-qr";
export const EVENT_REQUEST_RELOAD_SETTING_TAB = "reload-setting-tab";
@@ -35,6 +36,7 @@ declare global {
[EVENT_REQUEST_OPEN_P2P]: undefined;
[EVENT_REQUEST_OPEN_SETUP_URI]: undefined;
[EVENT_REQUEST_COPY_SETUP_URI]: undefined;
[EVENT_REQUEST_SHOW_SETUP_QR]: undefined;
[EVENT_REQUEST_RUN_DOCTOR]: string;
}
}

Submodule src/lib updated: 1e24d2af19...ad3f7ee995

View File

@@ -18,6 +18,7 @@ import {
type AUTO_MERGED,
type RemoteDBSettings,
type TweakValues,
type CouchDBCredentials,
} from "./lib/src/common/types.ts";
import { type FileEventItem } from "./common/types.ts";
import { type SimpleStore } from "./lib/src/common/utils.ts";
@@ -283,16 +284,14 @@ export default class ObsidianLiveSyncPlugin
$$connectRemoteCouchDB(
uri: string,
auth: {
username: string;
password: string;
},
auth: CouchDBCredentials,
disableRequestURI: boolean,
passphrase: string | false,
useDynamicIterationCount: boolean,
performSetup: boolean,
skipInfo: boolean,
compression: boolean
compression: boolean,
customHeaders: Record<string, string>
): Promise<
| string
| {

View File

@@ -81,9 +81,9 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
this._log("Collecting local files on the DB", LOG_LEVEL_VERBOSE);
const _DBEntries = [] as MetaEntry[];
// const _DBEntriesTask = [] as (() => Promise<MetaEntry | false>)[];
let count = 0;
for await (const doc of this.localDatabase.findAllNormalDocs()) {
// Fetch all documents from the database (including conflicts to prevent overwriting).
for await (const doc of this.localDatabase.findAllNormalDocs({ conflicts: true })) {
count++;
if (count % 25 == 0)
this._log(
@@ -200,9 +200,8 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
if (w && !(w.deleted || w._deleted)) {
if (!this.core.$$isFileSizeExceeded(w.size)) {
// Prevent applying the conflicted state to the storage.
const conflicted = await this.core.databaseFileAccess.getConflictedRevs(path);
if (conflicted.length > 0) {
this._log(`UPDATE STORAGE: ${path} has conflicts. skipped`, LOG_LEVEL_INFO);
if (w._conflicts?.length ?? 0 > 0) {
this._log(`UPDATE STORAGE: ${path} has conflicts. skipped (x)`, LOG_LEVEL_INFO);
return;
}
// await this.pullFile(path, undefined, false, undefined, false);
@@ -237,8 +236,7 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
runAll("SYNC DATABASE AND STORAGE", fileMap, async (e) => {
const { file, doc } = e;
// Prevent applying the conflicted state to the storage.
const conflicted = await this.core.databaseFileAccess.getConflictedRevs(file.path);
if (conflicted.length > 0) {
if (doc._conflicts?.length ?? 0 > 0) {
this._log(`SYNC DATABASE AND STORAGE: ${file.path} has conflicts. skipped`, LOG_LEVEL_INFO);
return;
}

View File

@@ -1,7 +1,16 @@
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_DEBUG, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { Notice, requestUrl, type RequestUrlParam, type RequestUrlResponse } from "../../deps.ts";
import { type EntryDoc, type FilePathWithPrefix } from "../../lib/src/common/types.ts";
import {
type CouchDBCredentials,
type EntryDoc,
type FilePathWithPrefix,
type JWTCredentials,
type JWTHeader,
type JWTParams,
type JWTPayload,
type PreparedJWT,
} from "../../lib/src/common/types.ts";
import { getPathFromTFile } from "../../common/utils.ts";
import {
disableEncryption,
@@ -13,7 +22,9 @@ import {
import { setNoticeClass } from "../../lib/src/mock_and_interop/wrapper.ts";
import { ObsHttpHandler } from "./APILib/ObsHttpHandler.ts";
import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.ts";
import { reactive, reactiveSource } from "octagonal-wheels/dataobject/reactive";
import { reactive, reactiveSource } from "octagonal-wheels/dataobject/reactive.js";
import { arrayBufferToBase64Single, writeString } from "../../lib/src/string_and_binary/convert.ts";
import { Refiner } from "octagonal-wheels/dataobject/Refiner";
setNoticeClass(Notice);
@@ -99,29 +110,166 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
}
}
_importKey(auth: JWTCredentials) {
if (auth.jwtAlgorithm == "HS256" || auth.jwtAlgorithm == "HS512") {
const key = (auth.jwtKey || "").trim();
if (key == "") {
throw new Error("JWT key is empty");
}
const binaryDerString = window.atob(key);
const binaryDer = new Uint8Array(binaryDerString.length);
for (let i = 0; i < binaryDerString.length; i++) {
binaryDer[i] = binaryDerString.charCodeAt(i);
}
const hashName = auth.jwtAlgorithm == "HS256" ? "SHA-256" : "SHA-512";
return crypto.subtle.importKey("raw", binaryDer, { name: "HMAC", hash: { name: hashName } }, true, [
"sign",
]);
} else if (auth.jwtAlgorithm == "ES256" || auth.jwtAlgorithm == "ES512") {
const pem = auth.jwtKey
.replace(/-----BEGIN [^-]+-----/, "")
.replace(/-----END [^-]+-----/, "")
.replace(/\s+/g, "");
// const pem = key.replace(/\s/g, "");
const binaryDerString = window.atob(pem);
const binaryDer = new Uint8Array(binaryDerString.length);
for (let i = 0; i < binaryDerString.length; i++) {
binaryDer[i] = binaryDerString.charCodeAt(i);
}
// const binaryDer = base64ToArrayBuffer(pem);
const namedCurve = auth.jwtAlgorithm == "ES256" ? "P-256" : "P-521";
const param = { name: "ECDSA", namedCurve };
return crypto.subtle.importKey("pkcs8", binaryDer, param, true, ["sign"]);
} else {
throw new Error("Supplied JWT algorithm is not supported.");
}
}
_currentCryptoKey = new Refiner<JWTCredentials, CryptoKey>({
evaluation: async (auth, previous) => {
return await this._importKey(auth);
},
});
_jwt = new Refiner<JWTParams, PreparedJWT>({
evaluation: async (params, previous) => {
const encodedHeader = btoa(JSON.stringify(params.header));
const encodedPayload = btoa(JSON.stringify(params.payload));
const buff = `${encodedHeader}.${encodedPayload}`.replace(/\+/g, "-").replace(/\//g, "_").replace(/=/g, "");
const key = await this._currentCryptoKey.update(params.credentials).value;
let token = "";
if (params.header.alg == "ES256" || params.header.alg == "ES512") {
const jwt = await crypto.subtle.sign(
{ name: "ECDSA", hash: { name: "SHA-256" } },
key,
writeString(buff)
);
token = (await arrayBufferToBase64Single(jwt))
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=/g, "");
} else if (params.header.alg == "HS256" || params.header.alg == "HS512") {
const jwt = await crypto.subtle.sign(
{ name: "HMAC", hash: { name: params.header.alg } },
key,
writeString(buff)
);
token = (await arrayBufferToBase64Single(jwt))
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=/g, "");
} else {
throw new Error("JWT algorithm is not supported.");
}
return {
...params,
token: `${buff}.${token}`,
} as PreparedJWT;
},
});
_jwtParams = new Refiner<JWTCredentials, JWTParams>({
evaluation(source, previous) {
const kid = source.jwtKid || undefined;
const sub = (source.jwtSub || "").trim();
if (sub == "") {
throw new Error("JWT sub is empty");
}
const algorithm = source.jwtAlgorithm || "";
if (!algorithm) {
throw new Error("JWT algorithm is not configured.");
}
if (algorithm != "HS256" && algorithm != "HS512" && algorithm != "ES256" && algorithm != "ES512") {
throw new Error("JWT algorithm is not supported.");
}
const header: JWTHeader = {
alg: source.jwtAlgorithm || "HS256",
typ: "JWT",
kid,
};
const iat = ~~(new Date().getTime() / 1000);
const exp = iat + (source.jwtExpDuration || 5) * 60; // 5 minutes
const payload = {
exp,
iat,
sub: source.jwtSub || "",
"_couchdb.roles": ["_admin"],
} satisfies JWTPayload;
return {
header,
payload,
credentials: source,
};
},
shouldUpdate(isDifferent, source, previous) {
if (isDifferent) {
return true;
}
if (!previous) {
return true;
}
// if expired.
const d = ~~(new Date().getTime() / 1000);
if (previous.payload.exp < d) {
// console.warn(`jwt expired ${previous.payload.exp} < ${d}`);
return true;
}
return false;
},
});
async $$connectRemoteCouchDB(
uri: string,
auth: { username: string; password: string },
auth: CouchDBCredentials,
disableRequestURI: boolean,
passphrase: string | false,
useDynamicIterationCount: boolean,
performSetup: boolean,
skipInfo: boolean,
compression: boolean
compression: boolean,
customHeaders: Record<string, string>
): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
const userNameAndPassword = auth.username && auth.password ? `${auth.username}:${auth.password}` : "";
if (this.authHeaderSource.value != userNameAndPassword) {
this.authHeaderSource.value = userNameAndPassword;
let authHeader = "";
if ("username" in auth) {
const userNameAndPassword = auth.username && auth.password ? `${auth.username}:${auth.password}` : "";
if (this.authHeaderSource.value != userNameAndPassword) {
this.authHeaderSource.value = userNameAndPassword;
}
authHeader = this.authHeader.value;
} else if ("jwtAlgorithm" in auth) {
const params = await this._jwtParams.update(auth).value;
const jwt = await this._jwt.update(params).value;
const token = jwt.token;
authHeader = `Bearer ${token}`;
}
const authHeader = this.authHeader.value;
// const _this = this;
const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {
adapter: "http",
auth,
auth: "username" in auth ? auth : undefined,
skip_setup: !performSetup,
fetch: async (url: string | Request, opts?: RequestInit) => {
let size = "";
@@ -146,9 +294,18 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
// --> native Fetch API.
try {
if (this.settings.enableDebugTools) {
// Issue #407
(opts!.headers as Headers).append("ngrok-skip-browser-warning", "123");
if (customHeaders) {
for (const [key, value] of Object.entries(customHeaders)) {
if (key && value) {
(opts!.headers as Headers).append(key, value);
}
}
// // Issue #407
// (opts!.headers as Headers).append("ngrok-skip-browser-warning", "123");
}
// debugger;
if (!("username" in auth)) {
(opts!.headers as Headers).append("authorization", authHeader);
}
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const response: Response = await fetch(url, opts);

View File

@@ -1,4 +1,4 @@
import { fireAndForget } from "octagonal-wheels/promises";
import { delay, fireAndForget } from "octagonal-wheels/promises";
import { __onMissingTranslation } from "../../lib/src/common/i18n";
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
@@ -6,6 +6,7 @@ import { eventHub } from "../../common/events";
import { enableTestFunction } from "./devUtil/testUtils.ts";
import { TestPaneView, VIEW_TYPE_TEST } from "./devUtil/TestPaneView.ts";
import { writable } from "svelte/store";
import type { FilePathWithPrefix } from "../../lib/src/common/types.ts";
export class ModuleDev extends AbstractObsidianModule implements IObsidianModule {
$everyOnloadStart(): Promise<boolean> {
@@ -98,9 +99,41 @@ export class ModuleDev extends AbstractObsidianModule implements IObsidianModule
}
async $everyOnLayoutReady(): Promise<boolean> {
if (!this.settings.enableDebugTools) return Promise.resolve(true);
if (await this.core.storageAccess.isExistsIncludeHidden("_SHOWDIALOGAUTO.md")) {
void this.core.$$showView(VIEW_TYPE_TEST);
}
// if (await this.core.storageAccess.isExistsIncludeHidden("_SHOWDIALOGAUTO.md")) {
// void this.core.$$showView(VIEW_TYPE_TEST);
// }
this.addCommand({
id: "test-create-conflict",
name: "Create conflict",
callback: async () => {
const filename = "test-create-conflict.md";
const content = `# Test create conflict\n\n`;
const w = await this.core.databaseFileAccess.store({
name: filename as FilePathWithPrefix,
path: filename as FilePathWithPrefix,
body: new Blob([content], { type: "text/markdown" }),
stat: {
ctime: new Date().getTime(),
mtime: new Date().getTime(),
size: content.length,
type: "file",
},
});
if (w) {
const id = await this.core.$$path2id(filename as FilePathWithPrefix);
const f = await this.core.localDatabase.getRaw(id);
console.log(f);
console.log(f._rev);
const revConflict = f._rev.split("-")[0] + "-" + (parseInt(f._rev.split("-")[1]) + 1).toString();
console.log(await this.core.localDatabase.bulkDocsRaw([f], { new_edits: false }));
console.log(
await this.core.localDatabase.bulkDocsRaw([{ ...f, _rev: revConflict }], { new_edits: false })
);
}
},
});
await delay(1);
return true;
}
testResults = writable<[boolean, string, string][]>([]);

View File

@@ -160,6 +160,10 @@ export class ModuleReplicateTest extends AbstractObsidianModule implements IObsi
}
async _dumpFileList(outFile?: string) {
if (!this.core || !this.core.storageAccess) {
this._log("No storage access", LOG_LEVEL_INFO);
return;
}
const files = this.core.storageAccess.getFiles();
const out = [] as any[];
const webcrypto = await getWebCrypto();

View File

@@ -94,6 +94,14 @@ export class ModuleObsidianSettings extends AbstractObsidianModule implements IO
region: settings.region,
secretKey: settings.secretKey,
useCustomRequestHandler: settings.useCustomRequestHandler,
bucketCustomHeaders: settings.bucketCustomHeaders,
couchDB_CustomHeaders: settings.couchDB_CustomHeaders,
useJWT: settings.useJWT,
jwtKey: settings.jwtKey,
jwtAlgorithm: settings.jwtAlgorithm,
jwtKid: settings.jwtKid,
jwtExpDuration: settings.jwtExpDuration,
jwtSub: settings.jwtSub,
};
settings.encryptedCouchDBConnection = await this.encryptConfigurationItem(
JSON.stringify(connectionSetting),

View File

@@ -191,6 +191,11 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractObsidianModule imp
delete saveData.couchDB_USER;
delete saveData.couchDB_PASSWORD;
delete saveData.passphrase;
delete saveData.jwtKey;
delete saveData.jwtKid;
delete saveData.jwtSub;
delete saveData.couchDB_CustomHeaders;
delete saveData.bucketCustomHeaders;
}
return saveData;
}

View File

@@ -9,7 +9,12 @@ import { configURIBase, configURIBaseQR } from "../../common/types.ts";
// import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.js";
import { decrypt, encrypt } from "../../lib/src/encryption/e2ee_v2.ts";
import { fireAndForget } from "../../lib/src/common/utils.ts";
import { EVENT_REQUEST_COPY_SETUP_URI, EVENT_REQUEST_OPEN_SETUP_URI, eventHub } from "../../common/events.ts";
import {
EVENT_REQUEST_COPY_SETUP_URI,
EVENT_REQUEST_OPEN_SETUP_URI,
EVENT_REQUEST_SHOW_SETUP_QR,
eventHub,
} from "../../common/events.ts";
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { decodeAnyArray, encodeAnyArray } from "../../common/utils.ts";
import qrcode from "qrcode-generator";
@@ -54,6 +59,7 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
});
eventHub.onEvent(EVENT_REQUEST_OPEN_SETUP_URI, () => fireAndForget(() => this.command_openSetupURI()));
eventHub.onEvent(EVENT_REQUEST_COPY_SETUP_URI, () => fireAndForget(() => this.command_copySetupURI()));
eventHub.onEvent(EVENT_REQUEST_SHOW_SETUP_QR, () => fireAndForget(() => this.encodeQR()));
return Promise.resolve(true);
}
async encodeQR() {

View File

@@ -35,7 +35,7 @@ import {
readAsBlob,
sizeToHumanReadable,
} from "../../../lib/src/common/utils.ts";
import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
import { arrayBufferToBase64Single, versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
import { Logger } from "../../../lib/src/common/logger.ts";
import {
balanceChunkPurgedDBs,
@@ -72,6 +72,7 @@ import {
EVENT_REQUEST_OPEN_SETUP_URI,
EVENT_REQUEST_RELOAD_SETTING_TAB,
EVENT_REQUEST_RUN_DOCTOR,
EVENT_REQUEST_SHOW_SETUP_QR,
eventHub,
} from "../../../common/events.ts";
import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
@@ -81,6 +82,7 @@ import { HiddenFileSync } from "../../../features/HiddenFileSync/CmdHiddenFileSy
import { EVENT_REQUEST_SHOW_HISTORY } from "../../../common/obsidianEvents.ts";
import { LocalDatabaseMaintenance } from "../../../features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts";
import { mount } from "svelte";
import { getWebCrypto } from "../../../lib/src/mods.ts";
export type OnUpdateResult = {
visibility?: boolean;
@@ -814,6 +816,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
return false;
};
const enableOnlySyncDisabled = enableOnly(() => !isAnySyncEnabled());
const combineOnUpdate = (func1: OnUpdateFunc, func2: OnUpdateFunc): OnUpdateFunc => {
return () => ({
...func1(),
...func2(),
});
};
const onlyOnP2POrCouchDB = () =>
({
visibility:
@@ -979,7 +987,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
eventHub.emitEvent(EVENT_REQUEST_COPY_SETUP_URI);
});
});
new Setting(paneEl)
.setName($msg("Setup.ShowQRCode"))
.setDesc($msg("Setup.ShowQRCode.Desc"))
.addButton((text) => {
text.setButtonText($msg("Setup.ShowQRCode")).onClick(() => {
eventHub.emitEvent(EVENT_REQUEST_SHOW_SETUP_QR);
});
});
});
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleReset")).then((paneEl) => {
new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameDiscardSettings"))
@@ -1444,6 +1461,10 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
new Setting(paneEl).autoWireText("bucket", { holdValue: true });
new Setting(paneEl).autoWireToggle("useCustomRequestHandler", { holdValue: true });
new Setting(paneEl).autoWireTextArea("bucketCustomHeaders", {
holdValue: true,
placeHolder: "x-custom-header: value\n x-custom-header2: value2",
});
new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameTestConnection"))
.addButton((button) =>
@@ -1465,6 +1486,7 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
"secretKey",
"bucket",
"useCustomRequestHandler",
"bucketCustomHeaders",
])
.addOnUpdate(onlyOnMinIO);
});
@@ -1511,20 +1533,119 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
holdValue: true,
onUpdate: enableOnlySyncDisabled,
});
new Setting(paneEl).autoWireText("couchDB_USER", {
new Setting(paneEl).autoWireToggle("useJWT", {
holdValue: true,
onUpdate: enableOnlySyncDisabled,
});
new Setting(paneEl).autoWireText("couchDB_USER", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => !this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("couchDB_PASSWORD", {
holdValue: true,
isPassword: true,
onUpdate: enableOnlySyncDisabled,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => !this.editingSettings.useJWT)
),
});
const algorithms = {
["HS256"]: "HS256",
["HS512"]: "HS512",
["ES256"]: "ES256",
["ES512"]: "ES512",
} as const;
new Setting(paneEl).autoWireDropDown("jwtAlgorithm", {
options: algorithms,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireTextArea("jwtKey", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
// eslint-disable-next-line prefer-const
let generatedKeyDivEl: HTMLDivElement;
new Setting(paneEl)
.setDesc("Generate ES256 Keypair for testing")
.addButton((button) =>
button.setButtonText("Generate").onClick(async () => {
const crypto = await getWebCrypto();
const keyPair = await crypto.subtle.generateKey(
{ name: "ECDSA", namedCurve: "P-256" },
true,
["sign", "verify"]
);
const pubKey = await crypto.subtle.exportKey("spki", keyPair.publicKey);
const privateKey = await crypto.subtle.exportKey("pkcs8", keyPair.privateKey);
const encodedPublicKey = await arrayBufferToBase64Single(pubKey);
const encodedPrivateKey = await arrayBufferToBase64Single(privateKey);
const privateKeyPem = `> -----BEGIN PRIVATE KEY-----\n> ${encodedPrivateKey}\n> -----END PRIVATE KEY-----`;
const publicKeyPem = `> -----BEGIN PUBLIC KEY-----\\n${encodedPublicKey}\\n-----END PUBLIC KEY-----`;
const title = $msg("Setting.GenerateKeyPair.Title");
const msg = $msg("Setting.GenerateKeyPair.Desc", {
public_key: publicKeyPem,
private_key: privateKeyPem,
});
await MarkdownRenderer.render(
this.plugin.app,
"## " + title + "\n\n" + msg,
generatedKeyDivEl,
"/",
this.plugin
);
})
)
.addOnUpdate(
combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
)
);
generatedKeyDivEl = this.createEl(
paneEl,
"div",
{ text: "" },
(el) => {},
visibleOnly(() => this.editingSettings.useJWT)
);
new Setting(paneEl).autoWireText("jwtKid", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("jwtSub", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireNumeric("jwtExpDuration", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("couchDB_DBNAME", {
holdValue: true,
onUpdate: enableOnlySyncDisabled,
});
new Setting(paneEl).autoWireTextArea("couchDB_CustomHeaders", { holdValue: true });
new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameTestDatabaseConnection"))
.setClass("wizardHidden")
@@ -1562,6 +1683,13 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
"couchDB_USER",
"couchDB_PASSWORD",
"couchDB_DBNAME",
"jwtAlgorithm",
"jwtExpDuration",
"jwtKey",
"jwtSub",
"jwtKid",
"useJWT",
"couchDB_CustomHeaders",
])
.addOnUpdate(onlyOnCouchDB);
});
@@ -1985,7 +2113,6 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
LEVEL_ADVANCED
).then((paneEl) => {
paneEl.addClass("wizardHidden");
new Setting(paneEl)
.autoWireText("settingSyncFile", { holdValue: true })
.addApplyButton(["settingSyncFile"]);
@@ -2330,6 +2457,11 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
pluginConfig.P2P_passphrase = redact(pluginConfig.P2P_passphrase);
pluginConfig.P2P_roomID = redact(pluginConfig.P2P_roomID);
pluginConfig.P2P_relays = redact(pluginConfig.P2P_relays);
pluginConfig.jwtKey = redact(pluginConfig.jwtKey);
pluginConfig.jwtSub = redact(pluginConfig.jwtSub);
pluginConfig.jwtKid = redact(pluginConfig.jwtKid);
pluginConfig.bucketCustomHeaders = redact(pluginConfig.bucketCustomHeaders);
pluginConfig.couchDB_CustomHeaders = redact(pluginConfig.couchDB_CustomHeaders);
const endpoint = pluginConfig.endpoint;
if (endpoint == "") {
pluginConfig.endpoint = "Not configured or AWS";
@@ -3383,6 +3515,7 @@ ${stringifyYaml(pluginConfig)}`;
const region = this.plugin.settings.region;
const endpoint = this.plugin.settings.endpoint;
const useCustomRequestHandler = this.plugin.settings.useCustomRequestHandler;
const customHeaders = this.plugin.settings.bucketCustomHeaders;
return new JournalSyncMinio(
id,
key,
@@ -3391,7 +3524,8 @@ ${stringifyYaml(pluginConfig)}`;
this.plugin.simpleStore,
this.plugin,
useCustomRequestHandler,
region
region,
customHeaders
);
}
async resetRemoteBucket() {

View File

@@ -443,4 +443,12 @@ span.ls-mark-cr::after {
justify-content: center;
align-items: center;
max-width: max-content;
}
.sls-keypair pre {
max-width: 100%;
overflow-x: auto;
white-space: pre-wrap;
word-break: break-all;
}

View File

@@ -10,6 +10,25 @@ Nevertheless, that being said, to be more honest, I still have not decided what
Note: Already you have noticed this, but let me mention it again, this is a significantly large update. If you have noticed anything, please let me know. I will try to fix it as soon as possible (Some address is on my [profile](https://github.com/vrtmrz)).
## 0.24.23
### New Feature
- Now, we can send custom headers to the server.
- They can be sent to either CouchDB or Object Storage.
- Authentication with JWT in CouchDB is now supported.
- I will describe steps later, but please refer to the [CouchDB document](https://docs.couchdb.org/en/stable/config/auth.html#authentication-configuration).
- A JWT keypair for testing can be generated in the setting dialogue.
### Improved
- The QR Code for set-up can be shown also from the setting dialogue now.
- Conflict checking for preventing unexpected overwriting on the boot-up process has been quite faster.
### Fixed
- Some bugs on Dev and Testing modules have been fixed.
## 0.24.22 ~~0.24.21~~
(Really sorry for the confusion. I have got a miss at releasing...).
@@ -74,54 +93,5 @@ Confession. I got the default values wrong. So scary and sorry.
- **NOW INDEED AND ACTUALLY** `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
## 0.24.16
### Improved
#### Peer-to-Peer
- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- No longer unexpected database broken, phew.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.
- Pseudo clients will not decrypt/encrypt anything, just relay the data. Hence, always settings are not compatible. Therefore, we have to accept the incompatibility for pseudo clients.
#### General
- New migration method has been implemented, that called `Doctor`.
- `Doctor` checks the difference between the ideal and actual values and encourages corrective action. To facilitate our decision, the reasons for this and the recommendations are also presented.
- This can be used not only during migration. We can invoke the doctor from the settings for trouble-shooting.
- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.
### Behaviour and default changed
- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
### Refactored
- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
## 0.24.15
### Fixed
- Now, even without WeakRef, Polyfill is used and the whole thing works without error. However, if you can switch WebView Engine, it is recommended to switch to a WebView Engine that supports WeakRef.
## 0.24.14
### Fixed
- Resolving conflicts of JSON files (and sensibly merging them) is now working fine, again!
- And, failure logs are more informative.
- More robust to release the event listeners on unwatching the local database.
### Refactored
- JSON file conflict resolution dialogue has been rewritten into svelte v5.
- Upgrade eslint.
- Remove unnecessary pragma comments for eslint.
Older notes are in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

View File

@@ -14,6 +14,57 @@ Thank you, and I hope your troubles will be resolved!
---
## 0.24.16
### Improved
#### Peer-to-Peer
- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- No longer unexpected database broken, phew.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.
- Pseudo clients will not decrypt/encrypt anything, just relay the data. Hence, always settings are not compatible. Therefore, we have to accept the incompatibility for pseudo clients.
#### General
- New migration method has been implemented, that called `Doctor`.
- `Doctor` checks the difference between the ideal and actual values and encourages corrective action. To facilitate our decision, the reasons for this and the recommendations are also presented.
- This can be used not only during migration. We can invoke the doctor from the settings for trouble-shooting.
- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.
### Behaviour and default changed
- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
### Refactored
- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
## 0.24.15
### Fixed
- Now, even without WeakRef, Polyfill is used and the whole thing works without error. However, if you can switch WebView Engine, it is recommended to switch to a WebView Engine that supports WeakRef.
## 0.24.14
### Fixed
- Resolving conflicts of JSON files (and sensibly merging them) is now working fine, again!
- And, failure logs are more informative.
- More robust to release the event listeners on unwatching the local database.
### Refactored
- JSON file conflict resolution dialogue has been rewritten into svelte v5.
- Upgrade eslint.
- Remove unnecessary pragma comments for eslint.
## 0.24.13
Sorry for the lack of replies. The ones that were not good are popping up, so I am just going to go ahead and get this one... However, they realised that refactoring and restructuring is about clarifying the problem. Your patience and understanding is much appreciated.