Add self-hosted-livesync-cli to src/apps/cli as a headless, and a dedicated version.

This commit is contained in:
vorotamoroz
2026-03-11 14:51:01 +01:00
parent 2f8bc4fef2
commit 0742773e1e
26 changed files with 2839 additions and 143 deletions

788
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -129,11 +129,13 @@
"@smithy/middleware-apply-body-checksum": "^4.3.9",
"@smithy/protocol-http": "^5.3.9",
"@smithy/querystring-builder": "^4.2.9",
"commander": "^14.0.3",
"diff-match-patch": "^1.0.5",
"fflate": "^0.8.2",
"idb": "^8.0.3",
"minimatch": "^10.2.2",
"octagonal-wheels": "^0.1.45",
"pouchdb-adapter-leveldb": "^9.0.0",
"qrcode-generator": "^1.4.4",
"trystero": "^0.22.0",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"

4
src/apps/cli/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
.livesync
test/*
!test/*.sh
node_modules

162
src/apps/cli/README.md Normal file
View File

@@ -0,0 +1,162 @@
# Self-hosted LiveSync CLI
Command-line version of Obsidian LiveSync plugin for syncing vaults without Obsidian.
## Features
- ✅ Sync Obsidian vaults using CouchDB without running Obsidian
- ✅ Compatible with Obsidian LiveSync plugin settings
- ✅ Supports all core sync features (encryption, conflict resolution, etc.)
- ✅ Lightweight and headless operation
- ✅ Cross-platform (Windows, macOS, Linux)
## Architecture
This CLI version is built using the same core as the Obsidian plugin:
```
CLI Main
└─ LiveSyncBaseCore<ServiceContext, IMinimumLiveSyncCommands>
├─ HeadlessServiceHub (All services without Obsidian dependencies)
└─ ServiceModules (Ported from main.ts)
├─ FileAccessCLI (Node.js FileSystemAdapter)
├─ StorageEventManagerCLI
├─ ServiceFileAccessCLI
├─ ServiceDatabaseFileAccess
├─ ServiceFileHandler
└─ ServiceRebuilder
```
### Key Components
1. **Node.js FileSystem Adapter** (`adapters/`)
- Platform-agnostic file operations using Node.js `fs/promises`
- Implements same interface as Obsidian's file system
2. **CLI Storage Event Manager** (`managers/`)
- File-based snapshot persistence (JSON)
- Console-based status updates
- Optional file watching (can be extended with chokidar)
3. **Service Modules** (`serviceModules/`)
- Direct port from `main.ts` `initialiseServiceModules`
- All core sync functionality preserved
4. **Main Entry Point** (`main.ts`)
- Command-line interface
- Settings management (JSON file)
- Graceful shutdown handling
## Installation
```bash
# Install dependencies (ensure you are in repository root directory, not src/apps/cli)
# due to shared dependencies with webapp and main library
npm install
# Build the project (ensure you are in `src/apps/cli` directory)
npm run build
```
## Usage
### Basic Usage
As you know, the CLI is designed to be used in a headless environment. Hence all operations are performed against a local vault directory and a settings file. Here are some example commands:
```bash
# Sync local database with CouchDB (no files will be changed).
node dist/cli/index.js /path/to/your-local-database --settings /path/to/settings.json sync
# Push files to local database
node dist/cli/index.js /path/to/your-local-database --settings /path/to/settings.json push /your/storage/file.md /vault/path/file.md
# Pull files from local database
node dist/cli/index.js /path/to/your-local-database --settings /path/to/settings.json pull /vault/path/file.md /your/storage/file.md
# Verbose logging
node dist/cli/index.js /path/to/your-local-database --settings /path/to/settings.json --verbose
```
### Configuration
The CLI uses the same settings format as the Obsidian plugin. Create a `.livesync/settings.json` file in your vault directory:
```json
{
"couchDB_URI": "http://localhost:5984",
"couchDB_USER": "admin",
"couchDB_PASSWORD": "password",
"couchDB_DBNAME": "obsidian-livesync",
"liveSync": true,
"syncOnSave": true,
"syncOnStart": true,
"encrypt": true,
"passphrase": "your-encryption-passphrase",
"usePluginSync": false,
"isConfigured": true
}
```
**Minimum required settings:**
- `couchDB_URI`: CouchDB server URL
- `couchDB_USER`: CouchDB username
- `couchDB_PASSWORD`: CouchDB password
- `couchDB_DBNAME`: Database name
- `isConfigured`: Set to `true` after configuration
### Command-line Options
```
Usage:
livesync-cli [database-path] [options]
Arguments:
database-path Path to the local database directory (required)
Options:
--settings, -s <path> Path to settings file (default: .livesync/settings.json in local database directory)
--verbose, -v Enable verbose logging
--help, -h Show this help message
sync Sync local database with CouchDB or Bucket
push <storagePath> <vaultPath> Push file to local database
pull <vaultPath> <storagePath> Pull file from local database
```
### Planned options:
- `put <vaultPath>`: Add/update file in local database from standard input
- `cat <vaultPath>`: Output file content to standard output
- `info <vaultPath>`: Show file metadata, conflicts, and, other information
- `ls <prefix>`: List files in local database with optional prefix filter
- `resolve <vaultPath> <revision>`: Resolve conflict for a file by choosing a specific revision
- `rm <vaultPath>`: Remove file from local database.
- `--immediate`: Perform sync after the command (e.g. `push`, `pull`, `put`, `rm`).
- `serve`: Start CLI in server mode, exposing REST APIs for remote, and batch operations.
## Use Cases
## Development
### Project Structure
```
src/apps/cli/
├── adapters/ # Node.js FileSystem Adapter
│ ├── NodeFileSystemAdapter.ts
│ ├── NodePathAdapter.ts
│ ├── NodeTypeGuardAdapter.ts
│ ├── NodeConversionAdapter.ts
│ ├── NodeStorageAdapter.ts
│ ├── NodeVaultAdapter.ts
│ └── NodeTypes.ts
├── managers/ # CLI-specific managers
│ ├── CLIStorageEventManagerAdapter.ts
│ └── StorageEventManagerCLI.ts
├── serviceModules/ # Service modules (ported from main.ts)
│ ├── CLIServiceModules.ts
│ ├── FileAccessCLI.ts
│ ├── ServiceFileAccessImpl.ts
│ └── DatabaseFileAccess.ts
├── main.ts # CLI entry point
└── README.md # This file
```

View File

@@ -0,0 +1,28 @@
import * as path from "path";
import type { UXFileInfoStub, UXFolderInfo } from "@lib/common/types";
import type { IConversionAdapter } from "@lib/serviceModules/adapters";
import type { NodeFile, NodeFolder } from "./NodeTypes";
/**
* Conversion adapter implementation for Node.js
*/
export class NodeConversionAdapter implements IConversionAdapter<NodeFile, NodeFolder> {
nativeFileToUXFileInfoStub(file: NodeFile): UXFileInfoStub {
return {
name: path.basename(file.path),
path: file.path,
stat: file.stat,
isFolder: false,
};
}
nativeFolderToUXFolder(folder: NodeFolder): UXFolderInfo {
return {
name: path.basename(folder.path),
path: folder.path,
isFolder: true,
children: [],
parent: path.dirname(folder.path) as any,
};
}
}

View File

@@ -0,0 +1,153 @@
import * as fs from "fs/promises";
import * as path from "path";
import type { FilePath, UXStat } from "@lib/common/types";
import type { IFileSystemAdapter } from "@lib/serviceModules/adapters";
import { NodePathAdapter } from "./NodePathAdapter";
import { NodeTypeGuardAdapter } from "./NodeTypeGuardAdapter";
import { NodeConversionAdapter } from "./NodeConversionAdapter";
import { NodeStorageAdapter } from "./NodeStorageAdapter";
import { NodeVaultAdapter } from "./NodeVaultAdapter";
import type { NodeFile, NodeFolder, NodeStat } from "./NodeTypes";
/**
* Complete file system adapter implementation for Node.js
*/
export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeFile, NodeFolder, NodeStat> {
readonly path: NodePathAdapter;
readonly typeGuard: NodeTypeGuardAdapter;
readonly conversion: NodeConversionAdapter;
readonly storage: NodeStorageAdapter;
readonly vault: NodeVaultAdapter;
private fileCache = new Map<string, NodeFile>();
constructor(private basePath: string) {
this.path = new NodePathAdapter();
this.typeGuard = new NodeTypeGuardAdapter();
this.conversion = new NodeConversionAdapter();
this.storage = new NodeStorageAdapter(basePath);
this.vault = new NodeVaultAdapter(basePath);
}
private resolvePath(p: FilePath | string): string {
return path.join(this.basePath, p);
}
private normalisePath(p: FilePath | string): string {
return this.path.normalisePath(p as string);
}
async getAbstractFileByPath(p: FilePath | string): Promise<NodeFile | null> {
const pathStr = this.normalisePath(p);
const cached = this.fileCache.get(pathStr);
if (cached) {
return cached;
}
return await this.refreshFile(pathStr);
}
async getAbstractFileByPathInsensitive(p: FilePath | string): Promise<NodeFile | null> {
const pathStr = this.normalisePath(p);
const exact = await this.getAbstractFileByPath(pathStr);
if (exact) {
return exact;
}
const lowerPath = pathStr.toLowerCase();
for (const [cachedPath, cachedFile] of this.fileCache.entries()) {
if (cachedPath.toLowerCase() === lowerPath) {
return cachedFile;
}
}
await this.scanDirectory();
for (const [cachedPath, cachedFile] of this.fileCache.entries()) {
if (cachedPath.toLowerCase() === lowerPath) {
return cachedFile;
}
}
return null;
}
async getFiles(): Promise<NodeFile[]> {
if (this.fileCache.size === 0) {
await this.scanDirectory();
}
return Array.from(this.fileCache.values());
}
async statFromNative(file: NodeFile): Promise<UXStat> {
return file.stat;
}
async reconcileInternalFile(p: string): Promise<void> {
// No-op in Node.js version
// This is used by Obsidian to sync internal file metadata
}
async refreshFile(p: string): Promise<NodeFile | null> {
const pathStr = this.normalisePath(p);
try {
const fullPath = this.resolvePath(pathStr);
const stat = await fs.stat(fullPath);
if (!stat.isFile()) {
this.fileCache.delete(pathStr);
return null;
}
const file: NodeFile = {
path: pathStr as FilePath,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
type: "file",
},
};
this.fileCache.set(pathStr, file);
return file;
} catch {
this.fileCache.delete(pathStr);
return null;
}
}
/**
* Helper method to recursively scan directory and populate file cache
*/
async scanDirectory(relativePath: string = ""): Promise<void> {
const fullPath = this.resolvePath(relativePath);
try {
const entries = await fs.readdir(fullPath, { withFileTypes: true });
for (const entry of entries) {
const entryRelativePath = path.join(relativePath, entry.name).replace(/\\/g, "/");
if (entry.isDirectory()) {
await this.scanDirectory(entryRelativePath);
} else if (entry.isFile()) {
const entryFullPath = this.resolvePath(entryRelativePath);
const stat = await fs.stat(entryFullPath);
const file: NodeFile = {
path: entryRelativePath as FilePath,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
type: "file",
},
};
this.fileCache.set(entryRelativePath, file);
}
}
} catch (error) {
// Directory doesn't exist or is not readable
console.error(`Error scanning directory ${fullPath}:`, error);
}
}
}

View File

@@ -0,0 +1,18 @@
import * as path from "path";
import type { FilePath } from "@lib/common/types";
import type { IPathAdapter } from "@lib/serviceModules/adapters";
import type { NodeFile } from "./NodeTypes";
/**
* Path adapter implementation for Node.js
*/
export class NodePathAdapter implements IPathAdapter<NodeFile> {
getPath(file: string | NodeFile): FilePath {
return (typeof file === "string" ? file : file.path) as FilePath;
}
normalisePath(p: string): string {
// Normalize path separators to forward slashes (like Obsidian)
return path.normalize(p).replace(/\\/g, "/");
}
}

View File

@@ -0,0 +1,124 @@
import * as fs from "fs/promises";
import * as path from "path";
import type { UXDataWriteOptions } from "@lib/common/types";
import type { IStorageAdapter } from "@lib/serviceModules/adapters";
import type { NodeStat } from "./NodeTypes";
/**
* Storage adapter implementation for Node.js
*/
export class NodeStorageAdapter implements IStorageAdapter<NodeStat> {
constructor(private basePath: string) {}
private resolvePath(p: string): string {
return path.join(this.basePath, p);
}
async exists(p: string): Promise<boolean> {
try {
await fs.access(this.resolvePath(p));
return true;
} catch {
return false;
}
}
async trystat(p: string): Promise<NodeStat | null> {
try {
const stat = await fs.stat(this.resolvePath(p));
return {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
type: stat.isDirectory() ? "folder" : "file",
};
} catch {
return null;
}
}
async stat(p: string): Promise<NodeStat | null> {
return await this.trystat(p);
}
async mkdir(p: string): Promise<void> {
await fs.mkdir(this.resolvePath(p), { recursive: true });
}
async remove(p: string): Promise<void> {
const fullPath = this.resolvePath(p);
const stat = await fs.stat(fullPath);
if (stat.isDirectory()) {
await fs.rm(fullPath, { recursive: true, force: true });
} else {
await fs.unlink(fullPath);
}
}
async read(p: string): Promise<string> {
return await fs.readFile(this.resolvePath(p), "utf-8");
}
async readBinary(p: string): Promise<ArrayBuffer> {
const buffer = await fs.readFile(this.resolvePath(p));
return buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) as ArrayBuffer;
}
async write(p: string, data: string, options?: UXDataWriteOptions): Promise<void> {
const fullPath = this.resolvePath(p);
await fs.mkdir(path.dirname(fullPath), { recursive: true });
await fs.writeFile(fullPath, data, "utf-8");
if (options?.mtime || options?.ctime) {
const atime = options.mtime ? new Date(options.mtime) : new Date();
const mtime = options.mtime ? new Date(options.mtime) : new Date();
await fs.utimes(fullPath, atime, mtime);
}
}
async writeBinary(p: string, data: ArrayBuffer, options?: UXDataWriteOptions): Promise<void> {
const fullPath = this.resolvePath(p);
await fs.mkdir(path.dirname(fullPath), { recursive: true });
await fs.writeFile(fullPath, new Uint8Array(data));
if (options?.mtime || options?.ctime) {
const atime = options.mtime ? new Date(options.mtime) : new Date();
const mtime = options.mtime ? new Date(options.mtime) : new Date();
await fs.utimes(fullPath, atime, mtime);
}
}
async append(p: string, data: string, options?: UXDataWriteOptions): Promise<void> {
const fullPath = this.resolvePath(p);
await fs.mkdir(path.dirname(fullPath), { recursive: true });
await fs.appendFile(fullPath, data, "utf-8");
if (options?.mtime || options?.ctime) {
const atime = options.mtime ? new Date(options.mtime) : new Date();
const mtime = options.mtime ? new Date(options.mtime) : new Date();
await fs.utimes(fullPath, atime, mtime);
}
}
async list(basePath: string): Promise<{ files: string[]; folders: string[] }> {
const fullPath = this.resolvePath(basePath);
try {
const entries = await fs.readdir(fullPath, { withFileTypes: true });
const files: string[] = [];
const folders: string[] = [];
for (const entry of entries) {
const entryPath = path.join(basePath, entry.name).replace(/\\/g, "/");
if (entry.isDirectory()) {
folders.push(entryPath);
} else if (entry.isFile()) {
files.push(entryPath);
}
}
return { files, folders };
} catch {
return { files: [], folders: [] };
}
}
}

View File

@@ -0,0 +1,15 @@
import type { ITypeGuardAdapter } from "@lib/serviceModules/adapters";
import type { NodeFile, NodeFolder } from "./NodeTypes";
/**
* Type guard adapter implementation for Node.js
*/
export class NodeTypeGuardAdapter implements ITypeGuardAdapter<NodeFile, NodeFolder> {
isFile(file: any): file is NodeFile {
return file && typeof file === "object" && "path" in file && "stat" in file && !file.isFolder;
}
isFolder(item: any): item is NodeFolder {
return item && typeof item === "object" && "path" in item && item.isFolder === true;
}
}

View File

@@ -0,0 +1,22 @@
import type { FilePath, UXStat } from "@lib/common/types";
/**
* Node.js file representation
*/
export type NodeFile = {
path: FilePath;
stat: UXStat;
};
/**
* Node.js folder representation
*/
export type NodeFolder = {
path: FilePath;
isFolder: true;
};
/**
* Node.js stat type (compatible with UXStat)
*/
export type NodeStat = UXStat;

View File

@@ -0,0 +1,118 @@
import * as fs from "fs/promises";
import * as path from "path";
import type { UXDataWriteOptions } from "@lib/common/types";
import type { IVaultAdapter } from "@lib/serviceModules/adapters";
import type { NodeFile, NodeFolder, NodeStat } from "./NodeTypes";
/**
* Vault adapter implementation for Node.js
*/
export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
constructor(private basePath: string) {}
private resolvePath(p: string): string {
return path.join(this.basePath, p);
}
async read(file: NodeFile): Promise<string> {
return await fs.readFile(this.resolvePath(file.path), "utf-8");
}
async cachedRead(file: NodeFile): Promise<string> {
// No caching in CLI version, just read directly
return await this.read(file);
}
async readBinary(file: NodeFile): Promise<ArrayBuffer> {
const buffer = await fs.readFile(this.resolvePath(file.path));
return buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) as ArrayBuffer;
}
async modify(file: NodeFile, data: string, options?: UXDataWriteOptions): Promise<void> {
const fullPath = this.resolvePath(file.path);
await fs.writeFile(fullPath, data, "utf-8");
if (options?.mtime || options?.ctime) {
const atime = options.mtime ? new Date(options.mtime) : new Date();
const mtime = options.mtime ? new Date(options.mtime) : new Date();
await fs.utimes(fullPath, atime, mtime);
}
}
async modifyBinary(file: NodeFile, data: ArrayBuffer, options?: UXDataWriteOptions): Promise<void> {
const fullPath = this.resolvePath(file.path);
await fs.writeFile(fullPath, new Uint8Array(data));
if (options?.mtime || options?.ctime) {
const atime = options.mtime ? new Date(options.mtime) : new Date();
const mtime = options.mtime ? new Date(options.mtime) : new Date();
await fs.utimes(fullPath, atime, mtime);
}
}
async create(p: string, data: string, options?: UXDataWriteOptions): Promise<NodeFile> {
const fullPath = this.resolvePath(p);
await fs.mkdir(path.dirname(fullPath), { recursive: true });
await fs.writeFile(fullPath, data, "utf-8");
if (options?.mtime || options?.ctime) {
const atime = options.mtime ? new Date(options.mtime) : new Date();
const mtime = options.mtime ? new Date(options.mtime) : new Date();
await fs.utimes(fullPath, atime, mtime);
}
const stat = await fs.stat(fullPath);
return {
path: p as any,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
type: "file",
},
};
}
async createBinary(p: string, data: ArrayBuffer, options?: UXDataWriteOptions): Promise<NodeFile> {
const fullPath = this.resolvePath(p);
await fs.mkdir(path.dirname(fullPath), { recursive: true });
await fs.writeFile(fullPath, new Uint8Array(data));
if (options?.mtime || options?.ctime) {
const atime = options.mtime ? new Date(options.mtime) : new Date();
const mtime = options.mtime ? new Date(options.mtime) : new Date();
await fs.utimes(fullPath, atime, mtime);
}
const stat = await fs.stat(fullPath);
return {
path: p as any,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
type: "file",
},
};
}
async delete(file: NodeFile | NodeFolder, force = false): Promise<void> {
const fullPath = this.resolvePath(file.path);
const stat = await fs.stat(fullPath);
if (stat.isDirectory()) {
await fs.rm(fullPath, { recursive: true, force });
} else {
await fs.unlink(fullPath);
}
}
async trash(file: NodeFile | NodeFolder, force = false): Promise<void> {
// In CLI, trash is the same as delete (no recycle bin)
await this.delete(file, force);
}
trigger(name: string, ...data: any[]): any {
// No-op in CLI version (no event system)
return undefined;
}
}

View File

@@ -0,0 +1,134 @@
import PouchDB from "pouchdb-core";
import HttpPouch from "pouchdb-adapter-http";
import mapreduce from "pouchdb-mapreduce";
import replication from "pouchdb-replication";
import LevelDBAdapter from "pouchdb-adapter-leveldb";
import find from "pouchdb-find";
import transform from "transform-pouch";
//@ts-ignore
import { findPathToLeaf } from "pouchdb-merge";
//@ts-ignore
import { adapterFun } from "pouchdb-utils";
//@ts-ignore
import { createError, MISSING_DOC, UNKNOWN_ERROR } from "pouchdb-errors";
import { mapAllTasksWithConcurrencyLimit, unwrapTaskResult } from "octagonal-wheels/concurrency/task";
PouchDB.plugin(LevelDBAdapter).plugin(HttpPouch).plugin(mapreduce).plugin(replication).plugin(find).plugin(transform);
type PurgeMultiResult = {
ok: true;
deletedRevs: string[];
documentWasRemovedCompletely: boolean;
};
type PurgeMultiParam = [docId: string, rev$$1: string];
function appendPurgeSeqs(db: PouchDB.Database, docs: PurgeMultiParam[]) {
return db
.get("_local/purges")
.then(function (doc: any) {
for (const [docId, rev$$1] of docs) {
const purgeSeq = doc.purgeSeq + 1;
doc.purges.push({
docId,
rev: rev$$1,
purgeSeq,
});
//@ts-ignore : missing type def
if (doc.purges.length > db.purged_infos_limit) {
//@ts-ignore : missing type def
doc.purges.splice(0, doc.purges.length - db.purged_infos_limit);
}
doc.purgeSeq = purgeSeq;
}
return doc;
})
.catch(function (err) {
if (err.status !== 404) {
throw err;
}
return {
_id: "_local/purges",
purges: docs.map(([docId, rev$$1], idx) => ({
docId,
rev: rev$$1,
purgeSeq: idx,
})),
purgeSeq: docs.length,
};
})
.then(function (doc) {
return db.put(doc);
});
}
/**
* purge multiple documents at once.
*/
PouchDB.prototype.purgeMulti = adapterFun(
"_purgeMulti",
function (
docs: PurgeMultiParam[],
callback: (
error: Error,
result?: {
[x: string]: PurgeMultiResult | Error;
}
) => void
) {
//@ts-ignore
if (typeof this._purge === "undefined") {
return callback(
//@ts-ignore: this ts-ignore might be hiding a `this` bug where we don't have "this" conext.
createError(UNKNOWN_ERROR, "Purge is not implemented in the " + this.adapter + " adapter.")
);
}
//@ts-ignore
// eslint-disable-next-line @typescript-eslint/no-this-alias
const self = this;
const tasks = docs.map(
(param) => () =>
new Promise<[PurgeMultiParam, PurgeMultiResult | Error]>((res, rej) => {
const [docId, rev$$1] = param;
self._getRevisionTree(docId, (error: Error, revs: string[]) => {
if (error) {
return res([param, error]);
}
if (!revs) {
return res([param, createError(MISSING_DOC)]);
}
let path;
try {
path = findPathToLeaf(revs, rev$$1);
} catch (error) {
//@ts-ignore
return res([param, error.message || error]);
}
self._purge(docId, path, (error: Error, result: PurgeMultiResult) => {
if (error) {
return res([param, error]);
} else {
return res([param, result]);
}
});
});
})
);
(async () => {
const ret = await mapAllTasksWithConcurrencyLimit(1, tasks);
const retAll = ret.map((e) => unwrapTaskResult(e)) as [PurgeMultiParam, PurgeMultiResult | Error][];
await appendPurgeSeqs(
self,
retAll.filter((e) => "ok" in e[1]).map((e) => e[0])
);
const result = Object.fromEntries(retAll.map((e) => [e[0][0], e[1]]));
return result;
})()
//@ts-ignore
.then((result) => callback(undefined, result))
.catch((error) => callback(error));
}
);
export { PouchDB };

452
src/apps/cli/main.ts Normal file
View File

@@ -0,0 +1,452 @@
#!/usr/bin/env node
/**
* Self-hosted LiveSync CLI
* Command-line version of Obsidian LiveSync plugin for syncing vaults without Obsidian
*/
if (!("localStorage" in globalThis)) {
const store = new Map<string, string>();
(globalThis as any).localStorage = {
getItem: (key: string) => (store.has(key) ? store.get(key)! : null),
setItem: (key: string, value: string) => {
store.set(key, value);
},
removeItem: (key: string) => {
store.delete(key);
},
clear: () => {
store.clear();
},
};
}
import * as fs from "fs/promises";
import * as path from "path";
import { NodeServiceContext, NodeServiceHub } from "./services/NodeServiceHub";
import { LiveSyncBaseCore } from "../../LiveSyncBaseCore";
import { ServiceContext } from "@lib/services/base/ServiceBase";
import { initialiseServiceModulesCLI } from "./serviceModules/CLIServiceModules";
import {
DEFAULT_SETTINGS,
LOG_LEVEL_VERBOSE,
type LOG_LEVEL,
type ObsidianLiveSyncSettings,
type FilePathWithPrefix,
} from "@lib/common/types";
import type { InjectableServiceHub } from "@lib/services/implements/injectable/InjectableServiceHub";
import type { InjectableSettingService } from "@/lib/src/services/implements/injectable/InjectableSettingService";
import { LOG_LEVEL_DEBUG, setGlobalLogFunction, defaultLoggerEnv } from "octagonal-wheels/common/logger";
import PouchDb from "pouchdb-core";
const SETTINGS_FILE = ".livesync/settings.json";
const VALID_COMMANDS = new Set(["sync", "push", "pull", "init-settings"] as const);
type CLICommand = "daemon" | "sync" | "push" | "pull" | "init-settings";
defaultLoggerEnv.minLogLevel = LOG_LEVEL_DEBUG;
// DI the log again.
// const recentLogEntries = reactiveSource<LogEntry[]>([]);
// const globalLogFunction = (message: any, level?: number, key?: string) => {
// const messageX =
// message instanceof Error
// ? new LiveSyncError("[Error Logged]: " + message.message, { cause: message })
// : message;
// const entry = { message: messageX, level, key } as LogEntry;
// recentLogEntries.value = [...recentLogEntries.value, entry];
// };
setGlobalLogFunction((msg, level) => {
console.log(`[${level}] ${typeof msg === "string" ? msg : JSON.stringify(msg)}`);
if (msg instanceof Error) {
console.error(msg);
}
});
interface CLIOptions {
databasePath?: string;
settingsPath?: string;
verbose?: boolean;
force?: boolean;
command: CLICommand;
commandArgs: string[];
}
function printHelp(): void {
console.log(`
Self-hosted LiveSync CLI
Usage:
livesync-cli [database-path] [options] [command] [command-args]
Arguments:
database-path Path to the local database directory (required)
Commands:
sync Run one replication cycle and exit
push <src> <dst> Push local file <src> into local database path <dst>
pull <src> <dst> Pull file <src> from local database into local file <dst>
init-settings [path] Create settings JSON from DEFAULT_SETTINGS
Options:
--settings, -s <path> Path to settings file (default: .livesync/settings.json in local database directory)
--force, -f Overwrite existing file on init-settings
--verbose, -v Enable verbose logging
--help, -h Show this help message
Examples:
livesync-cli ./my-database sync
livesync-cli ./my-database --settings ./custom-settings.json push ./note.md folder/note.md
livesync-cli ./my-database pull folder/note.md ./exports/note.md
livesync-cli init-settings ./data.json
livesync-cli ./my-database --verbose
`);
}
function parseArgs(): CLIOptions {
const args = process.argv.slice(2);
if (args.length === 0 || args.includes("--help") || args.includes("-h")) {
printHelp();
process.exit(0);
}
let databasePath: string | undefined;
let settingsPath: string | undefined;
let verbose = false;
let force = false;
let command: CLICommand = "daemon";
const commandArgs: string[] = [];
for (let i = 0; i < args.length; i++) {
const token = args[i];
switch (token) {
case "--settings":
case "-s": {
i++;
if (!args[i]) {
console.error(`Error: Missing value for ${token}`);
process.exit(1);
}
settingsPath = args[i];
break;
}
case "--verbose":
case "-v":
verbose = true;
break;
case "--force":
case "-f":
force = true;
break;
default: {
if (!databasePath) {
if (command === "daemon" && VALID_COMMANDS.has(token as any)) {
command = token as CLICommand;
break;
}
if (command === "init-settings") {
commandArgs.push(token);
break;
}
databasePath = token;
break;
}
if (command === "daemon" && VALID_COMMANDS.has(token as any)) {
command = token as CLICommand;
break;
}
commandArgs.push(token);
break;
}
}
}
if (!databasePath && command !== "init-settings") {
console.error("Error: database-path is required");
process.exit(1);
}
return {
databasePath,
settingsPath,
verbose,
force,
command,
commandArgs,
};
}
async function createDefaultSettingsFile(options: CLIOptions) {
const targetPath = options.settingsPath
? path.resolve(options.settingsPath)
: options.commandArgs[0]
? path.resolve(options.commandArgs[0])
: path.resolve(process.cwd(), "data.json");
if (!options.force) {
try {
await fs.stat(targetPath);
throw new Error(`Settings file already exists: ${targetPath} (use --force to overwrite)`);
} catch (ex: any) {
if (!(ex && ex?.code === "ENOENT")) {
throw ex;
}
}
}
const settings = {
...DEFAULT_SETTINGS,
useIndexedDBAdapter: false,
} as ObsidianLiveSyncSettings;
await fs.mkdir(path.dirname(targetPath), { recursive: true });
await fs.writeFile(targetPath, JSON.stringify(settings, null, 2), "utf-8");
console.log(`[Done] Created settings file: ${targetPath}`);
}
function toArrayBuffer(data: Buffer): ArrayBuffer {
return data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength) as ArrayBuffer;
}
function toVaultRelativePath(inputPath: string, vaultPath: string): string {
const stripped = inputPath.replace(/^[/\\]+/, "");
if (!path.isAbsolute(inputPath)) {
return stripped.replace(/\\/g, "/");
}
const resolved = path.resolve(inputPath);
const rel = path.relative(vaultPath, resolved);
if (rel.startsWith("..") || path.isAbsolute(rel)) {
throw new Error(`Path ${inputPath} is outside of the local database directory`);
}
return rel.replace(/\\/g, "/");
}
async function runCommand(
options: CLIOptions,
vaultPath: string,
core: LiveSyncBaseCore<ServiceContext, any>
): Promise<boolean> {
await core.services.control.activated;
if (options.command === "daemon") {
return true;
}
if (options.command === "sync") {
console.log("[Command] sync");
const result = await core.services.replication.replicate(true);
return !!result;
}
if (options.command === "push") {
if (options.commandArgs.length < 2) {
throw new Error("push requires two arguments: <src> <dst>");
}
const sourcePath = path.resolve(options.commandArgs[0]);
const destinationVaultPath = toVaultRelativePath(options.commandArgs[1], vaultPath);
const sourceData = await fs.readFile(sourcePath);
const sourceStat = await fs.stat(sourcePath);
console.log(`[Command] push ${sourcePath} -> ${destinationVaultPath}`);
await core.serviceModules.storageAccess.writeFileAuto(destinationVaultPath, toArrayBuffer(sourceData), {
mtime: sourceStat.mtimeMs,
ctime: sourceStat.ctimeMs,
});
const destinationPathWithPrefix = destinationVaultPath as FilePathWithPrefix;
const stored = await core.serviceModules.fileHandler.storeFileToDB(destinationPathWithPrefix, true);
return stored;
}
if (options.command === "pull") {
if (options.commandArgs.length < 2) {
throw new Error("pull requires two arguments: <src> <dst>");
}
const sourceVaultPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
const destinationPath = path.resolve(options.commandArgs[1]);
console.log(`[Command] pull ${sourceVaultPath} -> ${destinationPath}`);
const sourcePathWithPrefix = sourceVaultPath as FilePathWithPrefix;
const restored = await core.serviceModules.fileHandler.dbToStorage(sourcePathWithPrefix, null, true);
if (!restored) {
return false;
}
const data = await core.serviceModules.storageAccess.readFileAuto(sourceVaultPath);
await fs.mkdir(path.dirname(destinationPath), { recursive: true });
if (typeof data === "string") {
await fs.writeFile(destinationPath, data, "utf-8");
} else {
await fs.writeFile(destinationPath, new Uint8Array(data));
}
return true;
}
throw new Error(`Unsupported command: ${options.command}`);
}
async function main() {
const options = parseArgs();
if (options.command === "init-settings") {
await createDefaultSettingsFile(options);
return;
}
// Resolve vault path
const vaultPath = path.resolve(options.databasePath!);
// Check if vault directory exists
try {
const stat = await fs.stat(vaultPath);
if (!stat.isDirectory()) {
console.error(`Error: ${vaultPath} is not a directory`);
process.exit(1);
}
} catch (error) {
console.error(`Error: Vault directory ${vaultPath} does not exist`);
process.exit(1);
}
// Resolve settings path
const settingsPath = options.settingsPath
? path.resolve(options.settingsPath)
: path.join(vaultPath, SETTINGS_FILE);
console.log(`Self-hosted LiveSync CLI`);
console.log(`Vault: ${vaultPath}`);
console.log(`Settings: ${settingsPath}`);
console.log();
// Create service context and hub
const context = new NodeServiceContext(vaultPath);
const serviceHubInstance = new NodeServiceHub<NodeServiceContext>(vaultPath, context);
serviceHubInstance.API.addLog.setHandler((message: string, level: LOG_LEVEL) => {
const prefix = `[${level}]`;
if (level <= LOG_LEVEL_VERBOSE) {
if (!options.verbose) return;
}
console.log(`${prefix} ${message}`);
});
// Prevent replication result to be processed automatically.
serviceHubInstance.replication.processSynchroniseResult.addHandler(async () => {
console.log(`[Info] Replication result received, but not processed automatically in CLI mode.`);
return await Promise.resolve(true);
}, -100);
// Setup settings handlers
const settingService = serviceHubInstance.setting;
(settingService as InjectableSettingService<NodeServiceContext>).saveData.setHandler(
async (data: ObsidianLiveSyncSettings) => {
try {
await fs.writeFile(settingsPath, JSON.stringify(data, null, 2), "utf-8");
if (options.verbose) {
console.log(`[Settings] Saved to ${settingsPath}`);
}
} catch (error) {
console.error(`[Settings] Failed to save:`, error);
}
}
);
(settingService as InjectableSettingService<NodeServiceContext>).loadData.setHandler(
async (): Promise<ObsidianLiveSyncSettings | undefined> => {
try {
const content = await fs.readFile(settingsPath, "utf-8");
const data = JSON.parse(content);
if (options.verbose) {
console.log(`[Settings] Loaded from ${settingsPath}`);
}
// Force disable IndexedDB adapter in CLI environment
data.useIndexedDBAdapter = false;
return data;
} catch (error) {
if (options.verbose) {
console.log(`[Settings] File not found, using defaults`);
}
return undefined;
}
}
);
// Create LiveSync core
const core = new LiveSyncBaseCore(
serviceHubInstance,
(core: LiveSyncBaseCore<NodeServiceContext, any>, serviceHub: InjectableServiceHub<NodeServiceContext>) => {
return initialiseServiceModulesCLI(vaultPath, core, serviceHub);
},
() => [], // No extra modules
() => [], // No add-ons
() => [] // No serviceFeatures
);
// Setup signal handlers for graceful shutdown
const shutdown = async (signal: string) => {
console.log();
console.log(`[Shutdown] Received ${signal}, shutting down gracefully...`);
try {
await core.services.control.onUnload();
console.log(`[Shutdown] Complete`);
process.exit(0);
} catch (error) {
console.error(`[Shutdown] Error:`, error);
process.exit(1);
}
};
process.on("SIGINT", () => shutdown("SIGINT"));
process.on("SIGTERM", () => shutdown("SIGTERM"));
// Start the core
try {
console.log(`[Starting] Initializing LiveSync...`);
const loadResult = await core.services.control.onLoad();
if (!loadResult) {
console.error(`[Error] Failed to initialize LiveSync`);
process.exit(1);
}
await core.services.control.onReady();
console.log(`[Ready] LiveSync is running`);
console.log(`[Ready] Press Ctrl+C to stop`);
console.log();
// Check if configured
const settings = core.services.setting.currentSettings();
if (!settings.isConfigured) {
console.warn(`[Warning] LiveSync is not configured yet`);
console.warn(`[Warning] Please edit ${settingsPath} to configure CouchDB connection`);
console.warn();
console.warn(`Required settings:`);
console.warn(` - couchDB_URI: CouchDB server URL`);
console.warn(` - couchDB_USER: CouchDB username`);
console.warn(` - couchDB_PASSWORD: CouchDB password`);
console.warn(` - couchDB_DBNAME: Database name`);
console.warn();
} else {
console.log(`[Info] LiveSync is configured and ready`);
console.log(`[Info] Database: ${settings.couchDB_URI}/${settings.couchDB_DBNAME}`);
console.log();
}
const result = await runCommand(options, vaultPath, core);
if (!result) {
console.error(`[Error] Command '${options.command}' failed`);
process.exitCode = 1;
} else if (options.command !== "daemon") {
console.log(`[Done] Command '${options.command}' completed`);
}
if (options.command === "daemon") {
// Keep the process running
await new Promise(() => {});
} else {
await core.services.control.onUnload();
}
} catch (error) {
console.error(`[Error] Failed to start:`, error);
process.exit(1);
}
}
// Run main
main().catch((error) => {
console.error(`[Fatal Error]`, error);
process.exit(1);
});

View File

@@ -0,0 +1,133 @@
import type { FilePath, UXFileInfoStub, UXInternalFileInfoStub } from "@lib/common/types";
import type { FileEventItem } from "@lib/common/types";
import type { IStorageEventManagerAdapter } from "@lib/managers/adapters";
import type {
IStorageEventTypeGuardAdapter,
IStorageEventPersistenceAdapter,
IStorageEventWatchAdapter,
IStorageEventStatusAdapter,
IStorageEventConverterAdapter,
IStorageEventWatchHandlers,
} from "@lib/managers/adapters";
import type { FileEventItemSentinel } from "@lib/managers/StorageEventManager";
import type { NodeFile, NodeFolder } from "../adapters/NodeTypes";
import * as fs from "fs/promises";
import * as path from "path";
/**
* CLI-specific type guard adapter
*/
class CLITypeGuardAdapter implements IStorageEventTypeGuardAdapter<NodeFile, NodeFolder> {
isFile(file: any): file is NodeFile {
return file && typeof file === "object" && "path" in file && "stat" in file && !file.isFolder;
}
isFolder(item: any): item is NodeFolder {
return item && typeof item === "object" && "path" in item && item.isFolder === true;
}
}
/**
* CLI-specific persistence adapter (file-based snapshot)
*/
class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
private snapshotPath: string;
constructor(basePath: string) {
this.snapshotPath = path.join(basePath, ".livesync-snapshot.json");
}
async saveSnapshot(snapshot: (FileEventItem | FileEventItemSentinel)[]): Promise<void> {
try {
await fs.writeFile(this.snapshotPath, JSON.stringify(snapshot, null, 2), "utf-8");
} catch (error) {
console.error("Failed to save snapshot:", error);
}
}
async loadSnapshot(): Promise<(FileEventItem | FileEventItemSentinel)[] | null> {
try {
const content = await fs.readFile(this.snapshotPath, "utf-8");
return JSON.parse(content);
} catch {
return null;
}
}
}
/**
* CLI-specific status adapter (console logging)
*/
class CLIStatusAdapter implements IStorageEventStatusAdapter {
private lastUpdate = 0;
private updateInterval = 5000; // Update every 5 seconds
updateStatus(status: { batched: number; processing: number; totalQueued: number }): void {
const now = Date.now();
if (now - this.lastUpdate > this.updateInterval) {
if (status.totalQueued > 0 || status.processing > 0) {
console.log(
`[StorageEventManager] Batched: ${status.batched}, Processing: ${status.processing}, Total Queued: ${status.totalQueued}`
);
}
this.lastUpdate = now;
}
}
}
/**
* CLI-specific converter adapter
*/
class CLIConverterAdapter implements IStorageEventConverterAdapter<NodeFile> {
toFileInfo(file: NodeFile, deleted?: boolean): UXFileInfoStub {
return {
name: path.basename(file.path),
path: file.path,
stat: file.stat,
deleted: deleted,
isFolder: false,
};
}
toInternalFileInfo(p: FilePath): UXInternalFileInfoStub {
return {
name: path.basename(p),
path: p,
isInternal: true,
stat: undefined,
};
}
}
/**
* CLI-specific watch adapter (optional file watching with chokidar)
*/
class CLIWatchAdapter implements IStorageEventWatchAdapter {
constructor(private basePath: string) {}
async beginWatch(handlers: IStorageEventWatchHandlers): Promise<void> {
// File watching is not activated in the CLI.
// Because the CLI is designed for push/pull operations, not real-time sync.
console.log("[CLIWatchAdapter] File watching is not enabled in CLI version");
return Promise.resolve();
}
}
/**
* Composite adapter for CLI StorageEventManager
*/
export class CLIStorageEventManagerAdapter implements IStorageEventManagerAdapter<NodeFile, NodeFolder> {
readonly typeGuard: CLITypeGuardAdapter;
readonly persistence: CLIPersistenceAdapter;
readonly watch: CLIWatchAdapter;
readonly status: CLIStatusAdapter;
readonly converter: CLIConverterAdapter;
constructor(basePath: string) {
this.typeGuard = new CLITypeGuardAdapter();
this.persistence = new CLIPersistenceAdapter(basePath);
this.watch = new CLIWatchAdapter(basePath);
this.status = new CLIStatusAdapter();
this.converter = new CLIConverterAdapter();
}
}

View File

@@ -0,0 +1,28 @@
import { StorageEventManagerBase, type StorageEventManagerBaseDependencies } from "@lib/managers/StorageEventManager";
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
import type { IMinimumLiveSyncCommands, LiveSyncBaseCore } from "../../../LiveSyncBaseCore";
import type { ServiceContext } from "@lib/services/base/ServiceBase";
// import type { IMinimumLiveSyncCommands } from "@lib/services/base/IService";
export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEventManagerAdapter> {
core: LiveSyncBaseCore<ServiceContext, IMinimumLiveSyncCommands>;
constructor(
basePath: string,
core: LiveSyncBaseCore<ServiceContext, IMinimumLiveSyncCommands>,
dependencies: StorageEventManagerBaseDependencies
) {
const adapter = new CLIStorageEventManagerAdapter(basePath);
super(adapter, dependencies);
this.core = core;
}
/**
* Override _watchVaultRawEvents for CLI-specific logic
* In CLI, we don't have internal files like Obsidian's .obsidian folder
*/
protected override async _watchVaultRawEvents(path: string) {
// No-op in CLI version
// Internal file handling is not needed
}
}

16
src/apps/cli/package.json Normal file
View File

@@ -0,0 +1,16 @@
{
"name": "self-hosted-livesync-cli",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview",
"run": "node dist/index.cjs",
"buildRun": "npm run build && npm run",
"check": "svelte-check --tsconfig ./tsconfig.app.json && tsc -p tsconfig.node.json"
},
"dependencies": {},
"devDependencies": {}
}

View File

@@ -0,0 +1,104 @@
import type { InjectableServiceHub } from "@lib/services/implements/injectable/InjectableServiceHub";
import { ServiceRebuilder } from "@lib/serviceModules/Rebuilder";
import { ServiceFileHandler } from "../../../serviceModules/FileHandler";
import { StorageAccessManager } from "@lib/managers/StorageProcessingManager";
import type { LiveSyncBaseCore } from "../../../LiveSyncBaseCore";
import type { ServiceContext } from "@lib/services/base/ServiceBase";
import { FileAccessCLI } from "./FileAccessCLI";
import { ServiceFileAccessCLI } from "./ServiceFileAccessImpl";
import { ServiceDatabaseFileAccessCLI } from "./DatabaseFileAccess";
import { StorageEventManagerCLI } from "../managers/StorageEventManagerCLI";
import type { ServiceModules } from "@lib/interfaces/ServiceModule";
/**
* Initialize service modules for CLI version
* This is the CLI equivalent of ObsidianLiveSyncPlugin.initialiseServiceModules
*
* @param basePath - The base path of the vault directory
* @param core - The LiveSyncBaseCore instance
* @param services - The service hub
* @returns ServiceModules containing all initialized service modules
*/
export function initialiseServiceModulesCLI(
basePath: string,
core: LiveSyncBaseCore<ServiceContext, any>,
services: InjectableServiceHub<ServiceContext>
): ServiceModules {
const storageAccessManager = new StorageAccessManager();
// CLI-specific file access using Node.js FileSystemAdapter
const vaultAccess = new FileAccessCLI(basePath, {
storageAccessManager: storageAccessManager,
vaultService: services.vault,
settingService: services.setting,
APIService: services.API,
pathService: services.path,
});
// CLI-specific storage event manager
const storageEventManager = new StorageEventManagerCLI(basePath, core, {
fileProcessing: services.fileProcessing,
setting: services.setting,
vaultService: services.vault,
storageAccessManager: storageAccessManager,
APIService: services.API,
});
// Storage access using CLI file system adapter
const storageAccess = new ServiceFileAccessCLI({
API: services.API,
setting: services.setting,
fileProcessing: services.fileProcessing,
vault: services.vault,
appLifecycle: services.appLifecycle,
storageEventManager: storageEventManager,
storageAccessManager: storageAccessManager,
vaultAccess: vaultAccess,
});
// Database file access (platform-independent)
const databaseFileAccess = new ServiceDatabaseFileAccessCLI({
API: services.API,
database: services.database,
path: services.path,
storageAccess: storageAccess,
vault: services.vault,
});
// File handler (platform-independent)
const fileHandler = new (ServiceFileHandler as any)({
API: services.API,
databaseFileAccess: databaseFileAccess,
conflict: services.conflict,
setting: services.setting,
fileProcessing: services.fileProcessing,
vault: services.vault,
path: services.path,
replication: services.replication,
storageAccess: storageAccess,
});
// Rebuilder (platform-independent)
const rebuilder = new ServiceRebuilder({
API: services.API,
database: services.database,
appLifecycle: services.appLifecycle,
setting: services.setting,
remote: services.remote,
databaseEvents: services.databaseEvents,
replication: services.replication,
replicator: services.replicator,
UI: services.UI,
vault: services.vault,
fileHandler: fileHandler,
storageAccess: storageAccess,
control: services.control,
});
return {
rebuilder,
fileHandler,
databaseFileAccess,
storageAccess,
};
}

View File

@@ -0,0 +1,15 @@
import {
ServiceDatabaseFileAccessBase,
type ServiceDatabaseFileAccessDependencies,
} from "@lib/serviceModules/ServiceDatabaseFileAccessBase";
import type { DatabaseFileAccess } from "@lib/interfaces/DatabaseFileAccess";
/**
* CLI-specific implementation of ServiceDatabaseFileAccess
* Same as Obsidian version, no platform-specific changes needed
*/
export class ServiceDatabaseFileAccessCLI extends ServiceDatabaseFileAccessBase implements DatabaseFileAccess {
constructor(services: ServiceDatabaseFileAccessDependencies) {
super(services);
}
}

View File

@@ -0,0 +1,20 @@
import { FileAccessBase, type FileAccessBaseDependencies } from "@lib/serviceModules/FileAccessBase";
import { NodeFileSystemAdapter } from "../adapters/NodeFileSystemAdapter";
/**
* CLI-specific implementation of FileAccessBase
* Uses NodeFileSystemAdapter for Node.js file operations
*/
export class FileAccessCLI extends FileAccessBase<NodeFileSystemAdapter> {
constructor(basePath: string, dependencies: FileAccessBaseDependencies) {
const adapter = new NodeFileSystemAdapter(basePath);
super(adapter, dependencies);
}
/**
* Expose the adapter for accessing scanDirectory
*/
get nodeAdapter(): NodeFileSystemAdapter {
return this.adapter;
}
}

View File

@@ -0,0 +1,12 @@
import { ServiceFileAccessBase, type StorageAccessBaseDependencies } from "@lib/serviceModules/ServiceFileAccessBase";
import { NodeFileSystemAdapter } from "../adapters/NodeFileSystemAdapter";
/**
* CLI-specific implementation of ServiceFileAccess
* Uses NodeFileSystemAdapter for platform-specific operations
*/
export class ServiceFileAccessCLI extends ServiceFileAccessBase<NodeFileSystemAdapter> {
constructor(services: StorageAccessBaseDependencies<NodeFileSystemAdapter>) {
super(services);
}
}

View File

@@ -0,0 +1,211 @@
import { LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "@lib/common/logger";
import type { KeyValueDatabase } from "@lib/interfaces/KeyValueDatabase";
import type { IKeyValueDBService } from "@lib/services/base/IService";
import { ServiceBase, type ServiceContext } from "@lib/services/base/ServiceBase";
import type { InjectableAppLifecycleService } from "@lib/services/implements/injectable/InjectableAppLifecycleService";
import type { InjectableDatabaseEventService } from "@lib/services/implements/injectable/InjectableDatabaseEventService";
import type { IVaultService } from "@lib/services/base/IService";
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
import { createInstanceLogFunction } from "@lib/services/lib/logUtils";
import * as nodeFs from "node:fs";
import * as nodePath from "node:path";
class NodeFileKeyValueDatabase implements KeyValueDatabase {
private filePath: string;
private data = new Map<string, unknown>();
constructor(filePath: string) {
this.filePath = filePath;
this.load();
}
private asKeyString(key: IDBValidKey): string {
if (typeof key === "string") {
return key;
}
return JSON.stringify(key);
}
private load() {
try {
const loaded = JSON.parse(nodeFs.readFileSync(this.filePath, "utf-8")) as Record<string, unknown>;
this.data = new Map(Object.entries(loaded));
} catch {
this.data = new Map();
}
}
private flush() {
nodeFs.mkdirSync(nodePath.dirname(this.filePath), { recursive: true });
nodeFs.writeFileSync(this.filePath, JSON.stringify(Object.fromEntries(this.data), null, 2), "utf-8");
}
async get<T>(key: IDBValidKey): Promise<T> {
return this.data.get(this.asKeyString(key)) as T;
}
async set<T>(key: IDBValidKey, value: T): Promise<IDBValidKey> {
this.data.set(this.asKeyString(key), value);
this.flush();
return key;
}
async del(key: IDBValidKey): Promise<void> {
this.data.delete(this.asKeyString(key));
this.flush();
}
async clear(): Promise<void> {
this.data.clear();
this.flush();
}
private isIDBKeyRangeLike(value: unknown): value is { lower?: IDBValidKey; upper?: IDBValidKey } {
return typeof value === "object" && value !== null && ("lower" in value || "upper" in value);
}
async keys(query?: IDBValidKey | IDBKeyRange, count?: number): Promise<IDBValidKey[]> {
const allKeys = [...this.data.keys()];
let filtered = allKeys;
if (typeof query !== "undefined") {
if (this.isIDBKeyRangeLike(query)) {
const lower = query.lower?.toString() ?? "";
const upper = query.upper?.toString() ?? "\uffff";
filtered = filtered.filter((key) => key >= lower && key <= upper);
} else {
const exact = query.toString();
filtered = filtered.filter((key) => key === exact);
}
}
if (typeof count === "number") {
filtered = filtered.slice(0, count);
}
return filtered;
}
async close(): Promise<void> {
this.flush();
}
async destroy(): Promise<void> {
this.data.clear();
nodeFs.rmSync(this.filePath, { force: true });
}
}
export interface NodeKeyValueDBDependencies<T extends ServiceContext = ServiceContext> {
databaseEvents: InjectableDatabaseEventService<T>;
vault: IVaultService;
appLifecycle: InjectableAppLifecycleService<T>;
}
export class NodeKeyValueDBService<T extends ServiceContext = ServiceContext>
extends ServiceBase<T>
implements IKeyValueDBService
{
private _kvDB: KeyValueDatabase | undefined;
private _simpleStore: SimpleStore<any> | undefined;
private filePath: string;
private _log = createInstanceLogFunction("NodeKeyValueDBService");
get simpleStore() {
if (!this._simpleStore) {
throw new Error("SimpleStore is not initialized yet");
}
return this._simpleStore;
}
get kvDB() {
if (!this._kvDB) {
throw new Error("KeyValueDB is not initialized yet");
}
return this._kvDB;
}
constructor(context: T, dependencies: NodeKeyValueDBDependencies<T>, filePath: string) {
super(context);
this.filePath = filePath;
dependencies.databaseEvents.onResetDatabase.addHandler(this._everyOnResetDatabase.bind(this));
dependencies.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));
dependencies.databaseEvents.onDatabaseInitialisation.addHandler(this._everyOnInitializeDatabase.bind(this));
dependencies.databaseEvents.onUnloadDatabase.addHandler(this._onOtherDatabaseUnload.bind(this));
dependencies.databaseEvents.onCloseDatabase.addHandler(this._onOtherDatabaseClose.bind(this));
}
private async openKeyValueDB(): Promise<boolean> {
try {
this._kvDB = new NodeFileKeyValueDatabase(this.filePath);
return true;
} catch (ex) {
this._log("Failed to open Node key-value database", LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
}
private async _everyOnResetDatabase(): Promise<boolean> {
try {
await this._kvDB?.del("queued-files");
await this._kvDB?.destroy();
return await this.openKeyValueDB();
} catch (ex) {
this._log("Failed to reset Node key-value database", LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
}
private async _onOtherDatabaseUnload(): Promise<boolean> {
await this._kvDB?.close();
return true;
}
private async _onOtherDatabaseClose(): Promise<boolean> {
await this._kvDB?.close();
return true;
}
private _everyOnInitializeDatabase(): Promise<boolean> {
return this.openKeyValueDB();
}
private async _everyOnloadAfterLoadSettings(): Promise<boolean> {
if (!(await this.openKeyValueDB())) {
return false;
}
this._simpleStore = this.openSimpleStore<any>("os");
return true;
}
openSimpleStore<T>(kind: string): SimpleStore<T> {
const getDB = () => {
if (!this._kvDB) {
throw new Error("KeyValueDB is not initialized yet");
}
return this._kvDB;
};
const prefix = `${kind}-`;
return {
get: async (key: string): Promise<T> => {
return await getDB().get(`${prefix}${key}`);
},
set: async (key: string, value: any): Promise<void> => {
await getDB().set(`${prefix}${key}`, value);
},
delete: async (key: string): Promise<void> => {
await getDB().del(`${prefix}${key}`);
},
keys: async (from: string | undefined, to: string | undefined, count?: number): Promise<string[]> => {
const allKeys = (await getDB().keys(undefined, count)).map((e) => e.toString());
const lower = `${prefix}${from ?? ""}`;
const upper = `${prefix}${to ?? "\uffff"}`;
return allKeys
.filter((key) => key.startsWith(prefix))
.filter((key) => key >= lower && key <= upper)
.map((key) => key.substring(prefix.length));
},
db: Promise.resolve(getDB()),
} satisfies SimpleStore<T>;
}
}

View File

@@ -0,0 +1,206 @@
import type { AppLifecycleService, AppLifecycleServiceDependencies } from "@lib/services/base/AppLifecycleService";
import { ServiceContext } from "@lib/services/base/ServiceBase";
import * as nodePath from "node:path";
import { ConfigServiceBrowserCompat } from "@lib/services/implements/browser/ConfigServiceBrowserCompat";
import { SvelteDialogManagerBase, type ComponentHasResult } from "@lib/services/implements/base/SvelteDialog";
import { UIService } from "@lib/services/implements/base/UIService";
import { InjectableServiceHub } from "@lib/services/implements/injectable/InjectableServiceHub";
import { InjectableAppLifecycleService } from "@lib/services/implements/injectable/InjectableAppLifecycleService";
import { InjectableConflictService } from "@lib/services/implements/injectable/InjectableConflictService";
import { InjectableDatabaseEventService } from "@lib/services/implements/injectable/InjectableDatabaseEventService";
import { InjectableFileProcessingService } from "@lib/services/implements/injectable/InjectableFileProcessingService";
import { PathServiceCompat } from "@lib/services/implements/injectable/InjectablePathService";
import { InjectableRemoteService } from "@lib/services/implements/injectable/InjectableRemoteService";
import { InjectableReplicationService } from "@lib/services/implements/injectable/InjectableReplicationService";
import { InjectableReplicatorService } from "@lib/services/implements/injectable/InjectableReplicatorService";
import { InjectableTestService } from "@lib/services/implements/injectable/InjectableTestService";
import { InjectableTweakValueService } from "@lib/services/implements/injectable/InjectableTweakValueService";
import { InjectableVaultServiceCompat } from "@lib/services/implements/injectable/InjectableVaultService";
import { ControlService } from "@lib/services/base/ControlService";
import type { IControlService } from "@lib/services/base/IService";
import { HeadlessAPIService } from "@lib/services/implements/headless/HeadlessAPIService";
// import { HeadlessDatabaseService } from "@lib/services/implements/headless/HeadlessDatabaseService";
import type { ServiceInstances } from "@lib/services/ServiceHub";
import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
import { NodeSettingService } from "./NodeSettingService";
import { DatabaseService } from "@lib/services/base/DatabaseService";
import type { ObsidianLiveSyncSettings } from "@/lib/src/common/types";
export class NodeServiceContext extends ServiceContext {
vaultPath: string;
constructor(vaultPath: string) {
super();
this.vaultPath = vaultPath;
}
}
class NodeAppLifecycleService<T extends ServiceContext> extends InjectableAppLifecycleService<T> {
constructor(context: T, dependencies: AppLifecycleServiceDependencies) {
super(context, dependencies);
}
}
class NodeSvelteDialogManager<T extends ServiceContext> extends SvelteDialogManagerBase<T> {
openSvelteDialog<TValue, UInitial>(
component: ComponentHasResult<TValue, UInitial>,
initialData?: UInitial
): Promise<TValue | undefined> {
throw new Error("Method not implemented.");
}
}
type NodeUIServiceDependencies<T extends ServiceContext = ServiceContext> = {
appLifecycle: AppLifecycleService<T>;
config: ConfigServiceBrowserCompat<T>;
replicator: InjectableReplicatorService<T>;
APIService: HeadlessAPIService<T>;
control: IControlService;
};
class NodeDatabaseService<T extends NodeServiceContext> extends DatabaseService<T> {
protected override modifyDatabaseOptions(
settings: ObsidianLiveSyncSettings,
name: string,
options: PouchDB.Configuration.DatabaseConfiguration
): { name: string; options: PouchDB.Configuration.DatabaseConfiguration } {
const optionPass = {
...options,
prefix: this.context.vaultPath + nodePath.sep,
};
const passSettings = { ...settings, useIndexedDBAdapter: false };
return super.modifyDatabaseOptions(passSettings, name, optionPass);
}
}
class NodeUIService<T extends ServiceContext> extends UIService<T> {
override get dialogToCopy(): never {
throw new Error("Method not implemented.");
}
constructor(context: T, dependencies: NodeUIServiceDependencies<T>) {
const headlessConfirm = dependencies.APIService.confirm;
const dialogManager = new NodeSvelteDialogManager<T>(context, {
confirm: headlessConfirm,
appLifecycle: dependencies.appLifecycle,
config: dependencies.config,
replicator: dependencies.replicator,
control: dependencies.control,
});
super(context, {
appLifecycle: dependencies.appLifecycle,
dialogManager,
APIService: dependencies.APIService,
});
}
}
export class NodeServiceHub<T extends NodeServiceContext> extends InjectableServiceHub<T> {
constructor(basePath: string, context: T = new NodeServiceContext(basePath) as T) {
const runtimeDir = nodePath.join(basePath, ".livesync", "runtime");
const localStoragePath = nodePath.join(runtimeDir, "local-storage.json");
const keyValueDBPath = nodePath.join(runtimeDir, "keyvalue-db.json");
const API = new HeadlessAPIService<T>(context);
const conflict = new InjectableConflictService(context);
const fileProcessing = new InjectableFileProcessingService(context);
const setting = new NodeSettingService(context, { APIService: API }, localStoragePath);
const appLifecycle = new NodeAppLifecycleService<T>(context, {
settingService: setting,
});
const remote = new InjectableRemoteService(context, {
APIService: API,
appLifecycle,
setting,
});
const tweakValue = new InjectableTweakValueService(context);
const vault = new InjectableVaultServiceCompat(context, {
settingService: setting,
APIService: API,
});
const test = new InjectableTestService(context);
const databaseEvents = new InjectableDatabaseEventService(context);
const path = new PathServiceCompat(context, {
settingService: setting,
});
const database = new NodeDatabaseService<T>(context, {
API: API,
path,
vault,
setting,
});
const config = new ConfigServiceBrowserCompat<T>(context, {
settingService: setting,
APIService: API,
});
const replicator = new InjectableReplicatorService(context, {
settingService: setting,
appLifecycleService: appLifecycle,
databaseEventService: databaseEvents,
});
const replication = new InjectableReplicationService(context, {
APIService: API,
appLifecycleService: appLifecycle,
replicatorService: replicator,
settingService: setting,
fileProcessingService: fileProcessing,
databaseService: database,
});
const keyValueDB = new NodeKeyValueDBService(
context,
{
appLifecycle,
databaseEvents,
vault,
},
keyValueDBPath
);
const control = new ControlService(context, {
appLifecycleService: appLifecycle,
settingService: setting,
databaseService: database,
fileProcessingService: fileProcessing,
APIService: API,
replicatorService: replicator,
});
const ui = new NodeUIService<T>(context, {
appLifecycle,
config,
replicator,
APIService: API,
control,
});
const serviceInstancesToInit: Required<ServiceInstances<T>> = {
appLifecycle,
conflict,
database,
databaseEvents,
fileProcessing,
replication,
replicator,
remote,
setting,
tweakValue,
vault,
test,
ui,
path,
API,
config,
keyValueDB: keyValueDB as any,
control,
};
super(context, serviceInstancesToInit as any);
}
}

View File

@@ -0,0 +1,61 @@
import { EVENT_SETTING_SAVED } from "@lib/events/coreEvents";
import { EVENT_REQUEST_RELOAD_SETTING_TAB } from "@/common/events";
import { eventHub } from "@lib/hub/hub";
import { handlers } from "@lib/services/lib/HandlerUtils";
import type { ObsidianLiveSyncSettings } from "@lib/common/types";
import type { ServiceContext } from "@lib/services/base/ServiceBase";
import { SettingService, type SettingServiceDependencies } from "@lib/services/base/SettingService";
import * as nodeFs from "node:fs";
import * as nodePath from "node:path";
export class NodeSettingService<T extends ServiceContext> extends SettingService<T> {
private storagePath: string;
private localStore: Record<string, string> = {};
constructor(context: T, dependencies: SettingServiceDependencies, storagePath: string) {
super(context, dependencies);
this.storagePath = storagePath;
this.loadLocalStoreFromFile();
this.onSettingSaved.addHandler((settings) => {
eventHub.emitEvent(EVENT_SETTING_SAVED, settings);
return Promise.resolve(true);
});
this.onSettingLoaded.addHandler((settings) => {
eventHub.emitEvent(EVENT_REQUEST_RELOAD_SETTING_TAB);
return Promise.resolve(true);
});
}
private loadLocalStoreFromFile() {
try {
const loaded = JSON.parse(nodeFs.readFileSync(this.storagePath, "utf-8")) as Record<string, string>;
this.localStore = { ...loaded };
} catch {
this.localStore = {};
}
}
private flushLocalStoreToFile() {
nodeFs.mkdirSync(nodePath.dirname(this.storagePath), { recursive: true });
nodeFs.writeFileSync(this.storagePath, JSON.stringify(this.localStore, null, 2), "utf-8");
}
protected setItem(key: string, value: string) {
this.localStore[key] = value;
this.flushLocalStoreToFile();
}
protected getItem(key: string): string {
return this.localStore[key] ?? "";
}
protected deleteItem(key: string): void {
if (key in this.localStore) {
delete this.localStore[key];
this.flushLocalStoreToFile();
}
}
public saveData = handlers<{ saveData: (data: ObsidianLiveSyncSettings) => Promise<void> }>().binder("saveData");
public loadData = handlers<{ loadData: () => Promise<ObsidianLiveSyncSettings | undefined> }>().binder("loadData");
}

View File

@@ -0,0 +1,69 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
CLI_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)"
cd "$CLI_DIR"
CLI_ENTRY="${CLI_ENTRY:-$CLI_DIR/dist/index.cjs}"
RUN_BUILD="${RUN_BUILD:-1}"
REMOTE_PATH="${REMOTE_PATH:-test/push-pull.txt}"
WORK_DIR="$(mktemp -d "${TMPDIR:-/tmp}/livesync-cli-test.XXXXXX")"
trap 'rm -rf "$WORK_DIR"' EXIT
SETTINGS_FILE="${1:-$WORK_DIR/data.json}"
if [[ "$RUN_BUILD" == "1" ]]; then
echo "[INFO] building CLI..."
npm run build
fi
if [[ ! -f "$CLI_ENTRY" ]]; then
echo "[ERROR] CLI entry not found: $CLI_ENTRY" >&2
exit 1
fi
echo "[INFO] generating settings from DEFAULT_SETTINGS -> $SETTINGS_FILE"
node "$CLI_ENTRY" init-settings --force "$SETTINGS_FILE"
if [[ -n "${COUCHDB_URI:-}" && -n "${COUCHDB_USER:-}" && -n "${COUCHDB_PASSWORD:-}" && -n "${COUCHDB_DBNAME:-}" ]]; then
echo "[INFO] applying CouchDB env vars to generated settings"
SETTINGS_FILE="$SETTINGS_FILE" node <<'NODE'
const fs = require("node:fs");
const settingsPath = process.env.SETTINGS_FILE;
const data = JSON.parse(fs.readFileSync(settingsPath, "utf-8"));
data.couchDB_URI = process.env.COUCHDB_URI;
data.couchDB_USER = process.env.COUCHDB_USER;
data.couchDB_PASSWORD = process.env.COUCHDB_PASSWORD;
data.couchDB_DBNAME = process.env.COUCHDB_DBNAME;
data.isConfigured = true;
fs.writeFileSync(settingsPath, JSON.stringify(data, null, 2), "utf-8");
NODE
else
echo "[WARN] CouchDB env vars are not fully set. push/pull may fail unless generated settings are updated."
fi
VAULT_DIR="$WORK_DIR/vault"
mkdir -p "$VAULT_DIR/test"
SRC_FILE="$WORK_DIR/push-source.txt"
PULLED_FILE="$WORK_DIR/pull-result.txt"
printf 'push-pull-test %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" > "$SRC_FILE"
echo "[INFO] push -> $REMOTE_PATH"
node "$CLI_ENTRY" "$VAULT_DIR" --settings "$SETTINGS_FILE" push "$SRC_FILE" "$REMOTE_PATH"
echo "[INFO] pull <- $REMOTE_PATH"
node "$CLI_ENTRY" "$VAULT_DIR" --settings "$SETTINGS_FILE" pull "$REMOTE_PATH" "$PULLED_FILE"
if cmp -s "$SRC_FILE" "$PULLED_FILE"; then
echo "[PASS] push/pull roundtrip matched"
else
echo "[FAIL] push/pull roundtrip mismatch" >&2
echo "--- source ---" >&2
cat "$SRC_FILE" >&2
echo "--- pulled ---" >&2
cat "$PULLED_FILE" >&2
exit 1
fi

View File

@@ -0,0 +1,32 @@
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"module": "ESNext",
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"skipLibCheck": true,
/* Bundler mode */
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
/* Linting */
"strict": false,
"noUnusedLocals": false,
"noUnusedParameters": false,
"noFallthroughCasesInSwitch": true,
/* Path mapping */
"baseUrl": ".",
"paths": {
"@/*": ["../../*"],
"@lib/*": ["../../lib/src/*"]
}
},
"include": ["*.ts", "**/*.ts", "**/*.tsx"],
"exclude": ["node_modules", "dist"]
}

View File

@@ -0,0 +1,55 @@
import { defineConfig } from "vite";
import { svelte } from "@sveltejs/vite-plugin-svelte";
import path from "node:path";
import { readFileSync } from "node:fs";
const packageJson = JSON.parse(readFileSync("../../../package.json", "utf-8"));
const manifestJson = JSON.parse(readFileSync("../../../manifest.json", "utf-8"));
// https://vite.dev/config/
const defaultExternal = ["obsidian", "electron", "crypto", "pouchdb-adapter-leveldb", "commander", "punycode"];
export default defineConfig({
plugins: [svelte()],
resolve: {
alias: {
"@lib/worker/bgWorker.ts": "../../lib/src/worker/bgWorker.mock.ts",
"@lib/pouchdb/pouchdb-browser.ts": path.resolve(__dirname, "lib/pouchdb-node.ts"),
"@": path.resolve(__dirname, "../../"),
"@lib": path.resolve(__dirname, "../../lib/src"),
"../../src/worker/bgWorker.ts": "../../src/worker/bgWorker.mock.ts",
},
},
base: "./",
build: {
outDir: "dist",
emptyOutDir: true,
minify: false,
rollupOptions: {
input: {
index: path.resolve(__dirname, "main.ts"),
},
external: (id) => {
if (defaultExternal.includes(id)) return true;
if (id.startsWith(".") || id.startsWith("/")) return false;
if (id.startsWith("@/") || id.startsWith("@lib/")) return false;
if (id.endsWith(".ts") || id.endsWith(".js")) return false;
if (id === "fs" || id === "fs/promises" || id === "path" || id === "crypto") return true;
if (id.startsWith("pouchdb-")) return true;
if (id.startsWith("node:")) return true;
return false;
},
},
lib: {
entry: path.resolve(__dirname, "main.ts"),
formats: ["cjs"],
fileName: "index",
},
},
define: {
self: "globalThis",
global: "globalThis",
nonInteractive: "true",
// localStorage: "undefined", // Prevent usage of localStorage in the CLI environment
MANIFEST_VERSION: JSON.stringify(process.env.MANIFEST_VERSION || manifestJson.version || "0.0.0"),
PACKAGE_VERSION: JSON.stringify(process.env.PACKAGE_VERSION || packageJson.version || "0.0.0"),
},
});