mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-11-07 00:38:19 +00:00
Merge pull request #1312 from vrtmrz/docs/setup-selection-guide
docs: clarify backend selection, Fly.io costs, and Object Storage request scaling
This commit is contained in:
7 files changed
+255
-28
No files matched your search
@@ -34,37 +34,69 @@ This plug-in may be particularly useful for researchers, engineers, and develope
|
|||||||
Self-hosting lets us choose a server we trust and manage it ourselves. Please set up and maintain your own server, and keep it secure.
|
Self-hosting lets us choose a server we trust and manage it ourselves. Please set up and maintain your own server, and keep it secure.
|
||||||
|
|
||||||
<a id="setup-workflows"></a>
|
<a id="setup-workflows"></a>
|
||||||
|
<a id="what-you-need"></a>
|
||||||
|
|
||||||
### What you need
|
### 1. Choose a synchronisation backend
|
||||||
|
|
||||||
| Method | What you need | Nice to have |
|
Choose **CouchDB** or **Object Storage** for central storage, or use **WebRTC Peer-to-Peer (P2P)** without a central storage server. CouchDB and Object Storage are alternatives: choosing Object Storage does not require a CouchDB server. P2P can also run alongside either central backend; its connection is selected independently.
|
||||||
| --- | --- | --- |
|
|
||||||
| CouchDB | A CouchDB-compatible server. | — |
|
|
||||||
| Object Storage | An S3-compatible service or server. | — |
|
|
||||||
| Peer-to-Peer | Nothing. | Your own signalling server. |
|
|
||||||
|
|
||||||
For a CouchDB-compatible server, choose a setup guide for your environment:
|
> [!TIP]
|
||||||
|
> **You can switch backends at any time**: Self-hosted LiveSync supports multiple connection profiles via **Saved connections** in settings. You can begin with a hosted option such as Fly.io or Cloudflare R2, and switch to your own server later without rebuilding your Vault files.
|
||||||
|
|
||||||
- [Your own server](docs/setup_own_server.md): run CouchDB on a Raspberry Pi or another server you manage.
|
#### Backend comparison and decision guide
|
||||||
- [Docker](docker/README.md): deploy CouchDB with the supplied Docker stack.
|
|
||||||
- [Fly.io](docs/setup_flyio.md): manage your own CouchDB instance on hosted infrastructure.
|
|
||||||
- [IBM Cloudant (managed)](docs/setup_cloudant.md): use a database service managed by IBM.
|
|
||||||
|
|
||||||
The linked guides explain the setup and any provider-specific costs.
|
| Backend / Environment | LiveSync Mode<br>(Real-time continuous) | Sync while others offline | Mobile support<br>(iOS and Android) | Maintenance effort | Monthly cost example | Recommended for |
|
||||||
|
| :--- | :---: | :---: | :---: | :---: | :---: | :--- |
|
||||||
|
| **CouchDB (Self-hosted)**<br>*(Own VPS, NAS, Docker, or Raspberry Pi)* | ✅ Full | ✅ Yes | ⚠️ Requires HTTPS (public or private endpoint)[^private-https] | High (manage OS, Docker, and backups) | Free (on existing hardware) | Users with home servers or private infrastructure |
|
||||||
|
| **CouchDB (Fly.io)**<br>*(Cloud container PaaS)* | ✅ Full | ✅ Yes | ✅ Out of the box (HTTPS included) | Low (scripted setup via notebook) | ~USD 2.34 / mo in Ashburn[^fly-pricing]<br>*(see [cost details](docs/setup_flyio.md#costs-before-deployment))* | Users wanting real-time sync without managing hardware |
|
||||||
|
| **Object Storage (S3 / R2)**<br>*(Cloudflare R2, AWS S3, or MinIO)* | ❌ Periodic or on-save only | ✅ Yes | Managed services: HTTPS included<br>Self-hosted: configure HTTPS | Managed services: bucket configuration<br>Self-hosted: manage the server and backups | USD 0.00 within R2 free allowances[^r2-pricing]<br>*(see [cost guide](docs/setup_object_storage_costs.md))* | Users who prefer Object Storage without server maintenance |
|
||||||
|
| **WebRTC Peer-to-Peer**<br>*(Direct device-to-device)* | ✅ Full *(while connected)* | ❌ Both devices must be online | ⚠️ Limited by OS background tasks | None (free public signalling relay) | Free (no server required) | Users who prefer no central storage server |
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> **How Object Storage request counts affect costs**:
|
||||||
|
> Unlike traditional file-synchronisation tools that scan every single file on each sync (generating thousands of `LIST` or `HEAD` operations), LiveSync uses **Journal Sync**. It packages only recent changes into compressed delta journals.
|
||||||
|
> **Routine checks do not inspect each Vault file**. Whether your Vault contains 100 or 50,000 files, the journal listing step of an idle check uses one LIST request. Synchronisation also checks the bucket and reads or updates synchronisation parameters and device records. Metadata and Chunks travel together in journal packages, so initial and incremental transfer counts depend on those packages. See [Object Storage transfers, request scaling, and service costs](docs/setup_object_storage_costs.md) for a calculation example covering journal listing and transfers, with control requests identified separately.[^r2-pricing]
|
||||||
|
|
||||||
|
The figures above are calculation examples. Prices and free allowances may change. Check the linked official pricing pages and recalculate costs for your chosen configuration before deploying.
|
||||||
|
|
||||||
|
[^fly-pricing]: November 2026 ([Fly.io resource pricing](https://docs.fly.io/about/pricing)); one `shared-cpu-1x` Machine with 256 MB RAM and a 1 GB persistent volume in Ashburn (`iad`), running continuously for 30 days. Outbound data transfer and optional resources are additional.
|
||||||
|
[^r2-pricing]: November 2026 ([Cloudflare R2 pricing](https://developers.cloudflare.com/r2/pricing/)); Standard storage. Free allowances apply to the account's combined usage.
|
||||||
|
[^private-https]: A private HTTPS endpoint accessible only to your devices can be provided by services such as [Tailscale Serve](https://tailscale.com/docs/features/tailscale-serve). See the [Docker stack](docker/README.md#profile-details) for setup options.
|
||||||
|
|
||||||
|
<a id="prepare-your-chosen-backend"></a>
|
||||||
|
|
||||||
|
### 2. Prepare your chosen backend
|
||||||
|
|
||||||
|
Follow the setup guide for your selected method:
|
||||||
|
|
||||||
|
- **CouchDB**:
|
||||||
|
- **Self-hosted**: deploy using the [Docker stack](docker/README.md) or follow [Your own server](docs/setup_own_server.md).
|
||||||
|
- **Fly.io**: follow [Setup CouchDB on Fly.io](docs/setup_flyio.md) for a scripted cloud deployment.
|
||||||
|
- **IBM Cloudant**: follow [IBM Cloudant setup](docs/setup_cloudant.md) for managed CouchDB hosting.
|
||||||
|
- **Object Storage**:
|
||||||
|
- Follow [Object Storage Setup](docs/setup_object_storage.md) using Cloudflare R2, Amazon S3, or a self-hosted MinIO bucket.
|
||||||
|
- **Peer-to-Peer**:
|
||||||
|
- Follow [Peer-to-Peer Setup](docs/setup_p2p.md) after reviewing the [P2P communication model](docs/p2p.md). No server provisioning is required when using the public signalling relay.
|
||||||
|
|
||||||
<a id="3-minute-setup---couchdb-on-flyio"></a>
|
<a id="3-minute-setup---couchdb-on-flyio"></a>
|
||||||
|
|
||||||
For the Fly.io video walkthrough, see [automated setup](docs/setup_flyio.md#a-very-automated-setup).
|
For the Fly.io video walkthrough, see [automated setup](docs/setup_flyio.md#a-very-automated-setup).
|
||||||
|
|
||||||
P2P does not require a central data-storage server, but a signalling relay is still required for peer discovery. You can use the project's public signalling relay, so there is no server to provision. At least one device containing the required data must be online while another device synchronises. Follow the [Peer-to-Peer Setup](docs/setup_p2p.md) after reviewing the [P2P communication model](docs/p2p.md).
|
If you would like to test WebRTC connectivity between your devices before setting up P2P, try the experimental [WebPeer browser utility](https://vrtmrz.github.io/obsidian-livesync/webpeer/) ([documentation](src/apps/webpeer/README.md)).
|
||||||
|
|
||||||
If you would like to see how P2P setup and connectivity behave on your devices, try the guided connection check in the experimental [WebPeer browser utility](https://vrtmrz.github.io/obsidian-livesync/webpeer/). It uses an empty Vault and does not test note synchronisation. See the [WebPeer documentation](src/apps/webpeer/README.md) for its scope and limitations.
|
<a id="start-synchronising"></a>
|
||||||
|
|
||||||
### Start synchronising
|
### 3. Connect your devices and verify synchronisation
|
||||||
|
|
||||||
1. Install and configure Self-hosted LiveSync on your first device using [Quick Setup](docs/quick_setup.md) for CouchDB, [Object Storage Setup](docs/setup_object_storage.md), or [Peer-to-Peer Setup](docs/setup_p2p.md).
|
Regardless of your chosen backend, connecting your devices follows four straightforward steps:
|
||||||
2. Generate a Setup URI from that working device and follow the same guide to add your other devices.
|
|
||||||
3. Create test notes and confirm that they synchronise in both directions.
|
1. **Prepare your backend** using the guide selected in Step 2 (not required for P2P with the public relay).
|
||||||
|
2. **Set up your first device**: install Self-hosted LiveSync and configure your connection and encryption using the appropriate guide:
|
||||||
|
- For CouchDB: follow [Quick Setup](docs/quick_setup.md) to initialise the local and remote databases from your Vault.
|
||||||
|
- For Object Storage: follow [Object Storage Setup](docs/setup_object_storage.md) to initialise the local database and upload its journals to your bucket.
|
||||||
|
- For Peer-to-Peer: follow [Peer-to-Peer Setup](docs/setup_p2p.md) to initialise the local LiveSync database. P2P has no central remote database.
|
||||||
|
3. **Connect additional devices using a Setup URI**: on your working first device, run `Copy settings as a new Setup URI` from the command palette. Open that encrypted URI on your second device so it receives the identical encryption and connection profiles.
|
||||||
|
4. **Verify two-way synchronisation**: create a test note on your first device and confirm that it appears on the second. Then edit the note on the second device and confirm that the change reflects back on the first device. Configure optional features (such as customisation or hidden-file sync) only after this verification succeeds.
|
||||||
|
|
||||||
<a id="third-party-managed-couchdb-hosting"></a>
|
<a id="third-party-managed-couchdb-hosting"></a>
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -12,7 +12,7 @@ A document's original date and reviewed-version fields describe its baseline. Re
|
|||||||
| Choose a CouchDB setup | [Own server](setup_own_server.md), [Docker stack](../docker/README.md), [Fly.io](setup_flyio.md), or [IBM Cloudant (managed)](setup_cloudant.md) |
|
| Choose a CouchDB setup | [Own server](setup_own_server.md), [Docker stack](../docker/README.md), [Fly.io](setup_flyio.md), or [IBM Cloudant (managed)](setup_cloudant.md) |
|
||||||
| Configure CouchDB clients | [Quick setup](quick_setup.md) |
|
| Configure CouchDB clients | [Quick setup](quick_setup.md) |
|
||||||
| Find hosting services proposed by the community | [Third-party hosting listings](../README.md#hosting-services-proposed-by-the-community) |
|
| Find hosting services proposed by the community | [Third-party hosting listings](../README.md#hosting-services-proposed-by-the-community) |
|
||||||
| Set up S3-compatible storage | [Object Storage Setup](setup_object_storage.md) |
|
| Set up S3-compatible storage | [Object Storage Setup](setup_object_storage.md) and [transfers and service costs](setup_object_storage_costs.md) |
|
||||||
| Set up peer-to-peer synchronisation | [P2P communication model](p2p.md), then [Peer-to-Peer Setup](setup_p2p.md) |
|
| Set up peer-to-peer synchronisation | [P2P communication model](p2p.md), then [Peer-to-Peer Setup](setup_p2p.md) |
|
||||||
| Diagnose P2P connectivity | [Browser-assisted check](../src/apps/webpeer/README.md#p2p-connection-check), [network tips](tips/p2p-sync-tips.md), and [self-hosted TURN](../docker/coturn/README.md) |
|
| Diagnose P2P connectivity | [Browser-assisted check](../src/apps/webpeer/README.md#p2p-connection-check), [network tips](tips/p2p-sync-tips.md), and [self-hosted TURN](../docker/coturn/README.md) |
|
||||||
| Understand a setting or command | [Settings reference](settings.md) |
|
| Understand a setting or command | [Settings reference](settings.md) |
|
||||||
|
|||||||
@@ -102,6 +102,12 @@ recovery guidance, or diagnostics intended for users.
|
|||||||
CouchDB or Object Storage remote, with the current device first. Records
|
CouchDB or Object Storage remote, with the current device first. Records
|
||||||
describe participation, not current online activity. **Forget device** removes
|
describe participation, not current online activity. **Forget device** removes
|
||||||
an obsolete device record; it does not delete Vault files or revoke credentials.
|
an obsolete device record; it does not delete Vault files or revoke credentials.
|
||||||
|
- **Journal / Journal Sync:** The synchronisation mechanism used with
|
||||||
|
S3-compatible Object Storage and non-database remotes. Rather than
|
||||||
|
transferring individual files directly, Self-hosted LiveSync groups database
|
||||||
|
changes (Metadata documents and content Chunks) into compressed, time-stamped
|
||||||
|
journal packages. Devices track their synchronisation sequence using markers
|
||||||
|
and exchange only missing packages.
|
||||||
- **JWT Authentication:** An experimental CouchDB authentication option which
|
- **JWT Authentication:** An experimental CouchDB authentication option which
|
||||||
uses a JSON Web Token instead of standard credentials. It requires a private
|
uses a JSON Web Token instead of standard credentials. It requires a private
|
||||||
key or secret, algorithm, expiry duration, subject, and key ID.
|
key or secret, algorithm, expiry duration, subject, and key ID.
|
||||||
|
|||||||
+3
-1
@@ -11,7 +11,9 @@ Before starting:
|
|||||||
- prepare the remote service and a Setup URI; and
|
- prepare the remote service and a Setup URI; and
|
||||||
- keep the Setup URI and its passphrase separate from each other.
|
- keep the Setup URI and its passphrase separate from each other.
|
||||||
|
|
||||||
This walkthrough covers the recommended provisioned CouchDB path. Follow [Set up a CouchDB server](./setup_own_server.md) to prepare the server and Setup URI.
|
This walkthrough covers the CouchDB synchronisation path (using [your own server](./setup_own_server.md), the [Docker stack](../docker/README.md), or [Fly.io](./setup_flyio.md)).
|
||||||
|
- For S3-compatible cloud storage (such as Cloudflare R2 or Amazon S3), follow [Object Storage Setup](./setup_object_storage.md).
|
||||||
|
- For direct device-to-device synchronisation without a storage server, follow [Peer-to-Peer Setup](./setup_p2p.md).
|
||||||
|
|
||||||
## What a Setup URI contains
|
## What a Setup URI contains
|
||||||
|
|
||||||
|
|||||||
+30
-6
@@ -3,16 +3,32 @@
|
|||||||
|
|
||||||
This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
|
This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
|
||||||
|
|
||||||
The CouchDB instance is yours to manage. You can configure it yourself or set up another instance and switch to it, following the preservation and recovery guidance below.
|
The CouchDB instance is yours to manage, including its updates, credentials, and backups.
|
||||||
|
|
||||||
|
This guide prepares hosted CouchDB, then leads to [first-device setup](quick_setup.md#configure-couchdb-manually-on-the-first-device), [a Setup URI from that working device](quick_setup.md#create-a-setup-uri-for-another-device), and [additional-device setup with a two-way check](quick_setup.md#add-another-device). Review the costs below before deploying an application.
|
||||||
|
|
||||||
|
## Costs before deployment
|
||||||
|
|
||||||
|
Fly.io provides cloud infrastructure on which you run your own CouchDB container. A valid credit or debit card is required upon registration. A time-limited trial or sign-up credit does not provide ongoing free hosting.
|
||||||
|
|
||||||
|
For an example calculation, assume one Machine and a 1 GB persistent volume in Ashburn (`iad`), running continuously for 30 days. Applying the rates below gives a subtotal of approximately **USD 2.34**, with outbound data transfer and optional resources charged separately:[^fly-pricing]
|
||||||
|
|
||||||
|
| Resource | Basis of calculation | Monthly cost in this example |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| One `shared-cpu-1x` Machine (256 MB RAM) | Running continuously for 30 days (720 hours) | ~USD 2.19 |
|
||||||
|
| One 1 GB persistent volume | Provisioned storage (~USD 0.15/GB-month) | ~USD 0.15 |
|
||||||
|
| **Calculated subtotal** | Compute and persistent storage | **~USD 2.34 / month** |
|
||||||
|
|
||||||
|
*Note: Rates vary depending on the chosen Fly.io region, outbound network bandwidth, and optional dedicated IP addresses. The template configuration enables automatic machine suspension (`auto_stop_machines = true`), which may reduce running hours and compute costs when synchronisation is idle, though provisioned storage volumes continue to be billed. Check the current official pricing and recalculate costs for your chosen configuration before deploying.*
|
||||||
|
|
||||||
|
[^fly-pricing]: November 2026 ([Fly.io resource pricing](https://docs.fly.io/about/pricing)); Ashburn (`iad`), with 30 days of continuous Machine operation.
|
||||||
|
|
||||||
> [!WARNING]
|
> [!WARNING]
|
||||||
> Before replacing or deleting an existing CouchDB instance, back up every available Vault and the remote database, then [choose the authoritative copy](recovery.md#first-choose-the-authoritative-copy). Stop or suspend synchronisation on other devices while reviewing recovery options. Local files alone do not establish that every needed edit or revision has been preserved.
|
> Before replacing or deleting an existing CouchDB instance, back up every available Vault and the remote database, then [choose the authoritative copy](recovery.md#first-choose-the-authoritative-copy). Stop or suspend synchronisation on other devices while reviewing recovery options. Local files alone do not establish that every needed edit or revision has been preserved.
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> **What and why is the Fly.io?**
|
> **What and why is the Fly.io?**
|
||||||
> At some point, we started to experience problems related to our IBM Cloudant account. At the same time, Self-hosted LiveSync started to improve its functionality, requiring CouchDB in a more natural state to use all its features.
|
> Fly.io hosts the Machine and persistent volume while you administer CouchDB. For other environments, see [your own server](setup_own_server.md), the [Docker stack](../docker/README.md), or [IBM Cloudant](setup_cloudant.md).
|
||||||
>
|
|
||||||
> Fly.io charges according to current account and resource usage, including Machines, persistent volumes, and network usage. Check [Fly.io pricing](https://fly.io/pricing/) and the [pricing documentation](https://fly.io/docs/about/pricing/) before deploying. A time-limited trial does not provide ongoing free hosting.
|
|
||||||
|
|
||||||
## Required materials
|
## Required materials
|
||||||
|
|
||||||
@@ -267,8 +283,16 @@ Note: Each of these should also be repeated until finished in 200.
|
|||||||
|
|
||||||
#### 6. Use it from Self-hosted LiveSync
|
#### 6. Use it from Self-hosted LiveSync
|
||||||
|
|
||||||
Now the CouchDB is ready to use from Self-hosted LiveSync. Use your Fly.io app's hostname in `URI`, and the unique credentials configured above in `Username` and `Password`. The `Database name` could be anything you want.
|
Now CouchDB is ready for Self-hosted LiveSync. Use these values during manual setup:
|
||||||
Please refer to [Configure CouchDB manually on the first device](./quick_setup.md#configure-couchdb-manually-on-the-first-device).
|
|
||||||
|
| Prepared value | Setup dialogue field |
|
||||||
|
| --- | --- |
|
||||||
|
| `https://<your-app>.fly.dev` | `URL` |
|
||||||
|
| Your configured `COUCHDB_USER` | `Username` |
|
||||||
|
| Your configured `COUCHDB_PASSWORD` secret | `Password` |
|
||||||
|
| Your chosen database name, such as `obsidiannotes` | `Database Name` |
|
||||||
|
|
||||||
|
Please refer to [Configure CouchDB manually on the first device](./quick_setup.md#configure-couchdb-manually-on-the-first-device). After your first device synchronises a test note, generate a new Setup URI on that device and follow the additional-device steps in Quick setup. Confirm that a note created on each device reaches the other unchanged.
|
||||||
|
|
||||||
## Delete the Instance
|
## Delete the Instance
|
||||||
|
|
||||||
|
|||||||
@@ -2,18 +2,43 @@
|
|||||||
|
|
||||||
This guide establishes Object Storage synchronisation on a first device, generates a Setup URI for another device from that working device, and verifies synchronisation in both directions.
|
This guide establishes Object Storage synchronisation on a first device, generates a Setup URI for another device from that working device, and verifies synchronisation in both directions.
|
||||||
|
|
||||||
Object Storage uses the S3-compatible API. Prepare the following before starting:
|
Object Storage exchanges compressed [journals](glossary.md#user-facing-and-operational-terms) through an S3-compatible service or server (such as Cloudflare R2, Amazon S3, or MinIO). It supports manual, periodic, and event-triggered synchronisation; real-time LiveSync mode is unavailable. A CouchDB server is not required.
|
||||||
|
|
||||||
|
Before selecting a commercial storage service or configuring polling intervals, see [Object Storage transfers, request scaling, and service costs](setup_object_storage_costs.md).
|
||||||
|
|
||||||
|
## Before starting
|
||||||
|
|
||||||
|
Prepare the following:
|
||||||
|
|
||||||
- an HTTPS endpoint reachable by every device;
|
- an HTTPS endpoint reachable by every device;
|
||||||
- an access key and secret key with access to the selected bucket;
|
- an access key and secret key with access to the selected bucket;
|
||||||
- a bucket name and region;
|
- a bucket name and region;
|
||||||
- a unique bucket prefix when the bucket is shared; and
|
- a unique bucket prefix when the bucket is shared or reserved for this Vault;
|
||||||
|
- CORS configuration permitting Obsidian's web origins (`app://obsidian.md`, `capacitor://localhost`, `http://localhost`) with `ETag` exposed (or enable `Use internal API` during setup to bypass CORS restrictions); and
|
||||||
- separate passphrases for Vault encryption and Setup URI encryption.
|
- separate passphrases for Vault encryption and Setup URI encryption.
|
||||||
|
|
||||||
Back up every Vault involved, and do not use Obsidian Sync, iCloud synchronisation, or another synchronisation service on the same Vault.
|
Back up every Vault involved, and do not use Obsidian Sync, iCloud synchronisation, or another synchronisation service on the same Vault.
|
||||||
|
|
||||||
|
### Setting fields reference
|
||||||
|
|
||||||
|
Match your storage service credentials to the plug-in setup fields:
|
||||||
|
|
||||||
|
| Value from your service or server | Setup dialogue field | Setup URI generator variable |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| S3 API endpoint (not the admin web console) | `Endpoint URL` | `endpoint` |
|
||||||
|
| Access key ID | `Access Key ID` | `access_key` |
|
||||||
|
| Secret access key | `Secret Access Key` | `secret_key` |
|
||||||
|
| Existing bucket name | `Bucket Name` | `bucket` |
|
||||||
|
| Region specified by the provider (e.g., `auto` for R2) | `Region` | `region` |
|
||||||
|
| Folder prefix reserved for this Vault, such as `my-vault/` | `Folder Prefix` | `bucket_prefix` |
|
||||||
|
| Path-style addressing (required by some servers like MinIO) | `Use Path-Style Access` | `force_path_style` |
|
||||||
|
| Bypass CORS via Obsidian's native request API | `Use internal API` | `use_custom_request_handler` |
|
||||||
|
|
||||||
## Generate the initial Setup URI
|
## Generate the initial Setup URI
|
||||||
|
|
||||||
|
> [!TIP]
|
||||||
|
> If you prefer to configure settings directly in Obsidian rather than using the command-line Setup URI generator, see [Configure the first device manually](#configure-the-first-device-manually).
|
||||||
|
|
||||||
The public generator applies the Object Storage preset and records the connection as the selected remote profile. Run it from a trusted terminal:
|
The public generator applies the Object Storage preset and records the connection as the selected remote profile. Run it from a trusted terminal:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
@@ -101,6 +126,19 @@ Confirm that the first device's test note appears unchanged. Create a second ord
|
|||||||
|
|
||||||

|

|
||||||
|
|
||||||
|
## Configure the first device manually
|
||||||
|
|
||||||
|
If you prefer to configure settings directly in Obsidian rather than using the Deno Setup URI generator, use this manual procedure:
|
||||||
|
|
||||||
|
1. Install and enable Self-hosted LiveSync, then open onboarding from the `Welcome to Self-hosted LiveSync` Notice.
|
||||||
|
2. Select `I am setting this up for the first time`, then choose `Configure a remote manually` and select `Proceed with manual configuration`.
|
||||||
|
3. Configure `End-to-End Encryption` and enter a strong Vault encryption passphrase. Save this passphrase securely.
|
||||||
|
4. Select the S3-compatible Object Storage option and proceed to `S3/MinIO/R2 Configuration`.
|
||||||
|
5. Enter your bucket settings using the [Setting fields reference](#setting-fields-reference) table above.
|
||||||
|
6. Select `Test Settings and Continue`. If the connection test fails, verify the API endpoint, credentials, bucket permissions, and region. If CORS blocks the request or cannot be configured on your bucket, enable `Use internal API` to route requests through Obsidian's native HTTP handler.
|
||||||
|
7. Proceed to `Restart and Initialise Server`, then complete the confirmation steps as described in [Set up the first device](#set-up-the-first-device).
|
||||||
|
8. Once your test note synchronises, follow [Generate the second-device Setup URI](#generate-the-second-device-setup-uri) to connect subsequent devices easily.
|
||||||
|
|
||||||
## Safety notes
|
## Safety notes
|
||||||
|
|
||||||
- Treat the endpoint, bucket, prefix, access key, secret key, Vault passphrase, Setup URI, and Setup URI passphrase as sensitive.
|
- Treat the endpoint, bucket, prefix, access key, secret key, Vault passphrase, Setup URI, and Setup URI passphrase as sensitive.
|
||||||
|
|||||||
@@ -0,0 +1,125 @@
|
|||||||
|
# Object Storage transfers, request scaling, and service costs
|
||||||
|
|
||||||
|
This guide explains how Self-hosted LiveSync transfers data when using S3-compatible Object Storage, why API requests do not scale with the total number of files in your Vault, and how to calculate example costs from request counts before selecting a storage provider.
|
||||||
|
|
||||||
|
For setting up the plug-in with an existing bucket, see [Object Storage Setup](setup_object_storage.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How Journal Sync transfers data
|
||||||
|
|
||||||
|
Self-hosted LiveSync stores file properties in [Metadata](glossary.md#user-facing-and-operational-terms) and file contents in Chunks. When synchronising with Object Storage, it does not manage files as individual remote objects. Instead, it packages database changes into compressed, time-stamped **journals** using the [Journal Replicator](design_docs_of_journalsync_2nd.md) architecture.
|
||||||
|
|
||||||
|
### Why API requests do not scale with Vault file count
|
||||||
|
|
||||||
|
Traditional file-synchronisation tools often inspect every file in the remote storage to detect changes, issuing `LIST` or `HEAD` operations proportional to the total number of files in the Vault. In that model, a Vault with 10,000 files generates thousands of requests on every synchronisation cycle, quickly exhausting free allowances and incurring substantial API fees.
|
||||||
|
|
||||||
|
Self-hosted LiveSync works fundamentally differently:
|
||||||
|
|
||||||
|
- **Incremental journal tracking**: Each device tracks its synchronisation progress using sequence markers.
|
||||||
|
- **Journal listing**: The journal listing step uses `ListObjectsV2` to inspect journal packages after the last received journal.
|
||||||
|
- **Journal transfer**: Metadata and Chunks are packed together in journals. Each new journal package is uploaded with `PutObject`, and each missing package is downloaded with `GetObject`; individual Chunks are not separate remote objects.
|
||||||
|
- **Control requests**: Synchronisation also checks bucket availability (`HeadBucket`), reads synchronisation parameters and device participation data (`GetObject`), and can update device participation data (`PutObject`), even when no Vault files have changed.
|
||||||
|
|
||||||
|
Consequently, **whether your Vault contains 100 notes or 50,000 notes, the journal listing step of an idle check uses one LIST request**. The full request count also includes control requests, journal transfers, and any retries. Routine checks do not need a separate request for each unchanged Vault file.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Mapping activities to S3-compatible API operations
|
||||||
|
|
||||||
|
| Activity | Primary S3 API operation | Provider pricing category | What affects the request count |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| **Check for changes** | `ListObjectsV2` | Class A / Write-equivalent | Frequency of periodic sync checks and event triggers across active devices. |
|
||||||
|
| **Send changes** | `PutObject` | Class A / Write | Number of journal packages uploaded, each containing Metadata and/or Chunks. |
|
||||||
|
| **Receive changes** | `GetObject` | Class B / Read | Number of journal packages downloaded from other devices. |
|
||||||
|
| **Connection, compatibility, and device checks** | `HeadBucket`, `GetObject`, `PutObject` | Provider-specific read/write categories | Bucket availability checks and reads or updates of synchronisation parameters and device participation records, including during idle cycles. |
|
||||||
|
| **First-device initialisation** | `ListObjectsV2`, `PutObject` | Class A / Write | Listing and uploading journal packages containing the initial Metadata and Chunks, in addition to control requests. |
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> In most S3-compatible pricing models (including Amazon S3 and Cloudflare R2), `ListObjectsV2` is classified as a write-equivalent (Class A) operation, even though it reads metadata. Keeping periodic synchronisation intervals reasonable is the most effective way to manage request volume.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Estimating monthly request volume
|
||||||
|
|
||||||
|
To estimate your monthly request volume, consider three factors:
|
||||||
|
1. **Number of active devices**: Devices running Obsidian concurrently.
|
||||||
|
2. **Synchronisation interval**: The periodic polling interval configured in settings.
|
||||||
|
3. **Daily editing volume**: Number of modifications and new notes created per day.
|
||||||
|
|
||||||
|
### Calculation formula for routine checks
|
||||||
|
|
||||||
|
$$\text{Monthly listing requests} = \text{Devices} \times \text{Hours active/day} \times \left(\frac{60}{\text{Interval in minutes}}\right) \times 30\text{ days}$$
|
||||||
|
|
||||||
|
### Worked example
|
||||||
|
|
||||||
|
This example counts journal listing, uploads, and downloads. Bucket checks, synchronisation parameters, device participation records, and retries add further requests and are excluded from this subtotal.
|
||||||
|
|
||||||
|
Suppose you use two devices (for example, a desktop and a laptop):
|
||||||
|
|
||||||
|
- Both devices run Obsidian for 8 hours per day.
|
||||||
|
- Periodic synchronisation is set to once every **5 minutes** (12 checks per hour).
|
||||||
|
- For the journal transfer calculation, assume a combined 50 PUT requests and 100 GET requests per day across both devices. These are illustrative request counts; actual transfers depend on edit activity and journal packing.
|
||||||
|
|
||||||
|
```text
|
||||||
|
Routine listing checks:
|
||||||
|
2 devices × 8 hours/day × 12 checks/hour × 30 days = 5,760 LIST requests/month
|
||||||
|
|
||||||
|
Uploads: 50 PUT requests/day × 30 days = 1,500 PUT requests/month
|
||||||
|
Downloads: 100 GET requests/day × 30 days = 3,000 GET requests/month
|
||||||
|
|
||||||
|
Counted journal requests: 7,260 LIST/PUT and 3,000 GET requests/month
|
||||||
|
```
|
||||||
|
|
||||||
|
At the paid Amazon S3 Standard rates for US East (N. Virginia) below:[^s3-pricing]
|
||||||
|
|
||||||
|
```text
|
||||||
|
LIST: 5,760 ÷ 1,000 × USD 0.005 = USD 0.0288
|
||||||
|
PUT: 1,500 ÷ 1,000 × USD 0.005 = USD 0.0075
|
||||||
|
GET: 3,000 ÷ 1,000 × USD 0.0004 = USD 0.0012
|
||||||
|
|
||||||
|
Journal request subtotal in this example: USD 0.0375/month (about USD 0.04)
|
||||||
|
```
|
||||||
|
|
||||||
|
Add control requests, retries, storage, and data transfer when estimating the full cost. On Cloudflare R2 Standard, the counted journal requests fit the monthly free allowances if the account's other usage leaves enough allowance available.[^r2-pricing]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Initial upload and first-time device download
|
||||||
|
|
||||||
|
Initial setup transfers journal packages containing the existing Metadata and Chunks. A joining device downloads the existing packages which it has not yet processed:
|
||||||
|
|
||||||
|
1. **Initial Vault upload (first device)**:
|
||||||
|
- The first device splits files into Metadata documents and content Chunks, then groups those records into compressed journal packages.
|
||||||
|
- Each package is uploaded with one `PutObject` request. The number of requests depends on journal packing and the size of the records, rather than a one-to-one mapping to Vault files or Chunks.
|
||||||
|
2. **Initial Vault download (adding a second device)**:
|
||||||
|
- The joining device downloads journal packages with `GetObject` and applies the Metadata and Chunks they contain to its local database.
|
||||||
|
- The GET request count follows the number of downloaded packages; outbound data transfer depends on their total size. Control requests and retries add further operations.
|
||||||
|
|
||||||
|
### Financial impact of initial transfers
|
||||||
|
|
||||||
|
Calculate initial request charges from the number of API operations:
|
||||||
|
|
||||||
|
- **GET requests**: At the paid Amazon S3 rate used below, 5,000 GET requests cost USD 0.002, excluding storage and data transfer.[^s3-pricing] Cloudflare R2 Standard includes 10,000,000 free Class B operations every month.[^r2-pricing]
|
||||||
|
- **Egress bandwidth**: Cloudflare R2 charges zero egress fees.[^r2-pricing] For paid Amazon S3 usage, calculate outbound data transfer separately using the current rates for your region.[^s3-pricing]
|
||||||
|
- **Later synchronisation**: Once caught up, the device checks for new journal packages and downloads those it has not yet processed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Comparing provider allowances and costs
|
||||||
|
|
||||||
|
The table applies published rates checked in November 2026 to the journal requests counted in the worked example above. Prices and free allowances may change. Before choosing a provider or configuring synchronisation, check the linked official pricing pages and recalculate costs for your region, storage usage, request volume, and data transfer.
|
||||||
|
|
||||||
|
| Provider | Included monthly free allowances | Paid rates | Calculated journal request subtotal |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| **Cloudflare R2 Standard**[^r2-pricing] | • **1,000,000 Class A** (LIST, PUT)<br>• **10,000,000 Class B** (GET)<br>• **10 GB-month storage**<br>• **Zero egress fees** | • Class A: USD 4.50 / million<br>• Class B: USD 0.36 / million<br>• Storage: USD 0.015 / GB-month | **USD 0.00 for the counted journal requests**, assuming sufficient free allowance remains. The calculation uses 7,260 Class A and 3,000 Class B operations. |
|
||||||
|
| **Amazon S3 Standard (paid usage)**<br>US East (N. Virginia)[^s3-pricing] | — | • PUT/LIST: USD 0.005 / 1,000<br>• GET: USD 0.0004 / 1,000<br>• Storage: USD 0.023 / GB-month (first 50 TB)<br>• Internet data transfer charged separately | **About USD 0.04** for the counted journal requests. Other requests, storage, and data transfer are additional. |
|
||||||
|
|
||||||
|
### Practical recommendations
|
||||||
|
|
||||||
|
- **Cloudflare R2 may suit individual users** seeking a backend without server maintenance. The counted journal requests fit its monthly free allowances; include control requests, storage, and other account usage when comparing with the published limits.[^r2-pricing]
|
||||||
|
- **Avoid excessively short polling intervals** (such as polling every 10–15 seconds) unless necessary. A 1–5 minute interval provides timely synchronisation while preserving low request counts.
|
||||||
|
- **Batch saves and event triggers**: Enabling 'On Events' (sync on save or startup) allows you to lengthen the periodic interval without delaying synchronisation when active edits occur.
|
||||||
|
|
||||||
|
[^r2-pricing]: November 2026 ([Cloudflare R2 pricing](https://developers.cloudflare.com/r2/pricing/)); Standard storage.
|
||||||
|
[^s3-pricing]: November 2026 ([Amazon S3 pricing](https://aws.amazon.com/s3/pricing/)); S3 Standard paid usage in US East (N. Virginia).
|
||||||
Reference in new issue
Block a user