What is this?

The “Geek’s Cookbook” is a collection of guides for establishing your own highly-available docker container cluster (swarm). This swarm enables you to run self-hosted services such as GitLab, Plex, NextCloud, etc. Recent updates and additions are posted on the CHANGELOG.

Who is this for?

You already have a familiarity with concepts such as virtual machines, Docker containers, LetsEncrypt SSL certificates, databases, and command-line interfaces.

You’ve probably played with self-hosting some mainstream apps yourself, like Plex, OwnCloud, Wordpress or even SandStorm.

Why should I read this?

So if you’re familiar enough with the tools, and you’ve done self-hosting before, why would you read this book?

You want to upskill. You want to do container orchestration, LetsEncrypt certificates, git collaboration.

You want to play. You want a safe sandbox to test new tools, keeping the ones you want and tossing the ones you don’t.

You want reliability. Once you go from playing with a tool to actually using it, you want it to be available when you need it. Having to “quickly ssh into the host and restart the webserver” doesn’t cut it when your wife wants to know why her phone won’t sync!

What have you done for me lately? (CHANGELOG)

Check out recent change at CHANGELOG

What do you want from me?

I want your money.

No, seriously (but yes, I do want your money - see below), If the above applies to you, then you’re like me. I want everything I wrote above, so I ended up learning all this as I went along. I enjoy it, and I’m good at it. So I created this website, partly to make sure I documented my own setup properly.

Get in touch 👋

Come and say hi to me and the friendly geeks in the Discord chat or the Discourse forums - say hi, ask a question, or suggest a new recipe!

Tweet me up, I’m [@funkypenguin](https://twitter.com/funkypenguin)! 🐦

Contact me by a variety of channels

Buy my book 📖

I’m also writing the Geek Cookbook as a formal eBook, on Leanpub (https://leanpub.com/geeks-cookbook). Buy it for $0.99 (which is really just a token gesture of support) - you can get it for free (in PDF, mobi, or epub format), or pay me what you think it’s worth!

Donate / Support me 💰

The best way to support this work is to become a Patreon patron (for as little as $1/month!) - You get :

warm fuzzies,

access to the pre-mix repo,

an anonymous plug you can pull at any time,

and a bunch more loot based on tier

.. and I get some pocket money every month to buy wine, cheese, and cryptocurrency! 🍷 💰

Impulsively click here (NOW quick do it!) to patronize me, or instead thoughtfully and analytically review my Patreon page / history here and make up your own mind.

I also gratefully accept donations of most fine socialist/anarchist/hobbyist cryptocurrencies, including the list below (let me know if I’ve left out the coin-of-the-week, and I’ll happily add it):

-ist-currency | Address
Bitcoin | 1GBJfmqARmL66gQzUy9HtNWdmAEv74nfXj
Ethereum | 0x19e60ec49e1f053cfdfc193560ecfb3caed928f1
Litecoin | LYLEF7xTpeVbjjoZD3jGLVgvKSKTYDKbK8
:turtle: TurtleCoin | TRTLv2qCKYChMbU5sNkc85hzq2VcGpQidaowbnV2N6LAYrFNebMLepKKPrdif75x5hAizwfc1pX4gi5VsR9WQbjQgYcJm21zec4

Hire me 🏢

Need some system design work done? I do freelance consulting - contact me for details.

How to read this book

Structure

“Recipes” generally follow on from each other. I.e., if a particular recipe requires a mail server, that mail server would have been described in an earlier recipe.

Each recipe contains enough detail in a single page to take a project from start to completion.

When there are optional add-ons/integrations possible to a project (i.e., the addition of “smart LED bulbs” to Home Assistant), this will be reflected either as a brief “Chef’s note” after the recipe, or if they’re substantial enough, as a sub-page of the main project

Conventions

When creating swarm networks, we always explicitly set the subnet in the overlay network, to avoid potential conflicts (which docker won’t prevent, but which will generate errors) (https://github.com/moby/moby/issues/26912)

CHANGELOG

Subscribe to updates

Email : Sign up here (double-opt-in) to receive email updates on new and improve recipes!

Mastodon: https://mastodon.social/@geekcookbook_changes

RSS: https://mastodon.social/@geekcookbook_changes.atom

The #changelog channel in our Discord server

Recent additions to work-in-progress

Kubernetes recipes for UniFi controller, Miniflux, Kanboard and PrivateBin coming in March! (19 Mar 2019)

Recently added recipes

Added Kubernetes version of Miniflux recipe, a minimalistic RSS reader supporting the Fever API (26 Mar 2019)

Added Kubernetes version of Kanboard recipe, a lightweight, well-supported Kanban tool for visualizing your work (19 Mar 2019)

Added Minio, a high performance distributed object storage server, designed for large-scale private cloud infrastructure, but perfect for simple use cases where emulating AWS S3 is useful. (27 Jan 2019)

Added the beginning of the Kubernetes design, including a getting started on using Digital Ocean, and a WIP recipe for an MQTT broker (21 Jan 2019)

ElkarBackup, a beautiful GUI-based backup solution built on rsync/rsnapshot (1 Jan 2019)

Recent improvements

Added recipe for automated snapshots of Kubernetes Persistent Volumes, instructions for using Helm, and recipe for deploying Traefik, which completes the Kubernetes cluster design! (9 Feb 2019)

Added detailed description (and diagram) of our Kubernetes design, plus a simple load-balancer design to avoid the complexities/costs of permitting ingress access to a cluster (7 Feb 2019)

Added an introductory/explanatory page, including a children’s story, on Kubernetes (29 Jan 2019)

NextCloud updated to fix CalDAV/CardDAV service discovery behind Traefik reverse proxy (12 Dec 2018)

Welcome to Funky Penguin’s Geek Cookbook

Hello world,

I’m David.

I’ve spent 20+ years working with technology. My current role is Senior Infrastructure Architect at Prophecy Networks Ltd in New Zealand, with a specific interest in networking, systems, open-source, and business management.

I’ve had a book published, and I blog on topics that interest me.

Why Funky Penguin?

My first “real” job, out of high-school, was working the IT helpdesk in a typical pre-2000 organization in South Africa. I enjoyed experimenting with Linux, and cut my teeth by replacing the organization’s Exchange 5.5 mail platform with a 15-site qmail-ldap cluster, with amavis virus-scanning.

One of our suppliers asked me to quote to do the same for their organization. With nothing to loose, and half-expecting to be turned down, I quoted a generous fee, and chose a cheeky company name. The supplier immediately accepted my quote, and the name (“Funky Penguin”) stuck.

Technical Documentation

During the same “real” job above, I wanted to deploy jabberd, for internal instant messaging within the organization, and as a means to control the sprawl of ad-hoc instant-messaging among staff, using ICQ, MSN, and Yahoo Messenger.

To get management approval to deploy, I wrote a logger (with web UI) for jabber conversations (Bandersnatch), and a 75-page user manual (in Docbook XML) for a spunky Russian WinXP jabber client, JAJC.

Due to my contributions to phpList, I was approached in 2011 by Packt Publishing, to write a book about using PHPList.

Contact Me

Contact me by:

Jumping into our Discord server

Email (davidy@funkypenguin.co.nz)

Private, encrypted email with ProtonMail (funkypenguin@pm.me)

Twitter ([@funkypenguin](https://twitter.com/funkypenguin))

Or by using the form below:

HA Docker Swarm

This section introduces the HA Docker Swarm, which will be the basis for all the recipes discussed.

Design

In the design described below, the “private cloud” platform is:

Highly-available (can tolerate the failure of a single component)

Scalable (can add resource or capacity as required)

Portable (run it on your garage server today, run it in AWS tomorrow)

Secure (access protected with LetsEncrypt certificates)

Automated (requires minimal care and feeding)

Design Decisions

Where possible, services will be highly available.

This means that:

At least 3 docker swarm manager nodes are required, to provide fault-tolerance of a single failure.

GlusterFS is employed for share filesystem, because it too can be made tolerant of a single failure.

Where multiple solutions to a requirement exist, preference will be given to the most portable solution.

This means that:

Services are defined using docker-compose v3 YAML syntax

Services are portable, meaning a particular stack could be shut down and moved to a new provider with minimal effort.

Security

Under this design, the only inbound connections we’re permitting to our docker swarm are:

Network Flows

HTTP (TCP 80) : Redirects to https

HTTPS (TCP 443) : Serves individual docker containers via SSL-encrypted reverse proxy

Authentication

Where the proxied application provides a trusted level of authentication, or where the application requires public exposure,

High availability

Normal function

Assuming 3 nodes, under normal circumstances the following is illustrated:

All 3 nodes provide shared storage via GlusterFS, which is provided by a docker container on each node. (i.e., not running in swarm mode)

All 3 nodes participate in the Docker Swarm as managers.

The various containers belonging to the application “stacks” deployed within Docker Swarm are automatically distributed amongst the swarm nodes.

Persistent storage for the containers is provide via GlusterFS mount.

The traefik service (in swarm mode) receives incoming requests (on http and https), and forwards them to individual containers. Traefik knows the containers names because it’s able to access the docker socket.

All 3 nodes run keepalived, at different priorities. Since traefik is running as a swarm service and listening on TCP 80/443, requests made to the keepalived VIP and arriving at any of the swarm nodes will be forwarded to the traefik container (no matter which node it’s on), and then onto the target backend.

[picture]

HA function

Node failure

In the case of a failure (or scheduled maintenance) of one of the nodes, the following is illustrated:

The failed node no longer participates in GlusterFS, but the remaining nodes provide enough fault-tolerance for the cluster to operate.

The remaining two nodes in Docker Swarm achieve a quorum and agree that the failed node is to be removed.

The (possibly new) leader manager node reschedules the containers known to be running on the failed node, onto other nodes.

The traefik service is either restarted or unaffected, and as the backend containers stop/start and change IP, traefik is aware and updates accordingly.

The keepalived VIP continues to function on the remaining nodes, and docker swarm continues to forward any traffic received on TCP 80/443 to the appropriate node.

[picture]

HA function

Node restore

When the failed (or upgraded) host is restored to service, the following is illustrated:

GlusterFS regains full redundancy

Docker Swarm managers become aware of the recovered node, and will use it for scheduling new containers

Existing containers which were migrated off the node are not migrated backend

Keepalived VIP regains full redundancy

[picture]

HA function

Total cluster failure

A day after writing this, my environment suffered a fault whereby all 3 VMs were unexpectedly and simultaneously powered off.

Upon restore, docker failed to start on one of the VMs due to local disk space issue¹. However, the other two VMs started, established the swarm, mounted their shared storage, and started up all the containers (services) which were managed by the swarm.

In summary, although I suffered an unplanned power outage to all of my infrastructure, followed by a failure of a third of my hosts… ==all my platforms are 100% available with absolutely no manual intervention==.

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Since there’s no impact to availability, I can fix (or just reinstall) the failed node whenever convenient.

Virtual Machines

Let’s start building our cloud with virtual machines. You could use bare-metal machines as well, the configuration would be the same. Given that most readers (myself included) will be using virtual infrastructure, from now on I’ll be referring strictly to VMs.

I chose the “Atomic” CentOS/Fedora image for the VM layer because:

I want less responsibility for maintaining the system, including ensuring regular software updates and reboots. Atomic’s idempotent nature means the OS is largely read-only, and updates/rollbacks are “atomic” (haha) procedures, which can be easily rolled back if required.

For someone used to administrating servers individually, Atomic is a PITA. You have to employ tricky tricks to get it to install in a non-cloud environment. It’s not designed for tweaking or customizing beyond what cloud-config is capable of. For my purposes, this is good, because it forces me to change my thinking - to consider every daemon as a container, and every config as code, to be checked in and version-controlled. Atomic forces this thinking on you.

I want the design to be as “portable” as possible. While I run it on VPSs now, I may want to migrate it to a “cloud” provider in the future, and I’ll want the most portable, reproducible design.

Ingredients

!!! summary “Ingredients” 3 x Virtual Machines, each with:

* [ ] CentOS/Fedora Atomic
* [ ] At least 1GB RAM
* [ ] At least 20GB disk space (_but it'll be tight_)
* [ ] Connectivity to each other within the same subnet, and on a low-latency link (_i.e., no WAN links_)

Preparation

Install Virtual machines

Install / launch virtual machines.

The default username on CentOS atomic is “centos”, and you’ll have needed to supply your SSH key during the build process.

!!! tip If you’re not using a platform with cloud-init support (i.e., you’re building a VM manually, not provisioning it through a cloud provider), you’ll need to refer to trick #1 and trick #2 for a means to override the automated setup, apply a manual password to the CentOS account, and enable SSH password logins.

Prefer docker-latest

Run the following on each node to replace the default docker 1.12 with docker 1.13 (which we need for swarm mode):

systemctl disable docker --now
systemctl enable docker-latest --now
sed -i '/DOCKERBINARY/s/^#//g' /etc/sysconfig/docker

Upgrade Atomic

Finally, apply any Atomic host updates, and reboot, by running: atomic host upgrade && systemctl reboot.

Permit connectivity between VMs

By default, Atomic only permits incoming SSH. We’ll want to allow all traffic between our nodes, so add something like this to /etc/sysconfig/iptables:

# Allow all inter-node communication
-A INPUT -s 192.168.31.0/24 -j ACCEPT

And restart iptables with systemctl restart iptables

Enable host resolution

Depending on your hosting environment, you may have DNS automatically setup for your VMs. If not, it’s useful to set up static entries in /etc/hosts for the nodes. For example, I setup the following:

192.168.31.11   ds1     ds1.funkypenguin.co.nz
192.168.31.12   ds2     ds2.funkypenguin.co.nz
192.168.31.13   ds3     ds3.funkypenguin.co.nz

Set timezone

Set your local timezone, by running:

ln -sf /usr/share/zoneinfo/<your timezone> /etc/localtime

Serving

After completing the above, you should have:

[X] 3 x fresh atomic instances, at the latest releases,
    running Docker v1.13 (docker-latest)

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Shared Storage (Ceph)

While Docker Swarm is great for keeping containers running (and restarting those that fail), it does nothing for persistent storage. This means if you actually want your containers to keep any data persistent across restarts (hint: you do!), you need to provide shared storage to every docker node.

Design

Why not GlusterFS?

I originally provided shared storage to my nodes using GlusterFS (see the next recipe for details), but found it difficult to deal with because:

GlusterFS requires (n) “bricks”, where (n) has to be a multiple of your replica count. I.e., if you want 2 copies of everything on shared storage (the minimum to provide redundancy), you must have either 2, 4, 6 (etc..) bricks. The HA swarm design calls for minimum of 3 nodes, and so under GlusterFS, my third node can’t participate in shared storage at all, unless I start doubling up on bricks-per-node (which then impacts redundancy)

GlusterFS turns out to be a giant PITA when you want to restore a failed node. There are at least 14 steps to follow to replace a brick.

I’m pretty sure I messed up the 14-step process above anyway. My replaced brick synced with my “original” brick, but produced errors when querying status via the CLI, and hogged 100% of 1 CPU on the replaced node. Inexperienced with GlusterFS, and unable to diagnose the fault, I switched to a Ceph cluster instead.

Why Ceph?

I’m more familiar with Ceph - I use it in the OpenStack designs I manage

Replacing a failed node is easy, provided you can put up with the I/O load of rebalancing OSDs after the replacement.

CentOS Atomic includes the ceph client in the OS, so while the Ceph OSD/Mon/MSD are running under containers, I can keep an eye (and later, automatically monitor) the status of Ceph from the base OS.

Ingredients

!!! summary “Ingredients” 3 x Virtual Machines (configured earlier), each with:

* [X] CentOS/Fedora Atomic
* [X] At least 1GB RAM
* [X] At least 20GB disk space (_but it'll be tight_)
* [X] Connectivity to each other within the same subnet, and on a low-latency link (_i.e., no WAN links_)
* [ ] A second disk dedicated to the Ceph OSD

Preparation

SELinux

Since our Ceph components will be containerized, we need to ensure the SELinux context on the base OS’s ceph files is set correctly:

mkdir /var/lib/ceph
chcon -Rt svirt_sandbox_file_t /etc/ceph
chcon -Rt svirt_sandbox_file_t /var/lib/ceph

Setup Monitors

Pick a node, and run the following to stand up the first Ceph mon. Be sure to replace the values for MON_IP and CEPH_PUBLIC_NETWORK to those specific to your deployment:

docker run -d --net=host \
--restart always \
-v /etc/ceph:/etc/ceph \
-v /var/lib/ceph/:/var/lib/ceph/ \
-e MON_IP=192.168.31.11 \
-e CEPH_PUBLIC_NETWORK=192.168.31.0/24 \
--name="ceph-mon" \
ceph/daemon mon

Now copy the contents of /etc/ceph on this first node to the remaining nodes, and then run the docker command above (customizing MON_IP as you go) on each remaining node. You’ll end up with a cluster with 3 monitors (odd number is required for quorum, same as Docker Swarm), and no OSDs (yet)

Setup Managers

Since Ceph v12 (“Luminous”), some of the non-realtime cluster management responsibilities are delegated to a “manager”. Run the following on every node - only one node will be active, the others will be in standby:

docker run -d --net=host \
--privileged=true \
--pid=host \
-v /etc/ceph:/etc/ceph \
-v /var/lib/ceph/:/var/lib/ceph/ \
--name="ceph-mgr" \
--restart=always \
ceph/daemon mgr

Setup OSDs

Since we have a OSD-less mon-only cluster currently, prepare for OSD creation by dumping the auth credentials for the OSDs into the appropriate location on the base OS:

ceph auth get client.bootstrap-osd -o \
/var/lib/ceph/bootstrap-osd/ceph.keyring

On each node, you need a dedicated disk for the OSD. In the example below, I used /dev/vdd (the entire disk, no partitions) for the OSD.

Run the following command on every node:

docker run -d --net=host \
--privileged=true \
--pid=host \
-v /etc/ceph:/etc/ceph \
-v /var/lib/ceph/:/var/lib/ceph/ \
-v /dev/:/dev/ \
-e OSD_FORCE_ZAP=1
-e OSD_DEVICE=/dev/vdd \
-e OSD_TYPE=disk \
--name="ceph-osd" \
--restart=always \
ceph/daemon osd_ceph_disk

Watch the output by running docker logs ceph-osd -f, and confirm success.

!!! warning “Zapping the device” The Ceph OSD container will normally refuse to destroy a partition containing existing data, but above we are instructing ceph to zap (destroy) whatever is on the partition currently. Don’t run this against a device you care about, and if you’re unsure, omit the “OSD_FORCE_ZAP” variable

Setup MDSs

In order to mount our ceph pools as filesystems, we’ll need Ceph MDS(s). Run the following on each node:

docker run -d --net=host \
--name ceph-mds \
--restart always \
-v /var/lib/ceph/:/var/lib/ceph/ \
-v /etc/ceph:/etc/ceph \
-e CEPHFS_CREATE=1 \
-e CEPHFS_DATA_POOL_PG=256 \
-e CEPHFS_METADATA_POOL_PG=256 \
ceph/daemon mds

Apply tweaks

The ceph container seems to configure a pool default of 3 replicas (3 copies of each block are retained), which is one too many for our cluster (we are only protecting against the failure of a single node).

Run the following on any node to reduce the size of the pool to 2 replicas:

ceph osd pool set cephfs_data size 2
ceph osd pool set cephfs_metadata size 2

Disabled “scrubbing” (which can be IO-intensive, and is unnecessary on a VM) with:

ceph osd set noscrub
ceph osd set nodeep-scrub

Create credentials for swarm

In order to mount the ceph volume onto our base host, we need to provide cephx authentication credentials.

On one node, create a client for the docker swarm:

ceph auth get-or-create client.dockerswarm osd \
'allow rw' mon 'allow r' mds 'allow' > /etc/ceph/keyring.dockerswarm

Grab the secret associated with the new user (you’ll need this for the /etc/fstab entry below) by running:

ceph-authtool /etc/ceph/keyring.dockerswarm -p -n client.dockerswarm

Mount MDS volume

On each node, create a mountpoint for the data, by running mkdir /var/data, add an entry to fstab to ensure the volume is auto-mounted on boot, and ensure the volume is actually mounted if there’s a network / boot delay getting access to the gluster volume:

mkdir /var/data

MYHOST=`hostname -s`
echo -e "
# Mount cephfs volume \n
$MYHOST:6789:/      /var/data/      ceph      \
name=dockerswarm\
,secret=<YOUR SECRET HERE>\
,noatime,_netdev,context=system_u:object_r:svirt_sandbox_file_t:s0 \
0 2" >> /etc/fstab
mount -a

Install docker-volume plugin

Upstream bug for docker-latest reported at https://bugs.centos.org/view.php?id=13609

And the alpine fault: https://github.com/gliderlabs/docker-alpine/issues/317

Serving

After completing the above, you should have:

[X] Persistent storage available to every node
[X] Resiliency in the event of the failure of a single node

Chef’s Notes

Future enhancements to this recipe include:

Rather than pasting a secret key into /etc/fstab (which feels wrong), I’d prefer to be able to set “secretfile” in /etc/fstab (which just points ceph.mount to a file containing the secret), but under the current CentOS Atomic, we’re stuck with “secret”, per https://bugzilla.redhat.com/show_bug.cgi?id=1030402

This recipe was written with Ceph v11 “Jewel”. Ceph have subsequently releaesd v12 “Kraken”. I’ve updated the recipe for the addition of “Manager” daemons, but it should be noted that the only reader so far to attempt a Ceph install using CentOS Atomic and Ceph v12 had issues with OSDs, which lead him to move to Ubuntu 1604 instead.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Shared Storage (GlusterFS)

While Docker Swarm is great for keeping containers running (and restarting those that fail), it does nothing for persistent storage. This means if you actually want your containers to keep any data persistent across restarts (hint: you do!), you need to provide shared storage to every docker node.

Design

Why GlusterFS?

This GlusterFS recipe was my original design for shared storage, but I found it to be flawed, and I replaced it with a design which employs Ceph instead. This recipe is an alternate to the Ceph design, if you happen to prefer GlusterFS.

Ingredients

!!! summary “Ingredients” 3 x Virtual Machines (configured earlier), each with:

* [X] CentOS/Fedora Atomic
* [X] At least 1GB RAM
* [X] At least 20GB disk space (_but it'll be tight_)
* [X] Connectivity to each other within the same subnet, and on a low-latency link (_i.e., no WAN links_)
* [ ] A second disk, or adequate space on the primary disk for a dedicated data partition

Preparation

Create Gluster “bricks”

To build our Gluster volume, we need 2 out of the 3 VMs to provide one “brick”. The bricks will be used to create the replicated volume. Assuming a replica count of 2 (i.e., 2 copies of the data are kept in gluster), our total number of bricks must be divisible by our replica count. (I.e., you can’t have 3 bricks if you want 2 replicas. You can have 4 though - We have to have minimum 3 swarm manager nodes for fault-tolerance, but only 2 of those nodes need to run as gluster servers.)

On each host, run a variation following to create your bricks, adjusted for the path to your disk.

!!! note “The example below assumes /dev/vdb is dedicated to the gluster volume”

(
echo o # Create a new empty DOS partition table
echo n # Add a new partition
echo p # Primary partition
echo 1 # Partition number
echo   # First sector (Accept default: 1)
echo   # Last sector (Accept default: varies)
echo w # Write changes
) | sudo fdisk /dev/vdb

mkfs.xfs -i size=512 /dev/vdb1
mkdir -p /var/no-direct-write-here/brick1
echo '' >> /etc/fstab >> /etc/fstab
echo '# Mount /dev/vdb1 so that it can be used as a glusterfs volume' >> /etc/fstab
echo '/dev/vdb1 /var/no-direct-write-here/brick1 xfs defaults 1 2' >> /etc/fstab
mount -a && mount

!!! warning “Don’t provision all your LVM space” Atomic uses LVM to store docker data, and automatically grows Docker’s volumes as requried. If you commit all your free LVM space to your brick, you’ll quickly find (as I did) that docker will start to fail with error messages about insufficient space. If you’re going to slice off a portion of your LVM space in /dev/atomicos, make sure you leave enough space for Docker storage, where “enough” depends on how much you plan to pull images, make volumes, etc. I ate through 20GB very quickly doing development, so I ended up provisioning 50GB for atomic alone, with a separate volume for the brick.

Create glusterfs container

Atomic doesn’t include the Gluster server components. This means we’ll have to run glusterd from within a container, with privileged access to the host. Although convoluted, I’ve come to prefer this design since it once again makes the OS “disposable”, moving all the config into containers and code.

Run the following on each host:

docker run \
   -h glusterfs-server \
   -v /etc/glusterfs:/etc/glusterfs:z \
   -v /var/lib/glusterd:/var/lib/glusterd:z \
   -v /var/log/glusterfs:/var/log/glusterfs:z \
   -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
   -v /var/no-direct-write-here/brick1:/var/no-direct-write-here/brick1 \
   -d --privileged=true --net=host \
   --restart=always \
   --name="glusterfs-server" \
   gluster/gluster-centos

Create trusted pool

On a single node (doesn’t matter which), run docker exec -it glusterfs-server bash to launch a shell inside the container.

From the node, run gluster peer probe <other host>

Example output:

[root@glusterfs-server /]# gluster peer probe ds1
peer probe: success.
[root@glusterfs-server /]#

Run gluster peer status on both nodes to confirm that they’re properly connected to each other:

Example output:

[root@glusterfs-server /]# gluster peer status
Number of Peers: 1

Hostname: ds3
Uuid: 3e115ba9-6a4f-48dd-87d7-e843170ff499
State: Peer in Cluster (Connected)
[root@glusterfs-server /]#

Create gluster volume

Now we create a replicated volume out of our individual “bricks”.

Create the gluster volume by running

gluster volume create gv0 replica 2 \
 server1:/var/no-direct-write-here/brick1 \
 server2:/var/no-direct-write-here/brick1

Example output:

[root@glusterfs-server /]# gluster volume create gv0 replica 2 ds1:/var/no-direct-write-here/brick1/gv0  ds3:/var/no-direct-write-here/brick1/gv0
volume create: gv0: success: please start the volume to access data
[root@glusterfs-server /]#

Start the volume by running gluster volume start gv0

[root@glusterfs-server /]# gluster volume start gv0
volume start: gv0: success
[root@glusterfs-server /]#

The volume is only present on the host you’re shelled into though. To add the other hosts to the volume, run gluster peer probe <servername>. Don’t probe host from itself.

From one other host, run docker exec -it glusterfs-server bash to shell into the gluster-server container, and run gluster peer probe <original server name> to update the name of the host which started the volume.

Mount gluster volume

On the host (i.e., outside of the container - type exit if you’re still shelled in), create a mountpoint for the data, by running mkdir /var/data, add an entry to fstab to ensure the volume is auto-mounted on boot, and ensure the volume is actually mounted if there’s a network / boot delay getting access to the gluster volume:

mkdir /var/data
MYHOST=`hostname -s`
echo '' >> /etc/fstab >> /etc/fstab
echo '# Mount glusterfs volume' >> /etc/fstab
echo "$MYHOST:/gv0                /var/data      glusterfs       defaults,_netdev,context="system_u:object_r:svirt_sandbox_file_t:s0"  0  0"  >> /etc/fstab
mount -a

For some reason, my nodes won’t auto-mount this volume on boot. I even tried the trickery below, but they stubbornly refuse to automount.

echo -e "\n\n# Give GlusterFS 10s to start before \
mounting\nsleep 10s && mount -a" >> /etc/rc.local
systemctl enable rc-local.service

For non-gluster nodes, you’ll need to replace $MYHOST above with the name of one of the gluster hosts (I haven’t worked out how to make this fully HA yet)

Serving

After completing the above, you should have:

[X] Persistent storage available to every node
[X] Resiliency in the event of the failure of a single (gluster) node

Chef’s Notes

Future enhancements to this recipe include:

Migration of shared storage from GlusterFS to Ceph ()#2)

Correct the fact that volumes don’t automount on boot (#3)

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Keepalived

While having a self-healing, scalable docker swarm is great for availability and scalability, none of that is any good if nobody can connect to your cluster.

In order to provide seamless external access to clustered resources, regardless of which node they’re on and tolerant of node failure, you need to present a single IP to the world for external access.

Normally this is done using a HA loadbalancer, but since Docker Swarm aready provides the load-balancing capabilities (routing mesh), all we need for seamless HA is a virtual IP which will be provided by more than one docker node.

This is accomplished with the use of keepalived on at least two nodes.

Ingredients

Already deployed:
[X] At least 2 x CentOS/Fedora Atomic VMs
[X] low-latency link (i.e., no WAN links)

New:
[ ] 3 x IPv4 addresses (one for each node and one for the virtual IP)

Preparation

Enable IPVS module

On all nodes which will participate in keepalived, we need the “ip_vs” kernel module, in order to permit serivces to bind to non-local interface addresses.

Set this up once for both the primary and secondary nodes, by running:

echo "modprobe ip_vs" >> /etc/rc.local
modprobe ip_vs

Setup nodes

Assuming your IPs are as follows:

192.168.4.1 : Primary

192.168.4.2 : Secondary

192.168.4.3 : Virtual

Run the following on the primary

docker run -d --name keepalived --restart=always \
  --cap-add=NET_ADMIN --net=host \
  -e KEEPALIVED_UNICAST_PEERS="#PYTHON2BASH:['192.168.4.1', '192.168.4.2']" \
  -e KEEPALIVED_VIRTUAL_IPS=192.168.4.3 \
  -e KEEPALIVED_PRIORITY=200 \
  osixia/keepalived:1.3.5

And on the secondary:

docker run -d --name keepalived --restart=always \
  --cap-add=NET_ADMIN --net=host \
  -e KEEPALIVED_UNICAST_PEERS="#PYTHON2BASH:['192.168.4.1', '192.168.4.2']" \
  -e KEEPALIVED_VIRTUAL_IPS=192.168.4.3 \
  -e KEEPALIVED_PRIORITY=100 \
  osixia/keepalived:1.3.5

Serving

That’s it. Each node will talk to the other via unicast (no need to un-firewall multicast addresses), and the node with the highest priority gets to be the master. When ingress traffic arrives on the master node via the VIP, docker’s routing mesh will deliver it to the appropriate docker node.

Chef’s notes

Some hosting platforms (OpenStack, for one) won’t allow you to simply “claim” a virtual IP. Each node is only able to receive traffic targetted to its unique IP. In this case, keepalived is not the right solution, and a platform-specific load-balancing solution should be used. In OpenStack, this is Neutron’s “Load Balancer As A Service” (LBAAS) component. AWS and Azure would likely include similar protections.

More than 2 nodes can participate in keepalived. Simply ensure that each node has the appropriate priority set, and the node with the highest priority will become the master.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Docker Swarm Mode

For truly highly-available services with Docker containers, we need an orchestration system. Docker Swarm (as defined at 1.13) is the simplest way to achieve redundancy, such that a single docker host could be turned off, and none of our services will be interrupted.

Ingredients

3 x CentOS Atomic hosts (bare-metal or VMs). A reasonable minimum would be:

1 x vCPU

1GB repo_name

10GB HDD

Hosts must be within the same subnet, and connected on a low-latency link (i.e., no WAN links)

Preparation

Release the swarm!

Now, to launch my swarm:

docker swarm init

Yeah, that was it. Now I have a 1-node swarm.

[root@ds1 ~]# docker swarm init
Swarm initialized: current node (b54vls3wf8xztwfz79nlkivt8) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join \
    --token SWMTKN-1-2orjbzjzjvm1bbo736xxmxzwaf4rffxwi0tu3zopal4xk4mja0-bsud7xnvhv4cicwi7l6c9s6l0 \
    202.170.164.47:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

[root@ds1 ~]#

Run docker node ls to confirm that I have a 1-node swarm:

[root@ds1 ~]# docker node ls
ID                           HOSTNAME                STATUS  AVAILABILITY  MANAGER STATUS
b54vls3wf8xztwfz79nlkivt8 *  ds1.funkypenguin.co.nz  Ready   Active        Leader
[root@ds1 ~]#

Note that when I ran docker swarm init above, the CLI output gave me a command to run to join further nodes to my swarm. This would join the nodes as workers (as opposed to managers). Workers can easily be promoted to managers (and demoted again), but since we know that we want our other two nodes to be managers too, it’s simpler just to add them to the swarm as managers immediately.

On the first swarm node, generate the necessary token to join another manager by running docker swarm join-token manager:

[root@ds1 ~]# docker swarm join-token manager
To add a manager to this swarm, run the following command:

    docker swarm join \
    --token SWMTKN-1-2orjbzjzjvm1bbo736xxmxzwaf4rffxwi0tu3zopal4xk4mja0-cfm24bq2zvfkcwujwlp5zqxta \
    202.170.164.47:2377

[root@ds1 ~]#

Run the command provided on your second node to join it to the swarm as a manager. After adding the second node, the output of docker node ls (on either host) should reflect two nodes:

[root@ds2 davidy]# docker node ls
ID                           HOSTNAME                STATUS  AVAILABILITY  MANAGER STATUS
b54vls3wf8xztwfz79nlkivt8    ds1.funkypenguin.co.nz  Ready   Active        Leader
xmw49jt5a1j87a6ihul76gbgy *  ds2.funkypenguin.co.nz  Ready   Active        Reachable
[root@ds2 davidy]#

Repeat the process to add your third node.

Finally, docker node ls should reflect that you have 3 reachable manager nodes, one of whom is the “Leader”:

[root@ds3 ~]# docker node ls
ID                           HOSTNAME                      STATUS  AVAILABILITY  MANAGER STATUS
36b4twca7i3hkb7qr77i0pr9i    ds1.example.com  Ready   Active        Reachable
l14rfzazbmibh1p9wcoivkv1s *  ds3.example.com  Ready   Active        Reachable
tfsgxmu7q23nuo51wwa4ycpsj    ds2.example.com  Ready   Active        Leader
[root@ds3 ~]#

Setup automated cleanup

Docker swarm doesn’t do any cleanup of old images, so as you experiment with various stacks, and as updated containers are released upstream, you’ll soon find yourself loosing gigabytes of disk space to old, unused images.

To address this, we’ll run the “meltwater/docker-cleanup” container on all of our nodes. The container will clean up unused images after 30 minutes.

First, create docker-cleanup.env (mine is under /var/data/config/docker-cleanup), and exclude container images we know we want to keep:

KEEP_IMAGES=traefik,keepalived,docker-mailserver
DEBUG=1

Then create a docker-compose.yml as follows:

version: "3"

services:
  docker-cleanup:
    image: meltwater/docker-cleanup:latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/docker:/var/lib/docker
    networks:
      - internal
    deploy:
      mode: global
    env_file: /var/data/config/docker-cleanup/docker-cleanup.env

networks:
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.0.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Launch the cleanup stack by running docker stack deploy docker-cleanup -c <path-to-docker-compose.yml>

Setup automatic updates

If your swarm runs for a long time, you might find yourself running older container images, after newer versions have been released. If you’re the sort of geek who wants to live on the edge, configure shepherd to auto-update your container images regularly.

Create /var/data/config/shepherd/shepherd.env as follows:

# Don't auto-update Plex or Emby, I might be watching a movie! (Customize this for the containers you _don't_ want to auto-update)
BLACKLIST_SERVICES="plex_plex emby_emby"
# Run every 24 hours. I _really_ don't need new images more frequently than that!
SLEEP_TIME=1440

Then create /var/data/config/shepherd/shepherd.yml as follows:

version: "3"

services:
  shepherd-app:
    image: mazzolino/shepherd
    env_file : /var/data/config/shepherd/shepherd.env
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      - "traefik.enable=false"
    deploy:
      placement:
        constraints: [node.role == manager]

Launch shepherd by running docker stack deploy shepherd -c /var/data/config/shepherd/shepherd.yml, and then just forget about it, comfortable in the knowledge that every day, Shepherd will check that your images are the latest available, and if not, will destroy and recreate the container on the latest available image.

Tweaks

Add some handy bash auto-completion for docker. Without this, you’ll get annoyed that you can’t autocomplete docker stack deploy <blah> -c <blah.yml> commands.

cd /etc/bash_completion.d/
curl -O https://raw.githubusercontent.com/docker/cli/b75596e1e4d5295ac69b9934d1bd8aff691a0de8/contrib/completion/bash/docker

Install some useful bash aliases on each host

cd ~
curl -O https://raw.githubusercontent.com/funkypenguin/geek-cookbook/master/examples/scripts/gcb-aliases.sh
echo 'source ~/gcb-aliases.sh' >> ~/.bash_profile

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Traefik

The platforms we plan to run on our cloud are generally web-based, and each listening on their own unique TCP port. When a container in a swarm exposes a port, then connecting to any swarm member on that port will result in your request being forwarded to the appropriate host running the container. (Docker calls this the swarm “routing mesh”)

So we get a rudimentary load balancer built into swarm. We could stop there, just exposing a series of ports on our hosts, and making them HA using keepalived.

There are some gaps to this approach though:

No consideration is given to HTTPS. Implementation would have to be done manually, per-container.

No mechanism is provided for authentication outside of that which the container providers. We may not want to expose every interface on every container to the world, especially if we are playing with tools or containers whose quality and origin are unknown.

To deal with these gaps, we need a front-end load-balancer, and in this design, that role is provided by Traefik.

Ingredients

Docker swarm cluster with persistent shared storage

Preparation

Prepare the host

The traefik container is aware of the other docker containers in the swarm, because it has access to the docker socket at /var/run/docker.sock. This allows traefik to dynamically configure itself based on the labels found on containers in the swarm, which is hugely useful. To make this functionality work on our SELinux-enabled Atomic hosts, we need to add custom SELinux policy.

!!! tip The following is only necessary if you’re using SELinux!

Run the following to build and activate policy to permit containers to access docker.sock:

mkdir ~/dockersock
cd ~/dockersock
curl -O https://raw.githubusercontent.com/dpw/\
selinux-dockersock/master/Makefile
curl -O https://raw.githubusercontent.com/dpw/\
selinux-dockersock/master/dockersock.te
make && semodule -i dockersock.pp

Prepare traefik.toml

While it’s possible to configure traefik via docker command arguments, I prefer to create a config file (traefik.toml). This allows me to change traefik’s behaviour by simply changing the file, and keeps my docker config simple.

Create /var/data/traefik/, and then create traefik.toml inside it as follows:

checkNewVersion = true
defaultEntryPoints = ["http", "https"]

# This section enable LetsEncrypt automatic certificate generation / renewal
[acme]
email = "<your LetsEncrypt email address>"
storage = "acme.json" # or "traefik/acme/account" if using KV store
entryPoint = "https"
acmeLogging = true
onDemand = true
OnHostRule = true

[[acme.domains]]
  main = "<your primary domain>"

# Redirect all HTTP to HTTPS (why wouldn't you?)
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
      entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]

[web]
address = ":8080"
watch = true

[docker]
endpoint = "tcp://127.0.0.1:2375"
domain = "<your primary domain>"
watch = true
swarmmode = true

Prepare the docker service config

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Create /var/data/config/traefik/docker-compose.yml as follows:

version: "3"

services:
  traefik:
    image: traefik
    command: --web --docker --docker.swarmmode --docker.watch --docker.domain=funkypenguin.co.nz --logLevel=DEBUG
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
      - target: 8080
        published: 8080
        protocol: tcp
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /var/data/traefik/traefik.toml:/traefik.toml:ro
      - /var/data/traefik/acme.json:/acme.json
    labels:
      - "traefik.enable=false"
    networks:
      - public
    deploy:
      mode: global
      placement:
        constraints: [node.role == manager]
      restart_policy:
        condition: on-failure

networks:
  public:
    driver: overlay
    ipam:
      driver: default
      config:
      - subnet: 10.1.0.0/24

Docker won’t start an image with a bind-mount to a non-existent file, so prepare an empty acme.json (with the appropriate permissions) by running:

touch /var/data/traefik/acme.json
chmod 600 /var/data/traefik/acme.json

Traefik will populate acme.json itself when it runs, but it needs to exist before the container will start (Chicken, meet egg.)

Launch

Deploy traefik with docker stack deploy traefik -c /var/data/traefik/docker-compose.yml

Confirm traefik is running with docker stack ps traefik

Serving

You now have:

Frontend proxy which will dynamically configure itself for new backend containers

Automatic SSL support for all proxied resources

Chef’s Notes

Additional features I’d like to see in this recipe are:

Include documentation of oauth2_proxy container for protecting individual backends

Traefik webUI is available via HTTPS, protected with oauth_proxy

Pending a feature in docker-swarm to avoid NAT on routing-mesh-delivered traffic, update the design

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Create registry mirror

Although we now have shared storage for our persistent container data, our docker nodes don’t share any other docker data, such as container images. This results in an inefficiency - every node which participates in the swarm will, at some point, need the docker image for every container deployed in the swarm.

When dealing with large container (looking at you, GitLab!), this can result in several gigabytes of wasted bandwidth per-node, and long delays when restarting containers on an alternate node. (It also wastes disk space on each node, but we’ll get to that in the next section)

The solution is to run an official Docker registry container as a “pull-through” cache, or “registry mirror”. By using our persistent storage for the registry cache, we can ensure we have a single copy of all the containers we’ve pulled at least once. After the first pull, any subsequent pulls from our nodes will use the cached version from our registry mirror. As a result, services are available more quickly when restarting container nodes, and we can be more aggressive about cleaning up unused containers on our nodes (more later)

The registry mirror runs as a swarm stack, using a simple docker-compose.yml. Customize your mirror FQDN below, so that Traefik will generate the appropriate LetsEncrypt certificates for it, and make it available via HTTPS.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Create /var/data/config/registry/registry.yml as follows:

version: "3"

services:

  registry-mirror:
    image: registry:2
    networks:
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:<your mirror FQDN>
        - traefik.docker.network=traefik_public
        - traefik.port=5000
    ports:
      - 5000:5000
    volumes:
      - /var/data/registry/registry-mirror-data:/var/lib/registry
      - /var/data/registry/registry-mirror-config.yml:/etc/docker/registry/config.yml

networks:
  traefik_public:
    external: true

!!! note “Unencrypted registry” We create this registry without consideration for SSL, which will fail if we attempt to use the registry directly. However, we’re going to use the HTTPS-proxied version via Traefik, leveraging Traefik to manage the LetsEncrypt certificates required.

Create /var/data/registry/registry-mirror-config.yml as follows:

version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
  delete:
    enabled: true
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
proxy:
  remoteurl: https://registry-1.docker.io

Serving

Launch registry stack

Launch the registry stack by running docker stack deploy registry -c <path-to-docker-compose.yml>

Enable registry mirror and experimental features

To tell docker to use the registry mirror, and (while we’re here) in order to be able to watch the logs of any service from any manager node (an experimental feature in the current Atomic docker build), edit /etc/docker-latest/daemon.json on each node, and change from:

{
    "log-driver": "journald",
    "signature-verification": false
}

To:

{
    "log-driver": "journald",
    "signature-verification": false,
    "experimental": true,
    "registry-mirrors": ["https://<your registry mirror FQDN>"]
}

Then restart docker by running:

systemctl restart docker-latest

!!! tip "" Note the extra comma required after “false” above

Chef’s notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Chef’s Favorites (Docker)

The following recipes are the chef’s current favorites - these are recipes actively in use and updated by @funkypenguin hero: AutoPirate - A fully-featured recipe to automate finding, downloading, and organising your media 📺 🎥 🎵 📖

AutoPirate

Once the cutting edge of the “internet” (pre-world-wide-web and mosiac days), Usenet is now a murky, geeky alternative to torrents for file-sharing. However, it’s cool geeky, especially if you’re into having a fully automated media platform.

A good starter for the usenet scene is https://www.reddit.com/r/usenet/. Because it’s so damn complicated, a host of automated tools exist to automate the process of finding, downloading, and managing content. The tools included in this recipe are as follows:

[picture]

Autopirate Screenshot

This recipe presents a method to combine these tools into a single swarm deployment, and make them available securely.

Menu

Tools included in the AutoPirate stack are:

SABnzbd : downloads data from usenet servers based on .nzb definitions

NZBGet : downloads data from usenet servers based on .nzb definitions, but written in C++ and designed with performance in mind to achieve maximum download speed by using very little system resources (this is a popular alternative to SABnzbd)

RTorrent is a CLI-based torrent client, which when combined with ruTorrent becomes a powerful and fully browser-managed torrent client. (Yes, it’s not Usenet, but Sonarr/Radarr will let fulfill your watchlist using either Usenet or torrents, so it’s worth including)

NZBHydra : acts as a “meta-indexer”, so that your downloading tools (radarr, sonarr, etc) only need to be setup for a single indexes. Also produces interesting stats on indexers, which helps when evaluating which indexers are performing well.

NZBHydra2 : is a high-performance rewrite of the original NZBHydra, with extra features. While still in beta, this NZBHydra2 will eventually supercede NZBHydra

Sonarr : finds, downloads and manages TV shows

Radarr : finds, downloads and manages movies

Mylar : finds, downloads and manages comic books

Headphones : finds, downloads and manages music

Lazy Librarian : finds, downloads and manages ebooks

Ombi : provides an interface to request additions to a Plex/Emby library using the above tools

Jackett : Provides an local, caching, API-based interface to torrent trackers, simplifying the way your tools search for torrents.

Since this recipe is so long, and so many of the tools are optional to the final result (i.e., if you’re not interested in comics, you won’t want Mylar), I’ve described each individual tool on its own sub-recipe page (below), even though most of them are deployed very similarly.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Access to NZB indexers and Usenet servers

DNS entries configured for each of the NZB tools in this recipe that you want to use

Preparation

Setup data locations

We’ll need a unique directories for each tool in the stack, bind-mounted into our containers, so create them upfront, in /var/data/autopirate:

mkdir /var/data/autopirate
cd /var/data/autopirate
mkdir -p {lazylibrarian,mylar,ombi,sonarr,radarr,headphones,plexpy,nzbhydra,sabnzbd,nzbget,rtorrent,jackett}

Create a directory for the storage of your downloaded media, i.e., something like:

mkdir /var/data/media

Create a user to “own” the above directories, and note the uid and gid of the created user. You’ll need to specify the UID/GID in the environment variables passed to the container (in the example below, I used 4242 - twice the meaning of life).

Secure public access

What you’ll quickly notice about this recipe is that every web interface is protected by an OAuth proxy.

Why? Because these tools are developed by a handful of volunteer developers who are focused on adding features, not necessarily implementing robust security. Most users wouldn’t expose these tools directly to the internet, so the tools have rudimentary (if any) access control.

To mitigate the risk associated with public exposure of these tools (you’re on your smartphone and you want to add a movie to your watchlist, what do you do, hotshot?), in order to gain access to each tool you’ll first need to authenticate against your given OAuth provider.

This is tedious, but you only have to do it once. Each tool (Sonarr, Radarr, etc) to be protected by an OAuth proxy, requires unique configuration. I use github to provide my oauth, giving each tool a unique logo while I’m at it (make up your own random string for OAUTH2PROXYCOOKIE_SECRET)

For each tool, create /var/data/autopirate/.env, and set the following:

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=
PUID=4242
PGID=4242

Create at least /var/data/autopirate/authenticated-emails.txt, containing at least your own email address with your OAuth provider. If you wanted to grant access to a specific tool to other users, you’d need a unique authenticated-emails-.txt which included both normal email address as well as any addresses to be granted tool-specific access.

Setup components

Stack basics

Start with a swarm config file in docker-compose syntax, like this:

version: '3'

services:

And end with a stanza like this:

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.11.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Assemble the tools..

Now work your way through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

NZBHydra

NZBHydra2

Ombi

Jackett

End (launch the stack)

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

SABnzbd

Introduction

SABnzbd is the workhorse of the stack. It takes .nzb files as input (manually or from other autopirate stack tools), then connects to your chosen Usenet provider, downloads all the individual binaries referenced by the .nzb, and then tests/repairs/combines/uncompresses them all into the final result - media files.

[picture]

SABNZBD Screenshot

!!! tip “Sponsored Project” SABnzbd is one of my sponsored projects - a project I financially support on a regular basis because of its utility to me. It’s not sexy, but it’s consistent and reliable, and I enjoy the fruits of its labor near-daily.

Inclusion into AutoPirate

To include SABnzbd in your AutoPirate stack (The only reason you wouldn’t use SABnzbd, would be if you were using NZBGet instead), include the following in your autopirate.yml stack definition file:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

sabnzbd:
  image: linuxserver/sabnzbd:latest
  env_file : /var/data/config/autopirate/sabnzbd.env
  volumes:
   - /var/data/autopirate/sabnzbd:/config
   - /var/data/media:/media
  networks:
  - internal

sabnzbd_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/sabnzbd.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:sabnzbd.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://sabnzbd:8080
    -redirect-url=https://sabnzbd.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! warning “Important Note re hostname validation”

(**Updated 10 June 2018**) : In SABnzbd [2.3.3](https://sabnzbd.org/wiki/extra/hostname-check.html), hostname verification was added as a mandatory check. SABnzbd will refuse inbound connections which weren't addressed to its own (_initially, autodetected_) hostname. This presents a problem within Docker Swarm, where container hostnames are random and disposable.

You'll need to edit sabnzbd.ini (_only created after your first launch_), and **replace** the value in ```host_whitelist``` configuration (_it's comma-separated_) with the name of your service within the swarm definition, as well as your FQDN as accessed via traefik.

For example, mine simply reads ```host_whitelist = sabnzbd.funkypenguin.co.nz, sabnzbd```

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd (this page)

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

NZBGet

Introduction

NZBGet performs the same function as SABnzbd (downloading content from Usenet servers), but it’s lightweight and fast(er), written in C++ (as opposed to Python).

[picture]

NZBGet Screenshot

Inclusion into AutoPirate

To include NZBGet in your AutoPirate stack (The only reason you wouldn’t use NZBGet, would be if you were using SABnzbd instead), include the following in your autopirate.yml stack definition file:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

nzbget:
  image: linuxserver/nzbget
  env_file : /var/data/config/autopirate/nzbget.env
  volumes:
   - /var/data/autopirate/nzbget:/config
   - /var/data/media:/data
  networks:
  - internal

nzbget_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/nzbget.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:nzbget.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://nzbget:6789
    -redirect-url=https://nzbget.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! note NZBGet uses a 401 header to prompt for authentication. When you use OAuth2_proxy, this seems to break. Since we trust OAuth to authenticate us, we can just disable NZGet’s own authentication, by changing ControlPassword to null in nzbget.conf (i.e. ControlPassword=)

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet (this page)

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

RTorrent / ruTorrent

RTorrent is a popular CLI-based bittorrent client, and ruTorrent is a powerful web interface for rtorrent.

[picture]

Rtorrent Screenshot

Choose incoming port

When using a torrent client from behind NAT (which swarm, by nature, is), you typically need to set a static port for inbound torrent communications. In the example below, I’ve set the port to 36258. You’ll need to configure /var/data/autopirate/rtorrent/rtorrent/rtorrent.rc with the equivalent port.

Inclusion into AutoPirate

To include ruTorrent in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

rtorrent:
  image: linuxserver/rutorrent
  env_file : /var/data/config/autopirate/rtorrent.env
  ports:
   - 36258:36258
  volumes:
   - /var/data/media/:/media
   - /var/data/autopirate/rtorrent:/config
  networks:
  - internal

rtorrent_proxy:
  image: skippy/oauth2_proxy
  env_file : /var/data/config/autopirate/rtorrent.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:rtorrent.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://rtorrent:80
    -redirect-url=https://rtorrent.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent (this page)

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

Sonarr

Sonarr is a tool for finding, downloading and managing your TV series.

[picture]

Sonarr Screenshot

!!! tip “Sponsored Project” Sonarr is one of my sponsored projects - a project I financially support on a regular basis because of its utility to me. I forget it’s there until I (reliably) receive an email with new and exciting updates 😁

Inclusion into AutoPirate

To include Sonarr in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

sonarr:
  image: linuxserver/sonarr:latest
  env_file : /var/data/config/autopirate/sonarr.env
  volumes:
   - /var/data/autopirate/sonarr:/config
   - /var/data/media:/media
  networks:
  - internal

sonarr_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/sonarr.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:sonarr.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://sonarr:8989
    -redirect-url=https://sonarr.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr (this page)

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

Radarr

Radarr is a tool for finding, downloading and managing movies. Features include:

Adding new movies with lots of information, such as trailers, ratings, etc.

Can watch for better quality of the movies you have and do an automatic upgrade. eg. from DVD to Blu-Ray

Automatic failed download handling will try another release if one fails

Manual search so you can pick any release or to see why a release was not downloaded automatically

Full integration with SABnzbd and NZBGet

Automatically searching for releases as well as RSS Sync

Automatically importing downloaded movies

Recognizing Special Editions, Director’s Cut, etc.

Identifying releases with hardcoded subs

Importing movies from various online sources, such as IMDb Watchlists (A complete list can be found here)

Full integration with Kodi, Plex (notification, library update)

And a beautiful UI

Importing Metadata such as trailers or subtitles

[picture]

Radarr Screenshot

!!! tip “Sponsored Project” Sonarr is one of my sponsored projects - a project I financially support on a regular basis because of its utility to me. I forget it’s there until I (reliably) receive an email with new and exciting updates 😁

Inclusion into AutoPirate

To include Radarr in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

radarr:
  image: linuxserver/radarr:latest
  env_file : /var/data/config/autopirate/radarr.env
  volumes:
   - /var/data/autopirate/radarr:/config
   - /var/data/media:/media
  networks:
  - internal

radarr_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/radarr.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:radarr.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://radarr:7878
    -redirect-url=https://radarr.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr (this page)

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the autopirate “uber-recipe”, but has been split into its own page to reduce complexity.

Mylar

Mylar is a tool for downloading and managing digital comic books.

[picture]

Mylar Screenshot

Inclusion into AutoPirate

To include Mylar in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

mylar:
  image: linuxserver/mylar:latest
  env_file : /var/data/config/autopirate/mylar.env
  volumes:
   - /var/data/autopirate/mylar:/config
   - /var/data/media:/media
  networks:
  - internal

mylar_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/mylar.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:mylar.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://mylar:8090
    -redirect-url=https://mylar.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar (this page)

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the autopirate “uber-recipe”, but has been split into its own page to reduce complexity.

LazyLibrarian

LazyLibrarian is a tool to follow authors and grab metadata for all your digital reading needs. It uses a combination of Goodreads Librarything and optionally GoogleBooks as sources for author info and book info. Features include:

Find authors and add them to the database

List all books of an author and mark ebooks or audiobooks as ‘wanted’.

When processing the downloaded books it will save a cover picture (if available) and save all metadata into metadata.opf next to the bookfile (calibre compatible format)

AutoAdd feature for book management tools like Calibre which must have books in flattened directory structure, or use calibre to import your books into an existing calibre library

LazyLibrarian can also be used to search for and download magazines, and monitor for new issues

[picture]

Lazy Librarian Screenshot

Inclusion into AutoPirate

To include LazyLibrarian in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

lazylibrarian:
  image: linuxserver/lazylibrarian:latest
  env_file : /var/data/config/autopirate/lazylibrarian.env
  volumes:
   - /var/data/autopirate/lazylibrarian:/config
   - /var/data/media:/media
  networks:
  - internal

lazylibrarian_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/lazylibrarian.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:lazylibrarian.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://lazylibrarian:5299
    -redirect-url=https://lazylibrarian.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

calibre-server:
  image: regueiro/calibre-server
  volumes:
   - /var/data/media/Ebooks/calibre/:/opt/calibre/library
  networks:
  - internal

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian (this page)

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

The calibre-server container co-exists within the Lazy Librarian (LL) containers so that LL can automatically add a book to Calibre using the calibre-server interface. The calibre library can then be properly viewed using the calibre-web recipe.

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: AutoPirate - A fully-featured recipe to automate finding, downloading, and organising your media 📺 🎥 🎵 📖

!!! warning This is not a complete recipe - it’s a component of the autopirate “uber-recipe”, but has been split into its own page to reduce complexity.

Headphones

Headphones is an automated music downloader for NZB and Torrent, written in Python. It supports SABnzbd, NZBget, Transmission, µTorrent, Deluge and Blackhole.

[picture]

Headphones Screenshot

Inclusion into AutoPirate

To include Headphones in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

headphones:
  image: linuxserver/headphones:latest
  env_file : /var/data/config/autopirate/headphones.env
  volumes:
   - /var/data/autopirate/headphones:/config
   - /var/data/media:/media
  networks:
  - internal

headphones_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/headphones.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:headphones.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://headphones:8181
    -redirect-url=https://headphones.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones (this page)

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: AutoPirate - A fully-featured recipe to automate finding, downloading, and organising your media 📺 🎥 🎵 📖

!!! warning This is not a complete recipe - it’s a component of the autopirate “uber-recipe”, but has been split into its own page to reduce complexity.

Lidarr

Lidarr is an automated music downloader for NZB and Torrent. It performs the same function as Headphones, but is written using the same(ish) codebase as Radarr and Sonarr. It’s blazingly fast, and includes beautiful album/artist art. Lidarr supports SABnzbd, NZBGet, Transmission, µTorrent, Deluge and Blackhole (just like Sonarr / Radarr)

[picture]

Lidarr Screenshot

Inclusion into AutoPirate

To include Lidarr in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

lidarr:
  image: linuxserver/lidarr:latest
  env_file : /var/data/config/autopirate/lidarr.env
  volumes:
   - /var/data/autopirate/lidarr:/config
   - /var/data/media:/media
  networks:
  - internal

lidarr_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/lidarr.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:lidarr.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://lidarr:8181
    -redirect-url=https://lidarr.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr (this page)

NZBHydra

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

The addition of the Lidarr recipe was contributed by our very own @gpulido in Discord (http://chat.funkypenguin.co.nz) - Thanks Gabriel!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

NZBHydra

NZBHydra is a meta search for NZB indexers. It provides easy access to a number of raw and newznab based indexers. You can search all your indexers from one place and use it as indexer source for tools like Sonarr or CouchPotato. Features include:

Search by IMDB, TMDB, TVDB, TVRage and TVMaze ID (including season and episode) and filter by age and size. If an ID is not supported by an indexer it is attempted to be converted (e.g. TMDB to IMDB)

Query generation, meaning when you search for a movie using e.g. an IMDB ID a query will be generated for raw indexers. Searching for a series season 1 episode 2 will also generate queries for raw indexers, like s01e02 and 1x02

Grouping of results with the same title and of duplicate results, accounting for result posting time, size, group and poster. By default only one of the duplicates is shown. You can provide an indexer score to influence which one that might be

Compatible with Sonarr, CP, NZB 360, SickBeard, Mylar and Lazy Librarian (and others)

Statistics on indexers (average response time, share of results, access errors), searches and downloads per time of day and day of week, NZB download history and search history (both via internal GUI and API)

[picture]

NZBHydra Screenshot

Inclusion into AutoPirate

To include NZBHydra in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

nzbhydra:
  image: linuxserver/hydra:latest
  env_file : /var/data/config/autopirate/nzbhydra.env
  volumes:
   - /var/data/autopirate/nzbhydra:/config
  networks:
  - internal

nzbhydra_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/nzbhydra.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:nzbhydra.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://nzbhydra:5075
    -redirect-url=https://nzbhydra.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra (this page)

NZBHydra2

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

NZBHydra 2

NZBHydra 2 is a meta search for NZB indexers. It provides easy access to a number of raw and newznab based indexers. You can search all your indexers from one place and use it as an indexer source for tools like Sonarr, Radarr or CouchPotato.

!!! note NZBHydra 2 is a complete rewrite of NZBHydra (1). It’s currently in Beta. It works mostly fine but some functions might not be completely done and incompatibilities with some tools might still exist. You might want to run both in parallel for migration / testing purposes, but ultimately you’ll probably want to switch over to NZBHydra 2 exclusively.

[picture]

NZBHydra Screenshot

Features include:

Searches Anizb, BinSearch, NZBIndex and any newznab compatible indexers. Merges all results, filters them by a number of configurable restrictions, recognizes duplicates and returns them all in one place

Add results to NZBGet or SABnzbd

Support for all relevant media IDs (IMDB, TMDB, TVDB, TVRage, TVMaze) and conversion between them

Query generation, meaning a query will be generated if only a media ID is provided in the search and the indexer doesn’t support the ID or if no results were found

Compatible with Sonarr, Radarr, NZBGet, SABnzbd, nzb360, CouchPotato, Mylar, Lazy Librarian, Sick Beard, Jackett/Cardigann, Watcher, etc.

Search and download history and extensive stats. E.g. indexer response times, download shares, NZB age, etc.

Authentication and multi-user support

Automatic update of NZB download status by querying configured downloaders

RSS support with configurable cache times

Torrent support (Although I prefer Jackett for this):

For GUI searches, allowing you to download torrents to a blackhole folder

A separate Torznab compatible endpoint for API requests, allowing you to merge multiple trackers

Extensive configurability

Migration of database and settings from v1

Inclusion into AutoPirate

To include NZBHydra2 in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

nzbhydra2:
  image: linuxserver/hydra2:latest
  env_file : /var/data/config/autopirate/nzbhydra2.env
  volumes:
   - /var/data/autopirate/nzbhydra2:/config
  networks:
  - internal

nzbhydra2_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/nzbhydra2.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:nzbhydra2.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://nzbhydra2:5076
    -redirect-url=https://nzbhydra2.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2 (this page)

Ombi

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra2, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Note that NZBHydra2 can co-exist with NZBHydra (1), but if you want your tools (Sonarr, Radarr, etc) to use NZBHydra2, you’ll need to change both the target hostname (to “hydra2”) and the target port (to 5076).

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

Ombi

Ombi is a useful addition to the autopirate stack. Features include:

Lets users request Movies and TV Shows (whether it being the entire series, an entire season, or even single episodes.)

Easily manage your requests User management system (supports plex.tv, Emby and local accounts)

A landing page that will give you the availability of your Plex/Emby server and also add custom notification text to inform your users of downtime.

Allows your users to get custom notifications!

Will show if the request is already on plex or even if it’s already monitored. Automatically updates the status of requests when they are available on Plex/Emby

[picture]

Ombi Screenshot

Inclusion into AutoPirate

To include Ombi in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

ombi:
  image: linuxserver/ombi:latest
  env_file : /var/data/config/autopirate/ombi.env
  volumes:
   - /var/data/autopirate/ombi:/config
  networks:
  - internal

ombi_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/ombi.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:ombi.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://ombi:3579
    -redirect-url=https://ombi.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi (this page)

Jackett

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the autopirate “uber-recipe”, but has been split into its own page to reduce complexity.

Jackett

Jackett works as a proxy server: it translates queries from apps (Sonarr, Radarr, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software.

This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping & translation logic - removing the burden from other apps.

[picture]

Jackett Screenshot

Inclusion into AutoPirate

To include Jackett in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

jackett:
  image: linuxserver/jackett:latest
  env_file : /var/data/config/autopirate/jackett.env
  volumes:
   - /var/data/autopirate/jackett:/config
  networks:
  - internal

jackett_proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/config/autopirate/jackett.env
  networks:
    - internal
    - traefik_public
  deploy:
    labels:
      - traefik.frontend.rule=Host:jackett.example.com
      - traefik.docker.network=traefik_public
      - traefik.port=4180
  volumes:
    - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://jackett:9117
    -redirect-url=https://jackett.example.com
    -http-address=http://0.0.0.0:4180
    -email-domain=example.com
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett (this page)

Heimdall

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the autopirate “uber-recipe”, but has been split into its own page to reduce complexity.

Heimdall

Heimdall Application Dashboard is a dashboard for all your web applications. It doesn’t need to be limited to applications though, you can add links to anything you like.

Heimdall is an elegant solution to organise all your web applications. It’s dedicated to this purpose so you won’t lose your links in a sea of bookmarks.

Heimdall provides a single URL to manage access to all of your autopirate tools, and includes “enhanced” (i.e., display stats within Heimdall without launching the app) access to NZBGet, SABnzbd, and friends.

[picture]

Heimdall Screenshot

Inclusion into AutoPirate

To include Heimdall in your AutoPirate stack, include the following in your autopirate.yml stack definition file:

  heimdall:
    image: linuxserver/heimdall:latest
    env_file: /var/data/config/autopirate/heimdall.env
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/data/heimdall:/config
    networks:
      - internal

  heimdall_proxy:
    image: funkypenguin/oauth2_proxy:latest
    env_file : /var/data/config/autopirate/heimdall.env
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:heimdall.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/data/config/autopirate/authenticated-emails.txt:/authenticated-emails.txt
    command: |
      -cookie-secure=false
      -upstream=http://heimdall:80
      -redirect-url=https://heimdall.example.com
      -http-address=http://0.0.0.0:4180
      -email-domain=example.com
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

Assemble more tools..

Continue through the list of tools below, adding whichever tools your want to use, and finishing with the end section:

SABnzbd

NZBGet

RTorrent

Sonarr

Radarr

Mylar

Lazy Librarian

Headphones

Lidarr

NZBHydra

NZBHydra2

Ombi

Jackett

Heimdall (this page)

End (launch the stack)

Chef’s Notes 📓

In many cases, tools will integrate with each other. I.e., Radarr needs to talk to SABnzbd and NZBHydra, Ombi needs to talk to Radarr, etc. Since each tool runs within the stack under its own name, just refer to each tool by name (i.e. “radarr”), and docker swarm will resolve the name to the appropriate container. You can identify the tool-specific port by looking at the docker-compose service definition.

The inclusion of Heimdall was due to the efforts of @gkoerk in our Discord server. Thanks gkoerk!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s the conclusion to the AutoPirate “uber-recipe”, but has been split into its own page to reduce complexity.

Launch Autopirate stack

Launch the AutoPirate stack by running docker stack deploy autopirate -c <path -to-docker-compose.yml>

Confirm the container status by running “docker stack ps autopirate”, and wait for all containers to enter the “Running” state.

Log into each of your new tools at its respective HTTPS URL. You’ll be prompted to authenticate against your OAuth provider, and upon success, redirected to the tool’s UI.

Chef’s Notes 📓

This is a complex stack. Sing out in the comments if you found a flaw or need a hand :)

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Duplicity - A boring recipe to backup your exciting stuff. Boring is good.

Duplicity

Intro

[picture]

Duplicity Screenshot

Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.

So what does this mean for our stack? It means we can leverage Duplicity to backup all our data-at-rest to a wide variety of cloud providers, including, but not limited to:

acd_cli

Amazon S3

Backblaze B2

DropBox

ftp

Google Docs

Google Drive

Microsoft Azure

Microsoft Onedrive

Rackspace Cloudfiles

rsync

ssh/scp

SwiftStack

Ingredients

Docker swarm cluster with persistent shared storage

Credentials for one of the Duplicity’s supported upload destinations

Preparation

Setup data locations

We’ll need a folder to store a docker-compose .yml file, and an associated .env file. If you’re following my filesystem layout, create /var/data/config/duplicity (for the config), and /var/data/duplicity (for the metadata) as follows:

mkdir /var/data/config/duplicity
mkdir /var/data/duplicity
cd /var/data/config/duplicity

(Optional) Create Google Cloud Storage bucket

I didn’t already have an archival/backup provider, so I chose Google Cloud “cloud” storage for the low price-point - 0.7 cents per GB/month (Plus you start with $300 credit even when signing up for the free tier). You can use any destination supported by Duplicity’s URL scheme though, just make sure you specify the necessary environment variables.

Sign up, create an empty project, enable billing, and create a bucket. Give your bucket a unique name, example “jack-and-jills-bucket” (it’s unique across the entire Google Cloud)

Under “Storage” section > “Settings” > “Interoperability” tab > click “Enable interoperable access” and then “Create a new key” button and note both Access Key and Secret.

Prepare environment

Generate a random passphrase to use to encrypt your data. Save this somewhere safe, without it you won’t be able to restore!

Seriously, save. it. somewhere. safe.

Create duplicity.env, and populate with the following variables

SRC=/var/data/
DST=gs://jack-and-jills-bucket/yes-you-can-have-subdirectories
TMPDIR=/tmp
GS_ACCESS_KEY_ID=<YOUR GS ACCESS KEY>
GS_SECRET_ACCESS_KEY=<YOUR GS SECRET ACCESS KEY>

OPTIONS=--allow-source-mismatch --exclude /var/data/runtime --exclude /var/data/registry --exclude /var/data/duplicity --archive-dir=/archive
PASSPHRASE=<YOUR CHOSEN PASSPHRASE>

!!! note See the data layout reference for an explanation of the included/excluded paths above.

Run a test backup

Before we launch the automated daily backups, let’s run a test backup, as follows:

docker run --env-file duplicity.env -it --rm -v \
/var/data:/var/data:ro -v /var/data/duplicity/tmp:/tmp -v \
/var/data/duplicity/archive:/archive tecnativa/duplicity \
/etc/periodic/daily/jobrunner

You should see some activity, with a summary of bytes transferred at the end.

Run a test restore

Repeat after me: “If you don’t verify your backup, it’s not a backup”.

!!! warning Depending on what tier of storage you chose from your provider (i.e., Google Coldline, or Amazon S3), you may be charged for downloading data.

Run a variation of the following to confirm a file you expect to be backed up, is backed up. (I used traefik.yml from the traefik recipie, since this is likely to exist for every reader).

docker run --env-file duplicity.env -it --rm \
-v /var/data:/var/data:ro \
-v /var/data/duplicity/tmp:/tmp \
-v /var/data/duplicity/archive:/archive tecnativa/duplicity \
duplicity list-current-files \
\$DST | grep traefik.yml

Once you’ve identified a file to test-restore, use a variation of the following to restore it to /tmp (from the perspective of the container - it’s actually /var/data/duplicity/tmp)

docker run --env-file duplicity.env -it --rm \
-v /var/data:/var/data:ro \
-v /var/data/duplicity/tmp:/tmp \
-v /var/data/duplicity/archive:/archive \
tecnativa/duplicity duplicity restore \
--file-to-restore config/traefik/traefik.yml \
\$DST /tmp/traefik-restored.yml

Examine the contents of /var/data/duplicity/tmp/traefik-restored.yml to confirm it contains valid data.

Setup Docker Swarm

Now that we have confidence in our backup/restore process, let’s automate it by creating a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3"

services:
    backup:
        image: tecnativa/duplicity
        env_file: /var/data/config/duplicity/duplicity.env
        networks:
          - internal
        volumes:
          - /etc/localtime:/etc/localtime:ro
          - /var/data:/var/data:ro
          - /var/data/duplicity/tmp:/tmp
          - /var/data/duplicity/archive:/archive

networks:
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.10.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Duplicity stack

Launch Duplicity stack by running docker stack deploy duplicity -c <path -to-docker-compose.yml>

Nothing will happen. Very boring. But when the cron script fires (daily), duplicity will do its thing, and backup everything in /var/data to your cloud destination.

Chef’s Notes

Automatic backup can still fail if nobody checks that it’s running successfully. I’ll be working on an upcoming recipe to monitor the elements of the stack, including the success/failure of duplicity jobs.

The container provides the facility to specify an SMTP host and port, but not credentials, which makes it close to useless. As a result, I’ve left SMTP out of this recipe. To enable email notifications (if your SMTP server doesn’t require auth), add SMTP_HOST, SMTP_PORT, EMAIL_FROM and EMAIL_TO variables to duplicity.env

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Real heroes backup their 💾

Elkar Backup

Don’t be like Cameron. Backup your stuff.

!!! important Ongoing development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

ElkarBackup is a free open-source backup solution based on RSync/RSnapshot. It’s basically a web wrapper around rsync/rsnapshot, which means that your backups are just files on a filesystem, utilising hardlinks for tracking incremental changes. I find this result more reassuring than a blob of compressed, (encrypted?) data that more sophisticated backup solutions would produce for you.

[picture]

ElkarBackup Screenshot

Details

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/elkarbackup:

mkdir -p /var/data/elkarbackup/{backups,uploads,sshkeys,database-dump}
mkdir -p /var/data/runtime/elkarbackup/db
mkdir -p /var/data/config/elkarbackup

Prepare environment

Create /var/data/config/elkarbackup/elkarbackup.env, and populate with the following variables

SYMFONY__DATABASE__PASSWORD=password
EB_CRON=enabled
TZ='Etc/UTC'

#SMTP - Populate these if you want email notifications
#SYMFONY__MAILER__HOST=
#SYMFONY__MAILER__USER=
#SYMFONY__MAILER__PASSWORD=
#SYMFONY__MAILER__FROM=

# For mysql
MYSQL_ROOT_PASSWORD=password

#oauth2_proxy
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

Create /var/data/config/elkarbackup/elkarbackup-db-backup.env, and populate with the following, to setup the nightly database dump.

!!! note Running a daily database dump might be considered overkill, since ElkarBackup can be configured to backup its own database. However, making my own backup keeps the operation of this stack consistent with other stacks which employ MariaDB.

Also, did you ever hear about the guy who said "_I wish I had fewer backups"?

No, me either :shrug:

# For database backup (keep 7 days daily backups)
MYSQL_PWD=<same as SYMFONY__DATABASE__PASSWORD above>
MYSQL_USER=root
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3"

services:
  db:
    image: mariadb:10.4
    env_file: /var/data/config/elkarbackup/elkarbackup.env
    networks:
      - internal
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/data/runtime/elkarbackup/db:/var/lib/mysql

  db-backup:
    image: mariadb:10.4
    env_file: /var/data/config/elkarbackup/elkarbackup-db-backup.env
    volumes:
      - /var/data/elkarbackup/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        mysqldump -h db --all-databases | gzip -c > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.sql.gz
        (ls -t /dump/dump*.sql.gz|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.sql.gz)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - internal

  app:
    image: elkarbackup/elkarbackup
    env_file: /var/data/config/elkarbackup/elkarbackup.env
    networks:
      - internal
    volumes:
       - /etc/localtime:/etc/localtime:ro
       - /var/data/:/var/data
       - /var/data/elkarbackup/backups:/app/backups
       - /var/data/elkarbackup/uploads:/app/uploads
       - /var/data/elkarbackup/sshkeys:/app/.ssh

   proxy:
     image: funkypenguin/oauth2_proxy
     env_file: /var/data/config/elkarbackup/elkarbackup.env
     networks:
       - traefik_public
       - internal
     deploy:
       labels:
         - traefik.frontend.rule=Host:elkarbackup.example.com
         - traefik.port=4180
     volumes:
       - /var/data/config/traefik/authenticated-emails.txt:/authenticated-emails.txt
     command: |
       -cookie-secure=false
       -upstream=http://app:80
       -redirect-url=https://elkarbackup.example.com
       -http-address=http://0.0.0.0:4180
       -email-domain=example.com
       -provider=github
       -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.36.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch ElkarBackup stack

Launch the ElkarBackup stack by running docker stack deploy elkarbackup -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user “root” and the password default password “root”:

[picture]

ElkarBackup Login Screen

First thing you do, change your password, using the gear icon, and “Change Password” link:

[picture]

ElkarBackup Login Screen

Have a read of the Elkarbackup Docs - they introduce the concept of clients (hosts containing data to be backed up), jobs (what data gets backed up), policies (when is data backed up and how long is it kept).

At the very least, you want to setup a client called “localhost” with an empty path (i.e., the job path will be accessed locally, without SSH), and then add a job to this client to backup /var/data, excluding /var/data/runtime and /var/data/elkarbackup/backup (unless you like “backup-ception”)

Copying your backup data offsite

From the WebUI, you can download a script intended to be executed on a remote host, to backup your backup data to an offsite location. This is a Good Idea(tm), but needs some massaging for a Docker swarm deployment.

Here’s a variation to the standard script, which I’ve employed:

#!/bin/bash

REPOSITORY=/var/data/elkarbackup/backups
SERVER=<target host member of docker swarm>
SERVER_USER=elkarbackup
UPLOADS=/var/data/elkarbackup/uploads
TARGET=/srv/backup/elkarbackup

echo "Starting backup..."
echo "Date: " `date "+%Y-%m-%d (%H:%M)"`

ssh "$SERVER_USER@$SERVER" "cd '$REPOSITORY'; find . -maxdepth 2 -mindepth 2" | sed s/^..// | while read jobId
do
    echo Backing up job $jobId
    mkdir -p $TARGET/$jobId 2>/dev/null
    rsync -aH --delete "$SERVER_USER@$SERVER:$REPOSITORY/$jobId/" $TARGET/$jobId
done

echo Backing up uploads
rsync -aH --delete "$SERVER_USER@$SERVER":"$UPLOADS/" $TARGET/uploads

USED=`df -h . | awk 'NR==2 { print $3 }'`
USE=`df -h . | awk 'NR==2 { print $5 }'`
AVAILABLE=`df -h . | awk 'NR==2 { print $4 }'`

echo "Backup finished succesfully!"
echo "Date: " `date "+%Y-%m-%d (%H:%M)"`
echo ""
echo "**** INFO ****"
echo "Used disk space: $USED ($USE)"
echo "Available disk space: $AVAILABLE"
echo ""

!!! note You’ll note that I don’t use the script to create a mysql dump (since Elkar is running within a container anyway), rather I just rely on the database dump which is made nightly into /var/data/elkarbackup/database-dump/

Restoring data

Repeat after me : “It’s not a backup unless you’ve tested a restore”

!!! note I had some difficulty making restoring work well in the webUI. My attempts to “Restore to client” failed with an SSH error about “localhost” not found. I was able to download the backup from my web browser, so I considered it a successful restore, since I can retrieve the backed-up data either from the webUI or from the filesystem directly.

To restore files form a job, click on the “Restore” button in the WebUI, while on the Jobs tab:

[picture]

ElkarBackup Login Screen

This takes you to a list of backup names and file paths. You can choose to download the entire contents of the backup from your browser as a .tar.gz, or to restore the backup to the client. If you click on the name of the backup, you can also drill down into the file structure, choosing to restore a single file or directory.

!!! important Ongoing development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

Chef’s Notes

If you wanted to expose the ElkarBackup UI directly, you could remove the oauth2_proxy from the design, and move the traefik_public-related labels directly to the app service. You’d also need to add the traefik_public network to the app service.

The original inclusion of ElkarBackup was due to the efforts of @gpulido in our Discord server. Thanks Gabriel!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Emby

Emby (think “M.B.” or “Media Browser”) is best described as “like Plex but different” 😁 - It’s a bit geekier and less polished than Plex, but it allows for more flexibility and customization.

[picture]

Emby Screenshot

I’ve started experimenting with Emby as an alternative to Plex, because of the advanced parental controls it offers. Based on my experimentation thus far, I have a “kid-safe” profile which automatically logs in, and only displays kid-safe content, based on ratings.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need a location to store Emby’s library data, config files, logs and temporary transcoding space, so create /var/data/emby, and make sure it’s owned by the user and group who also own your media data.

mkdir /var/data/emby

Prepare environment

Create emby.env, and populate with PUID/GUID for the user who owns the /var/data/emby directory (above) and your actual media content (in this example, the media content is at /srv/data)

PUID=
GUID=

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3.0"

services:
  emby:
    image: emby/emby-server
    env_file: /var/data/config/emby/emby.env
    volumes:
      - /var/data/emby/emby:/config
      - /srv/data/:/data
    deploy:
      labels:
        - traefik.frontend.rule=Host:emby.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=8096
    networks:
        - traefik_public
        - internal
    ports:
      - 8096:8096

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.17.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Emby stack

Launch the stack by running docker stack deploy emby -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, and complete the wizard-based setup to complete deploying your Emby.

Chef’s Notes

I didn’t use an oauth2_proxy for this stack, because it would interfere with mobile client support.

Got an NVIDIA GPU? See this blog post re how to use your GPU to transcode your media!

We don’t bother exposing the HTTPS port for Emby, since Traefik is doing the SSL termination for us already.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Home Assistant

Home Assistant is a home automation platform written in Python, with extensive support for 3rd-party home-automation platforms including Xaomi, Phillips Hue, and a bazillion others.

[picture]

Home Assistant Screenshot

This recipie combines the extensibility of Home Assistant with the flexibility of InfluxDB (for time series data store) and Grafana (for beautiful visualisation of that data).

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/homeassistant:

mkdir /var/data/homeassistant
cd /var/data/homeassistant
mkdir -p {homeassistant,grafana,influxdb-backup}

Now create a directory for the influxdb realtime data:

mkdir /var/data/runtime/homeassistant/influxdb

Prepare environment

Create /var/data/config/homeassistant/grafana.env, and populate with the following - this is to enable grafana to work with oauth2_proxy without requiring an additional level of authentication:

GF_AUTH_BASIC_ENABLED=false

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3"

services:
    influxdb:
      image: influxdb
      networks:
        - internal
      volumes:
        - /var/data/homeassistant/influxdb:/var/lib/influxdb
        - /etc/localtime:/etc/localtime:ro

    homeassistant:
      image: homeassistant/home-assistant
      dns_search: hq.example.com
      volumes:
        - /var/data/homeassistant/homeassistant:/config
        - /etc/localtime:/etc/localtime:ro
      deploy:
        labels:
          - traefik.frontend.rule=Host:homeassistant.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=8123
      networks:
        - traefik_public
        - internal
      ports:
        - 8123:8123

    grafana-app:
      image: grafana/grafana
      env_file : /var/data/config/homeassistant/grafana.env
      volumes:
        - /var/data/homeassistant/grafana:/var/lib/grafana
        - /etc/localtime:/etc/localtime:ro
      networks:
        - internal

    grafana-proxy:
      image: a5huynh/oauth2_proxy
      env_file : /var/data/config/homeassistant/grafana.env
      dns_search: hq.example.com
      networks:
        - internal
        - traefik_public
      deploy:
        labels:
          - traefik.frontend.rule=Host:grafana.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=4180
      volumes:
        - /var/data/config/homeassistant/authenticated-emails.txt:/authenticated-emails.txt
      command: |
        -cookie-secure=false
        -upstream=http://grafana-app:3000
        -redirect-url=https://grafana.example.com
        -http-address=http://0.0.0.0:4180
        -email-domain=example.com
        -provider=github
        -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.13.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Home Assistant stack

Launch the Home Assistant stack by running docker stack deploy homeassistant -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, the password you created in configuration.yml as “frontend - api_key”. Then setup a bunch of sensors, and log into https://grafana.YOUR FQDN and create some beautiful graphs :)

Chef’s Notes

I tried to protect Home Assistant using oauth2_proxy, but HA is incompatible with the websockets implementation used by Home Assistant. Until this can be fixed, I suggest that geeks set frontend: api_key to a long and complex string, and rely on this to prevent malevolent internet miscreants from turning their lights on at 2am!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

iBeacons with Home assistant

!!! warning This is not a complete recipe - it’s an optional additional of the HomeAssistant “recipe”, since it only applies to a subset of users

One of the most useful features of Home Assistant is location awareness. I don’t care if someone opens my office door when I’m home, but you bet I care about (and want to be notified) it if I’m away!

Ingredients

HomeAssistant per recipe

iBeacon(s) - This recipe is for https://s.click.aliexpress.com/e/bzyLCnAp

LightBlue Explorer

Preparation

Write UUID to iBeacon

The iBeacons come with no UUID. We use the LightBlue Explorer app to pair with them (code is “123456”), and assign own own UUID.

Generate your own UUID, or get a random one at https://www.uuidgenerator.net/

Plug in your iBeacon, launch LightBlue Explorer, and find your iBeacon. The first time you attempt to interrogate it, you’ll be prompted to pair. Although it’s not recorded anywhere in the documentation (grr!), the pairing code is 123456

Having paired, you’ll be able to see the vital statistics of your iBeacon.

hero: Huginn - A recipe for self-hosted, hackable version of IFFTT / Zapier

Huginn

Huginn is a system for building agents that perform automated tasks for you online. They can read the web, watch for events, and take actions on your behalf. Huginn’s Agents create and consume events, propagating them along a directed graph. Think of it as a hackable version of IFTTT or Zapier on your own server.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Preparation

Setup data locations

Create the location for the bind-mount of the database, so that it’s persistent:

mkdir -p /var/data/huginn/database

Create email address

Strictly speaking, you don’t have to integrate Huginn with email. However, since we created our own mailserver stack earlier, it’s worth using it to enable emails within Huginn.

cd /var/data/docker-mailserver/
./setup.sh email add huginn@huginn.example.com my-password-here
# Setup MX and DKIM if they don't already exist:
./setup.sh config dkim
cat config/opendkim/keys/huginn.example.com/mail.txt

Prepare environment

Create /var/data/huginn/huginn.env, and populate with the following variables. Set the “INVITATION_CODE” variable if you want to require users to enter a code to sign up (protects the UI from abuse) (The full list of Huginn environment variables is available here)

# For huginn/huginn - essential
SMTP_DOMAIN=your-domain-here.com
SMTP_USER_NAME=you@gmail.com
SMTP_PASSWORD=somepassword
SMTP_SERVER=your-mailserver-here.com
SMTP_PORT=587
SMTP_AUTHENTICATION=plain
SMTP_ENABLE_STARTTLS_AUTO=true
INVITATION_CODE=<set an invitation code here>
POSTGRES_PORT_5432_TCP_ADDR=db
POSTGRES_PORT_5432_TCP_PORT=5432
DATABASE_USERNAME=huginn
DATABASE_PASSWORD=<database password>
DATABASE_ADAPTER=postgresql

# Optional extras for huginn/huginn, customize or append based on .env.example lined above
TWITTER_OAUTH_KEY=
TWITTER_OAUTH_SECRET=

# For postgres/postgres
POSTGRES_USER=huginn
POSTGRES_PASSWORD=<database password>
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:

  huginn:
    image: huginn/huginn
    env_file: /var/data/config/huginn/huginn.env
    volumes:
    - /etc/localtime:/etc/localtime:ro
    networks:
    - internal
    - traefik
    deploy:
      labels:
        - traefik.frontend.rule=Host:huginn.funkypenguin.co.nz
        - traefik.docker.network=traefik
        - traefik.port=3000

  db:
    env_file: /var/data/config/huginn/huginn.env
    image: postgres:latest
    volumes:
      - /var/data/runtime/huginn/database:/var/lib/postgresql/data
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal

  db-backup:
    image: postgres:latest
    env_file: /var/data/config/huginn/huginn.env
    volumes:
      - /var/data/huginn/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        pg_dump -Fc > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.psql
        (ls -t /dump/dump*.psql|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.psql)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - internal

networks:
  traefik:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.6.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Huginn stack

Launch the Huginn stack by running docker stack deploy huginn -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN. You’ll need to use the “Sign Up” button, and (optionally) enter your invitation code in order to create your account.

Chef’s Notes

I initially considered putting an oauth proxy in front of Huginn, but since the invitation code logic prevents untrusted access, and since using a proxy would break oauth for sevices like Twitter integration, I left it out.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Kanboard - A recipe to get your personal kanban on

Kanboard

Kanboard is a Kanban tool, developed by Frédéric Guillot. (Who also happens to be the developer of my favorite RSS reader, Miniflux)

!!! tip “Sponsored Project” Kanboard is one of my sponsored projects - a project I financially support on a regular basis because of its utility to me. I use it both in my DayJob(tm), and to manage my overflowing, overly-optimistic personal commitments! 😓

Features include:

Visualize your work

Limit your work in progress to be more efficient

Customize your boards according to your business activities

Multiple projects with the ability to drag and drop tasks

Reports and analytics

Fast and simple to use

Access from anywhere with a modern browser

Plugins and integrations with external services

Free, open source and self-hosted

Super simple installation

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry pointing your NextCloud url (kanboard.example.com) to your keepalived IP

Preparation

Setup data locations

Create the location for the bind-mount of the application data, so that it’s persistent:

mkdir -p /var/data/kanboard

Setup Environment

If you intend to use an OAuth proxy to further secure public access to your instance, create a kanboard.env file to hold your environment variables, and populate with your OAuth provider’s details (the cookie secret you can just make up):

# If you decide to protect kanboard with an oauth_proxy, complete these
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  kanboard:
    image: kanboard/kanboard
    volumes:
     - /var/data/kanboard/data:/var/www/app/data
     - /var/data/kanboard/plugins:/var/www/app/plugins
    networks:
    - internal
    deploy:
      labels:
        - traefik.frontend.rule=Host:kanboard.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=80

    proxy:
      image: a5huynh/oauth2_proxy
      env_file : /var/data/config/kanboard/kanboard.env
      networks:
        - internal
        - traefik_public
      deploy:
        labels:
          - traefik.frontend.rule=Host:kanboard.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=4180
      volumes:
        - /var/data/config/kanboard/authenticated-emails.txt:/authenticated-emails.txt
      command: |
        -cookie-secure=false
        -upstream=http://app
        -redirect-url=https://kanboard.example.com
        -http-address=http://0.0.0.0:4180
        -email-domain=example.com
        -provider=github
        -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.8.0/24

Serving

Launch Kanboard stack

Launch the Kanboard stack by running docker stack deploy kanboard -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN. Default credentials are admin/admin, after which you can change (under ‘profile’) and add more users.

Chef’s Notes

The default theme can be significantly improved by applying the ThemePlus plugin.

Kanboard becomes more useful when you integrate in/outbound email with MailGun, SendGrid, or Postmark.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Miniflux - A recipe for a lightweight minimalist RSS reader

Miniflux

Miniflux is a lightweight RSS reader, developed by Frédéric Guillot. (Who also happens to be the developer of the favorite Open Source Kanban app, Kanboard)

[picture]

Miniflux Screenshot

!!! tip “Sponsored Project” Miniflux is one of my sponsored projects - a project I financially support on a regular basis because of its utility to me. Although I get to process my RSS feeds less frequently than I’d like to!

I’ve reviewed Miniflux in detail on my blog, but features (among many) that I appreciate:

Compatible with the Fever API, read your feeds through existing mobile and desktop clients (This is the killer feature for me. I hardly ever read RSS on my desktop, I typically read on my iPhone or iPad, using Fiery Feeds or my new squeeze, Unread)

Send your bookmarks to Pinboard, Wallabag, Shaarli or Instapaper (I use this to automatically pin my bookmarks for collection on my blog)

Feeds can be configured to download a “full” version of the content (rather than an excerpt)

Use the Bookmarklet to subscribe to a website directly from any browsers

!!! abstract “2.0+ is a bit different” Some things changed when Miniflux 2.0 was released. For one thing, the only supported database is now postgresql (no more SQLite). External themes are gone, as is PHP (in favor of golang). It’s been a controversial change, but I’m keen on minimal and single-purpose, so I’m still very happy with the direction of development. The developer has laid out his opinions re the decisions he’s made in the course of development.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry pointing your Miniflux url (i.e. miniflux.example.com) to your keepalived IP

Preparation

Setup data locations

Create the location for the bind-mount of the application data, so that it’s persistent:

mkdir -p /var/data/miniflux/database-dump
mkdir -p /var/data/runtime/miniflux/database

Setup environment

Create /var/data/config/miniflux/miniflux.env something like this:

DATABASE_URL=postgres://miniflux:secret@db/miniflux?sslmode=disable
POSTGRES_USER=miniflux
POSTGRES_PASSWORD=secret

# This is necessary for the miniflux to update the db schema, even on an empty DB
RUN_MIGRATIONS=1

# Uncomment this on first run, else leave it commented out after adding your own user account
CREATE_ADMIN=1
ADMIN_USERNAME=admin
ADMIN_PASSWORD=test1234

# For backups
PGUSER=miniflux
PGPASSWORD=secret
PGHOST=db
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

The entire application is configured using environment variables, including the initial username. Once you’ve successfully deployed once, comment out CREATE_ADMIN and the two successive lines.

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  miniflux:
    image: miniflux/miniflux:2.0.7
    env_file: /var/data/config/miniflux/miniflux.env
    volumes:
     - /etc/localtime:/etc/localtime:ro
    networks:
    - internal
    - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:miniflux.example.com
        - traefik.port=8080
        - traefik.docker.network=traefik_public

  db:
    env_file: /var/data/config/miniflux/miniflux.env
    image: postgres:10.1
    volumes:
      - /var/data/runtime/miniflux/database:/var/lib/postgresql/data
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal

  db-backup:
    image: postgres:10.1
    env_file: /var/data/config/miniflux/miniflux.env
    volumes:
      - /var/data/miniflux/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        pg_dump -Fc > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.psql
        (ls -t /dump/dump*.psql|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.psql)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - internal

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.22.0/24

Serving

Launch Miniflux stack

Launch the Miniflux stack by running docker stack deploy miniflux -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, using the credentials you setup in the environment flie. After this, change your user/password as you see fit, and comment out the CREATE_ADMIN line in the env file (if you don’t, then an additional admin will be created the next time you deploy)

Chef’s Notes

Find the bookmarklet under the Settings -> Integration page.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Munin

Munin is a networked resource monitoring tool that can help analyze resource trends and “what just happened to kill our performance?” problems. It is designed to be very plug and play. A default installation provides a lot of graphs with almost no work.

[picture]

Munin Screenshot

Using Munin you can easily monitor the performance of your computers, networks, SANs, applications, weather measurements and whatever comes to mind. It makes it easy to determine “what’s different today” when a performance problem crops up. It makes it easy to see how you’re doing capacity-wise on any resources.

Munin uses the excellent ​RRDTool (written by Tobi Oetiker) and the framework is written in Perl, while plugins may be written in any language. Munin has a master/node architecture in which the master connects to all the nodes at regular intervals and asks them for data. It then stores the data in RRD files, and (if needed) updates the graphs. One of the main goals has been ease of creating new plugins (graphs).

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Prepare target nodes

Depending on what you want to monitor, you’ll want to install munin-node. On Ubuntu/Debian, you’ll use apt-get install munin-node, and on RHEL/CentOS, run yum install munin-node. Remember to edit /etc/munin/munin-node.conf, and set your node to allow the server to poll it, by adding cidr_allow x.x.x.x/x.

On CentOS Atomic, of course, you can’t install munin-node directly, but you can run it as a containerized instance. In this case, you can’t use swarm since you need the container running in privileged mode, so launch a munin-node container on each atomic host using:

docker run -d --name munin-node --restart=always \
  --privileged --net=host \
  -v /:/rootfs:ro \
  -v /sys:/sys:ro \
  -e ALLOW="cidr_allow 0.0.0.0/0" \
  -p 4949:4949 \
  --restart=always \
  funkypenguin/munin-node

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/munin:

mkdir /var/data/munin
cd /var/data/munin
mkdir -p {log,lib,run,cache}

Prepare environment

Create /var/data/config/munin/munin.env, and populate with the following variables. Use the OAUTH2 variables if you plan to use an oauth2_proxy to protect munin, and set at a minimum the MUNIN_USER, MUNIN_PASSWORD, and NODES values:

# Use these if you plan to protect the webUI with an oauth_proxy
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

MUNIN_USER=odin
MUNIN_PASSWORD=lokiisadopted
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USERNAME=smtp-username
SMTP_PASSWORD=smtp-password
SMTP_USE_TLS=false
SMTP_ALWAYS_SEND=false
SMTP_MESSAGE='[${var:group};${var:host}] -> ${var:graph_title} -> warnings: ${loop<,>:wfields  ${var:label}=${var:value}} / criticals: ${loop<,>:cfields  ${var:label}=${var:value}}'
ALERT_RECIPIENT=monitoring@example.com
ALERT_SENDER=alerts@example.com
NODES="node1:192.168.1.1 node2:192.168.1.2 node3:192.168.1.3"
SNMP_NODES="router1:10.0.0.254:9999"

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:

  munin:
    image: funkypenguin/munin-server
    env_file: /var/data/config/munin/munin.env
    networks:
      - internal
    volumes:
      - /var/data/munin/log:/var/log/munin
      - /var/data/munin/lib:/var/lib/munin
      - /var/data/munin/run:/var/run/munin
      - /var/data/munin/cache:/var/cache/munin

  proxy:
    image: funkypenguin/oauth2_proxy
    env_file: /var/data/config/munin/munin.env
    networks:
      - traefik_public
      - internal
    deploy:
      labels:
        - traefik.frontend.rule=Host:munin.example.com
        - traefik.docker.network=traefik
        - traefik.port=4180
    command: |
      -cookie-secure=false
      -upstream=http://munin:8080
      -redirect-url=https://munin.example.com
      -http-address=http://0.0.0.0:4180
      -email-domain=example.com
      -provider=github

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.24.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Munin stack

Launch the Munin stack by running docker stack deploy munin -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user and password password you specified in munin.env above.

Chef’s Notes

If you wanted to expose the Munin UI directly, you could remove the oauth2_proxy from the design, and move the traefik-related labels directly to the munin container. You’d also need to add the traefik_public network to the munin container.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Backup all your stuff. Share it. Privately.

NextCloud

!!! important Ongoing development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

NextCloud (a fork of OwnCloud, led by original developer Frank Karlitschek) is a suite of client-server software for creating and using file hosting services. It is functionally similar to Dropbox, although Nextcloud is free and open-source, allowing anyone to install and operate it on a private server. - https://en.wikipedia.org/wiki/Nextcloud

[picture]

NextCloud Screenshot

This recipe is based on the official NextCloud docker image, but includes seprate containers ofor the database (MariaDB), Redis (for transactional locking), Apache Solr (for full-text searching), automated database backup, (you do backup the stuff you care about, right?) and a separate cron container for running NextCloud’s 15-min crons.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry pointing your NextCloud url (nextcloud.example.com) to your keepalived IP

Preparation

Setup data locations

We’ll need several directories for static data to bind-mount into our container, so create them in /var/data/nextcloud (so that they can be backed up)

mkdir /var/data/nextcloud
cd /var/data/nextcloud
mkdir -p {html,apps,config,data,database-dump}

Now make more directories for runtime data (so that they can be not backed-up):

mkdir /var/data/runtime/nextcloud
cd /var/data/runtime/nextcloud
mkdir -p {db,redis}

Prepare environment

Create nextcloud.env, and populate with the following variables

NEXTCLOUD_ADMIN_USER=admin
NEXTCLOUD_ADMIN_PASSWORD=FVuojphozxMVyaYCUWomiP9b
MYSQL_HOST=db

# For mysql
MYSQL_ROOT_PASSWORD=<set to something secure>
MYSQL_DATABASE=nextcloud
MYSQL_USER=nextcloud
MYSQL_PASSWORD=set to something secure>

Now create a separate nextcloud-db-backup.env file, to capture the environment variables necessary to perform the backup. (If the same variables are shared with the mariadb container, they cause issues with database access)

# For database backup (keep 7 days daily backups)
MYSQL_PWD=<set to something secure, same as MYSQL_ROOT_PASSWORD above>
MYSQL_USER=root
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3.0"

services:
  nextcloud:
    image: nextcloud
    env_file: /var/data/config/nextcloud/nextcloud.env
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:nextcloud.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=80
    volumes:
    - /var/data/nextcloud/html:/var/www/html
    - /var/data/nextcloud/apps:/var/www/html/custom_apps
    - /var/data/nextcloud/config:/var/www/html/config
    - /var/data/nextcloud/data:/var/www/html/data

  db:
    image: mariadb:10
    env_file: /var/data/config/nextcloud/nextcloud.env
    networks:
      - internal
    volumes:
      - /var/data/runtime/nextcloud/db:/var/lib/mysql

  db-backup:
    image: mariadb:10
    env_file: /var/data/config/nextcloud/nextcloud.env
    volumes:
      - /var/data/nextcloud/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        mysqldump -h db --all-databases | gzip -c > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.sql.gz
        (ls -t /dump/dump*.sql.gz|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.sql.gz)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - internal

  redis:
    image: redis:alpine
    networks:
      - internal
    volumes:
      - /var/data/runtime/nextcloud/redis:/data

  cron:
    image: nextcloud
    volumes:
      - /var/data/nextcloud/:/var/www/html
    user: www-data
    networks:
      - internal
    entrypoint: |
      bash -c 'bash -s <<EOF
        trap "break;exit" SIGHUP SIGINT SIGTERM
        while [ ! -f /var/www/html/config/config.php ]; do
          sleep 1
        done
        while true; do
          php -f /var/www/html/cron.php
          sleep 15m
        done
      EOF'

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.12.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch NextCloud stack

Launch the NextCloud stack by running docker stack deploy nextcloud -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user “admin” and the password you specified in nextcloud.env.

Enable redis

To make NextCloud a little snappier, edit /var/data/nextcloud/config/config.php (now that it’s been created on the first container launch), and add the following:

  'redis' => array(
     'host' => 'redis',
     'port' => 6379,
      ),

Use service discovery

Want to use Calendar/Contacts on your iOS device? Want to avoid dictating long, rambling URL strings to your users, like https://nextcloud.batcave.com/remote.php/dav/principals/users/USERNAME/ ?

Huzzah! NextCloud supports service discovery for CalDAV/CardDAV, allowing you to simply tell your device the primary URL of your server (nextcloud.batcave.org, for example), and have the device figure out the correct WebDAV path to use.

We (and anyone else using the NextCloud Docker image) are using an SSL-terminating reverse proxy (Traefik) in front of our NextCloud container. In fact, it’s not possible to setup SSL within the NextCloud container.

When using a reverse proxy, your device requests a URL from your proxy (https://nextcloud.batcave.com/.well-known/caldav), and the reverse proxy then passes that request unencrypted to the internal URL of the NextCloud instance (i.e., http://172.16.12.123/.well-known/caldav)

The Apache webserver on the NextCloud container (knowing it was spoken to via HTTP), responds with a 301 redirect to http://nextcloud.batcave.com/remote.php/dav/. See the problem? You requested an HTTPS (encrypted) url, and in return, you received a redirect to an HTTP (unencrypted) URL. Any sensible client (iOS included) will refuse such schenanigans.

To correct this, we need to tell NextCloud to always redirect the .well-known URLs to an HTTPS location. This can only be done after deploying NextCloud, since it’s only on first launch of the container that the .htaccess file is created in the first place.

To make NextCloud service discovery work with Traefik reverse proxy, edit /var/data/nextcloud/html/.htaccess, and change this:

RewriteRule ^\.well-known/carddav /remote.php/dav/ [R=301,L]
RewriteRule ^\.well-known/caldav /remote.php/dav/ [R=301,L]

To this:

RewriteRule ^\.well-known/carddav https://%{SERVER_NAME}/remote.php/dav/ [R=301,L]
RewriteRule ^\.well-known/caldav https://%{SERVER_NAME}/remote.php/dav/ [R=301,L]

Then restart your container with docker service update nextcloud_nextcloud --force to restart apache.

Your can test for success by running curl -i https://nextcloud.batcave.org/.well-known/carddav. You should get a 301 redirect to your equivalent of https://nextcloud.batcave.org/remote.php/dav/, as below:

[davidy:~] % curl -i https://nextcloud.batcave.org/.well-known/carddav
HTTP/2 301
content-type: text/html; charset=iso-8859-1
date: Wed, 12 Dec 2018 08:30:11 GMT
location: https://nextcloud.batcave.org/remote.php/dav/

Note that this .htaccess can be overwritten by NextCloud, and you may have to reapply the change in future. I’ve created an issue requesting a permanent fix.

!!! important Ongoing development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

Chef’s Notes

Since many of my other recipes use PostgreSQL, I’d have preferred to use Postgres over MariaDB, but MariaDB seems to be the preferred database type.

I’m not the first user to stumble across the service discovery bug with reverse proxies.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

OwnTracks

OwnTracks allows you to keep track of your own location. You can build your private location diary or share it with your family and friends. OwnTracks is open-source and uses open protocols for communication so you can be sure your data stays secure and private.

[picture]

OwnTracks Screenshot

Using a smartphone app, OwnTracks allows you to collect and analyse your own location data without sharing this data with a cloud provider (i.e. Apple, Google). Potential use cases are:

Sharing family locations without relying on Apple Find-My-friends

Performing automated actions in HomeAssistant when you arrive/leave home

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need a directory so store OwnTracks’ data , so create /var/data/owntracks:

mkdir /var/data/owntracks

Prepare environment

Create owntracks.env, and populate with the following variables

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

OTR_USER=recorder
OTR_PASSWD=yourpassword
MQTTHOSTNAME=owntracks.example.com
HOSTLIST=owntracks.example.com

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3.0"

services:
    owntracks-app:
      image: funkypenguin/owntracks
      env_file : /var/data/config/owntracks/owntracks.env
      volumes:
        - /var/data/owntracks:/owntracks
      networks:
        - internal
      ports:
        - 1883:1883
        - 8883:8883
        - 8083:8083

    owntracks-proxy:
      image: a5huynh/oauth2_proxy
      env_file : /var/data/config/owntracks/owntracks.env
      networks:
        - internal
        - traefik_public
      deploy:
        labels:
              - traefik.frontend.rule=Host:owntracks.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=4180
      volumes:
        - /var/data/config/owntracks/authenticated-emails.txt:/authenticated-emails.txt
      command: |
        -cookie-secure=false
        -upstream=http://owntracks-app:8083
        -redirect-url=https://owntracks.example.com
        -http-address=http://0.0.0.0:4180
        -email-domain=example.com
        -provider=github
        -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.15.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch OwnTracks stack

Launch the OwnTracks stack by running docker stack deploy owntracks -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user “root” and the password you specified in gitlab.env.

Chef’s Notes

If you wanted to expose the OwnTracks Web UI directly, you could remove the oauth2_proxy from the design, and move the traefik-related labels directly to the wekan container. You’d also need to add the traefik network to the owntracks container.

I’m using my own image rather than owntracks/recorderd, because of a potentially swarm-breaking bug I found in the official container. If this gets resolved (or if I was mistaken) I’ll update the recipe accordingly.

By default, you’ll get a fully accessible, unprotected MQTT broker. This may not be suitable for public exposure, so you’ll want to look into securing mosquitto with TLS and ACLs.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

phpIPAM

phpIPAM is an open-source web IP address management application (IPAM). Its goal is to provide light, modern and useful IP address management. It is php-based application with MySQL database backend, using jQuery libraries, ajax and HTML5/CSS3 features.

[picture]

phpIPAM Screenshot

phpIPAM fulfils a non-sexy, but important role - It helps you manage your IP address allocation.

Why should you care about this?

You probably have a home network, with 20-30 IP addresses, for your family devices, your IoT devices, your smart TV, etc. If you want to (a) monitor them, and (b) audit who does what, you care about what IPs they’re assigned by your DHCP server.

You could simple keep track of all devices with leases in your DHCP server, but what happens if your (hypothetical?) Ubiquity Edge Router X crashes and burns due to lack of disk space, and you loose track of all your leases? Well, you have to start from scratch, is what!

And that HomeAssistant config, which you so carefully compiled, refers to each device by IP/DNS name, so you’d better make sure you recreate it consistently!

Enter phpIPAM. A tool designed to help home keeps as well as large organisations keep track of their IP (and VLAN, VRF, and AS number) allocations.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname (i.e. “phpipam.your-domain.com”) you intend to use for phpIPAM, pointed to your keepalived IPIP

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/phpipam:

mkdir /var/data/phpipam/databases-dump -p
mkdir /var/data/runtime/phpipam -p

Prepare environment

Create phpipam.env, and populate with the following variables

# Setup for github, phpipam application
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

# For MariaDB/MySQL database
MYSQL_ROOT_PASSWORD=imtoosecretformyshorts
MYSQL_DATABASE=phpipam
MYSQL_USER=phpipam
MYSQL_PASSWORD=secret

# phpIPAM-specific variables
MYSQL_ENV_MYSQL_USER=phpipam
MYSQL_ENV_MYSQL_PASSWORD=secret
MYSQL_ENV_MYSQL_DB=phpipam
MYSQL_ENV_MYSQL_HOST=db

# For backup
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Additionally, create phpipam-backup.env, and populate with the following variables:

# For MariaDB/MySQL database
MYSQL_ROOT_PASSWORD=imtoosecretformyshorts
MYSQL_DATABASE=phpipam
MYSQL_USER=phpipam
MYSQL_PASSWORD=secret

# For backup
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Create nginx.conf

I usually protect my stacks using an oauth proxy container in front of the app. This protects me from either accidentally exposing a platform to the world, or having a insecure platform accessed and abused.

In the case of phpIPAM, the oauth_proxy creates an additional complexity, since it passes the “Authorization” HTTP header to the phpIPAM container. phpIPAH then examines the header, determines that the provided username (my email address associated with my oauth provider) doesn’t match a local user account, and denies me access without the opportunity to retry.

The (dirty) solution I’ve come up with is to insert an Nginx instance in the path between the oauth_proxy and the phpIPAM container itself. Nginx can remove the authorization header, so that phpIPAM can prompt me to login with a web-based form.

Create /var/data/phpipam/nginx.conf as follows:

upstream app-upstream {
    server app:80;
}

server {
    listen 80;
    server_name ~.;

    # Just redirect everything to the upstream
    # Yes, it's embarassing. We are just a mechanism to strip an AUTH header :(
    location ^~ / {
        proxy_pass http://app-upstream;
    proxy_set_header       Authorization "";
    }

}

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:

  db:
    image: mariadb:10
    env_file: /var/data/config/phpipam/phpipam.env
    networks:
      - internal
    volumes:
      - /var/data/runtime/phpipam/db:/var/lib/mysql

  proxy:
    image: funkypenguin/oauth2_proxy
    env_file: /var/data/config/phpipam/phpipam.env
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:phpipam.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    volumes:
      - /var/data/config/phpipam/authenticated-emails.txt:/authenticated-emails.txt
    command: |
      -cookie-secure=false
      -upstream=http://nginx
      -redirect-url=https://phpipam.example.com
      -http-address=http://0.0.0.0:4180
      -email-domain=example.com
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

  # Wait, what? Why do we have an oauth_proxy _and_ an nginx frontend for a simple webapp?
  # Well, it's a long story. Basically, the phpipam container sees the "auth" headers passed by the
  # oauth_proxy, and decides to use these exclusively to authenticate users. So no web-based login form, just "access denied"
  # To work around this, we add nginx reverse proxy to the mix. A PITA, but an easy way to solve without altering the PHPIPAM code
  nginx:
    image: nginx:latest
    networks:
      - internal
    volumes:
      - /var/data/phpipam/nginx.conf:/etc/nginx/conf.d/default.conf:ro

  app:
    image: pierrecdn/phpipam
    env_file: /var/data/config/phpipam/phpipam.env
    networks:
      - internal

  db-backup:
    image: mariadb:10
    env_file: /var/data/config/phpipam/phpipam.env
    volumes:
      - /var/data/phpipam/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        mysqldump -h db --all-databases | gzip -c > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.sql.gz
        (ls -t /dump/dump*.sql.gz|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.sql.gz)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
      - internal

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.47.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch phpIPAM stack

Launch the phpIPAM stack by running docker stack deploy phpipam -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, and follow the on-screen prompts to set your first user/password.

Chef’s Notes

If you wanted to expose the phpIPAM UI directly, you could remove the oauth2_proxy and the nginx services from the design, and move the traefik_public-related labels directly to the phpipam container. You’d also need to add the traefik_public network to the phpipam container.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: A recipe to manage your Media 🎥 📺 🎵

Plex

Plex is a client-server media player system and software suite comprising two main components (a media server and client applications)

[picture]

Plex Screenshot

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

A DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need a directories to bind-mount into our container for Plex to store its library, so create /var/data/plex:

mkdir /var/data/plex

Prepare environment

Create plex.env, and populate with the following variables. Set PUID and GUID to the UID and GID of the user who owns your media files, on the local filesystem

EDGE=1
VERSION=latest
PUID=42
PGID=42

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3.0"

services:
  plex:
    image: linuxserver/plex
    env_file: plex.env
    volumes:
      - /var/data/config/plex:/config
      - /var/data/media:/media
    deploy:
      labels:
        - traefik.frontend.rule=Host:plex.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=32400
    networks:
        - traefik_public
        - internal
    ports:
      - 32469:32469
      - 32400:32400
      - 32401:32401
      - 3005:3005
      - 8324:8324
      - 1900:1900/udp
      - 32410:32410/udp
      - 32412:32412/udp
      - 32413:32413/udp
      - 32414:32414/udp

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.16.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Plex stack

Launch the Plex stack by running docker stack deploy plex -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN (You’ll need to setup a plex.tv login for remote access / discovery to work from certain clients)

Chef’s Notes

Plex uses port 32400 for remote access, using your plex.tv user/password to authenticate you. The inclusion of the traefik proxy in this recipe is simply to allow you to use the web client (as opposed to a client app) by connecting directly to your instance, as opposed to browsing your media via https://plex.tv/web

Got an NVIDIA GPU? See this blog post re how to use your GPU to transcode your media!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

PrivateBin

PrivateBin is a minimalist, open source online pastebin where the server (can) has zero knowledge of pasted data. We all need to paste data / log files somewhere when it doesn’t make sense to paste it inline. With PasteBin, you can own the hosting, access, and eventual deletion of this data.

[picture]

PrivateBin Screenshot

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need a single location to bind-mount into our container, so create /var/data/privatebin, and make it world-writable (there might be a more secure way to do this!)

mkdir /var/data/privatebin
chmod 777 /var/data/privatebin/

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  app:
    image: privatebin/nginx-fpm-alpine
    volumes:
      - /var/data/privatebin:/srv/data
    networks:
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:privatebin.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=4180

networks:
  traefik_public:
    external: true

Serving

Launch PrivateBin stack

Launch the PrivateBin stack by running docker stack deploy privatebin -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user “root” and the password you specified in gitlab.env.

Chef’s Notes

The PrivateBin repo explains how to tweak configuration options, or to use a database instead of file storage, if your volume justifies it :)

The inclusion of PrivateBin was due to the efforts of @gkoerk in our Discord server. Thanks Jerry!!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Swarmprom

Swarmprom is a starter kit for Docker Swarm monitoring with Prometheus, Grafana, cAdvisor, Node Exporter, Alert Manager and Unsee. And it’s damn sexy. See for yourself:

[picture]

Swarmprom Screenshot

So what do all these components do?

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud.

Grafana is a tool to make data beautiful.

cAdvisor cAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a running daemon that collects, aggregates, processes, and exports information about running containers.

Node Exporter is a Prometheus exporter for hardware and OS metrics

Alert Manager Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integrations such as email, Slack, etc.

Unsee is an alert dashboard for Alert Manager

How does this magic work?

I’d encourage you to spend some time reading https://github.com/stefanprodan/swarmprom. Stefan has included detailed explanations about which elements perform which functions, as well as how to customize your stack. (This is only a starting point, after all)

Ingredients

Docker swarm cluster on 17.09.0 or newer (doesn’t work with CentOS Atomic, unfortunately) with persistent shared storage

Traefik configured per design

DNS entry for the hostnames you intend to use, pointed to your keepalived IP

Preparation

This is basically a rehash of stefanprodan’s instructions to match the way I’ve configured other recipes.

Setup oauth provider

Grafana includes decent login protections, but from what I can see, Prometheus, AlertManager, and Unsee do no authentication. In order to expose these publicly for your own consumption (my assumption for the rest of this recipe), you’ll want to prepare to run oauth_proxy containers in front of each of the 4 web UIs in this recipe.

Setup metrics

Edit (or create, depending on your OS) /etc/docker/daemon.json, and add the following, to enable the experimental export of metrics to Prometheus:

{
  "metrics-addr" : "0.0.0.0:9323",
  "experimental" : true
}

Restart docker with systemctl restart docker

Setup and populate data locations

We’ll need several files to bind-mount into our containers, so create directories for them and get the latest copies:

mkdir -p /var/data/swarmprom/dockerd-exporter/
cd /var/data/swarmprom/dockerd-exporter/
wget https://raw.githubusercontent.com/stefanprodan/swarmprom/master/dockerd-exporter/Caddyfile

mkdir -p /var/data/swarmprom/prometheus/rules/
cd /var/data/swarmprom/prometheus/rules/
wget https://raw.githubusercontent.com/stefanprodan/swarmprom/master/prometheus/rules/swarm_task.rules.yml
wget https://raw.githubusercontent.com/stefanprodan/swarmprom/master/prometheus/rules/swarm_node.rules.yml

# Directories for holding runtime data
mkdir /var/data/runtime/swarmprom/grafana/
mkdir /var/data/runtime/swarmprom/alertmanager/
mkdir /var/data/runtime/prometheus

chown nobody:nogroup /var/data/runtime/prometheus

Prepare Grafana

Grafana will make all the data we collect from our swarm beautiful.

Create /var/data/swarmprom/grafana.env, and populate with the following variables

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

# Disable basic auth (it conflicts with oauth_proxy)
GF_AUTH_BASIC_ENABLED=false

# Set this to the real-world URL to your grafana install (else you get screwy CSS thanks to oauth_proxy)
GF_SERVER_ROOT_URL=https://grafana.example.com
GF_SERVER_DOMAIN=grafana.example.com

# Set your default admin/pass here
GF_SECURITY_ADMIN_USER=admin
GF_SECURITY_ADMIN_PASSWORD=ilovemybatmanunderpants

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), based on the original swarmprom docker-compose.yml file

???+ note “This example is 274 lines long. Click here to collapse it for better readability”

!!! tip
        I share (_with my [patreon patrons](https://www.patreon.com/funkypenguin)_) a private "_premix_" git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a ```git pull``` and a ```docker stack deploy``` 👍

  ```
  version: "3.3"

  networks:
    net:
      driver: overlay
      attachable: true

  volumes:
      prometheus: {}
      grafana: {}
      alertmanager: {}

  configs:
    dockerd_config:
      file: /var/data/swarmprom/dockerd-exporter/Caddyfile
    node_rules:
      file: /var/data/swarmprom/prometheus/rules/swarm_node.rules.yml
    task_rules:
      file: /var/data/swarmprom/prometheus/rules/swarm_task.rules.yml

  services:
    dockerd-exporter:
      image: stefanprodan/caddy
      networks:
        - internal
      environment:
        - DOCKER_GWBRIDGE_IP=172.18.0.1
      configs:
        - source: dockerd_config
          target: /etc/caddy/Caddyfile
      deploy:
        mode: global
        resources:
          limits:
            memory: 128M
          reservations:
            memory: 64M

    cadvisor:
      image: google/cadvisor
      networks:
        - internal
      command: -logtostderr -docker_only
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock:ro
        - /:/rootfs:ro
        - /var/run:/var/run
        - /sys:/sys:ro
        - /var/lib/docker/:/var/lib/docker:ro
      deploy:
        mode: global
        resources:
          limits:
            memory: 128M
          reservations:
            memory: 64M

    grafana:
      image: stefanprodan/swarmprom-grafana:5.0.1
      networks:
        - internal
      env_file: /var/data/config/swarmprom/grafana.env
      environment:
        - GF_USERS_ALLOW_SIGN_UP=false
        - GF_SMTP_ENABLED=${GF_SMTP_ENABLED:-false}
        - GF_SMTP_FROM_ADDRESS=${GF_SMTP_FROM_ADDRESS:-grafana@test.com}
        - GF_SMTP_FROM_NAME=${GF_SMTP_FROM_NAME:-Grafana}
        - GF_SMTP_HOST=${GF_SMTP_HOST:-smtp:25}
        - GF_SMTP_USER=${GF_SMTP_USER}
        - GF_SMTP_PASSWORD=${GF_SMTP_PASSWORD}
      volumes:
        - /var/data/runtime/swarmprom/grafana:/var/lib/grafana
      deploy:
        mode: replicated
        replicas: 1
        placement:
          constraints:
            - node.role == manager
        resources:
          limits:
            memory: 128M
          reservations:
            memory: 64M

    grafana-proxy:
      image: a5huynh/oauth2_proxy
      env_file : /var/data/config/swarmprom/grafana.env
      networks:
        - internal
        - traefik_public
      deploy:
        labels:
          - traefik.frontend.rule=Host:grafana.swarmprom.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=4180
      volumes:
        - /var/data/config/swarmprom/authenticated-emails.txt:/authenticated-emails.txt
      command: |
        -cookie-secure=false
        -upstream=http://grafana:3000
        -redirect-url=https://grafana.swarmprom.example.com
        -http-address=http://0.0.0.0:4180
        -email-domain=example.com
        -provider=github
        -authenticated-emails-file=/authenticated-emails.txt

    alertmanager:
      image: stefanprodan/swarmprom-alertmanager:v0.14.0
      networks:
        - internal
      environment:
        - SLACK_URL=${SLACK_URL:-https://hooks.slack.com/services/TOKEN}
        - SLACK_CHANNEL=${SLACK_CHANNEL:-general}
        - SLACK_USER=${SLACK_USER:-alertmanager}
      command:
        - '--config.file=/etc/alertmanager/alertmanager.yml'
        - '--storage.path=/alertmanager'
      volumes:
        - /var/data/runtime/swarmprom/alertmanager:/alertmanager
      deploy:
        mode: replicated
        replicas: 1
        placement:
          constraints:
            - node.role == manager
        resources:
          limits:
            memory: 128M
          reservations:
            memory: 64M

    alertmanager-proxy:
      image: a5huynh/oauth2_proxy
      env_file : /var/data/config/swarmprom/alertmanager.env
      networks:
        - internal
        - traefik_public
      deploy:
        labels:
          - traefik.frontend.rule=Host:alertmanager.swarmprom.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=4180
      volumes:
        - /var/data/config/swarmprom/authenticated-emails.txt:/authenticated-emails.txt
      command: |
        -cookie-secure=false
        -upstream=http://alertmanager:9093
        -redirect-url=https://alertmanager.swarmprom.example.com
        -http-address=http://0.0.0.0:4180
        -email-domain=example.com
        -provider=github
        -authenticated-emails-file=/authenticated-emails.txt

    unsee:
      image: cloudflare/unsee:v0.8.0
      networks:
        - internal
      environment:
        - "ALERTMANAGER_URIS=default:http://alertmanager:9093"
      deploy:
        mode: replicated
        replicas: 1

    unsee-proxy:
      image: a5huynh/oauth2_proxy
      env_file : /var/data/config/swarmprom/unsee.env
      networks:
        - internal
        - traefik_public
      deploy:
        labels:
          - traefik.frontend.rule=Host:unsee.swarmprom.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=4180
      volumes:
        - /var/data/config/swarmprom/authenticated-emails.txt:/authenticated-emails.txt
      command: |
        -cookie-secure=false
        -upstream=http://unsee:8080
        -redirect-url=https://unsee.swarmprom.example.com
        -http-address=http://0.0.0.0:4180
        -email-domain=example.com
        -provider=github
        -authenticated-emails-file=/authenticated-emails.txt

    node-exporter:
      image: stefanprodan/swarmprom-node-exporter:v0.15.2
      networks:
        - internal
      environment:
        - NODE_ID={{.Node.ID}}
      volumes:
        - /proc:/host/proc:ro
        - /sys:/host/sys:ro
        - /:/rootfs:ro
        - /etc/hostname:/etc/nodename
      command:
        - '--path.sysfs=/host/sys'
        - '--path.procfs=/host/proc'
        - '--collector.textfile.directory=/etc/node-exporter/'
        - '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
        # no collectors are explicitely enabled here, because the defaults are just fine,
        # see https://github.com/prometheus/node_exporter
        # disable ipvs collector because it barfs the node-exporter logs full with errors on my centos 7 vm's
        - '--no-collector.ipvs'
      deploy:
        mode: global
        resources:
          limits:
            memory: 128M
          reservations:
            memory: 64M

    prometheus:
      image: stefanprodan/swarmprom-prometheus:v2.2.0-rc.0
      networks:
        - internal
      command:
        - '--config.file=/etc/prometheus/prometheus.yml'
        - '--web.console.libraries=/etc/prometheus/console_libraries'
        - '--web.console.templates=/etc/prometheus/consoles'
        - '--storage.tsdb.path=/prometheus'
        - '--storage.tsdb.retention=24h'
      volumes:
        - /var/data/runtime/swarmprom/prometheus:/prometheus
      configs:
        - source: node_rules
          target: /etc/prometheus/swarm_node.rules.yml
        - source: task_rules
          target: /etc/prometheus/swarm_task.rules.yml
      deploy:
        mode: replicated
        replicas: 1
        placement:
          constraints:
            - node.role == manager
        resources:
          limits:
            memory: 2048M
          reservations:
            memory: 128M

    prometheus-proxy:
      image: a5huynh/oauth2_proxy
      env_file : /var/data/config/swarmprom/prometheus.env
      networks:
        - internal
        - traefik_public
      deploy:
        labels:
          - traefik.frontend.rule=Host:prometheus.swarmprom.example.com
          - traefik.docker.network=traefik_public
          - traefik.port=4180
      volumes:
        - /var/data/config/swarmprom/authenticated-emails.txt:/authenticated-emails.txt
      command: |
        -cookie-secure=false
        -upstream=http://prometheus:9090
        -redirect-url=https://prometheus.swarmprom.example.com
        -http-address=http://0.0.0.0:4180
        -email-domain=example.com
        -provider=github
        -authenticated-emails-file=/authenticated-emails.txt

  networks:
    traefik_public:
      external: true
    internal:
      driver: overlay
      ipam:
        config:
          - subnet: 172.16.29.0/24
  ```

!!! note
    Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you're creating/removing stacks a lot. See [my list](/reference/networks/) here.

Serving

Launch Swarmprom stack

Launch the Swarm stack by running docker stack deploy swarmprom -c <path -to-docker-compose.yml>

Log into your new grafana instance, check out your beautiful graphs. Move onto drooling over Prometheus, AlertManager, and Unsee.

Chef’s Notes

Pay close attention to the grafana.env config. If you encounter errors about basic auth failed, or failed CSS, it’s likely due to misconfiguration of one of the grafana environment variables.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: How to setup a TurtleCoin Mining Pool

TurtleCoin Mining Pool

Cryptocurrency miners will “pool” their GPU resources (“hashpower”) into aggregate “mining pools”, so that by the combined effort of all the miners, the pool will receive a reward for the blocks “mined” into the blockchain, and this reward will be distributed among the miners.

[picture]

Turtle Pool Screenshot

This recipe illustrates how to build a mining pool for TurtleCoin, one of many CryptoNote currencies (which include Monero), but the principles can be applied to most mineable coins.

The end result is a mining pool which looks like this: https://trtl.heigh-ho.funkypenguin.co.nz/

!!! question “WTF is a TurtleCoin and why do I want it?”"

In my opinion - because it's a fun, no-BS project with a [silly origin story](https://turtlecoin.lol/#story), a [friendly, welcoming community](http://chat.turtlecoin.lol/), and you'll learn more about cryptocurrency/blockchain than you expect.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostnames (pool and api) you intend to use, pointed to your keepalived IP

At least 16GB disk space (12GB used, 4GB for future growth)

Preparation

Create user account

The TurtleCoin pool elements won’t (and shouldn’t) run as root, but they’ll need access to write data to some parts of the filesystem (like logs, etc).

To manage access control, we’ll want to create a local user on each docker node with the same UID.

useradd -u 3506 turtle-pool

!!! question “Why 3506?” I’m glad you asked. TurtleCoin hard-forked at block 350K to avoid ASIC miners. The Ninja Turtles’ human friend April O’Neil works at Channel 6 News. 350 + 6 = 3506 😁. Aren’t you glad you asked?

Setup Redis

The pool uses Redis for in-memory and persistent storage. This comes in handy for the Docker Swarm deployment, since while the various pool modules weren’t designed to run as microservices, the fact that they all rely on Redis for data storage makes this possible.

!!! warning “Playing it safe”

Be aware that by default, Redis stores some data **only** in memory, and writes to the filesystem at default intervals (_can be up to 5 minutes by default_). Given we don't **want** to loose 5 minutes of miner's data if we restart Redis (_what happens if we found a block during those 5 minutes but haven't paid any miners yet?_), we want to ensure that Redis runs in "appendonly" mode, which ensures that every change is immediately written to disk.

We also want to make sure that we retain all Redis logs persistently (_We're dealing with people's cryptocurrency here, it's a good idea to keep persistent logs for debugging/auditing_)

Create directories to hold Redis data. We use separate directories for future flexibility - One day, we may want to backup the data but not the logs, or move the data to an SSD partition but leave the logs on slower, cheaper disk.

mkdir -p /var/data/turtle-pool/redis/config
mkdir -p /var/data/turtle-pool/redis/data
mkdir -p /var/data/turtle-pool/redis/logs
chown turtle-pool /var/data/turtle-pool/redis/data
chown turtle-pool /var/data/turtle-pool/redis/logs

Create /var/data/turtle-pool/redis/config/redis.conf using http://download.redis.io/redis-stable/redis.conf as a guide. The following are the values I changed from default on my deployment (but I’m not a Redis expert!):

appendonly yes
appendfilename "appendonly.aof"
loglevel notice
logfile "/logs/redis.log"
protected-mode no

I also had to disable the following line, by commenting it out (thus ensuring Redis container will respond to the other containers):

bind 127.0.0.1

Setup Nginx

We’ll run a simple Nginx container to serve the static front-end of the web UI.

The simplest way to get the frontend is just to clone the upstream turtle-pool repo, and mount the “/website” subdirectory into Nginx.

git clone https://github.com/turtlecoin/turtle-pool.git /var/data/turtle-pool/nginx/

Edit /var/data/turtle-pool/nginx/website/config.js, and change at least the following:

var api = "https://<CHOOSE A FQDN TO USE FOR YOUR API>";
var poolHost = "<SET TO THE PUBLIC DNS NAME FOR YOUR POOL SERVER";

Setup TurtleCoin daemon

The first thing we’ll need to participate in the great and powerful TurtleCoin network is a node. The node is responsible for communicating with the rest of the nodes in the blockchain, allowing our miners to receive new blocks to try to find.

As of June 2018, the TurtleCoin daemon is not what you’d call “stable”. Certain known circumstances (like unusual blocks) can cause the daemon to crash, and permanently corrupt its stored blockchain data. When (not if) this happens, the only solution is to delete all the stored blockchain data, and allow the daemon to resync (from checkpoints and then from other nodes). This resyncing process takes time, during which your miners can’t mine. To partially mitigate this risk, we run 2 extra copies of the daemon, which we’ll then start/stop at alternate intervals, so that in the event of a corruption of the daemon’s data, we have hopefully-safe recent copies of the blockchain, allowing us to recover and restore quickly.

Create directories to hold the blockchain data for all 3 daemons:

mkdir -p /var/data/runtime/turtle-pool/daemon/1
mkdir -p /var/data/runtime/turtle-pool/daemon/failsafe1
mkdir -p /var/data/runtime/turtle-pool/daemon/failsafe2

Create cron entries (in /etc/cron.d/turtle-pool-failsafe) to start/stop the failsafe daemons:

# Each TurtleCoin failsafe daemon gets to run ~50% of every hour, but they're never running at the same time
# (to prevent a doomsday block from killing them both)
0 * * * *       root    docker service scale turtle-pool_daemon-failsafe1=0
1 * * * *       root    docker service scale turtle-pool_daemon-failsafe2=1
30 * * * *      root    docker service scale turtle-pool_daemon-failsafe2=0
31 * * * *      root    docker service scale turtle-pool_daemon-failsafe1=1

Setup TurtleCoin wallet

Our pool needs a wallet to be able to (a) receive rewards for blocks discovered, and (b) pay out our miners for their share of the reward.

Create directories to hold wallet data

mkdir -p /var/data/turtle-pool/wallet/config
mkdir -p /var/data/turtle-pool/wallet/container
mkdir -p /var/data/turtle-pool/wallet/logs
chown -R turtle-pool /var/data/turtle-pool/wallet/container
chown -R turtle-pool /var/data/turtle-pool/wallet/logs

Now create the initial wallet. You’ll want to secure your wallet password, so the command below will prompt you for the key (no output from the prompt), and insert it into an environment variable. This means that the key won’t be stored in plaintext in your bash history!

read PASS

After having entered your password (you can confirm by running env | grep PASS), run the following to run the wallet daemon once, with the instruction to create a new wallet container:

docker run \
 -v /var/data/turtle-pool/wallet/container:/container \
 --rm -ti --entrypoint /usr/local/bin/walletd funkypenguin/turtlecoind \
 --container-file /container/wallet.container \
 --container-password $PASS \
 --generate-container

You’ll get a lot of output. The following are relevant lines from a successful run with the extra stuff stripped out:

2018-May-01 11:14:57.662664 INFO    walled v0.4.3.1257 ()
2018-May-01 11:14:59.868087 INFO    Generating new wallet
2018-May-01 11:14:59.919178 INFO    Container initialized with view secret key, public view key <your view public key will be here>
2018-May-01 11:14:59.920932 INFO    New wallet added TRTL<your wallet's public address>, creation timestamp 0
2018-May-01 11:14:59.932367 INFO    Container shut down
2018-May-01 11:14:59.932419 INFO    Loading container...
2018-May-01 11:14:59.961814 INFO    Consumer added, consumer 0x55b0fb5bc070, count 1
2018-May-01 11:14:59.961996 INFO    Starting...
2018-May-01 11:14:59.962173 INFO    Container loaded, view public key <your view public key will be here>, wallet count 1, actual balance 0.00, pending balance 0.00
2018-May-01 11:14:59.962508 INFO    New wallet is generated. Address: TRTL<your wallet's public address>
2018-May-01 11:14:59.962581 INFO    Saving container...
2018-May-01 11:14:59.962683 INFO    Stopping...
2018-May-01 11:14:59.962862 INFO    Stopped

Take careful note of your wallet password, public view key, and wallet address (which starts with TRTL)

Create /var/data/turtle-pool/wallet/config/wallet.conf, containing the following:

bind-address = 0.0.0.0
container-file = /container/wallet.container
container-password = <ENTER YOUR CONTAINER PASSWORD HERE>
rpc-password = <CHOOSE A PASSWORD TO ALLOW POOL TO TALK TO WALLET>
log-file = /dev/stdout
log-level = 3
daemon-address = daemon

Setup TurtleCoin mining pool

Following the convention we’ve set above, create directories to hold pool data:

mkdir -p /var/data/turtle-pool/pool/config
mkdir -p /var/data/turtle-pool/pool/logs
chown -R turtle-pool /var/data/turtle-pool/pool/logs

Now create /var/data/turtle-pool/pool/config/config.json, using https://github.com/turtlecoin/turtle-pool/blob/master/config.json as a guide, and adjusting at least the following:

Send logs to /logs/, so that they can potentially be stored / backed up separately from the config:

"logging": {
    "files": {
        "level": "debug",
        "directory": "/logs",
        "flushInterval": 5
    },

Set the “poolAddress” field to your wallet address

"poolServer": {
    "enabled": true,
    "clusterForks": "auto",
    "poolAddress": "<SET THIS TO YOUR WALLET ADDRESS GENERATED ABOVE>",

Add the “host” value to the api section, since our API will run on its own container, and choose a password you’ll use for the webUI admin page

"api": {
    "enabled": true,
    "hashrateWindow": 600,
    "updateInterval": 5,
    "host": "pool-api",
    "port": 8117,
    "blocks": 30,
    "payments": 30,
    "password": "<PASSWORD FOR ADMIN PAGE ACCESS>"

Set the host value for the daemon:

"daemon": {
    "host": "daemon",
    "port": 11898
},

Set the host value for the wallet, and set your container password (you recorded it earlier, remember?)

"wallet": {
    "host": "wallet",
    "port": 8070,
    "password": "<SET ME TO YOUR WALLET RPC PASSWORD>"
},

Set the host value for Redis:

"redis": {
    "host": "redis",
    "port": 6379
},

That’s it! The above config files mean each element of the pool will be able to communicate with the other elements within the docker swarm, by name.

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  daemon:
    image: funkypenguin/turtlecoind
    volumes:
      - /var/data/runtime/turtle-pool/daemon/1:/var/lib/turtlecoind/
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal
      - traefik_public
    ports:
      - 11897:11897
      labels:
        - traefik.frontend.rule=Host:explorer.trtl.heigh-ho.funkypenguin.co.nz
        - traefik.docker.network=traefik_public
        - traefik.port=11898

  daemon-failsafe1:
    image: funkypenguin/turtlecoind
    volumes:
      - /var/data/runtime/turtle-pool/daemon/failsafe1:/var/lib/turtlecoind/
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal

  daemon-failsafe2:
    image: funkypenguin/turtlecoind
    volumes:
      - /var/data/runtime/turtle-pool/daemon/failsafe2:/var/lib/turtlecoind/
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal

  pool-pool:
    image: funkypenguin/turtle-pool
    volumes:
      - /var/data/turtle-pool/pool/config:/config:ro
      - /var/data/turtle-pool/pool/logs:/logs
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal
    ports:
      - 3333:3333
      - 5555:5555
      - 7777:7777
    entrypoint: |
      node init.js -module=pool -config=/config/config.json

  pool-api:
    image: funkypenguin/turtle-pool
    volumes:
      - /var/data/turtle-pool/pool/config:/config:ro
      - /var/data/turtle-pool/pool/logs:/logs
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:api.trtl.heigh-ho.funkypenguin.co.nz
        - traefik.docker.network=traefik_public
        - traefik.port=8117
    entrypoint: |
      node init.js -module=api -config=/config/config.json

  pool-unlocker:
    image: funkypenguin/turtle-pool
    volumes:
      - /var/data/turtle-pool/pool/config:/config:ro
      - /var/data/turtle-pool/pool/logs:/logs
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal
    entrypoint: |
      node init.js -module=unlocker -config=/config/config.json

  pool-payments:
    image: funkypenguin/turtle-pool
    volumes:
      - /var/data/turtle-pool/pool/config:/config:ro
      - /var/data/turtle-pool/pool/logs:/logs
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal
    entrypoint: |
      node init.js -module=payments -config=/config/config.json

  pool-charts:
    image: funkypenguin/turtle-pool
    volumes:
      - /var/data/turtle-pool/pool/config:/config:ro
      - /var/data/turtle-pool/pool/logs:/logs
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal
    entrypoint: |
      node init.js -module=chartsDataCollector -config=/config/config.json

  wallet:
    image: funkypenguin/turtlecoind
    volumes:
      - /var/data/turtle-pool/wallet/config:/config:ro
      - /var/data/turtle-pool/wallet/container:/container
      - /var/data/turtle-pool/wallet/logs:/logs
      - /etc/localtime:/etc/localtime:ro
    networks:
      - internal
    entrypoint: |
      walletd --config /config/wallet.conf | tee /logs/walletd.log

  redis:
    volumes:
      - /var/data/turtle-pool/redis/config:/config:ro
      - /var/data/turtle-pool/redis/data:/data
      - /var/data/turtle-pool/redis/logs:/logs
      - /etc/localtime:/etc/localtime:ro
    image: redis
    entrypoint: |
      redis-server /config/redis.conf
    networks:
      - internal

  nginx:
    volumes:
      - /var/data/turtle-pool/nginx/website:/usr/share/nginx/html:ro
      - /etc/localtime:/etc/localtime:ro
    image: nginx
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:trtl.heigh-ho.funkypenguin.co.nz
        - traefik.docker.network=traefik_public
        - traefik.port=80

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.21.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch the Turtle! 🐢

Launch the Turtle pool stack by running docker stack deploy turtle-pool -c <path -to-docker-compose.yml>, and then run "docker stack ps turtle-pool``` to ensure the stack has come up properly. (See troubleshooting if not)

The first thing that’ll happen is that TurtleCoind will start syncing the blockchain from the bootstrap data. You can watch this happening with docker service logs turtle-pool_daemon -f. While the daemon is syncing, it won’t respond to requests, so walletd, the pool, etc will be non-functional.

You can watch the various elements of the pool doing their thing, by running tail -f /var/data/turtle-pool/pool/logs/*.log

So how do I mine to it?

That.. is another recipe. Start with the “CryptoMiner” uber-recipe for GPU/rig details, grab a copy of xmr-stack (patched for the forked TurtleCoin) from https://github.com/turtlecoin/trtl-stak/releases, and follow your nose. Jump into the TurtleCoin discord (below) #mining channel for help.

What to do if it breaks?

TurtleCoin is a baby cryptocurrency. There are scaling issues to solve, and large amounts components of this recipe are under rapid development. So, elements may break/change in time, and this recipe itself is a work-in-progress.

Jump into the TurtleCoin Discord server to ask questions, contribute, and send/receive some TRTL tips!

Chef’s Notes

Because Docker Swarm performs ingress NAT for its load-balanced “routing mesh”, the source address of inbound miner traffic is rewritten to a (common) docker node IP address. This means it’s not possible to determine the actual source IP address of a miner. Which, in turn, means that any one misconfigured miner could trigger an IP ban, and lock out all other miners for 5 minutes at a time.

Two possible solutions to this are (1) disable banning, or (2) update the pool banning code to ban based on a combination of IP address and miner wallet address. I’ll be working on a change to implement #2 if this becomes a concern.

The traefik labels in the docker-compose are to permit automatic LetsEncrypt SSL-protected proxying of your pool UI and API addresses.

After a power fault in my datacenter caused daemon DB corruption, I added a second daemon, running in parallel to the first. The failsafe daemon runs once an hour, syncs with the running daemons, and shuts down again, providing a safely halted version of the daemon DB for recovery.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Also, you could send me some :turtle: ❤️ to TRTLv2qCKYChMbU5sNkc85hzq2VcGpQidaowbnV2N6LAYrFNebMLepKKPrdif75x5hAizwfc1pX4gi5VsR9WQbjQgYcJm21zec4

Your comments? 💬

Recipies (Docker)

Now follows individual recipes.

hero: Heroic Hero

BookStack

BookStack is a simple, self-hosted, easy-to-use platform for organising and storing information.

A friendly middle ground between heavyweights like MediaWiki or Confluence and Gollum, BookStack relies on a database backend (so searching and versioning is easy), but limits itself to a pre-defined, 3-tier structure (book, chapter, page). The result is a lightweight, approachable personal documentation stack, which includes search and Markdown editing.

[picture]

BookStack Screenshot

I like to protect my public-facing web UIs with an oauth_proxy, ensuring that if an application bug (or a user misconfiguration) exposes the app to unplanned public scrutiny, I have a second layer of defense.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/bookstack:

mkdir -p /var/data/bookstack/database-dump
mkdir -p /var/data/runtime/bookstack/db

Prepare environment

Create bookstack.env, and populate with the following variables. Set the oauth_proxy variables provided by your OAuth provider (if applicable.)

# For oauth-proxy (optional)
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

# For MariaDB/MySQL database
MYSQL_RANDOM_ROOT_PASSWORD=true
MYSQL_DATABASE=bookstack
MYSQL_USER=bookstack
MYSQL_PASSWORD=secret

# Bookstack-specific variables
DB_HOST=bookstack_db:3306
DB_DATABASE=bookstack
DB_USERNAME=bookstack
DB_PASSWORD=secret

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:

  db:
    image: mariadb:10
    env_file: /var/data/config/bookstack/bookstack.env
    networks:
      - internal
    volumes:
      - /var/data/runtime/bookstack/db:/var/lib/mysql

  proxy:
    image: a5huynh/oauth2_proxy
    env_file : /var/data/config/bookstack/bookstack.env
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:bookstack.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    volumes:
      - /var/data/config/bookstack/authenticated-emails.txt:/authenticated-emails.txt
    command: |
      -cookie-secure=false
      -upstream=http://app
      -redirect-url=https://bookstack.example.com
      -http-address=http://0.0.0.0:4180
      -email-domain=example.com
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

  app:
    image: solidnerd/bookstack
    env_file: /var/data/config/bookstack/bookstack.env
    networks:
      - internal

  db-backup:
    image: mariadb:10
    env_file: /var/data/config/bookstack/bookstack.env
    volumes:
      - /var/data/bookstack/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        mysqldump -h db --all-databases | gzip -c > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.sql.gz
        (ls -t /dump/dump*.sql.gz|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.sql.gz)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - internal

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.33.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Bookstack stack

Launch the BookStack stack by running docker stack deploy bookstack -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, authenticate with oauth_proxy, and then login with username ‘admin@admin.com’ and password ‘password’.

Chef’s Notes

If you wanted to expose the BookStack UI directly, you could remove the oauth2_proxy from the design, and move the traefik_public-related labels directly to the bookstack container. You’d also need to add the traefik_public network to the bookstack container.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: We dig dig digga-dig dig!

CryptoMiner

This is a diversion from my usual recipes - recently I’ve become interested in cryptocurrency, both in mining, and in investing.

I honestly didn’t expect to enjoy the mining process as much as I did. Part of the enjoyment was getting my hands dirty with hardware.

Since a mining rig relies on hardware, we can’t really use a docker swarm for this one!

[picture]

CryptoMiner Screenshot

This recipe isn’t for everyone - if you just want to make some money from cryptocurrency, then you’re better off learning to invest or trade. However, if you want to (ideally) make money and you like tinkering, playing with hardware, optimising and monitoring, read on!

Ingredients

Suitable system guts (CPU, motherboard, RAM, PSU) for your mining rig

AMD / Nvidia GPUs (yes, plural, since although you can start with just one, you’ll soon get hooked!)

A friendly operating system (Ubuntu/Debian/CentOS7) are known to work

Patience and time

Preparation

For readability, I’ve split this recipe into multiple sub-recipes, which can be found below, or in the navigation links on the right-hand side:

Build your mining rig 💻

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges or wallets 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

Ultimately I hope to move all the configuration / mining executables into docker containers, but for now, they’re running on a CentOS7 host for direct access to GPUs. (Apparently it may be possible to pass-thru the GPUs to docker containers, but I wanted stability first, before abstracting my hardware away from my miners)

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

Mining Rig

Hardware

You can surely find a better tutorial on how to build a mining rig than this one. However, to summarise what I’ve learned:

You want a beefy power supply, with lots of PCI-e 8pin and 6pin cables.

You need 1 x PCI express (PCI-e) port per GPU

You don’t need powerful CPU or much RAM - the GPUs do all the mining work. My current guts (minus the PSU) are 5 years old.

Do I need a open-air rig?

Initially, no. You can use any old PC chassis. But as soon as you want more than one GPU, you’re going to start to run into cooling problems.

You don’t need anything fancy. Here’s a photo of the rig my wife built me:

[picture]

My mining rig, naked

I recommend this design (with the board with little holes in it) - it takes up more space, but I have more room to place extra components (PSUs, hard drives, etc), as illustrated below:

!!! note You’ll note the hard drives in the picture - that’s not part of the mining requirements, it’s because my rig doubles as my Plex server ;)

[picture]

My mining rig, populated

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻 (This page)

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges or wallets 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

Pro-tip : You’re going to spend some time overclocking. Which is going to make your mining host unstable.

Yes. It’s the ultimate #firstworldproblem, but if you have a means to remotely reboot your host, use it! You can thank me later.

(I hooked up a remote-controlled outlet to my rig, so that I can power-cycle it without having to crawl under the desk!)

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

AMD GPU

Ingredients

AMD drivers for your GPU

Linux version of “atiflash” command

A VBIOS rom compatible with your GPU model and memory manufacturer

Preparation

Install the drivers

There are links on the AMD driver download page (linked above) to drivers for RHEL/CentOS6, RHEL/CentOS7, and Ubuntu 16.04. As I write this, the latest version is amdgpu-pro-17.50-511655.

!!! note You’ll find reference online to the “blockchain” drivers. These were an earlier, beta release which have been superseded by version 17.50 and later. You can ignore these.

Uncompress the drivers package, and run the following:

./amdgpu-install --opencl=legacy --headless

If you have a newer (than my 5-year-old one!) motherboard/CPU, you can also try the following, for ROCm support (which might allow you some more software-based overclocking powers):

./amdgpu-install --opencl=legacy,rocm --headless

Reboot upon completion.

Flash the BIOS

Yes, this sounds scary, but it’s not as bad as it sounds, if you want better performance from your GPUs, you have to flash your GPU BIOS.

Why flash BIOS?

Here’s my noob-level version of why:

GPU-mining performance is all about the memory speed of your GPU - you get the best mining from the fastest internal timings. So you want to optimize your GPU to do really fast memory work, which is not how it’s designed by default.

The processor on your GPU sits almost idle, so you lower the power to the processor (undervolt) to save some power.

As it turns out, the factory memory timings of the RX5xx series were particularly poor.

As an aside, here’s an illustration re why you’d want to flash your BIOS. Below is the mining throughput of 2 AMD RX580s I purchased together. Guess which one had its BIOS flashed?

ETH: GPU0 30.115 Mh/s, GPU1 22.176 Mh/s

Here’s the power consumption of the two GPUs while doing the above test:

GPU1 (original ROM)

GFX Clocks and Power:
        1750 MHz (MCLK)
        1411 MHz (SCLK)
        144.107 W (VDDC)
        16.0 W (VDDCI)
        171.161 W (max GPU)
        172.209 W (average GPU)

GPU Temperature: 67 C
GPU Load: 100 %

GPU0 (flashed ROM)

GFX Clocks and Power:
        2050 MHz (MCLK)
        1150 MHz (SCLK)
        87.155 W (VDDC)
        16.0 W (VDDCI)
        117.152 W (max GPU)
        116.1 W (average GPU)

GPU Temperature: 62 C
GPU Load: 100 %

So, by flashing the BIOS, I gained 8 MH/s (a 36% increase), while reducing power consumption by ~40W!

How to flash AMD GPU BIOS?

Get atiflash for linux.

Identify which card you want to flash, by running ./atiflash -i

Example output below:

[root@kvm ~]# ./atiflash -i

adapter bn dn dID       asic           flash      romsize test    bios p/n
======= == == ==== =============== ============== ======= ==== ================
   0    01 00 67DF Ellesmere       M25P20/c         40000 pass 113-1E3660EU-O55
[root@kvm ~]#

Save the original, factory ROM, by running ./atiflash -s <adapter number> <filename to save>

Example below:

[root@kvm ~]# ./atiflash -s 0 rx580-4gb-299-1E366-101SA.orig.rom
0x40000 bytes saved, checksum = 0x7FBF

Now find an appropriate ROM to flash onto the card, and run ```atiflash -p

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes a range of RX580-compatible ROMs, some of which I’ve tweaked for my own GPUs. 👍

Example below:

[root@kvm ~]# ./atiflash -f -p 0 Insan1ty\ R9\ 390X\ BIOS\ v1.81/R9\ 290X/MEM\ MOD\ --\ ELPIDA/290X_ELPIDA_MOD_V1.8.rom
Old SSID: E285
New SSID: 9395
Old P/N: 113-E285FOC-U005
New P/N: 113-GRENADA_XT_C671_D5_8GB_HY_W
Old DeviceID: 67B1
New DeviceID: 67B0
Old Product Name: C67111 Hawaii PRO OC GDDR5 4GB 64Mx32 300e/150m
New Product Name: C67130 Grenada XT A0 GDDR5 8GB 128Mx32 300e/150m
Old BIOS Version: 015.044.000.011.000000
New BIOS Version: 015.049.000.000.000000
Flash type: M25P10/c
Burst size is 256
20000/20000h bytes programmed
20000/20000h bytes verified

Restart System To Complete VBIOS Update.
[root@kvm ~]#

Reboot the system, hold onto your butts, and wait for your newly-flashed GPU to fire up.

If it goes wrong

The safest way to do this is to run more than one GPU, and to flash the GPUs one-at-a-time, rebooting after each. That way, even if you make your GPU totally unresponsive, you’ll still get access to your system to flash it back to the factory ROM.

That said, it’s very unlikely that a flashed GPU won’t let you boot at all though. In the (legion) cases where I overclocked my RX580 too far, I was able choose to boot into rescue mode in CentOS7 (bypassing the framebuffer / drm initialisation), and reflash my card back to its original BIOS.

Mooar tweaking! 🔧

If you want to tweak the BIOS yourself, download the Polaris bios editor and tweak away!

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻

Setup your AMD (this page) or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges or wallets 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

My two RX580 cards (bought alongside each other) perform slightly differently. GPU0 works with a 2050Mhz memory clock, but GPU1 only works at 2000Mhz. Anything over 2000Mhz causes system instability. YMMV.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

NVidia GPU

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

Ingredients

Nvidia drivers for your GPU

Some form of X11 GUI preconfigured on your linux host (yes, it’s a PITA, but it’s necessary for overclocking)

Preparation

Install kernel-devel and gcc

The nVidia drivers will need the kernel development packages for your OS installed, as well as gcc. Run the following (for CentOS - there will be an Ubuntu equivalent):

yum install kernel-devel-$(uname -r) gcc

Remove nouveau

Your host probably already includes nouveau, free/libre drivers for Nvidia graphics card. These won’t cut it for mining, so blacklist them to avoid conflict with the dirty, proprietary Nvidia drivers:

echo 'blacklist nouveau' >> /etc/modprobe.d/blacklist.conf
dracut /boot/initramfs-$(uname -r).img $(uname -r) --force
systemctl disable gdm
reboot

Install Nvidia drivers

Download and uncompress the Nvidia drivers, and execute the installation as root, with a command something like this:

bash NVIDIA-Linux-x86_64-352.30.run

Update your X11 config by running:

nvidia-xconfig

Enable GUID

systemctl enable gdm
ln -s '/usr/lib/systemd/system/gdm.service' '/etc/systemd/system/display-manager.service'
reboot

Overclock

Preparation

!!! warning Like overclocking itself, this process is still a work in progress. YMMV.

Of course, you want to squeeze the optimal performance out of your GPU. This is where the X11 environment is required - to adjust GPU clock/memory settings, you need to use the nvidia-settings command, which (stupidly) requires an X11 display, even if you’re just using the command line.

The following command: configures X11 for a “fake” screen so that X11 will run, even on a headless machine managed by SSH only, and ensures that the PCI bus ID of every NVidia device is added to the xorg.conf file (to avoid errors about “(EE) no screens found(EE)”)

nvidia-xconfig -a --allow-empty-initial-configuration --cool-bits=28 --use-display-device="DFP-0" --connected-monitor="DFP-0" --enable-all-gpus --separate-x-screens

!!! note The script below was taken from https://github.com/Cyclenerd/ethereum_nvidia_miner

Make a directory for your overclocking script. Mine happens to be /root/overclock/, but use whatever you like.

Create settings.conf as follows:

# Known to work with Nvidia 1080ti, but probably not optimal. It's an eternal work-in-progress.
MY_WATT="200"
MY_CLOCK="100"
MY_MEM="400"
MY_FAN="60"

Then create nvidia-overclock.sh as follows:

#!/usr/bin/env bash

#
# nvidia-overclock.sh
# Author: Nils Knieling - https://github.com/Cyclenerd/ethereum_nvidia_miner
#
# Overclocking with nvidia-settings
#

# Load global settings settings.conf
if ! source ~/overclock/settings.conf; then
    echo "FAILURE: Can not load global settings 'settings.conf'"
    exit 9
fi

export DISPLAY=:0

# Graphics card 1 to 6
for MY_DEVICE in {0..5}
do
    # Check if card exists
    if nvidia-smi -i $MY_DEVICE >> /dev/null 2>&1; then
        nvidia-settings -a "[gpu:$MY_DEVICE]/GPUPowerMizerMode=1"
        # Fan speed
        nvidia-settings -a "[gpu:$MY_DEVICE]/GPUFanControlState=1"
        nvidia-settings -a "[fan:$MY_DEVICE]/GPUTargetFanSpeed=$MY_FAN"
        # Graphics clock
        nvidia-settings -a "[gpu:$MY_DEVICE]/GPUGraphicsClockOffset[3]=$MY_CLOCK"
        # Memory clock
        nvidia-settings -a "[gpu:$MY_DEVICE]/GPUMemoryTransferRateOffset[3]=$MY_MEM"
                # Set watt/powerlimit. This is also set in miner.sh at autostart.
                sudo nvidia-smi -i "$MY_DEVICE" -pl "$MY_WATT"
    fi
done

echo
echo "Done"
echo

Start your engine!

Once you’ve got X11 running correctly, execute ,/nvidia-overclock.sh, and you should see something like the following:

[root@kvm overclock]# ./nvidia-overclock.sh
  Attribute 'GPUPowerMizerMode' (kvm.funkypenguin.co.nz:0[gpu:0]) assigned value 1.
  Attribute 'GPUFanControlState' (kvm.funkypenguin.co.nz:0[gpu:0]) assigned value 1.
  Attribute 'GPUTargetFanSpeed' (kvm.funkypenguin.co.nz:0[fan:0]) assigned value 60.
  Attribute 'GPUGraphicsClockOffset' (kvm.funkypenguin.co.nz:0[gpu:0]) assigned value 100.
  Attribute 'GPUMemoryTransferRateOffset' (kvm.funkypenguin.co.nz:0[gpu:0]) assigned value 400.

Power limit for GPU 00000000:04:00.0 was set to 150.00 W from 150.00 W.
All done.

Done

[root@kvm overclock]#

Play with changing your settings.conf file until you break it, and then go back one revision :)

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻

Setup your AMD or Nvidia (this page) GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges or wallets 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

Mining Pools

You and your puny GPUs don’t have a snowball’s chance of mining a block on your own. Your only option is to join a mass mining conglomerate (a mining pool), throw in your share of the effort for a share of the reward.

Preparation

Setup accounts at mining pools

This’ll save you some frustration later… Next time you’re watching a movie or doing something mindless, visit http://whattomine.com/, and take note of the 10-15 most profitable coins for your GPU type(s).

On your exchanges, identify the “deposit address” for each popular coin, and note them down for the next step.

!!! note
If you’re wanting to mine directly to a wallet for long-term holding, then substitute your wallet public address for this deposit address.

Now work your way through the following list of pools, creating an account on each one you want as you go. In the case of each pool/coin, setup your “payout address” to match your change address for the coin (above).

Mining Pool Hub (Lots of coins)

NiceHash (Ethereum, Decred)

suprnova - Lots of coins, but, you generally need a separate login for each pool. You also need to create a worker in each pool with a common username and password, for Minerhotel.

nanopool (Ethereum, Ethereum Classic, SiaCoin, ZCash, Monero, Pascal and Electroneum)

slushpool (BTC and ZCash)

Serving

Avoid fees

As noted by IronicBadger here, the name of the game is avoiding fees where possible. Here are a few tips:

Mining Pool Hub is a popular pool for multiple coins, and it allows you (for a fee) to auto-exchange the coins you mine for coins that you actually want.

NiceHash will allow you to send your earned bitcoin (whatever you mine, they pay you in bitcoin) to coinbase for free.

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges (This page) or wallets 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

Wallet

You may be mining a particular coin, and want to hold onto it, in the hopes of long-term growth. The safest place to stick that coin, therefore, is an a wallet.

Preparation

Get your wallets

Your favorite coin probably has a link to various desktop wallets on their website. All you have to do is download the wallet to your desktop, fire it up, and then backup your public/private key somewhere safe.

Wallets I use

I mine most of my coins to Exchanges, but I do have the following wallets:

Jaxx on my iPhone for popular coins (BTC, ETH, etc)

Eleos wallet for ZClassic

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges or wallets (This page) 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

Exchange

You may be mining a particular coin, and want to hold onto it, in the hopes of long-term growth. In that case, stick it in a wallet and be done with it.

You may also not care too much about the coin (you’re mining for money, right?), in which case you want to “cash out” your coins into something you can spend.

In this case, you’ll want to configure your mining pool to send your coin-of-choice to an exchange, so that you can turn it into a different coin, or extract it into FIAT (oldschool, cave-man currency).

Preparation

Get verified at exchanges

Most exchanges (Binance is currently a notable exception) require some sort of verification of your ID before they’ll let you trade, or withdraw coins as FIAT. So, you may as well get yourself verified in anticipation (it can take a while during periods of increased crypto-hype).

Here are (referral) links to exchanges I’ve used personally:

Cryptopia : Trades obscure altcoins that other exchanges don’t, and can withdraw to USD and NZD

Binance : Doesn’t require verification for small-time traders/miners

Coinbase : Beginner’s exchange. Coins mined in Nicehash can be sent to coinbase with zero fees.

Send coins to Exchanges

Now simply configure your mining pool (or your miner) to send your coins to your wallet’s deposit address for each coin. Note that every coin has a unique wallet address.

!!! warning Don’t try to send one coin (i.e., LTC) to a different coin’s (i.e. BTC) wallet. You will loose your money and be sad 😞

Withdraw coins to FIAT (cash)

Once you have enough coins in your exchange wallet, you can “trade” them into the real-world currency of your choice. For example, if you mined 100 Ella to Cryptopia, you could trade it for NZDT or USDT, and withdraw it to your bank account.

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges (This page) or wallets 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Minerhotel

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

So, you have GPUs. You can mine cryptocurrency. But what cryptocurrency should you mine?

You could manually keep track of whattomine, and launch/stop miners based on profitability/convenience, as you see fit.

You can automate the process of mining the most profitable coin based on your GPUs’ capabilities and the current market prices, and do better things with your free time! (receiving alerts, of course, if anything stops working!)

This recipe covers option #2 😁

Miner hotel is a collection of scripts and config files to fully automate your mining across AMD or Nvidia cards.

Ingredients

Latest Minerhotel release for Linux

Time and patience

Preparation

Unpack Minerhotel

Unpack the minerhotel release. You can technically unpack it anywhere, but this guide, and all pre-configured miners, expect an installation at /opt/minerhotel.

Prepare miner.config

Copy /opt/minerhotel/miner.config.example to /opt/minerhotel/miner.config, and start making changes. Here’s a rundown of the variables:

WALLET<WHATEVER> : Set these WALLET variables to your wallet addresses for all the currencies you want to mine. Your miner will fail to start without the wallet variable, but it won’t confirm it’s a valid wallet. Now, double-check to confirm the wallet is correct, and you’re not just mining coins to /dev/null, or someone else’s wallet! You can either use your exchange wallet address or your own wallet.

WORKER : Set this to the name you’ll use to define your miner in the various pools you mine. Some pools (i.e. NiceHash) auto-create workers based on whatever worked name you specify, whereas others (Supernova.cc) will refuse to authenticate you unless you’ve manually created the worker/password in their UI first.

SUPRUSER : Set this to your supernova.cc login username (not your worker name) (optional, only use this if you want to use supernova.cc)

SUPRPASS : Set this to the password you’ve configured within Supernova.cc for your worker as defined by the WORKER variable. Note that this require syou to use the same worker name and password across all your supernova.cc pools (optional, only necessary if you want to use supernova.cc)

MPHUSER : Set this to your miningpoolhub login username (optional, only necessary if you want to use miningpoolhub.com)

TBFUSER : Set this to your theblocksfactory login username (optional, only necessary if you want to use theblocksfactory.com)

VERTPOOLUSER/VERTPOOLPASS : Set these to your vertpool user/password (optional, only necessary if you want to use vertpool.org)

Install services

Run /opt/minerhotel/scripts/install-services.sh to install the necessary services for systemd

Run /opt/minerhotel/scripts/fixmods.sh to correctly set the filesystem permissions for the various miner executables

!!! note fixmods.sh doesn’t correctly set permissions on subdirectories, so until this is fixed, you also need to run chmod 755 /opt/minerhotel/bin/claymore/ethdcrminer64

Setup whattomine-linux

For the whattomine bot to select the most profitable coin to mine for your GPUs, you’ll need to feed your cookie from https://whattomine.com

Start by installing this addon for Chrome, or this addon for firefox

Then visit http://whattomine.com/ and tweak settings for you GPUs, power costs, etc.

Grab the cookie per the whattomine README, and paste it (about 2200 characters) into /opt/minerhotel/whattomine/config.json

Ensure that only the coins/miners that you want are enabled in config.json - delete the others, or put a dash (“-”) after the ones you want to disable. Set the service names as defined in /opt/minerhotel/services/

Test miners

Before trusting the whattomine service to automatically launch your miners, test each one first by starting them manually, and then checking their status.

For example, to test the miner-amd-eth-ethhash-ethermine miner, run

systemctl start miner-amd-eth-ethhash-ethermine.service to start the service

And then watch the output by running journalctl -u miner-amd-eth-ethhash-ethermine -f

When you’re satisfied it’s working correctly (without errors and with a decent hashrate), stop the miner again by running systemctl stop miner-amd-eth-ethhash-ethermine, and move onto testing the next one.

Serving

Launch whattomine

Finally, run systemctl start minerhotel-whattomine and then journalctl -u minerhotel-whattomine -f to watch the output. Within a minute, you should see whattomime launching the most profitable miner, as illustrated below:

Jan 29 13:49:38 kvm.funkypenguin.co.nz whattomine-linux[2057]: 2018-01-29T13:49:38+1300 <INF> whattomine.js Loading whattominebot
Jan 29 13:49:38 kvm.funkypenguin.co.nz whattomine-linux[2057]: 2018-01-29T13:49:38+1300 <INF> whattomine.js Starting whattominebot now.
Jan 29 13:50:45 kvm.funkypenguin.co.nz whattomine-linux[2057]: 2018-01-29T13:50:45+1300 <INF> whattomine.js Mining Ethereum|ETH|Ethash|0.0089|0.00093|100
Jan 29 13:50:45 kvm.funkypenguin.co.nz whattomine-linux[2057]: 2018-01-29T13:50:45+1300 <INF> whattomine.js Could not find a miner for Ubiq.
Jan 29 13:51:39 kvm.funkypenguin.co.nz whattomine-linux[2057]: 2018-01-29T13:51:39+1300 <INF> whattomine.js Mining Ethereum|ETH|Ethash|0.0089|0.00094|100
Jan 29 13:51:39 kvm.funkypenguin.co.nz whattomine-linux[2057]: 2018-01-29T13:51:39+1300 <INF> whattomine.js Could not find a miner for Ubiq.

!!! note The messages about “Could not find miner” can be ignored, they indicate that one of the preferred coins on whattomine does not have a miner defined.

To make whattomine start automatically in future, run systemctl enable minerhotel-whattomine

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨 (This page)

Send your coins to exchanges or wallets 💹

Monitor your empire :heartbeat:

Profit! 💰

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Monitor

!!! warning This is not a complete recipe - it’s a component of the cryptominer “uber-recipe”, but has been split into its own page to reduce complexity.

So, you’re a miner! But if you’re not actively mining, are you still a miner? This page details how to measure your mining activity, and how to raise an alert when a profit-affecting issue affects your miners.

Ingredients

InfluxDB+Grafana instance, for visualising data

Icinga, Nagios etc for alarming on GPU/miner status

Asi MPM (iOS) for monitoring your miner/pool status

Altpocket, CoinTracking, etc for managing your crypto-asset portfolio (referral links)

Preparation

Visualising performance

[picture]

Visualise mining performance

Since Minerhotel switches currency based on what’s most profitable in the moment, it’s hard to gauge the impact of changes (overclocking, tweaking, mining pools) over time.

I hacked up a bash script which grabs performance data from the output of the miners, and throws it into an InfluxDB database, which can then be visualized using Grafana.

Here’s an early version of the script (it’s since been updated for clockspeed and power usage too):

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes up-to-date versions of the InfluxDB /Grafana script mentioned above, as well as pre-setup Grafana graphs, so that patrons can simply “git pull” and start monitoring 👍

Alarming on failure

[picture]

Visualise mining performance

GPU mining can fail in subtle ways. On occasion, I’ve tweaked my GPUs to the point that the miner will start, but one or all GPUs will report a zero hash rate. I wanted to be alerted to such profit-affecting issues, so I wrote a bash script (intended to be executed by NRPE from Icinga, Nagios, etc).

The script tests the output of the currently active miner, and ensures the GPUs have a valid hashrate.

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes up-to-date versions of the Icinga scripts mentioned above, so that patrons can simply “git pull” and start monitoring 👍

Monitoring pool/miner status

I’ve tried several iOS apps for monitoring my performance across various. The most useful app I’ve found thus far is Asi MPM. It requires a little upfront effort to configure for all your coins/pools, but thereafter it’s a handy way to keep tabs on your new obsession!

Track your portfolio

Now that you’ve got your coins happily cha-chinging into you wallets (and potentially various exchanges), you’ll want to monitor the performance of your portfolio over time.

Web Apps

There’s a detailed breakdown of porfolio-management apps here.

Personally, I use:

Altpocket (A free web app which can auto-sync with certain exchanges and wallets)

CoinTracking - The top crypto-portfolio manager, by far. But it’s expensive when you get to > 200 trades. You get what you pay for ;)

Mobile Apps

I’ve found the following iOS apps to be useful in tracking my portfolio (really more for investing than mining though, since portfolio tracking requires a manual entry for each trade)

Delta (iOS) - Track your portfolio (losses/gains) and alert you to changes in the coins you watch

Bitscreener )(iOS) - Track multiple currencies on a watchlist, and quickly view news/discussion per coin

!!! note Some of the links above are referral links. I get some goodies when you use them.

Continue your adventure

Now, continue to the next stage of your grand mining adventure:

Build your mining rig 💻

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges or wallets 💹

Monitor your empire :heartbeat: (this page)

Profit! 💰

Chef’s Notes

Ultimately I hope to move all the configuration / mining executables into docker containers, but for now, they’re running on a CentOS7 host for direct access to GPUs. (Apparently it may be possible to pass-thru the GPUs to docker containers, but I wanted stability first, before abstracting my hardware away from my miners)

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Profit! 💰

Well, that’s it really. You’re a cryptominer. Welcome to the party.

Your adventure has only just begun!

To recap, you did all this:

Build your mining rig 💻

Setup your AMD or Nvidia GPUs 🎨

Sign up for mining pools :swimmer:

Setup your miners with Miner Hotel 🏨

Send your coins to exchanges or wallets 💹

Monitor your empire :heartbeat:

Profit! (This page) 💰

What next?

Get in touch and share your experience - there’s a special discord channel if you’re the IM type, else post a comment/thread at the kitchen :)

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Manage your ebook collection. Like a BOSS.

Calibre-Web

The AutoPirate recipe includes Lazy Librarian, a tool for tracking, finding, and downloading eBooks. However, after the eBooks are downloaded, Lazy Librarian is not much use for organising, tracking, and actually reading them.

Calibre-Web could be described as “Plex (or Emby) for eBooks” - it’s a web-based interface to manage your eBook library, screenshot below:

[picture]

Calibre-Web Screenshot

Of course, you probably already manage your eBooks using the excellent Calibre, but this is primarily a (powerful) desktop application. Calibre-Web is an alternative way to manage / view your existing Calibre database, meaning you can continue to use Calibre on your desktop if you wish.

As a long-time Kindle user, Calibre-Web brings (among others) the following features which appeal to me:

Filter and search by titles, authors, tags, series and language

Create custom book collection (shelves) Support for editing eBook metadata and deleting eBooks from Calibre library

Support for converting eBooks from EPUB to Kindle format (mobi/azw)

Send eBooks to Kindle devices with the click of a button

Support for reading eBooks directly in the browser (.txt, .epub, .pdf, .cbr, .cbt, .cbz)

Upload new books in PDF, epub, fb2 format

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need a directory to store some config data for Calibre-Web, container, so create /var/data/calibre-web, and ensure the directory is owned by the same use which owns your Calibre data (below)

mkdir /var/data/calibre-web
chown calibre:calibre /var/data/calibre-web # for example

Ensure that your Calibre library is accessible to the swarm (i.e., exists on shared storage), and that the same user who owns the config directory above, also owns the actual calibre library data (including the ebooks managed by Calibre).

Prepare environment

We’ll use an oauth-proxy to protect the UI from public access, so create calibre-web.env, and populate with the following variables:

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=<make this a random string>
PUID=
PGID=

Follow the instructions to setup your oauth provider. You need to setup a unique key/secret for each instance of the proxy you want to run, since in each case the callback URL will differ.

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  app:
    image: technosoft2000/calibre-web
    env_file : /var/data/config/calibre-web/calibre-web.env
    volumes:
     - /var/data/calibre-web:/config
     - /srv/data/Archive/Ebooks/calibre:/books
    networks:
    - internal

  proxy:
    image: a5huynh/oauth2_proxy
    env_file : /var/data/config/calibre-web/calibre-web.env
    dns_search: hq.example.com
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:calibre-web.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    volumes:
      - /var/data/config/calibre-web/authenticated-emails.txt:/authenticated-emails.txt
    command: |
      -cookie-secure=false
      -upstream=http://app:8083
      -redirect-url=https://calibre-web.example.com
      -http-address=http://0.0.0.0:4180
      -email-domain=example.com
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.18.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Calibre-Web

Launch the Calibre-Web stack by running docker stack deploy calibre-web -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN. You’ll be directed to the initial GUI configuraition. Set the first field (Location of Calibre database) to “/books/”, and when complete, login using defaults username of “admin” with password “admin123”.

Chef’s Notes

Yes, Calibre does provide a server component. But it’s not as fully-featured as Calibre-Web (i.e., you can’t use it to send ebooks directly to your Kindle)

A future enhancement might be integrating this recipe with the filestore for NextCloud, so that the desktop database (Calibre) can be kept synced with Calibre-Web.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Collabora Online

!!! important Development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

Collabora Online Development Edition (or “CODE”), is the lightweight, or “home” edition of the commercially-supported Collabora Online platform. It

It’s basically the LibreOffice interface in a web-browser. CODE is not a standalone app, it’s a backend intended to be accessed via “WOPI” from an existing interface (in our case, NextCloud)

[picture]

CODE Screenshot

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname (i.e. “collabora.your-domain.com”) you intend to use for LDAP Account Manager, pointed to your keepalived IP

NextCloud installed and operational

Docker-compose installed on your node(s) - this is a special case which needs to run outside of Docker Swarm

Preparation

Explanation for complexity

Due to the clever magic that Collabora does to present a “headless” LibreOffice UI to the browser, the CODE docker container requires system capabilities which cannot be granted under Docker Swarm (specifically, MKNOD).

So we have to run Collabora itself in the next best thing to Docker swarm - a docker-compose stack. Using docker-compose will at least provide us with consistent and version-able configuration files.

This presents another problem though - Docker Swarm with Traefik is superb at making all our stacks “just work” with ingress routing and LetsEncyrpt certificates. We don’t want to have to do this manually (like a cave-man), so we engage in some trickery to allow us to still use our swarmed Traefik to terminate SSL.

We run a single swarmed Nginx instance, which forwards all requests to an upstream, with the target IP of the docker0 interface, on port 9980 (the port exposed by the CODE container)

We attach the necessary labels to the Nginx container to instruct Trafeik to setup a front/backend for collabora.<ourdomain>. Now incoming requests to https://collabora.<ourdomain> will hit Traefik, be forwarded to nginx (wherever in the swarm it’s running), and then to port 9980 on the same node that nginx is running on.

What if we’re running multiple nodes in our swarm, and nginx ends up on a different node to the one running Collabora via docker-compose? Well, either constrain nginx to the same node as Collabora (example below), or just launch an instance of Collabora on every node then. It’s just a rendering / GUI engine after all, it doesn’t hold any persistent data.

Here’s a (highly technical) diagram to illustrate:

[picture]

CODE traffic flow

Setup data locations

We’ll need a directory for holding config to bind-mount into our containers, so create /var/data/collabora, and /var/data/config/collabora for holding the docker/swarm config

mkdir /var/data/collabora/
mkdir /var/data/config/collabora/

Prepare environment

Create /var/data/config/collabora/collabora.env, and populate with the following variables, customized for your installation.

!!! warning Note the following:

1. Variables are in lower-case, unlike our standard convention. This is to align with the CODE container
2. Set domain to your [NextCloud](/recipes/nextcloud/) domain, and escape all the periods as per the example
3. Set your server_name to collabora.<yourdomain\>. Escaping periods is unnecessary
4. Your password cannot include triangular brackets - the entrypoint script will insert this password into an XML document, and triangular brackets will make bad(tm) things happen 🔥

username=admin
password=ilovemypassword
domain=nextcloud\.batcave\.com
server_name=collabora.batcave.com
termination=true

Create docker-compose.yml

Create /var/data/config/collabora/docker-compose.yml as follows:

version: "3.0"

services:
  local-collabora:
    image: funkypenguin/collabora
    # the funkypenguin version has a patch to include "termination" behind SSL-terminating reverse proxy (traefik), see CODE PR #50.
    # Once merged, the official container can be used again.
    #image: collabora/code
    env_file: /var/data/config/collabora/collabora.env
    volumes:
      - /var/data/collabora/loolwsd.xml:/etc/loolwsd/loolwsd.xml-new
    cap_add:
      - MKNOD
    ports:
      - 9980:9980

Create nginx.conf

Create /var/data/config/collabora/nginx.conf as follows, changing the server_name value to match the environment variable you established above.

upstream collabora-upstream {
    # Run collabora under docker-compose, since it needs MKNOD cap, which can't be provided by Docker Swarm.
    # The IP here is the typical IP of docker0 - change if yours is different.
    server 172.17.0.1:9980;
}

server {
    listen 80;
    server_name collabora.batcave.com;

    # static files
    location ^~ /loleaflet {
        proxy_pass http://collabora-upstream;
        proxy_set_header Host $http_host;
    }

    # WOPI discovery URL
    location ^~ /hosting/discovery {
        proxy_pass http://collabora-upstream;
        proxy_set_header Host $http_host;
    }

    # Main websocket
    location ~ /lool/(.*)/ws$ {
        proxy_pass http://collabora-upstream;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_read_timeout 36000s;
    }

    # Admin Console websocket
    location ^~ /lool/adminws {
    proxy_buffering off;
        proxy_pass http://collabora-upstream;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_read_timeout 36000s;
    }

    # download, presentation and image upload
    location ~ /lool {
        proxy_pass https://collabora-upstream;
        proxy_set_header Host $http_host;
    }
}

Create loolwsd.xml

Until we understand how to pass trusted network parameters to the entrypoint script using environment variables, we have to maintain a manually edited version of loolwsd.xml, and bind-mount it into our collabora container.

The way we do this is we mount /var/data/collabora/loolwsd.xml as /etc/loolwsd/loolwsd.xml-new, then allow the container to create its default /etc/loolwsd/loolwsd.xml, copy this default over our /var/data/collabora/loolwsd.xml as /etc/loolwsd/loolwsd.xml-new, and then update the container to use our /var/data/collabora/loolwsd.xml as /etc/loolwsd/loolwsd.xml instead (confused yet?)

Create an empty /var/data/collabora/loolwsd.xml by running touch /var/data/collabora/loolwsd.xml. We’ll populate this in the next section…

Setup Docker Swarm

Create /var/data/config/collabora/collabora.yml as follows, changing the traefik frontend_rule as necessary:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3.0"

services:

  nginx:
    image: nginx:latest
    networks:
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:collabora.batcave.com
        - traefik.docker.network=traefik_public
        - traefik.port=80
        - traefik.frontend.passHostHeader=true
        # uncomment this line if you want to force nginx to always run on one node (i.e., the one running collabora)
      #placement:
      #  constraints:
      #     - node.hostname == ds1
    volumes:
      - /var/data/collabora/nginx.conf:/etc/nginx/conf.d/default.conf:ro

networks:
  traefik_public:
    external: true

Serving

Generate loolwsd.xml

Well. This is awkward. There’s no documented way to make Collabora work with Docker Swarm, so we’re doing a bit of a hack here, until I understand how to pass these arguments via environment variables.

Launching Collabora is (for now) a 2-step process. First.. we launch collabora itself, by running:

cd /var/data/config/collabora/
docker-compose -d up

Output looks something like this:

root@ds1:/var/data/config/collabora# docker-compose up -d
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

Pulling local-collabora (funkypenguin/collabora:latest)...
latest: Pulling from funkypenguin/collabora
7b8b6451c85f: Pull complete
ab4d1096d9ba: Pull complete
e6797d1788ac: Pull complete
e25c5c290bde: Pull complete
4b8e1b074e06: Pull complete
f51a3d1fb75e: Pull complete
8b826e2ae5ad: Pull complete
Digest: sha256:6cd38cb5cbd170da0e3f0af85cecf07a6bc366e44555c236f81d5b433421a39d
Status: Downloaded newer image for funkypenguin/collabora:latest
Creating collabora_local-collabora_1 ...
Creating collabora_local-collabora_1 ... done
root@ds1:/var/data/config/collabora#

Now exec into the container (from another shell session), by running exec <container name> -it /bin/bash. Make a copy of /etc/loolwsd/loolwsd, by running cp /etc/loolwsd/loolwsd.xml /etc/loolwsd/loolwsd.xml-new, and then exit the container with exit.

Delete the collabora container by hitting CTRL-C in the docker-compose shell, running docker-compose rm, and then altering this line in docker-compose.yml:

      - /var/data/collabora/loolwsd.xml:/etc/loolwsd/loolwsd.xml-new

To this:

      - /var/data/collabora/loolwsd.xml:/etc/loolwsd/loolwsd.xml

Edit /var/data/collabora/loolwsd.xml, find the storage.filesystem.wopi section, and add lines like this to the existing allow rules (to allow IPv6-enabled hosts to still connect with their IPv4 addreses):

<host desc="Regex pattern of hostname to allow or deny." allow="true">::ffff:10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="Regex pattern of hostname to allow or deny." allow="true">::ffff:172\.1[6789]\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="Regex pattern of hostname to allow or deny." allow="true">::ffff:172\.2[0-9]\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="Regex pattern of hostname to allow or deny." allow="true">::ffff:172\.3[01]\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="Regex pattern of hostname to allow or deny." allow="true">::ffff:192\.168\.[0-9]{1,3}\.[0-9]{1,3}</host>

Find the net.post_allow section, and add a line like this:

<host desc="RFC1918 private addressing in inet6 format">::ffff:10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="RFC1918 private addressing in inet6 format">::ffff:172\.1[6789]\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="RFC1918 private addressing in inet6 format">::ffff:172\.2[0-9]\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="RFC1918 private addressing in inet6 format">::ffff:172\.3[01]\.[0-9]{1,3}\.[0-9]{1,3}</host>
<host desc="RFC1918 private addressing in inet6 format">::ffff:192\.168\.[0-9]{1,3}\.[0-9]{1,3}</host>

Find these 2 lines:

<ssl desc="SSL settings">
    <enable type="bool" default="true">true</enable>

And change to:

<ssl desc="SSL settings">
    <enable type="bool" default="true">false</enable>

Now re-launch collabora (with the correct with loolwsd.xml) under docker-compose, by running:

docker-compose -d up

Once collabora is up, we launch the swarm stack, by running:

docker stack deploy collabora -c /var/data/config/collabora/collabora.yml

Visit https://collabora.<yourdomain>/l/loleaflet/dist/admin/admin.html and confirm you can login with the user/password you specified in collabora.env

Integrate into NextCloud

In NextCloud, Install the Collabora Online app (https://apps.nextcloud.com/apps/richdocuments), and then under Settings -> Collabora Online, set your Collabora Online Server to https://collabora.<your domain>

[picture]

CODE Screenshot

Now browse your NextCloud files. Click the plus (+) sign to create a new document, and create either a new document, spreadsheet, or presentation. Name your document and then click on it. If Collabora is setup correctly, you’ll shortly enter into the rich editing interface provided by Collabora :)

!!! important Development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

Chef’s Notes

Yes, this recipe is complicated. And you probably only care if you feel strongly about using Open Source rich document editing in the browser, vs using something like Google Docs. It works impressively well however, once it works. I hope to make this recipe simpler once the CODE developers have documented how to pass optional parameters as environment variables.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Ghost - A recipe for beautiful online publication.

Ghost

Ghost is “a fully open source, hackable platform for building and running a modern online publication.”

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Preparation

Setup data locations

Create the location for the bind-mount of the application data, so that it’s persistent:

mkdir -p /var/data/ghost

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  ghost:
    image: ghost:1-alpine
    volumes:
     - /etc/localtime:/etc/localtime:ro
     - /var/data/ghost/:/var/lib/ghost/content
    networks:
    - traefik
    deploy:
      labels:
        - traefik.frontend.rule=Host:ghost.example.com
        - traefik.docker.network=traefik
        - traefik.port=2368

networks:
  traefik:
    external: true

Serving

Launch Ghost stack

Launch the Ghost stack by running docker stack deploy ghost -c <path -to-docker-compose.yml>

Create your first administrative account at https://YOUR-FQDN/admin/

Chef’s Notes

If I wasn’t committed to a static-site-generated blog, Ghost is the platform I’d use for my blog.

A default using the SQlite database takes 548k of space:

[root@ds1 ghost]# du -sh /var/data/ghost/
548K    /var/data/ghost/
[root@ds1 ghost]#

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Gitlab - A recipe for a self-hosted GitHub alternative

GitLab

GitLab is a self-hosted alternative to GitHub. The most common use case is (a set of) developers with the desire for the rich feature-set of GitHub, but with unlimited private repositories.

Docker does maintain an official “Omnibus” container, but for this recipe I prefer the “dockerized gitlab” project, since it allows distribution of the various Gitlab components across multiple swarm nodes.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/gitlab:

cd /var/data
mkdir gitlab
cd gitlab
mkdir -p {postgresql,redis,gitlab}

Prepare environment

You’ll need to know the following:

Choose a password for postgresql, you’ll need it for DB_PASS in the compose file (below)

Generate 3 passwords using pwgen -Bsv1 64. You’ll use these for the XXX_KEY_BASE environment variables below

Create gitlab.env, and populate with at least the following variables (the full set is available at https://github.com/sameersbn/docker-gitlab#available-configuration-parameters):

DB_USER=gitlab
DB_PASS=gitlabdbpass
DB_NAME=gitlabhq_production
DB_EXTENSION=pg_trgm
DB_ADAPTER=postgresql
DB_HOST=postgresql
TZ=Pacific/Auckland
REDIS_HOST=redis
REDIS_PORT=6379
GITLAB_TIMEZONE=Auckland
GITLAB_HTTPS=true
SSL_SELF_SIGNED=false
GITLAB_HOST=gitlab.example.com
GITLAB_PORT=443
GITLAB_SSH_PORT=2222
GITLAB_SECRETS_DB_KEY_BASE=CFf7sS3kV2nGXBtMHDsTcjkRX8PWLlKTPJMc3lRc6GCzJDdVljZ85NkkzJ8mZbM5
GITLAB_SECRETS_SECRET_KEY_BASE=h2LBVffktDgb6BxM3B97mDSjhnSNwLc5VL2Hqzq9cdrvBtVw48WSp5wKj5HZrJM5
GITLAB_SECRETS_OTP_KEY_BASE=t9LPjnLzbkJ7Nt6LZJj6hptdpgG58MPJPwnMMMDdx27KSwLWHDrz9bMWXQMjq5mp
GITLAB_ROOT_PASSWORD=changeme

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  redis:
    image: sameersbn/redis:latest
    command:
    - --loglevel warning
    volumes:
    - /var/data/gitlab/redis:/var/lib/redis:Z
    networks:
    - internal

  postgresql:
    image: sameersbn/postgresql:9.6-2
    env_file: /var/data/config/gitlab/gitlab.env
    volumes:
    - /var/data/gitlab/postgresql:/var/lib/postgresql:Z
    networks:
    - internal

  gitlab:
    image: sameersbn/gitlab:latest
    env_file: /var/data/config/gitlab/gitlab.env
    networks:
    - internal
    - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:gitlab.example.com
        - traefik.docker.network=traefik
        - traefik.port=80
      restart_policy:
        delay: 10s
        max_attempts: 10
        window: 60s
    ports:
    - "2222:22"
    volumes:
    - /var/data/gitlab/gitlab:/home/git/data:Z

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.2.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch gitlab

Launch the mail server stack by running docker stack deploy gitlab -c <path -to-docker-compose.yml>

Log into your new instance at https://[your FQDN], with user “root” and the password you specified in gitlab.env.

Chef’s Notes

A few comments on decisions taken in this design:

I use the sameersbn/gitlab:latest image, rather than a specific version. This lets me execute updates simply by redeploying the stack (and why wouldn’t I want the latest version?)

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Gitlab Runner

Some features of GitLab require a “runner” (in the sense of a “gopher” or a “minion”). A runner “registers” itself with a GitLab instance, and is given tasks to run. Tasks include running Continuous Integration (CI) builds, and building container images.

While a runner isn’t strictly required to use GitLab, if you want to do CI, you’ll need at least one. There are many ways to deploy a runner - this recipe focuses on the docker container model.

Ingredients

Docker swarm cluster with persistent shared storage

GitLab installation (see previous recipe)

Preparation

Setup data locations

We’ll need several directories to bind-mount into our runner containers, so create them in /var/data/gitlab:

cd /var/data
mkdir gitlab
cd gitlab
mkdir -p {runners/1,runners/2}

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  1:
    image: gitlab/gitlab-runner
    volumes:
    - /var/data/gitlab-runner/1:/var/data/gitlab/runners/1
    networks:
    - internal

  2:
    image: gitlab/gitlab-runner
    volumes:
    - /var/data/gitlab-runner/:/var/data/gitlab/runners/2
    networks:
    - internal

networks:
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.23.0/24

Configure runners

From your GitLab UI, you can retrieve a “token” necessary to register a new runner. To register the runner, you can either create config.toml in each runner’s bind-mounted folder (example below), or just “docker exec” into each runner container and execute gitlab-container register to interactively generate config.toml.

Sample runner config.toml:

concurrent = 1
check_interval = 0

[[runners]]
  name = "myrunner1"
  url = "https://gitlab.example.com"
  token = "<long string here>"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "ruby:2.1"
    privileged = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]

Serving

Launch runners

Launch the mail server stack by running docker stack deploy gitlab-runner -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user “root” and the password you specified in gitlab.env.

Chef’s Notes

You’ll note that I setup 2 runners. One is locked to a single project (this cookbook build), and the other is a shared runner. I wanted to ensure that one runner was always available to run CI for this project, even if I’d tied up another runner on something heavy-duty, like a container build. Customize this to your use case.

Originally I deployed runners in the same stack as GitLab, but I found that they would frequently fail to start properly when I launched the stack. I think that this was because the runners started so quickly (and GitLab starts so slowly!), that they always started up reporting that the GitLab instance was invalid or unavailable. I had issues with CI builds stuck permanently in a “pending” state, which were only resolved by restarting the runner. Having the runners deployed in a separate stack to GitLab avoids this problem.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Gollum - A recipe for your own git-based wiki

Gollum

Gollum is a simple wiki system built on top of Git. A Gollum Wiki is simply a git repository (either bare or regular) of a specific nature:

A Gollum repository’s contents are human-editable, unless the repository is bare.

Pages are unique text files which may be organized into directories any way you choose.

Other content can also be included, for example images, PDFs and headers/footers for your pages.

Gollum pages:

May be written in a variety of markups.

Can be edited with your favourite system editor or IDE (changes will be visible after committing) or with the built-in web interface.

Can be displayed in all versions (commits).

[picture]

Gollum Screenshot

As you’ll note in the (real world) screenshot above, my requirements for a personal wiki are:

Portable across my devices

Supports images

Full-text search

Supports inter-note links

Revision control

Gollum meets all these requirements, and as an added bonus, is extremely fast and lightweight.

!!! note Since Gollum itself offers no user authentication, this design secures gollum behind an oauth2 proxy, so that in order to gain access to the Gollum UI at all, oauth2 authentication (to GitHub, GitLab, Google, etc) must have already occurred.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Preparation

Setup data locations

We’ll need an empty git repository in /var/data/gollum for our data:

mkdir /var/data/gollum
cd /var/data/gollum
git init

Prepare environment

Choose an oauth provider, and obtain a client ID and secret

Create gollum.env, and populate with the following variables (you can make the cookie secret whatever you like)

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  app:
    image: dakue/gollum
    volumes:
     - /var/data/gollum:/gollum
    networks:
    - internal
    command: |
      --allow-uploads
      --emoji
      --user-icons gravatar

  proxy:
    image: a5huynh/oauth2_proxy
    env_file : /var/data/config/gollum/gollum.env
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:gollum.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    volumes:
      - /var/data/config/gollum/authenticated-emails.txt:/authenticated-emails.txt
    command: |
      -cookie-secure=false
      -upstream=http://app:4567
      -redirect-url=https://gollum.example.com
      -http-address=http://0.0.0.0:4180
      -email-domain=example.com
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.9.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Gollum stack

Launch the Gollum stack by running docker stack deploy gollum -c <path-to-docker-compose.yml>

Authenticate against your OAuth provider, and then start editing your wiki!

Chef’s Notes

In the current implementation, Gollum is a “single user” tool only. The contents of the wiki are saved as markdown files under /var/data/gollum, and all the git commits are currently “Anonymous”

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Automate your social life! (what could go wrong?)

InstaPy

InstaPy is an Instagram bot, developed by Tim Grossman. Tim describes his motivation and experiences developing the bot here.

What’s an Instagram bot? Basically, you feed the bot your Instagram user/password, and it executes follows/unfollows/likes/comments on your behalf based on rules you set. (I set my bot to like one photo tagged with “#penguin” per-run)

[picture]

InstaPy Screenshot

Great power, right? A client (yes, you can hire me!) asked me to integrate InstaPy into their swarm, and this recipe is the result.

Ingredients

Docker swarm cluster with persistent shared storage

Preparation

Setup data locations

We need a data location to store InstaPy’s config, as well as its log files. Create /var/data/instapy per below

mkdir -p /var/data/instapy/logs

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'
services:
  web:
    command: ["./wait-for", "selenium:4444", "--", "python", "docker_quickstart.py"]
    environment:
      - PYTHONUNBUFFERED=0

    # Modify the image to whatever Tim's image tag ends up as. I used funkypenguin/instapy for my build
    image: funkypenguin/instapy:latest

    # When using swarm, you can't use relative paths, so the following needs to be set to the full filesystem path to your logs and docker_quickstart.py
    # Bind-mount docker_quickstart.py, since now that we're using a public image, we can't "bake" our credentials into the image anymore
    volumes:
      - /var/data/instapy/logs:/code/logs
      - var/data/instapy/instapy.py:/code/docker_quickstart.py:ro

    # This section allows docker to restart the container when it exits (either normally or abnormally), which ensures that
    # InstaPy keeps re-running. Tweak the delay to avoid being banned for excessive activity
    deploy:
      restart_policy:
        condition: any
        delay: 3600s

  selenium:
    image: selenium/standalone-chrome-debug
    ports:
      - "5900:5900"

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Command your bot

Create a variation of https://github.com/timgrossmann/InstaPy/blob/master/docker_quickstart.py at /var/data/instapy/instapy.py (the file we bind-mounted in the swarm config above)

Change at least the following:

insta_username = ''
insta_password = ''

Here’s an example of my config, set to like a single penguin-pic per run:

insta_username = 'funkypenguin'
insta_password = 'followmemypersonalbrandisawesome'

dont_like = ['food','girl','batman','gotham','dead','nsfw','porn','slut','baby','tv','athlete','nhl','hockey','estate','music','band','clothes']
friend_list = ['therock','ruinporn']

# If you want to enter your Instagram Credentials directly just enter
# username=<your-username-here> and password=<your-password> into InstaPy
# e.g like so InstaPy(username="instagram", password="test1234")

bot = InstaPy(username='insta_username', password='insta_password', selenium_local_session=False)
bot.set_selenium_remote_session(selenium_url='http://selenium:4444/wd/hub')
bot.login()
bot.set_upper_follower_count(limit=10000)
bot.set_lower_follower_count(limit=10)
bot.set_comments([u'Cool :penguin:!', u'Awesome :penguin:!!', u'Nice :penguin:!!'])
bot.set_dont_include(friend_list)
bot.set_dont_like(dont_like)
#bot.set_ignore_if_contains(ignore_words)

# OK, so go through my feed and like stuff, interacting with people I follow
# bot.like_by_feed(amount=3, randomize=True, unfollow=True, interact=True)

# Now find posts tagged as #penguin, and like 'em, commenting 50% of the time
bot.set_do_comment(True, percentage=50)
bot.set_comments([u'Cool :penguin:!', u'Awesome :penguin:!!', u'Nice :penguin:!!'])
bot.like_by_tags(['#penguin'], amount=1)

# goodnight, sweet bot
bot.end()

Serving

Destroy all humans

Launch the bot by running docker stack deploy instapy -c <path -to-docker-compose.yml>

While you’re waiting for Docker to pull down the images, educate yourself on the risk of a robotic uprising:

After swarm deploys, you won’t see much, but you can monitor what InstaPy is doing, by running docker service logs instapy_web.

You can also watch the bot at work by VNCing to your docker swarm, password “secret”. You’ll see Selenium browser window cycling away, interacting with all your real/fake friends on Instagram :)

Chef’s Notes

Amazingly, my bot has ended up tagging more non-penguins than actual penguins. I don’t understand how Instagrammers come up with their hashtags!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

KeyCloak

KeyCloak is “an open source identity and access management solution.” Using a local database, or a variety of backends (think OpenLDAP), you can provide Single Sign-On (SSO) using OpenID, OAuth 2.0, and SAML.

!!! important Development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

[picture]

KeyCloak Screenshot

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname (i.e. “keycloak.your-domain.com”) you intend to use for LDAP Account Manager, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container for both runtime and backup data, so create them as follows

mkdir /var/data/runtime/keycloak/database
mkdir /var/data/keycloak/database-dump

Prepare environment

Create /var/data/keycloak/keycloak.env, and populate with the following variables, customized for your own domain structure.

# Technically, this could be auto-detected, but we prefer to be prescriptive
DB_VENDOR=postgres
DB_DATABASE=keycloak
DB_ADDR=keycloak-db
DB_USER=keycloak
DB_PASSWORD=myuberpassword
KEYCLOAK_USER=admin
KEYCLOAK_PASSWORD=ilovepasswords

# This is required to run keycloak behind traefik
PROXY_ADDRESS_FORWARDING=true

# What's our hostname?
KEYCLOAK_HOSTNAME=keycloak.batcave.com

# Tell Postgress what user/password to create
POSTGRES_USER=keycloak
POSTGRES_PASSWORD=myuberpassword

Create /var/data/keycloak/keycloak-backup.env, and populate with the following, so that your database can be backed up to the filesystem, daily:

PGHOST=keycloak-db
PGUSER=keycloak
PGPASSWORD=myuberpassword
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  keycloak:
    image: jboss/keycloak
    env_file: /var/data/config/keycloak/keycloak.env
    volumes:
      - /etc/localtime:/etc/localtime:ro
    networks:
    - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:keycloak.batcave.com
        - traefik.port=8080
        - traefik.docker.network=traefik_public

  keycloak-db:
    env_file: /var/data/config/keycloak/keycloak.env
    image: postgres:10.1
    volumes:
      - /var/data/runtime/keycloak/database:/var/lib/postgresql/data
      - /etc/localtime:/etc/localtime:ro
    networks:
    - traefik_public

  keycloak-db-backup:
    image: postgres:10.1
    env_file: /var/data/config/keycloak/keycloak-backup.env
    volumes:
      - /var/data/keycloak/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        pg_dump -Fc > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.psql
        (ls -t /dump/dump*.psql|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.psql)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - traefik_public

networks:
  traefik_public:
    external: true

!!! warning Normally, we set unique static subnets for every stack you deploy, and put the non-public facing components (like databases) in an dedicated <stack>_internal network. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

However, KeyCloak's JBOSS startup script assumes a single interface, and will crash in a ball of 🔥 if you try to assign multiple interfaces to the container. This means that we can't use a "keycloak_internal" network for our supporting containers. This is why unlike our other recipes, all the supporting services are prefixed with "keycloak-".

Serving

Launch KeyCloak stack

Launch the OpenLDAP stack by running docker stack deploy keycloak -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, and login with the user/password you defined in keycloak.env.

Integrating into OpenLDAP

KeyCloak gets really sexy when you integrate it into your OpenLDAP stack (also, it’s great not to have to play with ugly LDAP tree UIs).

You’ll need to have completed the OpenLDAP recipe

You start in the “Master” realm - but mouseover the realm name, to a dropdown box allowing you add an new realm:

[picture]

KeyCloak Add Realm Screenshot

Enter a name for your new realm, and click “Create”:

[picture]

KeyCloak Add Realm Screenshot

Once in the desired realm, click on User Federation, and click Add Provider. On the next page (“Required Settings”), set the following:

Edit Mode : Writeable

Vendor : Other

Connection URL : ldap://openldap

Users DN : ou=People,<your base DN>

Authentication Type : simple

Bind DN : cn=admin,<your base DN>

Bind Credential : <your chosen admin password>

Save your changes, and then navigate back to “User Federation” > Your LDAP name > Mappers:

[picture]

KeyCloak Add Realm Screenshot

For each of the following mappers, click the name, and set the “Read Only” flag to “Off” (this enables 2-way sync between KeyCloak and OpenLDAP)

last name

username

email

first name

[picture]

KeyCloak Add Realm Screenshot

!!! important Development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

Chef’s Notes

I wanted to be able to add multiple networks to KeyCloak (i.e., a dedicated overlay network for LDAP authentication), but the entrypoint used by the container produces an error when more than one network is configured. This could theoretically be corrected in future, with a PR, but the GitHub repo has no issues enabled, so I wasn’t sure where to start.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

OpenLDAP

!!! important Development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

LDAP is probably the most ubiquitous authentication backend, before the current era of “stupid social sign-ons”. Many of the recipes featured in the cookbook (NextCloud, Kanboard, Gitlab, etc) offer LDAP integration.

Big deal, who cares?

If you’re the only user of your tools, it probably doesn’t bother you too much to setup new user accounts for every tool. As soon as you start sharing tools with collaborators (think 10 staff using NextCloud), you suddenly feel the pain of managing a growing collection of local user accounts per-service.

Enter OpenLDAP - the most crusty, PITA, fiddly platform to setup (yes, I’m a little bitter, dynamic configuration backend!), but hugely useful for one job - a Lightweight Protocol for managing a Directory used for Access (see what I did there?)

The nice thing about OpenLDAP is, like MySQL, once you’ve setup the server, you probably never have to interact directly with it. There are many tools which will let you interact with your LDAP database via a(n ugly) UI.

This recipe combines the raw power of OpenLDAP with the flexibility and featureset of LDAP Account Manager.

[picture]

OpenLDAP Screenshot

What’s the takeaway?

What you’ll end up with is a directory structure which will allow integration with popular tools (NextCloud, Kanboard, Gitlab, etc), as well as with KeyCloak (an upcoming recipe), for true SSO.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname (i.e. “lam.your-domain.com”) you intend to use for LDAP Account Manager, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/openldap:

mkdir /var/data/openldap/openldap
mkdir /var/data/runtime/openldap/

!!! note “Why 2 directories?” For rationale, see my data layout explanation

Prepare environment

Create /var/data/openldap/openldap.env, and populate with the following variables, customized for your own domain structure. Take care with LDAP_DOMAIN, this is core to your directory structure, and can’t easily be changed later.

LDAP_DOMAIN=batcave.gotham
LDAP_ORGANISATION=BatCave Inc
LDAP_ADMIN_PASSWORD=supermansucks
LDAP_TLS=false

# Use these if you plan to protect the LDAP Account Manager webUI with an oauth_proxy
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

!!! note I use an OAuth proxy to protect access to the web UI, when the sensitivity of the protected data (i.e. my authentication store) warrants it, or if I don’t necessarily trust the security of the webUI.

Create authenticated-emails.txt, and populate with the email addresses (matched to GitHub user accounts, in my case) to which you want grant access, using OAuth2.

Create config.cfg

The Dockerized version of LDAP Account Manager is a little fiddly. In order to maintain a config file which persists across container restarts, we need to present the container with a copy of /var/www/html/config/lam.conf, tweaked for our own requirements.

Create /var/data/openldap/lam/config/config.cfg as follows:

???+ note “Much scroll, very text. Click here to collapse it for better readability”

```
# password to add/delete/rename configuration profiles (default: lam)
password: {SSHA}54haBZN/kfgNVJ+W3YJrI2dCic4= iCXkNA==

# default profile, without ".conf"
default: batcave

# log level
logLevel: 4

# log destination
logDestination: SYSLOG

# session timeout in minutes
sessionTimeout: 30

# list of hosts which may access LAM
allowedHosts:

# list of hosts which may access LAM Pro self service
allowedHostsSelfService:

# encrypt session data
encryptSession: true

# Password: minimum password length
passwordMinLength: 0

# Password: minimum uppercase characters
passwordMinUpper: 0

# Password: minimum lowercase characters
passwordMinLower: 0

# Password: minimum numeric characters
passwordMinNumeric: 0

# Password: minimum symbolic characters
passwordMinSymbol: 0

# Password: minimum character classes (0-4)
passwordMinClasses: 0

# Password: checked rules
checkedRulesCount: -1

# Password: must not contain part of user name
passwordMustNotContain3Chars: false

# Password: must not contain user name
passwordMustNotContainUser: false

# Email format (default/unix)
mailEOL: default

# PHP error reporting (default/system)
errorReporting: default

# License
license:
```

Create <profile>.cfg

While config.cfg (above) defines application-level configuration, <profile>.cfg is used to configure “domain-specific” configuration. You probably only need a single profile, but LAM could theoretically be used to administer several totally unrelated LDAP servers, ergo the concept of “profiles”.

Create yours profile (you chose a default profile in config.cfg above, remember?) by creating /var/data/openldap/lam/config/<profile>.conf, as follows:

???+ note “Much scroll, very text. Click here to collapse it for better readability”

```
# LDAP Account Manager configuration
#
# Please do not modify this file manually. The configuration can be done completely by the LAM GUI.
#
###################################################################################################

# server address (e.g. ldap://localhost:389 or ldaps://localhost:636)
ServerURL: ldap://openldap:389

# list of users who are allowed to use LDAP Account Manager
# names have to be separated by semicolons
# e.g. admins: cn=admin,dc=yourdomain,dc=org;cn=root,dc=yourdomain,dc=org
Admins: cn=admin,dc=batcave,dc=gotham

# password to change these preferences via webfrontend (default: lam)
Passwd: {SSHA}h39N9+gg/Qf1K/986VkKrjWlkcI= S/IAUQ==

# suffix of tree view
# e.g. dc=yourdomain,dc=org
treesuffix: dc=batcave,dc=gotham

# default language (a line from config/language)
defaultLanguage: en_GB.utf8

# Path to external Script
scriptPath:

# Server of external Script
scriptServer:

# Access rights for home directories
scriptRights: 750

# Number of minutes LAM caches LDAP searches.
cachetimeout: 5

# LDAP search limit.
searchLimit: 0

# Module settings

modules: posixAccount_user_minUID: 10000
modules: posixAccount_user_maxUID: 30000
modules: posixAccount_host_minMachine: 50000
modules: posixAccount_host_maxMachine: 60000
modules: posixGroup_group_minGID: 10000
modules: posixGroup_group_maxGID: 20000
modules: posixGroup_pwdHash: SSHA
modules: posixAccount_pwdHash: SSHA

# List of active account types.
activeTypes: user,group

types: suffix_user: ou=People,dc=batcave,dc=gotham
types: attr_user: #uid;#givenName;#sn;#uidNumber;#gidNumber
types: modules_user: inetOrgPerson,posixAccount,shadowAccount

types: suffix_group: ou=Groups,dc=batcave,dc=gotham
types: attr_group: #cn;#gidNumber;#memberUID;#description
types: modules_group: posixGroup

# Password mail subject
lamProMailSubject: Your password was reset

# Password mail text
lamProMailText: Dear @@givenName@@ @@sn@@,+::++::+your password was reset to: @@newPassword@@+::++::++::+Best regards+::++::+deskside support+::+

serverDisplayName:

# enable TLS encryption
useTLS: no

# follow referrals
followReferrals: false

# paged results
pagedResults: false

referentialIntegrityOverlay: false

# time zone
timeZone: Europe/London

scriptUserName:

scriptSSHKey:

scriptSSHKeyPassword:

# Access level for this profile.
accessLevel: 100

# Login method.
loginMethod: list

# Search suffix for LAM login.
loginSearchSuffix: dc=batcave,dc=gotham

# Search filter for LAM login.
loginSearchFilter: uid=%USER%

# Bind DN for login search.
loginSearchDN:

# Bind password for login search.
loginSearchPassword:

# HTTP authentication for LAM login.
httpAuthentication: false

# Password mail from
lamProMailFrom:

# Password mail reply-to
lamProMailReplyTo:

# Password mail is HTML
lamProMailIsHTML: false

# Allow alternate address
lamProMailAllowAlternateAddress: true

jobsBindPassword:

jobsBindUser:

jobsDatabase:

jobsDBHost:

jobsDBPort:

jobsDBUser:

jobsDBPassword:

jobsDBName:

jobToken: 190339140545

pwdResetAllowSpecificPassword: true

pwdResetAllowScreenPassword: true

pwdResetForcePasswordChange: true

pwdResetDefaultPasswordOutput: 2

twoFactorAuthentication: none

twoFactorAuthenticationURL: https://localhost

twoFactorAuthenticationInsecure:

twoFactorAuthenticationLabel:

twoFactorAuthenticationOptional:

twoFactorAuthenticationCaption:
tools: tool_hide_toolOUEditor: false
tools: tool_hide_toolProfileEditor: false
tools: tool_hide_toolSchemaBrowser: false
tools: tool_hide_toolServerInformation: false
tools: tool_hide_toolTests: false
tools: tool_hide_toolPDFEditor: false
tools: tool_hide_toolFileUpload: false
tools: tool_hide_toolMultiEdit: false
```

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this, at (/var/data/config/openldap/openldap.yml)

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  openldap:
    image: osixia/openldap
    env_file: /var/data/config/openldap/openldap.env
    networks:
    - traefik_public
    - auth_internal
    volumes:
    - /var/data/runtime/openldap/:/var/lib/ldap
    - /var/data/openldap/openldap/:/etc/ldap/slapd.d

  lam:
    image: jacksgt/ldap-account-manager
    networks:
    - auth_internal
    volumes:
    - /var/data/openldap/lam/config/config.cfg:/var/www/html/config/config.cfg
    - /var/data/openldap/lam/config/batcave.conf:/var/www/html/config/batcave.conf

  lam-proxy:
    image: funkypenguin/oauth2_proxy
    env_file: /var/data/config/openldap/openldap.env
    networks:
      - traefik_public
      - auth_internal
    deploy:
      labels:
        - traefik.frontend.rule=Host:lam.batcave.com
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    command: |
      -cookie-secure=false
      -upstream=http://lam:80
      -redirect-url=https://lam.batcave.com
      -http-address=http://0.0.0.0:4180
      -email-domain=batcave.com
      -provider=github

networks:
  # Used to expose lam-proxy to external access, and openldap to keycloak
  traefik_public:
    external: true

  # Used to expose openldap to other apps which want to talk to LDAP, including LAM
  auth_internal:
    external: true

!!! warning Normally, we set unique static subnets for every stack you deploy, and put the non-public facing components (like databases) in an dedicated <stack>_internal network. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

However, you're likely to want to use OpenLdap with KeyCloak, whose JBOSS startup script assumes a single interface, and will crash in a ball of 🔥 if you try to assign multiple interfaces to the container.

Since we're going to want KeyCloak to be able to talk to OpenLDAP, we have no choice but to leave the OpenLDAP container on the "traefik_public" network. We can, however, create **another** overlay network (_auth_internal, see below_), add it to the openldap container, and use it to provide OpenLDAP access to our other stacks.

Create another stack config file (/var/data/config/openldap/auth.yml) containing just the auth_internal network, and a dummy container:

version: '3'

services:
  helloworld:
    image: hello-world
    networks:
      - internal

networks:
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.39.0/24

Serving

Launch OpenLDAP stack

Create the auth_internal overlay network, by running docker stack deploy auth -c /var/data/config/openldap/auth.yml`, then launch the OpenLDAP stack by runningdocker stack deploy openldap -c /var/data/config/openldap/openldap.yml```

Log into your new LAM instance at https://YOUR-FQDN.

On first login, you’ll be prompted to create the “ou=People” and “ou=Group” elements. Proceed to create these.

You’ve now setup your OpenLDAP directory structure, and your administration interface, and hopefully won’t have to interact with the “special” LDAP Account Manager interface much again!

Create your users using the “New User” button.

!!! important Development of this recipe is sponsored by The Common Observatory. Thanks guys!

[![Common Observatory](../images/common_observatory.png)](https://www.observe.global/)

Chef’s Notes

The KeyCloak](/recipes/keycloak/) recipe illustrates how to integrate KeyCloak with your LDAP directory, giving you a cleaner interface to manage users, and a raft of SSO / OAuth features.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Docker-mailserver - A recipe for a self-contained mailserver and friends ✉️

Mail Server

Many of the recipes that follow require email access of some kind. It’s normally possible to use a hosted service such as SendGrid, or just a gmail account. If (like me) you’d like to self-host email for your stacks, then the following recipe provides a full-stack mail server running on the docker HA swarm.

Of value to me in choosing docker-mailserver were:

Automatically renews LetsEncrypt certificates

Creation of email accounts across multiple domains (i.e., the same container gives me mailbox wekan@wekan.example.com, and gitlab@gitlab.example.com)

The entire configuration is based on flat files, so there’s no database or persistence to worry about

docker-mailserver doesn’t include a webmail client, and one is not strictly needed. Rainloop can be added either as another service within the stack, or as a standalone service. Rainloop will be covered in a future recipe.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

LetsEncrypt authorized email address for domain

Access to manage DNS records for domains

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/docker-mailserver:

cd /var/data
mkdir docker-mailserver
cd docker-mailserver
mkdir {maildata,mailstate,config,letsencrypt,rainloop}

Get LetsEncrypt certificate

Decide on the FQDN to assign to your mailserver. You can service multiple domains from a single mailserver - i.e., bob@dev.example.com and daphne@prod.example.com can both be served by mail.example.com.

The docker-mailserver container can renew our LetsEncrypt certs for us, but it can’t generate them. To do this, we need to run certbot (from a container) to request the initial certs and create the appropriate directory structure.

In the example below, since I’m already using Traefik to manage the LE certs for my web platforms, I opted to use the DNS challenge to prove my ownership of the domain. The certbot client will prompt you to add a DNS record for domain verification.

docker run -ti --rm -v \
"$(pwd)"/letsencrypt:/etc/letsencrypt certbot/certbot \
--manual --preferred-challenges dns certonly \
-d mail.example.com

Get setup.sh

docker-mailserver comes with a handy bash script for managing the stack (which is just really a wrapper around the container.) It’ll make our setup easier, so download it into the root of your configuration/data directory, and make it executable:

curl -o setup.sh \
https://raw.githubusercontent.com/tomav/docker-mailserver/master/setup.sh \
chmod a+x ./setup.sh

Create email accounts

For every email address required, run ./setup.sh email add <email> <password> to create the account. The command returns no output.

You can run ./setup.sh email list to confirm all of your addresses have been created.

Create DKIM DNS entries

Run ./setup.sh config dkim to create the necessary DKIM entries. The command returns no output.

Examine the keys created by opendkim to identify the DNS TXT records required:

for i in `find config/opendkim/keys/ -name mail.txt`; do \
echo $i; \
cat $i; \
done

You’ll end up with something like this:

config/opendkim/keys/gitlab.example.com/mail.txt
mail._domainkey IN  TXT ( "v=DKIM1; k=rsa; "
      "p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCYuQqDg2ZG8ZOfI1PvarF1Gcr5cJnCR8BeCj5HYgeRohSrxKL5utPEF/AWAxXYwnKpgYN837fu74GfqsIuOhu70lPhGV+O2gFVgpXYWHELvIiTqqO0QgarIN63WE2gzE4s0FckfLrMuxMoXr882wuzuJhXywGxOavybmjpnNHhbQIDAQAB" )  ; ----- DKIM key mail for gitlab.example.com
[root@ds1 mail]#

Create the necessary DNS TXT entries for your domain(s). Note that although opendkim splits the record across two lines, the actual record should be concatenated on creation. I.e., the DNS TXT record above should read:

"v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCYuQqDg2ZG8ZOfI1PvarF1Gcr5cJnCR8BeCj5HYgeRohSrxKL5utPEF/AWAxXYwnKpgYN837fu74GfqsIuOhu70lPhGV+O2gFVgpXYWHELvIiTqqO0QgarIN63WE2gzE4s0FckfLrMuxMoXr882wuzuJhXywGxOavybmjpnNHhbQIDAQAB"

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3.2 - because we need to expose mail ports in “host mode”), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3.2'

services:
  mail:
    image: tvial/docker-mailserver:latest
    ports:
      - target: 25
        published: 25
        protocol: tcp
        mode: host
      - target: 587
        published: 587
        protocol: tcp
        mode: host
      - target: 993
        published: 993
        protocol: tcp
        mode: host
      - target: 995
        published: 995
        protocol: tcp
        mode: host
    volumes:
      - /var/data/docker-mailserver/maildata:/var/mail
      - /var/data/docker-mailserver/mailstate:/var/mail-state
      - /var/data/docker-mailserver/config:/tmp/docker-mailserver
      - /var/data/docker-mailserver/letsencrypt:/etc/letsencrypt
    env_file: /var/data/docker-mailserver/docker-mailserver.env
    networks:
      - internal
    deploy:
      replicas: 1

    rainloop:
    image: hardware/rainloop
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:rainloop.example.com
        - traefik.docker.network=traefik_public
        - traefik.port=8888
    volumes:
      - /var/data/mailserver/rainloop:/rainloop/data

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.2.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot.

A sample docker-mailserver.env file looks like this:

ENABLE_SPAMASSASSIN=1
ENABLE_CLAMAV=1
ENABLE_POSTGREY=1
ONE_DIR=1
OVERRIDE_HOSTNAME=mail.example.com
OVERRIDE_DOMAINNAME=mail.example.com
POSTMASTER_ADDRESS=admin@example.com
PERMIT_DOCKER=network
SSL_TYPE=letsencrypt

Serving

Launch mailserver

Launch the mail server stack by running docker stack deploy docker-mailserver -c <path-to-docker-mailserver.yml>

Chef’s Notes

One of the elements of this design which I didn’t appreciate at first is that since the config is entirely file-based, setup.sh can be run on any container host, provided it has the shared data mounted. This means that even though docker-mailserver was not designed with docker swarm in mind, it works perfectl with swarm. I.e., from any node, regardless of where the container is actually running, you’re able to add/delete email addresses, view logs, etc.

If you’re using sieve with Rainloop, take note of the workaround identified by ggilley

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Minio

Minio is a high performance distributed object storage server, designed for large-scale private cloud infrastructure.

However, at its simplest, Minio allows you to expose a local filestructure via the Amazon S3 API. You could, for example, use it to provide access to “buckets” (folders) of data on your filestore, secured by access/secret keys, just like AWS S3. You can further interact with your “buckets” with common tools, just as if they were hosted on S3.

Under a more advanced configuration, Minio runs in distributed mode, with features including high-availability, mirroring, erasure-coding, and “bitrot detection”.

[picture]

Minio Screenshot

Possible use-cases:

Sharing files (protected by user accounts with secrets) via HTTPS, either as read-only or read-write, in such a way that the bucket could be mounted to a remote filesystem using common S3-compatible tools, like goofys. Ever wanted to share a folder with friends, but didn’t want to open additional firewall ports etc?

Simulating S3 in a dev environment

Mirroring an S3 bucket locally

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We’ll need a directory to hold our minio file store, as well as our minio client config, so create a structure at /var/data/minio:

mkdir /var/data/minio
cd /var/data/minio
mkdir -p {mc,data}

Prepare environment

Create minio.env, and populate with the following variables

MINIO_ACCESS_KEY=<some random, complex string>
MINIO_SECRET_KEY=<another random, complex string>

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3.1'

services:
  app:
    image: minio/minio
    env_file: /var/data/config/minio/minio.env
    volumes:
     - /var/data/minio/data:/data
    networks:
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:minio.example.com
        - traefik.port=9000
    command:  minio server /data

networks:
  traefik_public:
    external: true

Serving

Launch Minio stack

Launch the Minio stack by running docker stack deploy minio -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with the access key and secret key you specified in minio.env.

If you created /var/data/minio, you’ll see nothing. If you referenced existing data, you should see all subdirectories in your existing folder represented as buckets.

If all you need is single-user access to your data, you’re done! 🎉

If, however, you want to expose data to multiple users, at different privilege levels, you’ll need the minio client to create some users and (potentially) policies…

Setup minio client

To administer the Minio server, we need the Minio client. While it’s possible to download the minio client and run it locally, it’s just as easy to do it within a small (5Mb) container.

I created an alias on my docker nodes, allowing me to run mc quickly:

alias mc='docker run -it -v /docker/minio/mc/:/root/.mc/ --network traefik_public minio/mc'

Now I use the alias to launch the client shell, and connect to my minio instance (I could also use the external, traefik-provided URL)

root@ds1:~# mc config host add minio http://app:9000 admin iambatman
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.
Added `minio` successfully.
root@ds1:~#

Add (readonly) user

Use mc to add a (readonly or readwrite) user, by running mc admin user add minio <access key> <secret key> <access level>

Example:

root@ds1:~# mc admin user add minio spiderman peterparker readonly
Added user `spiderman` successfully.
root@ds1:~#

Confirm by listing your users (admin is excluded from the list):

root@node1:~# mc admin user list minio
enabled    spiderman             readonly
root@node1:~#

Make a bucket accessible to users

By default, all buckets have no “policies” attached to them, and so can only be accessed by the administrative user. Having created some readonly/read-write users above, you’ll be wanting to grant them access to buckets.

The simplest permission scheme is “on or off”. Either a bucket has a policy, or it doesn’t. (I believe you can apply policies to subdirectories of buckets in a more advanced configuration)

After no policy, the most restrictive policy you can attach to a bucket is “download”. This policy will allow authenticated users to download contents from the bucket. Apply the “download” policy to a bucket by running mc policy download minio/<bucket name>, i.e.:

root@ds1:# mc policy download minio/comics
Access permission for `minio/comics` is set to `download`
root@ds1:#

Advanced bucketing

There are some clever complexities you can achieve with user/bucket policies, including:

A public bucket, which requires no authentication to read or even write (for a public dropbox, for example)

A special bucket, hidden from most users, but available to VIP users by application of a custom “canned policy”

Mount a minio share remotely

Having setup your buckets, users, and policies - you can give out your minio external URL, and user access keys to your remote users, and they can S3-mount your buckets, interacting with them based on their user policy (read-only or read/write)

I tested the S3 mount using goofys, “a high-performance, POSIX-ish Amazon S3 file system written in Go”.

First, I created ~/.aws/credentials, as follows:

[default]
aws_access_key_id=spiderman
aws_secret_access_key=peterparker

And then I ran (in the foreground, for debugging), goofys --f -debug_s3 --debug_fuse --endpoint=https://traefik.example.com <bucketname> <local mount point>

To permanently mount an S3 bucket using goofys, I’d add something like this to /etc/fstab:

goofys#bucket   /mnt/mountpoint        fuse     _netdev,allow_other,--file-mode=0666    0       0

Chef’s Notes

There are many S3-filesystem-mounting tools available, I just picked Goofys because it’s simple. Google is your friend :)

Some applications (like NextCloud) can natively mount S3 buckets

Some backup tools (like Duplicity) can backup directly to S3 buckets

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Piwik

Piwik is a rich open-source web analytics platform, which can be coupled with commercial plugins for additional features. It’s most simply described as “self-hosted Google Analytics”.

[picture]

Piwik Screenshot

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Preparation

Limitation of docker-swarm

The docker-swarm load-balancer is a problem for deploying piwik, since it rewrites the source address of every incoming packet to whichever docker node received the packet into the swarm. Which is a PITA for analytics, since the original source IP of the request is obscured.

The issue is tracked at #25526, and there is a workaround, but it requires running the piwik “app” container on every swarm node…

Prepare environment

Create piwik.env, and populate with the following variables

MYSQL_ROOT_PASSWORD=set-me-and-use-me-when-setting-up-piwik

Setup docker swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
  db:
    image: mysql
    volumes:
      - /var/data/piwik/mysql/runtime:/var/lib/mysql
    env_file: /var/data/piwik/piwik.env
    networks:
    - internal
  app:
    image: piwik:apache
    volumes:
      - /var/data/piwik/config:/var/www/html/config
    networks:
    - internal
    - traefik
    deploy:
      mode: global
      labels:
        - traefik.frontend.rule=Host:piwik.example.com
        - traefik.docker.network=traefik
        - traefik.port=80
  cron:
    image: piwik:apache
    volumes:
      - /var/data/piwik/config:/var/www/html/config
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      while /bin/true; do
        su -s "/bin/bash" -c "/usr/local/bin/php /var/www/html/console core:archive" www-data
        sleep 3600
      done
      EOF'
    networks:
    - internal

networks:
  traefik:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.4.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch the Piwik stack by running docker stack deploy piwik -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, and follow the wizard to complete the setup.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: A recipe for a sexy view of your Docker Swarm

Portainer

Portainer is a lightweight sexy UI for visualizing your docker environment. It also happens to integrate well with Docker Swarm clusters, which makes it a great fit for our stack.

[picture]

Portainer Screenshot

This is a “lightweight” recipe, because Portainer is so “lightweight”. But it is shiny…

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

Create a folder to store portainer’s persistent data:

mkdir /var/data/portainer

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3"

services:
  app:
    image: portainer/portainer
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/data/portainer:/data
    networks:
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:portainer.funkypenguin.co.nz
        - traefik.port=9000
      placement:
        constraints: [node.role == manager]
    command: -H unix:///var/run/docker.sock

networks:
  traefik_public:
    external: true

Serving

Launch Portainer stack

Launch the Portainer stack by running docker stack deploy portainer -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN. You’ll be prompted to set your admin user/password.

Chef’s Notes

I wanted to use oauth2_proxy to provide an additional layer of security for Portainer, but the proxy seems to break the authentication mechanism, effectively making the stack so secure, that it can’t be logged into!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Realms

Realms is a git-based wiki (like Gollum, but with basic authentication and registration)

[picture]

Realms Screenshot

Features include:

Built with Bootstrap 3.

Markdown (w/ HTML Support).

Syntax highlighting (Ace Editor).

Live preview.

Collaboration (TogetherJS / Firepad).

Drafts saved to local storage.

Handlebars for templates and logic.

!!! warning “Project likely abandoned”

In my limited trial, Realms seems _less_ useful than [Gollum](/recipes/gollum/) for my particular use-case (_i.e., you're limited to markdown syntax only_), but other users may enjoy the basic user authentication and registration features, which Gollum lacks.

Also of note is that the docker image is 1.17GB in size, and the handful of commits to the [source GitHub repo](https://github.com/scragg0x/realms-wiki/commits/master)  in the past year has listed TravisCI build failures. This has many of the hallmarks of an abandoned project, to my mind.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

Since we’ll start with a basic Realms install, let’s just create a single directory to hold the realms (SQLite) data:

mkdir /var/data/realms/

Create realms.env, and populate with the following variables (if you intend to use an oauth_proxy to double-secure your installation, which I recommend)

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3"

services:
  realms:
    image: realms/realms-wiki:latest
    env_file: /var/data/config/realms/realms.env
    volumes:
      - /var/data/realms:/home/wiki/data
    networks:
      - internal

  realms_proxy:
    image: funkypenguin/oauth2_proxy:latest
    env_file : /var/data/config/realms/realms.env
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:realms.funkypenguin.co.nz
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    volumes:
      - /var/data/config/realms/authenticated-emails.txt:/authenticated-emails.txt
    command: |
      -cookie-secure=false
      -upstream=http://realms:5000
      -redirect-url=https://realms.funkypenguin.co.nz
      -http-address=http://0.0.0.0:4180
      -email-domain=funkypenguin.co.nz
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.35.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Realms stack

Launch the Wekan stack by running docker stack deploy realms -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, authenticate against oauth_proxy, and you’re immediately presented with Realms wiki, waiting for a fresh edit ;)

Chef’s Notes

If you wanted to expose the Realms UI directly, you could remove the oauth2_proxy from the design, and move the traefik_public-related labels directly to the realms container. You’d also need to add the traefik_public network to the realms container.

The inclusion of Realms was due to the efforts of @gkoerk in our Discord server. Thanks gkoerk!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Tiny Tiny RSS

Tiny Tiny RSS is a self-hosted, AJAX-based RSS reader, which rose to popularity as a replacement for Google Reader. It supports geeky advanced features, such as:

Plugins and themeing in a drop-in fashion

Filtering (discard all articles with title matching “trump”)

Sharing articles via a unique public URL/feed

[picture]

Tiny Tiny RSS Screenshot

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/ttrss:

mkdir /var/data/ttrss
cd /var/data/ttrss
mkdir -p {database,database-dump}

Prepare environment

Create ttrss.env, and populate with the following variables, customizing at least the database password (POSTGRES_PASSWORD and DB_PASS) and the TTRSS_SELF_URL to point to your installation.

# Variables for postgres:latest
POSTGRES_USER=ttrss
POSTGRES_PASSWORD=mypassword
DB_EXTENSION=pg_trgm

# Variables for pg_dump running in postgres/latest (used for db-backup)
PGUSER=ttrss
PGPASSWORD=mypassword
PGHOST=db
BACKUP_NUM_KEEP=3
BACKUP_FREQUENCY=1d

# Variables for funkypenguin/docker-ttrss
DB_USER=ttrss
DB_PASS=mypassword
DB_PORT=5432
DB_PORT_5432_TCP_ADDR=db
DB_PORT_5432_TCP_PORT=5432
TTRSS_SELF_URL=https://ttrss.example.com
TTRSS_REPO=https://github.com/funkypenguin/tt-rss.git
S6_BEHAVIOUR_IF_STAGE2_FAILS=2

Setup docker swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:
    db:
      image: postgres:latest
      env_file: /var/data/ttrss/ttrss.env
      volumes:
        - /var/data/ttrss/database:/var/lib/postgresql/data
      networks:
        - internal

    app:
      image: funkypenguin/docker-ttrss
      env_file: /var/data/ttrss/ttrss.env
      deploy:
        labels:
          - traefik.frontend.rule=Host:ttrss.funkypenguin.co.nz
          - traefik.docker.network=traefik
          - traefik.port=8080
      networks:
        - internal
        - traefik

    db-backup:
      image: postgres:latest
      env_file: /var/data/ttrss/ttrss.env
      volumes:
        - /var/data/ttrss/database-dump:/dump
        - /etc/localtime:/etc/localtime:ro
      entrypoint: |
        bash -c 'bash -s <<EOF
        trap "break;exit" SIGHUP SIGINT SIGTERM
        sleep 2m
        while /bin/true; do
          pg_dump -Fc > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.psql
          (ls -t /dump/dump*.psql|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.psql)|sort|uniq -u|xargs rm -- {}
          sleep $$BACKUP_FREQUENCY
        done
        EOF'
      networks:
      - internal

networks:
  traefik:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.5.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch TTRSS stack

Launch the TTRSS stack by running docker stack deploy ttrss -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN - the first user you create will be an administrative user.

Chef’s Notes

There are several TTRSS containers available on docker hub, none of them “official”. I chose x86dev’s container for its features - such as my favorite skins and plugins, and the daily automatic updates from the “rolling release” master. Some of the features of the container I use are due to a PR I submitted:

Docker swarm looses the docker-compose concept of “dependencies” between containers. In the case of this stack, the application server typically starts up before the database container, which causes the database autoconfiguration scripts to fail, and brings up the app in a broken state. To prevent this, I include “wait-for”, which (combined with “S6_BEHAVIOUR_IF_STAGE2_FAILS=2”), will cause the app container to restart (and attempt to auto-configure itself) until the database is ready.

The upstream git URL changed recently, but my experience of the new repository is that it’s SO slow, that the initial “git clone” on setup of the container times out. To work around this, I created my own repo, cloned upstream, pushed it into my repo, and pointed the container at my own repo with TTRSS_REPO. I don’t get the latest code changes, but at least the app container starts up. When upstream git is performing properly, I’ll remove TTRSS_REPO to revert back to the “rolling release”.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Read-it-later, mate!

Wallabag

Wallabag is a self-hosted webapp which allows you to save URLs to “read later”, similar to Instapaper or Pocket. Like Instapaper (but not Pocket, sadly), Wallabag allows you to annotate any pages you grab for your own reference.

All saved data (pages, annotations, images, tags, etc) are stored on your own server, and can be shared/exported in a variety of formats, including ePub and PDF.

[picture]

Wallabag Screenshot

There are plugins for Chrome and Firefox, as well as apps for iOS, Android, etc. Wallabag will also integrate nicely with my favorite RSS reader, Miniflux (for which there is an existing recipe).

Here’s a video which shows off the UI a bit more.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Setup data locations

We need a filesystem location to store images that Wallabag downloads from the original sources, to re-display when you read your articles, as well as nightly database dumps (which you should backup), so create something like this:

mkdir -p /var/data/wallabag
cd /var/data/wallabag
mkdir -p {images,db-dump}

Prepare environment

Create wallabag.env, and populate with the following variables. The only variable you have to change is SYMFONY__ENV__DOMAIN_NAME - this must be the URL that your Wallabag instance will be available at (else you’ll have no CSS)

# For the DB container
POSTGRES_PASSWORD=wallabag
POSTGRES_USER=wallabag

# For the wallabag container
SYMFONY__ENV__DATABASE_DRIVER=pdo_pgsql
SYMFONY__ENV__DATABASE_HOST=db
SYMFONY__ENV__DATABASE_PORT=5432
SYMFONY__ENV__DATABASE_NAME=wallabag
SYMFONY__ENV__DATABASE_USER=wallabag
SYMFONY__ENV__DATABASE_PASSWORD=wallabag
SYMFONY__ENV__DOMAIN_NAME=https://wallabag.example.com
SYMFONY__ENV__DATABASE_DRIVER_CLASS=Wallabag\CoreBundle\Doctrine\DBAL\Driver\CustomPostgreSQLDriver
SYMFONY__ENV__MAILER_HOST=127.0.0.1
SYMFONY__ENV__MAILER_USER=~
SYMFONY__ENV__MAILER_PASSWORD=~
SYMFONY__ENV__FROM_EMAIL=wallabag@example.com
SYMFONY__ENV__FOSUSER_REGISTRATION=false

# If you decide to protect wallabag with an oauth_proxy, complete these
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

Now create wallabag-backup.env in the same folder, with the following contents. (This is necessary to prevent environment variables required for backup from breaking the DB container)

# For database backups
PGUSER=wallabag
PGPASSWORD=wallabag
PGHOST=db
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'
services:

  wallabag:
    image: wallabag/wallabag
    env_file: /var/data/config/wallabag/wallabag.env
    networks:
      - internal
    volumes:
      - /var/data/wallabag/images:/var/www/wallabag/web/assets/images

  wallabag_proxy:
   image: a5huynh/oauth2_proxy
   env_file: /var/data/config/wallabag/wallabag.env
   networks:
     - internal
     - traefik_public
   deploy:
     labels:
       - traefik.frontend.rule=Host:wallabag.example.com
       - traefik.docker.network=traefik_public
       - traefik.port=4180
   volumes:
     - /var/data/config/wallabag/authenticated-emails.txt:/authenticated-emails.txt
   command: |
     -cookie-secure=false
     -upstream=http://wallabag:80
     -redirect-url=https://wallabag.example.com
     -http-address=http://0.0.0.0:4180
     -email-domain=example.com
     -provider=github
     -authenticated-emails-file=/authenticated-emails.txt

  db:
    image: postgres
    env_file: /var/data/config/wallabag/wallabag.env
    dns_search:
      - hq.example.com
    volumes:
      - /var/data/runtime/wallabag/data:/var/lib/postgresql/data
    networks:
      - internal

  db-backup:
    image: postgres:latest
    env_file: /var/data/config/wallabag/wallabag-backup.env
    volumes:
      - /var/data/wallabag/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        pg_dump -Fc > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.psql
        (ls -t /dump/dump*.psql|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.psql)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - internal

  redis:
    image: redis:alpine
    networks:
      - internal

  import-instapaper:
    image: wallabag/wallabag
    env_file: /var/data/config/wallabag/wallabag.env
    networks:
      - internal
    command: |
      import instapaper

  import-pocket:
    image: wallabag/wallabag
    env_file: /var/data/config/wallabag/wallabag.env
    networks:
      - internal
    command: |
      import pocket

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.21.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Wallabag stack

Launch the Wallabag stack by running docker stack deploy wallabag -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user “wallabag” and default password “wallabag”.

Enable asynchronous imports

You’ll have noticed redis, plus the pocket/instapaper-importing containers included in the .yml above. Redis is there to allow asynchronous imports, and pocket and instapaper are there since they’re likely the most popular platform you’d want to import from. Other possibilities (you’ll need to adjust the .yml) are readability, firefox, chrome, and wallabag_v1 and wallabag_v2.

Even with all these elements in place, you still need to enable Redis under Internal Settings -> Import, via the admin user in the webUI. Here’s a screenshot to help you find it:

[picture]

Wallabag Imports

Chef’s Notes

If you wanted to expose the Wallabag UI directly (required for the iOS/Android apps), you could remove the oauth2_proxy from the design, and move the traefik-related labels directly to the wallabag container. You’d also need to add the traefik_public network to the wallabag container. I found the iOS app to be unreliable and clunky, so elected to leave my oauth_proxy enabled, and to simply use the webUI on my mobile devices instead. YMMMV.

I’ve not tested the email integration, but you’d need an SMTP server listening on port 25 (since we can’t change the port) to use it

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Wekan

Wekan is an open-source kanban board which allows a card-based task and to-do management, similar to tools like WorkFlowy or Trello.

[picture]

Wekan Screenshot

Wekan allows to create Boards, on which Cards can be moved around between a number of Columns. Boards can have many members, allowing for easy collaboration, just add everyone that should be able to work with you on the board to it, and you are good to go! You can assign colored Labels to cards to facilitate grouping and filtering, additionally you can add members to a card, for example to assign a task to someone.

There’s a video of the developer showing off the app, as well as a functional demo.

!!! note For added privacy, this design secures wekan behind an oauth2 proxy, so that in order to gain access to the wekan UI at all, oauth2 authentication (to GitHub, GitLab, Google, etc) must have already occurred.

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

Preparation

Setup data locations

We’ll need several directories to bind-mount into our container, so create them in /var/data/wekan:

mkdir /var/data/wekan
cd /var/data/wekan
mkdir -p {wekan-db,wekan-db-dump}

Prepare environment

You’ll need to know the following:

Choose an oauth provider, and obtain a client ID and secret

Create wekan.env, and populate with the following variables

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=
MONGO_URL=mongodb://wekandb:27017/wekan
ROOT_URL=https://wekan.example.com
MAIL_URL=smtp://wekan@wekan.example.com:password@mail.example.com:587/
MAIL_FROM="Wekan <wekan@wekan.example.com>"

# Mongodb specific database dump details
BACKUP_NUM_KEEP=7
BACKUP_FREQUENCY=1d

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: '3'

services:

  wekandb:
    image: mongo:latest
    command: mongod --smallfiles --oplogSize 128
    networks:
      - internal
    volumes:
      - /var/data/runtime/wekan/database:/data/db
      - /var/data/wekan/database-dump:/dump

  proxy:
    image: a5huynh/oauth2_proxy
    env_file: /var/data/config/wekan/wekan.env
    networks:
      - traefik
      - internal
    volumes:
      - /var/data/oauth_proxy/authenticated-emails.txt:/authenticated-emails.txt
    deploy:
      labels:
        - traefik.frontend.rule=Host:wekan.example.com
        - traefik.docker.network=traefik
        - traefik.port=4180
    command: |
      -cookie-secure=false
      -upstream=http://wekan:80
      -redirect-url=https://wekan.example.com
      -http-address=http://0.0.0.0:4180
      -email-domain=example.com
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

  wekan:
    image: wekanteam/wekan:latest
    networks:
      - internal
    env_file: /var/data/config/wekan/wekan.env

  db-backup:
    image: mongo:latest
    env_file : /var/data/config/wekan/wekan.env
    volumes:
      - /var/data/wekan/database-dump:/dump
      - /etc/localtime:/etc/localtime:ro
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 2m
      while /bin/true; do
        mongodump -h db --gzip --archive=/dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.mongo.gz
        (ls -t /dump/dump*.mongo.gz|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.mongo.gz)|sort|uniq -u|xargs rm -- {}
        sleep $$BACKUP_FREQUENCY
      done
      EOF'
    networks:
    - internal

networks:
  traefik:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.3.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Wekan stack

Launch the Wekan stack by running docker stack deploy wekan -c <path -to-docker-compose.yml>

Log into your new instance at https://YOUR-FQDN, with user “root” and the password you specified in gitlab.env.

Chef’s Notes

If you wanted to expose the Wekan UI directly, you could remove the oauth2_proxy from the design, and move the traefik-related labels directly to the wekan container. You’d also need to add the traefik network to the wekan container.

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

hero: Terminal in a browser, baby! 💻

Wetty

Wetty is a responsive, modern terminal, in your web browser. Yes, your browser. When combined with secure authentication and SSL encryption, it becomes a useful tool for quick and easy remote access.

[picture]

Wetty Screenshot

Why would you need SSH in a browser window?

Need shell access to a node with no external access? Deploy Wetty behind an oauth_proxy with a SSL-terminating reverse proxy (traefik), and suddenly you have the means to SSH to your private host from any web browser (protected by your oauth_proxy of course, and your OAuth provider’s 2FA)

Here are some other possible use cases:

Access to SSH / CLI from an environment where outgoing SSH is locked down, or SSH client isn’t / can’t be installed. (i.e., a corporate network)

Access to long-running processes inside a tmux session (like irrsi)

Remote access to a VM / container running Kali linux, for penetration testing

Ingredients

Docker swarm cluster with persistent shared storage

Traefik configured per design

DNS entry for the hostname you intend to use, pointed to your keepalived IP

Preparation

Prepare environment

Create wetty.env, and populate with the following variables per the oauth_proxy instructions:

OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=

# To use WeTTY to SSH to a host besides the (mostly useless) alpine container it comes with
SSHHOST=batcomputer.batcave.com
SSHUSER=batman

Setup Docker Swarm

Create a docker swarm config file in docker-compose syntax (v3), something like this:

!!! tip I share (with my patreon patrons) a private “premix” git repository, which includes necessary docker-compose and env files for all published recipes. This means that patrons can launch any recipe with just a git pull and a docker stack deploy 👍

version: "3"
services:
  wetty:
    image: krishnasrinivas/wetty
    env_file : /var/data/config/wetty/wetty.env
    networks:
        - internal
  proxy:
    image: funkypenguin/oauth2_proxy:latest
    env_file: /var/data/config/wetty/wetty.env
    networks:
      - internal
      - traefik_public
    deploy:
      labels:
        - traefik.frontend.rule=Host:wetty.funkypenguin.co.nz
        - traefik.docker.network=traefik_public
        - traefik.port=4180
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/data/config/wetty/authenticated-emails.txt:/authenticated-emails.txt
    command: |
      -cookie-secure=false
      -upstream=http://wetty:3000
      -redirect-url=https://wetty.funkypenguin.co.nz
      -http-address=http://0.0.0.0:4180
      -provider=github
      -authenticated-emails-file=/authenticated-emails.txt

networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.45.0/24

!!! note Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you’re creating/removing stacks a lot. See my list here.

Serving

Launch Wetty stack

Launch the Wetty stack by running docker stack deploy wetty -c <path -to-docker-compose.yml>

Browse to your new browser-cli-terminal at https://YOUR-FQDN. Authenticate with your OAuth provider, and then proceed to login, either to the remote host you specified (batcomputer.batcave.com, in the example above), or using user and password “term” to log directly into the Wetty alpine container (from which you can establish egress SSH)

Chef’s Notes

You could set SSHHOST to the IP of the “docker0” interface on your host, which is normally 172.17.0.1. (Or run /sbin/ip route|awk '/default/ { print $3 }' in the container) This would then provide you the ability to remote-manage your swarm with only web access to Wetty.

The inclusion of Wetty was due to the efforts of @gpulido in our Discord server. Thanks Gabriel!

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Reference

Now follows useful elements which are not full recipes.

OAuth proxy

Some of the platforms we use on our swarm may have strong, proven security to prevent abuse. Techniques such as rate-limiting (to defeat brute force attacks) or even support 2-factor authentication (tiny-tiny-rss or Wallabag support this).

Other platforms may provide no authentication (Traefik’s web UI for example), or minimal, un-proven UI authentication which may have been added as an afterthought.

Still platforms may hold such sensitive data (i.e., NextCloud), that we’ll feel more secure by putting an additional authentication layer in front of them.

This is the role of the OAuth proxy.

How does it work?

Normally, Traefik proxies web requests directly to individual web apps running in containers. The user talks directly to the webapp, and the webapp is responsible for ensuring appropriate authentication.

When employing the OAuth proxy , the proxy sits in the middle of this transaction - traefik sends the web client to the OAuth proxy, the proxy authenticates the user against a 3rd-party source (GitHub, Google, etc), and then passes authenticated requests on to the web app in the container.

Illustrated below: OAuth proxy

The advantage under this design is additional security. If I’m deploying a web app which I expect only myself to require access to, I’ll put the oauth_proxy in front of it. The overhead is negligible, and the additional layer of security is well-worth it.

Ingredients

Preparation

OAuth provider

OAuth Proxy currently supports the following OAuth providers:

Google (default)

Azure

Facebook

GitHub

GitLab

LinkedIn

MyUSA

Follow the instructions to setup your oauth provider. You need to setup a unique key/secret for each instance of the proxy you want to run, since in each case the callback URL will differ.

Authorized emails file

There are a variety of options with oauth_proxy re which email addresses (authenticated against your oauth provider) should be permitted access. You can permit access based on email domain (*@gmail.com), individual email address (batman@gmail.com), or based on provider-specific groups (i.e., a GitHub organization)

The most restrictive configuration allows access on a per-email address basis, which is illustrated below:

I created /var/data/oauth_proxy/authenticated-emails.txt, and add my own email address to the first line.

Configure stack

You’ll need to define a service for the oauth_proxy in every stack which you want to protect. Here’s an example from the Wekan recipe:

proxy:
  image: a5huynh/oauth2_proxy
  env_file : /var/data/wekan/wekan.env
  networks:
    - traefik
    - internal
  deploy:
    labels:
      - traefik.frontend.rule=Host:wekan.funkypenguin.co.nz
      - traefik.docker.network=traefik
      - traefik.port=4180
  volumes:
    - /var/data/oauth_proxy/authenticated-emails.txt:/authenticated-emails.txt
  command: |
    -cookie-secure=false
    -upstream=http://wekan:80
    -redirect-url=https://wekan.funkypenguin.co.nz
    -http-address=http://0.0.0.0:4180
    -email-domain=funkypenguin.co.nz
    -provider=github
    -authenticated-emails-file=/authenticated-emails.txt

Note above how: * Labels are required to tell Traefik to forward the traffic to the proxy, rather than the backend container running the app * An environment file is defined, but.. * The redirect URL must still be passed to the oauth_proxy in the command argument

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Data layout

The applications deployed in the stack utilize a combination of data-at-rest (static config, files, etc) and runtime data (live database files). The realtime data can’t be backed up with a simple copy-paste, so where we employ databases, we also include containers to perform a regular export of database data to a filesystem location.

So that we can confidently backup all our data, I’ve setup a data layout as follows:

Configuration data

Configuration data goes into /var/data/config/[recipe name], and is typically only a docker-compose .yml, and a .env file

Runtime data

Realtime data (typically database files or files-in-use) are stored in /var/data/realtime/[recipe-name], and are excluded from backup (They change constantly, and cannot be safely restored).

Static data

Static data goes into /var/data/[recipe name], and includes anything that can be safely backed up while a container is running. This includes database exports of the runtime data above.

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Networks

In order to avoid IP addressing conflicts as we bring swarm networks up/down, we will statically address each docker overlay network, and record the details below:

Network | Range
Traefik | unspecified
Docker-cleanup | 172.16.0.0/24
Mail Server | 172.16.1.0/24
Gitlab | 172.16.2.0/24
Wekan | 172.16.3.0/24
Piwik | 172.16.4.0/24
Tiny Tiny RSS | 172.16.5.0/24
Huginn | 172.16.6.0/24
Unifi | 172.16.7.0/24
Kanboard | 172.16.8.0/24
Gollum | 172.16.9.0/24
Duplicity | 172.16.10.0/24
Autopirate | 172.16.11.0/24
Nextcloud | 172.16.12.0/24
Portainer | 172.16.13.0/24
Home-Assistant | 172.16.14.0/24
OwnTracks | 172.16.15.0/24
Plex | 172.16.16.0/24
Emby | 172.16.17.0/24
Calibre-Web | 172.16.18.0/24
Wallabag | 172.16.19.0/24
InstaPy | 172.16.20.0/24
Turtle Pool | 172.16.21.0/24
MiniFlux | 172.16.22.0/24
Gitlab Runner | 172.16.23.0/24
Munin | 172.16.24.0/24
Masari Mining Pool | 172.16.25.0/24
Athena Mining Pool | 172.16.26.0/24
Bookstack | 172.16.33.0/24
Swarmprom | 172.16.34.0/24
Realms | 172.16.35.0/24
ElkarBackup | 172.16.36.0/24
Mayan EDMS | 172.16.37.0/24
Shaarli | 172.16.38.0/24
OpenLDAP | 172.16.39.0/24
MatterMost | 172.16.40.0/24
PrivateBin | 172.16.41.0/24
Mayan EDMS | 172.16.42.0/24
Hack MD | 172.16.43.0/24
FlightAirMap | 172.16.44.0/24
Wetty | 172.16.45.0/24
FileBrowser | 172.16.46.0/24
phpIPAM | 172.16.47.0/24
Dozzle | 172.16.48.0/24

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Containers

In the course of creating these recipes, I’ve often ended up creating containers with my own tweaks or changes. Below is a list of all the containers I’ve built. All of them are automatic builds, and the Dockerfiles and build logs are publicly available:

Name | Description | Badges
funkypenguin/athena Size | Athena cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/alertmanager-discord Size | AlertManager-compatible webhook to send Prometheus alerts to a Discord channel | Docker Pulls Docker Stars
funkypenguin/aeon Size | Aeon cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/bittube Size | BitTube cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/cryptonote-nodejs-pool Size | nodeJS-based mining pool for cryptonote-based mining pools, supporting advanced features like email/telegram notifications | Docker Pulls Docker Stars
funkypenguin/conceal-core Size | Conceal cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/git-docker Size | Git client in a docker container, for use on immutable OS (Atomic) hosts | Docker Pulls Docker Stars
funkypenguin/home-assistant Size | home-assistant | Docker Pulls Docker Stars
funkypenguin/htpc-cron Size | htpc-cron | Docker Pulls Docker Stars
funkypenguin/kepl Size | KEPL cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/koson Size | koson | Docker Pulls Docker Stars
funkypenguin/loki Size | loki | Docker Pulls Docker Stars
funkypenguin/masari Size | Masari cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/monero Size | Monero cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/monkeytips Size | MonkeyTips cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/minio Size | minio | Docker Pulls Docker Stars
funkypenguin/mqtt-certbot-dns Size | mqtt-certbot-dns | Docker Pulls Docker Stars
funkypenguin/munin-server Size | munin-server | Docker Pulls Docker Stars
funkypenguin/munin-node Size | munin-node | Docker Pulls Docker Stars
funkypenguin/mwlib Size | mwlib | Docker Pulls Docker Stars
funkypenguin/mqttwarn Size | mqttwarn | Docker Pulls Docker Stars
funkypenguin/nginx-proxy-letsencrypt Size | nginx-proxy-letsencrypt | Docker Pulls Docker Stars
funkypenguin/nzbdrone Size | nzbdrone | Docker Pulls Docker Stars
funkypenguin/owntracks Size | Owntracks | Docker Pulls Docker Stars
funkypenguin/oauth2_proxy Size | OAuth2 proxy supporting self-signed upstream certs | Docker Pulls Docker Stars
funkypenguin/plex Size | plex | Docker Pulls Docker Stars
funkypenguin/radarrsync Size | Python script to sync multiple Radarr instances | Docker Pulls Docker Stars
funkypenguin/ryo-currency Size | RYO cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/rtorrent Size | rtorrent | Docker Pulls Docker Stars
funkypenguin/sabnzbd Size | sabnzbd | Docker Pulls Docker Stars
funkypenguin/turtlecoind Size | turtlecoin | Docker Pulls Docker Stars
funkypenguin/temasek Size | temasek | Docker Pulls Docker Stars
funkypenguin/turtle-pool Size | turtle-pool | Docker Pulls Docker Stars
funkypenguin/turtlecoin Size | turtlecoin | Docker Pulls Docker Stars
funkypenguin/x-cash Size | X-CASH cryptocurrency daemon/services | Docker Pulls Docker Stars
funkypenguin/xmrig-cpu Size | xmrig-cpu | Docker Pulls Docker Stars

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Introduction

Our HA platform design relies on Atomic OS, which only contains bare minimum elements to run containers.

So how can we use git on this system, to push/pull the changes we make to config files? With a container, of course!

git-docker

I made a simple container which just basically executes git in the CWD:

To use it transparently, add an alias for the “git” command, or just download it with the rest of the handy aliases:

alias git='docker run -v $PWD:/var/data -v \
/var/data/git-docker/data/.ssh:/root/.ssh funkypenguin/git-docker git'

Setup SSH key

If you plan to actually push using git, you’ll need to setup an SSH keypair. You could copy across whatever keypair you currently use, but it’s probably more appropriate to generate a specific keypair for this purpose.

Generate your new SSH keypair by running:

mkdir -p /var/data/git-docker/data/.ssh
chmod 600 /var/data/git-docker/data/.ssh
docker run -v /var/data/git-docker/data/.ssh:/root/.ssh funkypenguin/git-docker ssh-keygen -t ed25519 -f /root/.ssh/id_ed25519

The output will look something like this:

Generating public/private ed25519 key pair.
Enter passphrase (empty for no passphrase): Enter same passphrase again: Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_ed25519.
Your public key has been saved in /root/.ssh/id_ed25519.pub.
The key fingerprint is:
SHA256:uZtriS7ypx7Q4kr+w++nHhHpcRfpf5MhxP3Wpx3H3hk root@a230749d8d8a
The key's randomart image is:
+--[ED25519 256]--+
|         .o .    |
|      .  ..o .   |
|     + ....   ...|
|   .. + .o . . E=|
|  o .o  S . . ++B|
| . o  .  . . +..+|
| .o .. ...  . .  |
|o..o..+.oo       |
|...=OX+.+.       |
+----[SHA256]-----+

Now add the contents of /var/data/git-docker/data/.ssh/id_ed25519.pub to your git account, and off you go - just run “git” from your Atomic host as usual, and pretend that you have the client installed!

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

OpenVPN

Sometimes you need an OpenVPN tunnel between your docker hosts and some other environment. I needed this to provide connectivity between swarm-deployed services like Home Assistant, and my IOT devices within my home LAN.

OpenVPN is one application which doesn’t really work in a swarm-type deployment, since each host will typically require a unique certificate/key to connect to the VPN anyway.

In my case, I needed each docker node to connect via OpenVPN back to a pfsense instance, but there were a few gotchas related to OpenVPN at CentOS Atomic which I needed to address first.

SELinux for OpenVPN

Yes, SELinux. Install a custom policy permitting a docker container to create tun interfaces, like this:

cat << EOF > docker-openvpn.te
module docker-openvpn 1.0;

require {
    type svirt_lxc_net_t;
    class tun_socket create;
}

#============= svirt_lxc_net_t ==============
allow svirt_lxc_net_t self:tun_socket create;

EOF

checkmodule -M -m -o docker-openvpn.mod docker-openvpn.te
semodule_package -o docker-openvpn.pp -m docker-openvpn.mod
semodule -i docker-openvpn.pp

Insert the tun module

Even with the SELinux policy above, I still need to insert the “tun” module into the running kernel at the host-level, before a docker container can use it to create a tun interface.

Run the following to auto-insert the tun module on boot:

cat << EOF >> /etc/rc.d/rc.local
# Insert the "tun" module so that the vpn-client container can access /dev/net/tun
/sbin/modprobe tun
EOF
chmod 755 /etc/rc.d/rc.local

Connect the VPN

Finally, for each node, I exported client credentials, and SCP’d them over to the docker node, into /root/my-vpn-configs-here/. I also had to use the NET_ADMIN cap-add parameter, as illustrated below:

docker run -d --name vpn-client \
  --restart=always --cap-add=NET_ADMIN --net=host \
  --device /dev/net/tun \
  -v /root/my-vpn-configs-here:/vpn:z \
  ekristen/openvpn-client --config /vpn/my-host-config.ovpn

Now every time my node boots, it establishes a VPN tunnel back to my pfsense host and (by using custom configuration directives in OpenVPN) is assigned a static VPN IP.

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬

Troubleshooting

Having difficulty with a recipe? Here are some tips..

Why is my stack not launching?

Run docker stack ps <stack name> --no-trunc for more details on why individual containers failed to launching

Attaching to running container

Need to debug why your oauth2_proxy container can’t talk to its upstream app? Start by identifying which node the proxy container is running on, using docker ps <stack name>.

SSH to the host node, and attach to the container using docker exec -it <continer id> /bin/bash (substitute /bin/ash for /bin/bash, in the case of an Alpine container), and then try to telnet to your upstream host.

Watching logs of container

Need to see what a particular container is doing? Run docker service logs -f <stack name>_<container name> to watch a particular service. As the service dies and is recreated, the logs will continue to be displayed.

Visually monitoring containers with ctop

For a visual “top-like” display of your container’s activity (as well as a detailed per-container view), try using ctop.

To execute, simply run docker run --rm -ti --name ctop -v /var/run/docker.sock:/var/run/docker.sock quay.io/vektorlab/ctop:latest

Example:

Chef’s Notes

Tip your waiter (donate) 👏

Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏

Your comments? 💬