Add docker-swarm mastodon recipe

Signed-off-by: David Young <davidy@funkypenguin.co.nz>

_snippets/common-links.md

@@ -7,6 +7,7 @@
 [cert_cka]:         https://www.credly.com/badges/cd307d51-544b-4bc6-97b0-9015e40df40d
 [cert_ckad]:        https://www.credly.com/badges/9ed9280a-fb92-46ca-b307-8f74a2cccf1d
 [cert_cks]:         https://www.credly.com/badges/93fa53da-1f38-47a9-b6ee-dce6a8fad9fc
+[community/mastodon]: /community/mastodon/
 [contact]:          https://www.funkypenguin.co.nz/contact
 [cyberchef]:        /recipes/cyberchef/
 [discord]:          https://chat.funkypenguin.co.nz
@@ -25,6 +26,7 @@
 [lazylibrarian]:    /recipes/autopirate/lazylibrarian/
 [lidarr]:           /recipes/autopirate/lidarr/
 [linx]:             /recipes/linx/
+[mastodon]:         /recipes/mastodon/
 [mealie]:           /recipes/mealie/
 [miniflux]:         /recipes/miniflux/
 [minio]:            /recipes/minio/

manuscript/community/mastodon.md

@@ -7,11 +7,11 @@ icon: material/mastodon
 
 Mastondon is a self-hosted / open-source microblogging platform (*heavily inspired by Twitter*), which supports federation, rather than centralization. Like email, any user on any Mastodon instance can follow, "toot" (*not tweet!*), and reply to any user on any _other_ instance.
 
-Our community Mastodon server is sooo [FKNY](https://so.fnky.nz/web/directory), but if you're already using Mastdon on another server, you can seamlessly interact with us from there too, thanks to the magic of federation!
+Our community Mastodon server is sooo [FKNY](https://so.fnky.nz/web/directory), but if you're already using Mastodon on another server (*or your [own instance][mastodon]*), you can seamlessly interact with us from there too, thanks to the magic of federation!
 
 !!! question "This is dumb, there's nobody here"
     * Give it time. The first time you get a federated reply from someone on another instance, it just "clicks" (*at least, it did for me*)
-    * Follow some folks (I'm [@funkypenguin](https://so.fnky.nz/@funkypenguin))
+    * Follow some folks (I'm [funkypenguin@so.fnky.nz](https://so.fnky.nz/@funkypenguin))
     * Install [mobile client](https://joinmastodon.org/apps)
 
 ## How do I Mastodon?

manuscript/recent-changes.md

@@ -8,13 +8,13 @@ hide:
 
 Recipe                   | Description                                                                                                      | Date
 -------------------------|------------------------------------------------------------------------------------------------------------------|--------------
+[Mastodon][mastodon] | Federated social network. Think "*twitter but like email*"      | _5 Aug 2022_
 [Kavita][kavita] | "Rocket-fueled" reader for manga/comics/ebooks, able to save reading position across devices/sessions      | _27 Jul 2022_
 [Authelia][authelia] | Authentication and two factor authorization server with Authelia      | _1 Nov 2021_
 [Prowlarr][prowlarr]     | An indexer manager/proxy built on the popular arr .net/reactjs base stack to integrate with the [AutoPirate][autopirate] friends | _27 Oct 2021_
 [Archivebox][archivebox] | Website Archiving service to save websites to view offline      | _19 Oct 2021_
 [Readarr][readarr] | [Autopirate][autopirate] component to grab and manage eBooks (*think "Sonarr/Radarr for books*")                     | _18 Oct 2021_
-[Paperless-NG][paperless-ng] | Organize, index, search, and view all your documents                         | _14 Oct 2021_
+
-[RSS-Bridge][rss-bridge] | Consume content from non-open platforms (*Facebook, Instagram, etc*) in your feed reader                         | _1 Oct 2021_
 
 ## Recent updates
 

manuscript/recipes/kubernetes/mastodon.md

@@ -0,0 +1 @@
+hello

manuscript/recipes/mastodon.md

@@ -0,0 +1,397 @@
+---
+title: Install Mastodon in Docker Swarm
+description: How to install your own Mastodon instance using Docker Swarm
+---
+
+# Install Mastodon in Docker Swarm
+
+[Mastodon](https://joinmastodon.org/) is an open-source, federated (*i.e., decentralized*) social network, inspired by Twitter's "microblogging" format, and used by upwards of 4.4M early-adopters, to share links, pictures, video and text.
+
+![Mastodon Screenshot](/images/mastodon.png){ loading=lazy }
+
+!!! question "Why would I run my own instance?"
+    That's a good question. After all, there are all sorts of public instances available, with a [range of themes and communities](https://joinmastodon.org/communities). You may want to run your own instance because you like the tech, because you just think it's cool :material-emoticon-cool-outline:
+    
+    You may also have realized that since Mastodon is **federated**, users on your instance can follow, toot, and interact with users on any other instance!
+
+    If you're **not** into that much effort / pain, you're welcome to [join our instance][community/mastodon] :material-mastodon:
+
+## Mastodon requirements
+
+!!! summary "Ingredients"
+    Already deployed:
+
+    * [X] [Docker swarm cluster](/docker-swarm/design/) with [persistent shared storage](/docker-swarm/shared-storage-ceph/)
+    * [X] [Traefik](/docker-swarm/traefik/) configured per design
+
+    New:
+
+    * [ ] DNS entry for your epic new social network, pointed to your [keepalived](/docker-swarm/keepalived/) IP
+    * [ ] An S3-compatible bucket for serving media (*I use [Backblaze B2](https://www.backblaze.com/b2/docs/s3_compatible_api.html)*)
+    * [ ] An SMTP gateway for delivering email notifications (*I use [Mailgun](https://www.mailgun.com/)*)
+    * [ ] A business card, with the title "[*I'm CEO, Bitch*](https://nextshark.com/heres-the-story-behind-mark-zuckerbergs-im-ceo-bitch-business-card/)"
+
+### Setup data locations
+
+First, we create a directory to hold the Mastodon docker-compose configuration:
+
+```bash
+mkdir /var/data/config/mastodon
+```
+
+Then we setup directories to hold all the various data:
+
+```bash
+mkdir -p /var/data/runtime/mastodon/redis
+mkdir -p /var/data/runtime/mastodon/elasticsearch
+mkdir -p /var/data/runtime/mastodon/postgres 
+```
+
+!!! question "Why `/var/data/runtime/mastodon` and not just `/var/data/mastodon`?"
+    The data won't be able to be backed up by a regular filesystem backup, because it'll be in use. We still need to store it **somewhere** though, so we use `/var/data/runtime`, which is excluded from automated backups. See [Data Layout](/reference/data_layout/) for details.
+
+
+
+### Setup Mastodon enviroment
+
+Create `/var/data/config/mastodon/mastodon.env` something like the example below..
+
+```yaml title="/var/data/config/mastodon/mastodon.env"
+# This is a sample configuration file. You can generate your configuration
+# with the `rake mastodon:setup` interactive setup wizard, but to customize
+# your setup even further, you'll need to edit it manually. This sample does
+# not demonstrate all available configuration options. Please look at
+# https://docs.joinmastodon.org/admin/config/ for the full documentation.
+
+# Note that this file accepts slightly different syntax depending on whether
+# you are using `docker-compose` or not. In particular, if you use
+# `docker-compose`, the value of each declared variable will be taken verbatim,
+# including surrounding quotes.
+# See: https://github.com/mastodon/mastodon/issues/16895
+
+# Federation
+# ----------
+# This identifies your server and cannot be changed safely later
+# ----------
+LOCAL_DOMAIN=example.com  # (1)!
+
+# Redis
+# -----
+REDIS_HOST=redis
+REDIS_PORT=6379
+
+# PostgreSQL
+# ----------
+DB_HOST=db
+DB_USER=postgres
+DB_NAME=postgres
+DB_PASS=tootmeupbuttercup # (2)!
+DB_PORT=5432
+
+# Elasticsearch (optional)
+# ------------------------
+ES_ENABLED=false  # (3)!
+ES_HOST=es
+ES_PORT=9200
+# Authentication for ES (optional)
+ES_USER=elastic
+ES_PASS=password
+
+# Secrets
+# -------
+# Make sure to use `rake secret` to generate secrets
+# -------
+SECRET_KEY_BASE=imafreaksecretbaby  # (4)!
+OTP_SECRET=imtoosecretformysocks  
+
+# Web Push
+# --------
+# Generate with `rake mastodon:webpush:generate_vapid_key`
+# docker run -it tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key
+# --------
+VAPID_PRIVATE_KEY=  # (5)!
+VAPID_PUBLIC_KEY= 
+
+# Sending mail # (6)!
+# ------------
+SMTP_SERVER=smtp.mailgun.org
+SMTP_PORT=587
+SMTP_LOGIN=
+SMTP_PASSWORD=
+SMTP_FROM_ADDRESS=notifications@example.com
+
+# File storage (optional)  # (7)!
+# -----------------------
+S3_ENABLED=true
+S3_BUCKET=files.example.com
+AWS_ACCESS_KEY_ID=
+AWS_SECRET_ACCESS_KEY=
+S3_ALIAS_HOST=files.example.com
+
+# IP and session retention
+# -----------------------
+# Make sure to modify the scheduling of ip_cleanup_scheduler in config/sidekiq.yml
+# to be less than daily if you lower IP_RETENTION_PERIOD below two days (172800).
+# -----------------------
+IP_RETENTION_PERIOD=31556952
+SESSION_RETENTION_PERIOD=31556952
+```
+
+1. Set this to the FQDN you plan to use for your instance.
+2. It doesn't matter what this is set to, since we're using `POSTGRES_HOST_AUTH_METHOD=trust`, but I've left it in for completeness and consistency with Mastodon's docs
+3. Only enable this if you have enough resources for an Elasticsearch instance for full-text indexing
+4. Generate these with `docker run -it tootsuite/mastodon bundle exec rake secret`
+5. Generate these with `docker run -it tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key`
+6. You'll need to complete this if you want to send email
+7. You'll need to complete this if you want to host media elsewhere
+
+### Mastodon Docker Swarm config
+
+Create a docker swarm config file in docker-compose syntax (v3), something like this example:
+
+--8<-- "premix-cta.md"
+
+```yaml title="/var/data/config/mastodon/mastodon.yml"
+version: '3.5'
+services:
+  db:
+    image: postgres:14-alpine
+    networks:
+      - internal
+    healthcheck:
+      test: ['CMD', 'pg_isready', '-U', 'postgres']
+    volumes:
+      - /var/data/runtime/mastodon/postgres:/var/lib/postgresql/data    
+    environment:
+      - 'POSTGRES_HOST_AUTH_METHOD=trust'
+
+  redis:
+    image: redis:6-alpine
+    networks:
+      - internal
+    healthcheck:
+      test: ['CMD', 'redis-cli', 'ping']
+    volumes:
+      - /var/data/runtime/mastodon/redis:/data
+
+  # es:
+  #   image: docker.elastic.co/elasticsearch/elasticsearch:7.17.4
+  #   environment:
+  #     - "ES_JAVA_OPTS=-Xms512m -Xmx512m -Des.enforce.bootstrap.checks=true"
+  #     - "xpack.license.self_generated.type=basic"
+  #     - "xpack.security.enabled=false"
+  #     - "xpack.watcher.enabled=false"
+  #     - "xpack.graph.enabled=false"
+  #     - "xpack.ml.enabled=false"
+  #     - "bootstrap.memory_lock=true"
+  #     - "cluster.name=es-mastodon"
+  #     - "discovery.type=single-node"
+  #     - "thread_pool.write.queue_size=1000"
+  #   networks:
+  #      - internal
+  #   healthcheck:
+  #      test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"]
+  #   volumes:
+  #      - /var/data/runtime/mastodon/elasticsearch:/usr/share/elasticsearch/data
+  #   ulimits:
+  #     memlock:
+  #       soft: -1
+  #       hard: -1
+  #     nofile:
+  #       soft: 65536
+  #       hard: 65536
+  #   ports:
+  #     - '9200:9200'
+
+  web:
+    image: tootsuite/mastodon
+    env_file: /var/data/config/mastodon/mastodon.env
+    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
+    networks:
+      - internal
+      - traefik_public
+    healthcheck:
+      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
+    volumes:
+      - /var/data/mastodon:/mastodon/public/system
+    deploy:
+      labels:
+        # traefik
+        - traefik.enable=true
+        - traefik.docker.network=traefik_public
+
+        # traefikv2
+        - "traefik.http.routers.mastodon.rule=Host(`mastodon.example.com`)"
+        - "traefik.http.routers.mastodon.entrypoints=https"
+        - "traefik.http.services.mastodon.loadbalancer.server.port=3000"
+
+  streaming:
+    image: tootsuite/mastodon
+    env_file: /var/data/config/mastodon/mastodon.env
+    command: node ./streaming
+    networks:
+      - internal
+      - traefik_public
+    healthcheck:
+      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']
+    deploy:
+      labels:
+        # traefik
+        - traefik.enable=true
+        - traefik.docker.network=traefik_public
+
+        # traefikv2
+        - "traefik.http.routers.mastodon.rule=Host(`mastodon.example.com`) && PathPrefix(`/api/v1/streaming`))"
+        - "traefik.http.routers.mastodon.entrypoints=https"
+        - "traefik.http.services.mastodon.loadbalancer.server.port=3000"
+
+  sidekiq:
+    image: tootsuite/mastodon
+    env_file: /var/data/config/mastodon/mastodon.env
+    command: bundle exec sidekiq
+    networks:
+      - internal
+    volumes:
+      - /var/data/mastodon:/mastodon/public/system
+    healthcheck:
+      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
+
+  ## Uncomment to enable federation with tor instances along with adding the following ENV variables
+  ## http_proxy=http://privoxy:8118
+  ## ALLOW_ACCESS_TO_HIDDEN_SERVICE=true
+  # tor:
+  #   image: sirboops/tor
+  #   networks:
+  #      - internal
+  #
+  # privoxy:
+  #   image: sirboops/privoxy
+  #   volumes:
+  #     - /var/data/mastodon/privoxy:/opt/config
+  #   networks:
+  #     - internal
+
+networks:
+  traefik_public:
+    external: true
+  internal:
+    driver: overlay
+    ipam:
+      config:
+        - subnet: 172.16.9.0/24
+```
+
+--8<-- "reference-networks.md"
+
+## Pre-warming
+
+Unlike most recipes, we can't just deploy Mastodon into Docker Swarm, and trust it to setup its database itself. We have to "pre-warm" it using docker-compose, per the official docs (*Docker Swarm is not officially supported*)
+
+### Start with docker-compose
+
+From the `/var/data/config/mastodon` directory, run the following to start up the Mastodon environment using docker-compose. This will result in a **broken** environment, since the database isn't configured yet, but it provides us the opportunity to configure it.
+
+```bash
+docker-compose -f mastodon.yml up -d
+```
+
+The output should look something like this:
+
+```bash
+root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml up -d
+WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
+WARNING: The Docker Engine you're using is running in swarm mode.
+
+Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
+
+To deploy your application across the swarm, use `docker stack deploy`.
+
+Creating mastodon_sidekiq_1   ... done
+Creating mastodon_db_1        ... done
+Creating mastodon_redis_1     ... done
+Creating mastodon_streaming_1 ... done
+Creating mastodon_web_1       ... done
+root@raphael:/var/data/config/mastodon#
+```
+
+### Create database
+
+Run the following to create the database. You can expect this to take a few minutes, and produce a **lot** of output:
+
+```bash
+cd /var/data/config/mastodon
+docker-compose -f mastodon.yml run --rm web bundle exec rake db:migrate
+```
+
+### Create admin user
+
+Next, decide on your chosen username, and create your admin user:
+
+```bash
+cd /var/data/config/mastodon
+docker-compose -f mastodon.yml run --rm web bin/tootctl accounts \
+create <username> --email <email address> --confirmed --role admin
+``` 
+
+The password will be output on completion:
+
+```bash
+root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml run --rm web bin/tootctl accounts create batman --email batman@batcave.org --confirmed --role admin
+WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
+OK
+New password: c6eb8e0d10cd6f0aa874b7a384177a08
+root@raphael:/var/data/config/mastodon#
+```
+
+### Turn off docker-compose
+
+We've setup the essestials now, everything else can be configured either via the UI or via the `.env` file, so tear down the docker-compose environment with:
+
+```bash
+docker-compose -f mastodon.yml down
+```
+
+The output should look like this:
+
+```
+root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml down
+WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
+Stopping mastodon_streaming_1 ... done
+Stopping mastodon_web_1       ... done
+Stopping mastodon_db_1        ... done
+Stopping mastodon_redis_1     ... done
+Stopping mastodon_sidekiq_1   ... done
+Removing mastodon_streaming_1 ... done
+Removing mastodon_web_1       ... done
+Removing mastodon_db_1        ... done
+Removing mastodon_redis_1     ... done
+Removing mastodon_sidekiq_1   ... done
+Removing network mastodon_internal
+Network traefik_public is external, skipping
+root@raphael:/var/data/config/mastodon#
+```
+
+## :material-mastodon: Launch Mastodon!
+
+Launch the Mastodon stack by running 
+
+```bash
+docker stack deploy mastodon -c /var/data/config/mastodon/mastodon.yml
+```
+
+Now hit the URL you defined in your config, and you should see your beautiful new Mastodon instance! Login with your configured credentials, navigate to **Preferences**, and have fun tweaking and tooting away!
+
+Once you're done, "toot" me by mentioning [funkypenguin@so.fnky.nz](https://so.fnky.nz/@funkypenguin) in a toot! :wave:
+
+
+## Summary 
+
+What have we achieved? Even though we had to jump through some extra hoops to setup database and users, we now have a fully-swarmed Mastodon instance, ready to federate with the world! :material-mastodon:
+
+!!! summary "Summary"
+    Created:
+
+    * [X] Mastodon configured, running, and ready to toot!
+
+
+--8<-- "recipe-footer.md"

mkdocs.yml

@@ -51,6 +51,7 @@ nav:
         - Shared Storage (GlusterFS): docker-swarm/shared-storage-gluster.md
         - Keepalived: docker-swarm/keepalived.md
         - Docker Swarm Mode: docker-swarm/docker-swarm-mode.md
+      - Essentials:
         - Traefik: docker-swarm/traefik.md
         - Traefik Forward Auth: 
             - docker-swarm/traefik-forward-auth/index.md 
@@ -61,7 +62,7 @@ nav:
         - Registry: docker-swarm/registry.md
         - Mail Server: recipes/mail.md
         - Duplicity: recipes/duplicity.md
-      - Chef's Favorites:
+      - Recipes:
         - Auto Pirate:
           - recipes/autopirate/index.md
           - Headphones: recipes/autopirate/headphones.md
@@ -80,25 +81,6 @@ nav:
           - SABnzbd: recipes/autopirate/sabnzbd.md
           - Sonarr: recipes/autopirate/sonarr.md
           - End: recipes/autopirate/end.md      
-        - Emby: recipes/emby.md
-        - Home Assistant: recipes/homeassistant.md
-        - Huginn: recipes/huginn.md
-        - Jellyfin: recipes/jellyfin.md
-        - Kanboard: recipes/kanboard.md
-        - Keycloak: 
-          - recipes/keycloak/index.md   
-          - OIDC Provider: recipes/keycloak/setup-oidc-provider.md        
-          - OpenLDAP: recipes/keycloak/authenticate-against-openldap.md
-        - Miniflux: recipes/miniflux.md
-        - Munin: recipes/munin.md
-        - NextCloud: recipes/nextcloud.md
-        - phpIPAM: recipes/phpipam.md
-        - Plex: recipes/plex.md
-        - Portainer: recipes/portainer.md        
-        - PrivateBin: recipes/privatebin.md
-        - Restic: recipes/restic.md          
-        - Swarmprom: recipes/swarmprom.md    
-      - Recipes:
         - Archive Box: recipes/archivebox.md
         - Bitwarden: recipes/bitwarden.md
         - Bookstack: recipes/bookstack.md
@@ -107,30 +89,47 @@ nav:
         - CyberChef: recipes/cyberchef.md
         - Duplicati: recipes/duplicati.md
         - ElkarBackup: recipes/elkarbackup.md
+        - Emby: recipes/emby.md
+        - Home Assistant: recipes/homeassistant.md
+        - Huginn: recipes/huginn.md        
         - Funkwhale: recipes/funkwhale.md
         - Ghost: recipes/ghost.md
         - GitLab: recipes/gitlab.md
         - GitLab Runner: recipes/gitlab-runner.md
         - Gollum: recipes/gollum.md
         - InstaPy: recipes/instapy.md
+        - Jellyfin: recipes/jellyfin.md
+        - Keycloak: 
+          - recipes/keycloak/index.md      
+          - OIDC Provider: recipes/keycloak/setup-oidc-provider.md        
+          - OpenLDAP: recipes/keycloak/authenticate-against-openldap.md 
+        - Kanboard: recipes/kanboard.md
+        - Kavita: recipes/kavita.md
         - Keycloak: 
           - recipes/keycloak/index.md   
           - OIDC Provider: recipes/keycloak/setup-oidc-provider.md        
           - OpenLDAP: recipes/keycloak/authenticate-against-openldap.md
         - Komga: recipes/komga.md
-        - Kavita: recipes/kavita.md
         - Linx: recipes/linx.md
+        - Mastodon: recipes/mastodon.md
         - Mealie: recipes/mealie.md
+        - Miniflux: recipes/miniflux.md        
         - Minio: recipes/minio.md
+        - Munin: recipes/munin.md        
+        - NextCloud: recipes/nextcloud.md        
         - NightScout: recipes/nightscout.md
         - OpenLDAP: recipes/openldap.md
         - OwnTracks: recipes/owntracks.md
-        - Photoprism: recipes/photoprism.md
         - Paperless NG: recipes/paperless-ng.md
+        - Photoprism: recipes/photoprism.md
+        - Plex: recipes/plex.md        
+        - phpIPAM: recipes/phpipam.md        
         - Portainer: recipes/portainer.md  
+        - PrivateBin: recipes/privatebin.md
         - Realms: recipes/realms.md
         - Restic: recipes/restic.md
         - RSS Bridge: recipes/rss-bridge.md   
+        - Swarmprom: recipes/swarmprom.md              
         - Tiny Tiny RSS: recipes/tiny-tiny-rss.md
         - Traefik: docker-swarm/traefik.md
         - Traefik Forward Auth: 
@@ -227,7 +226,7 @@ nav:
       #   - Istio: recipes/kubernetes/wip.md
       #   - Jaeger: kubernetes/wip.md
       #   - Kiali: kubernetes/wip.md
-      #   - Minio: kubernetes/wip.md
+        # - Mastodon: recipes/kubernetes/mastodon.md
       #   - NGINX Ingress: kubernetes/wip.md
       #   - Polaris: kubernetes/wip.md
       #   - Portainer: kubernetes/wip.md
@@ -315,6 +314,8 @@ theme:
 
 extra:
   social:
+    - icon: 'fontawesome/brands/mastodon'
+      link: 'https://so.funkypenguin.co.nz/'
     - icon: 'fontawesome/brands/github'
       link: 'https://github.com/funkypenguin'
     - icon: 'fontawesome/brands/twitter'