diff --git a/_snippets/common-links.md b/_snippets/common-links.md index 8fac0f5..456a20c 100644 --- a/_snippets/common-links.md +++ b/_snippets/common-links.md @@ -7,6 +7,7 @@ [cert_cka]: https://www.credly.com/badges/cd307d51-544b-4bc6-97b0-9015e40df40d [cert_ckad]: https://www.credly.com/badges/9ed9280a-fb92-46ca-b307-8f74a2cccf1d [cert_cks]: https://www.credly.com/badges/93fa53da-1f38-47a9-b6ee-dce6a8fad9fc +[community/mastodon]: /community/mastodon/ [contact]: https://www.funkypenguin.co.nz/contact [cyberchef]: /recipes/cyberchef/ [discord]: https://chat.funkypenguin.co.nz @@ -25,6 +26,7 @@ [lazylibrarian]: /recipes/autopirate/lazylibrarian/ [lidarr]: /recipes/autopirate/lidarr/ [linx]: /recipes/linx/ +[mastodon]: /recipes/mastodon/ [mealie]: /recipes/mealie/ [miniflux]: /recipes/miniflux/ [minio]: /recipes/minio/ diff --git a/manuscript/community/mastodon.md b/manuscript/community/mastodon.md index 9dfd123..947b6ea 100644 --- a/manuscript/community/mastodon.md +++ b/manuscript/community/mastodon.md @@ -7,11 +7,11 @@ icon: material/mastodon Mastondon is a self-hosted / open-source microblogging platform (*heavily inspired by Twitter*), which supports federation, rather than centralization. Like email, any user on any Mastodon instance can follow, "toot" (*not tweet!*), and reply to any user on any _other_ instance. -Our community Mastodon server is sooo [FKNY](https://so.fnky.nz/web/directory), but if you're already using Mastdon on another server, you can seamlessly interact with us from there too, thanks to the magic of federation! +Our community Mastodon server is sooo [FKNY](https://so.fnky.nz/web/directory), but if you're already using Mastodon on another server (*or your [own instance][mastodon]*), you can seamlessly interact with us from there too, thanks to the magic of federation! !!! question "This is dumb, there's nobody here" * Give it time. The first time you get a federated reply from someone on another instance, it just "clicks" (*at least, it did for me*) - * Follow some folks (I'm [@funkypenguin](https://so.fnky.nz/@funkypenguin)) + * Follow some folks (I'm [funkypenguin@so.fnky.nz](https://so.fnky.nz/@funkypenguin)) * Install [mobile client](https://joinmastodon.org/apps) ## How do I Mastodon? diff --git a/manuscript/images/mastodon.png b/manuscript/images/mastodon.png new file mode 100644 index 0000000..21e6e71 Binary files /dev/null and b/manuscript/images/mastodon.png differ diff --git a/manuscript/recent-changes.md b/manuscript/recent-changes.md index e3a8102..aa4c2da 100644 --- a/manuscript/recent-changes.md +++ b/manuscript/recent-changes.md @@ -8,13 +8,13 @@ hide: Recipe | Description | Date -------------------------|------------------------------------------------------------------------------------------------------------------|-------------- +[Mastodon][mastodon] | Federated social network. Think "*twitter but like email*" | _5 Aug 2022_ [Kavita][kavita] | "Rocket-fueled" reader for manga/comics/ebooks, able to save reading position across devices/sessions | _27 Jul 2022_ [Authelia][authelia] | Authentication and two factor authorization server with Authelia | _1 Nov 2021_ [Prowlarr][prowlarr] | An indexer manager/proxy built on the popular arr .net/reactjs base stack to integrate with the [AutoPirate][autopirate] friends | _27 Oct 2021_ [Archivebox][archivebox] | Website Archiving service to save websites to view offline | _19 Oct 2021_ [Readarr][readarr] | [Autopirate][autopirate] component to grab and manage eBooks (*think "Sonarr/Radarr for books*") | _18 Oct 2021_ -[Paperless-NG][paperless-ng] | Organize, index, search, and view all your documents | _14 Oct 2021_ -[RSS-Bridge][rss-bridge] | Consume content from non-open platforms (*Facebook, Instagram, etc*) in your feed reader | _1 Oct 2021_ + ## Recent updates diff --git a/manuscript/recipes/kubernetes/mastodon.md b/manuscript/recipes/kubernetes/mastodon.md new file mode 100644 index 0000000..b6fc4c6 --- /dev/null +++ b/manuscript/recipes/kubernetes/mastodon.md @@ -0,0 +1 @@ +hello \ No newline at end of file diff --git a/manuscript/recipes/mastodon.md b/manuscript/recipes/mastodon.md new file mode 100644 index 0000000..59b30b6 --- /dev/null +++ b/manuscript/recipes/mastodon.md @@ -0,0 +1,397 @@ +--- +title: Install Mastodon in Docker Swarm +description: How to install your own Mastodon instance using Docker Swarm +--- + +# Install Mastodon in Docker Swarm + +[Mastodon](https://joinmastodon.org/) is an open-source, federated (*i.e., decentralized*) social network, inspired by Twitter's "microblogging" format, and used by upwards of 4.4M early-adopters, to share links, pictures, video and text. + +![Mastodon Screenshot](/images/mastodon.png){ loading=lazy } + +!!! question "Why would I run my own instance?" + That's a good question. After all, there are all sorts of public instances available, with a [range of themes and communities](https://joinmastodon.org/communities). You may want to run your own instance because you like the tech, because you just think it's cool :material-emoticon-cool-outline: + + You may also have realized that since Mastodon is **federated**, users on your instance can follow, toot, and interact with users on any other instance! + + If you're **not** into that much effort / pain, you're welcome to [join our instance][community/mastodon] :material-mastodon: + +## Mastodon requirements + +!!! summary "Ingredients" + Already deployed: + + * [X] [Docker swarm cluster](/docker-swarm/design/) with [persistent shared storage](/docker-swarm/shared-storage-ceph/) + * [X] [Traefik](/docker-swarm/traefik/) configured per design + + New: + + * [ ] DNS entry for your epic new social network, pointed to your [keepalived](/docker-swarm/keepalived/) IP + * [ ] An S3-compatible bucket for serving media (*I use [Backblaze B2](https://www.backblaze.com/b2/docs/s3_compatible_api.html)*) + * [ ] An SMTP gateway for delivering email notifications (*I use [Mailgun](https://www.mailgun.com/)*) + * [ ] A business card, with the title "[*I'm CEO, Bitch*](https://nextshark.com/heres-the-story-behind-mark-zuckerbergs-im-ceo-bitch-business-card/)" + +### Setup data locations + +First, we create a directory to hold the Mastodon docker-compose configuration: + +```bash +mkdir /var/data/config/mastodon +``` + +Then we setup directories to hold all the various data: + +```bash +mkdir -p /var/data/runtime/mastodon/redis +mkdir -p /var/data/runtime/mastodon/elasticsearch +mkdir -p /var/data/runtime/mastodon/postgres +``` + +!!! question "Why `/var/data/runtime/mastodon` and not just `/var/data/mastodon`?" + The data won't be able to be backed up by a regular filesystem backup, because it'll be in use. We still need to store it **somewhere** though, so we use `/var/data/runtime`, which is excluded from automated backups. See [Data Layout](/reference/data_layout/) for details. + + + +### Setup Mastodon enviroment + +Create `/var/data/config/mastodon/mastodon.env` something like the example below.. + +```yaml title="/var/data/config/mastodon/mastodon.env" +# This is a sample configuration file. You can generate your configuration +# with the `rake mastodon:setup` interactive setup wizard, but to customize +# your setup even further, you'll need to edit it manually. This sample does +# not demonstrate all available configuration options. Please look at +# https://docs.joinmastodon.org/admin/config/ for the full documentation. + +# Note that this file accepts slightly different syntax depending on whether +# you are using `docker-compose` or not. In particular, if you use +# `docker-compose`, the value of each declared variable will be taken verbatim, +# including surrounding quotes. +# See: https://github.com/mastodon/mastodon/issues/16895 + +# Federation +# ---------- +# This identifies your server and cannot be changed safely later +# ---------- +LOCAL_DOMAIN=example.com # (1)! + +# Redis +# ----- +REDIS_HOST=redis +REDIS_PORT=6379 + +# PostgreSQL +# ---------- +DB_HOST=db +DB_USER=postgres +DB_NAME=postgres +DB_PASS=tootmeupbuttercup # (2)! +DB_PORT=5432 + +# Elasticsearch (optional) +# ------------------------ +ES_ENABLED=false # (3)! +ES_HOST=es +ES_PORT=9200 +# Authentication for ES (optional) +ES_USER=elastic +ES_PASS=password + +# Secrets +# ------- +# Make sure to use `rake secret` to generate secrets +# ------- +SECRET_KEY_BASE=imafreaksecretbaby # (4)! +OTP_SECRET=imtoosecretformysocks + +# Web Push +# -------- +# Generate with `rake mastodon:webpush:generate_vapid_key` +# docker run -it tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key +# -------- +VAPID_PRIVATE_KEY= # (5)! +VAPID_PUBLIC_KEY= + +# Sending mail # (6)! +# ------------ +SMTP_SERVER=smtp.mailgun.org +SMTP_PORT=587 +SMTP_LOGIN= +SMTP_PASSWORD= +SMTP_FROM_ADDRESS=notifications@example.com + +# File storage (optional) # (7)! +# ----------------------- +S3_ENABLED=true +S3_BUCKET=files.example.com +AWS_ACCESS_KEY_ID= +AWS_SECRET_ACCESS_KEY= +S3_ALIAS_HOST=files.example.com + +# IP and session retention +# ----------------------- +# Make sure to modify the scheduling of ip_cleanup_scheduler in config/sidekiq.yml +# to be less than daily if you lower IP_RETENTION_PERIOD below two days (172800). +# ----------------------- +IP_RETENTION_PERIOD=31556952 +SESSION_RETENTION_PERIOD=31556952 +``` + +1. Set this to the FQDN you plan to use for your instance. +2. It doesn't matter what this is set to, since we're using `POSTGRES_HOST_AUTH_METHOD=trust`, but I've left it in for completeness and consistency with Mastodon's docs +3. Only enable this if you have enough resources for an Elasticsearch instance for full-text indexing +4. Generate these with `docker run -it tootsuite/mastodon bundle exec rake secret` +5. Generate these with `docker run -it tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key` +6. You'll need to complete this if you want to send email +7. You'll need to complete this if you want to host media elsewhere + +### Mastodon Docker Swarm config + +Create a docker swarm config file in docker-compose syntax (v3), something like this example: + +--8<-- "premix-cta.md" + +```yaml title="/var/data/config/mastodon/mastodon.yml" +version: '3.5' +services: + db: + image: postgres:14-alpine + networks: + - internal + healthcheck: + test: ['CMD', 'pg_isready', '-U', 'postgres'] + volumes: + - /var/data/runtime/mastodon/postgres:/var/lib/postgresql/data + environment: + - 'POSTGRES_HOST_AUTH_METHOD=trust' + + redis: + image: redis:6-alpine + networks: + - internal + healthcheck: + test: ['CMD', 'redis-cli', 'ping'] + volumes: + - /var/data/runtime/mastodon/redis:/data + + # es: + # image: docker.elastic.co/elasticsearch/elasticsearch:7.17.4 + # environment: + # - "ES_JAVA_OPTS=-Xms512m -Xmx512m -Des.enforce.bootstrap.checks=true" + # - "xpack.license.self_generated.type=basic" + # - "xpack.security.enabled=false" + # - "xpack.watcher.enabled=false" + # - "xpack.graph.enabled=false" + # - "xpack.ml.enabled=false" + # - "bootstrap.memory_lock=true" + # - "cluster.name=es-mastodon" + # - "discovery.type=single-node" + # - "thread_pool.write.queue_size=1000" + # networks: + # - internal + # healthcheck: + # test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"] + # volumes: + # - /var/data/runtime/mastodon/elasticsearch:/usr/share/elasticsearch/data + # ulimits: + # memlock: + # soft: -1 + # hard: -1 + # nofile: + # soft: 65536 + # hard: 65536 + # ports: + # - '9200:9200' + + web: + image: tootsuite/mastodon + env_file: /var/data/config/mastodon/mastodon.env + command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000" + networks: + - internal + - traefik_public + healthcheck: + test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1'] + volumes: + - /var/data/mastodon:/mastodon/public/system + deploy: + labels: + # traefik + - traefik.enable=true + - traefik.docker.network=traefik_public + + # traefikv2 + - "traefik.http.routers.mastodon.rule=Host(`mastodon.example.com`)" + - "traefik.http.routers.mastodon.entrypoints=https" + - "traefik.http.services.mastodon.loadbalancer.server.port=3000" + + streaming: + image: tootsuite/mastodon + env_file: /var/data/config/mastodon/mastodon.env + command: node ./streaming + networks: + - internal + - traefik_public + healthcheck: + test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1'] + deploy: + labels: + # traefik + - traefik.enable=true + - traefik.docker.network=traefik_public + + # traefikv2 + - "traefik.http.routers.mastodon.rule=Host(`mastodon.example.com`) && PathPrefix(`/api/v1/streaming`))" + - "traefik.http.routers.mastodon.entrypoints=https" + - "traefik.http.services.mastodon.loadbalancer.server.port=3000" + + sidekiq: + image: tootsuite/mastodon + env_file: /var/data/config/mastodon/mastodon.env + command: bundle exec sidekiq + networks: + - internal + volumes: + - /var/data/mastodon:/mastodon/public/system + healthcheck: + test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"] + + ## Uncomment to enable federation with tor instances along with adding the following ENV variables + ## http_proxy=http://privoxy:8118 + ## ALLOW_ACCESS_TO_HIDDEN_SERVICE=true + # tor: + # image: sirboops/tor + # networks: + # - internal + # + # privoxy: + # image: sirboops/privoxy + # volumes: + # - /var/data/mastodon/privoxy:/opt/config + # networks: + # - internal + +networks: + traefik_public: + external: true + internal: + driver: overlay + ipam: + config: + - subnet: 172.16.9.0/24 +``` + +--8<-- "reference-networks.md" + +## Pre-warming + +Unlike most recipes, we can't just deploy Mastodon into Docker Swarm, and trust it to setup its database itself. We have to "pre-warm" it using docker-compose, per the official docs (*Docker Swarm is not officially supported*) + +### Start with docker-compose + +From the `/var/data/config/mastodon` directory, run the following to start up the Mastodon environment using docker-compose. This will result in a **broken** environment, since the database isn't configured yet, but it provides us the opportunity to configure it. + +```bash +docker-compose -f mastodon.yml up -d +``` + +The output should look something like this: + +```bash +root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml up -d +WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm. +WARNING: The Docker Engine you're using is running in swarm mode. + +Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node. + +To deploy your application across the swarm, use `docker stack deploy`. + +Creating mastodon_sidekiq_1 ... done +Creating mastodon_db_1 ... done +Creating mastodon_redis_1 ... done +Creating mastodon_streaming_1 ... done +Creating mastodon_web_1 ... done +root@raphael:/var/data/config/mastodon# +``` + +### Create database + +Run the following to create the database. You can expect this to take a few minutes, and produce a **lot** of output: + +```bash +cd /var/data/config/mastodon +docker-compose -f mastodon.yml run --rm web bundle exec rake db:migrate +``` + +### Create admin user + +Next, decide on your chosen username, and create your admin user: + +```bash +cd /var/data/config/mastodon +docker-compose -f mastodon.yml run --rm web bin/tootctl accounts \ +create --email --confirmed --role admin +``` + +The password will be output on completion: + +```bash +root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml run --rm web bin/tootctl accounts create batman --email batman@batcave.org --confirmed --role admin +WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm. +OK +New password: c6eb8e0d10cd6f0aa874b7a384177a08 +root@raphael:/var/data/config/mastodon# +``` + +### Turn off docker-compose + +We've setup the essestials now, everything else can be configured either via the UI or via the `.env` file, so tear down the docker-compose environment with: + +```bash +docker-compose -f mastodon.yml down +``` + +The output should look like this: + +``` +root@raphael:/var/data/config/mastodon# docker-compose -f mastodon.yml down +WARNING: Some services (streaming, web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm. +Stopping mastodon_streaming_1 ... done +Stopping mastodon_web_1 ... done +Stopping mastodon_db_1 ... done +Stopping mastodon_redis_1 ... done +Stopping mastodon_sidekiq_1 ... done +Removing mastodon_streaming_1 ... done +Removing mastodon_web_1 ... done +Removing mastodon_db_1 ... done +Removing mastodon_redis_1 ... done +Removing mastodon_sidekiq_1 ... done +Removing network mastodon_internal +Network traefik_public is external, skipping +root@raphael:/var/data/config/mastodon# +``` + +## :material-mastodon: Launch Mastodon! + +Launch the Mastodon stack by running + +```bash +docker stack deploy mastodon -c /var/data/config/mastodon/mastodon.yml +``` + +Now hit the URL you defined in your config, and you should see your beautiful new Mastodon instance! Login with your configured credentials, navigate to **Preferences**, and have fun tweaking and tooting away! + +Once you're done, "toot" me by mentioning [funkypenguin@so.fnky.nz](https://so.fnky.nz/@funkypenguin) in a toot! :wave: + + +## Summary + +What have we achieved? Even though we had to jump through some extra hoops to setup database and users, we now have a fully-swarmed Mastodon instance, ready to federate with the world! :material-mastodon: + +!!! summary "Summary" + Created: + + * [X] Mastodon configured, running, and ready to toot! + + +--8<-- "recipe-footer.md" \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index b81353c..165dcf6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -51,6 +51,7 @@ nav: - Shared Storage (GlusterFS): docker-swarm/shared-storage-gluster.md - Keepalived: docker-swarm/keepalived.md - Docker Swarm Mode: docker-swarm/docker-swarm-mode.md + - Essentials: - Traefik: docker-swarm/traefik.md - Traefik Forward Auth: - docker-swarm/traefik-forward-auth/index.md @@ -61,7 +62,7 @@ nav: - Registry: docker-swarm/registry.md - Mail Server: recipes/mail.md - Duplicity: recipes/duplicity.md - - Chef's Favorites: + - Recipes: - Auto Pirate: - recipes/autopirate/index.md - Headphones: recipes/autopirate/headphones.md @@ -79,26 +80,7 @@ nav: - Rtorrent: recipes/autopirate/rtorrent.md - SABnzbd: recipes/autopirate/sabnzbd.md - Sonarr: recipes/autopirate/sonarr.md - - End: recipes/autopirate/end.md - - Emby: recipes/emby.md - - Home Assistant: recipes/homeassistant.md - - Huginn: recipes/huginn.md - - Jellyfin: recipes/jellyfin.md - - Kanboard: recipes/kanboard.md - - Keycloak: - - recipes/keycloak/index.md - - OIDC Provider: recipes/keycloak/setup-oidc-provider.md - - OpenLDAP: recipes/keycloak/authenticate-against-openldap.md - - Miniflux: recipes/miniflux.md - - Munin: recipes/munin.md - - NextCloud: recipes/nextcloud.md - - phpIPAM: recipes/phpipam.md - - Plex: recipes/plex.md - - Portainer: recipes/portainer.md - - PrivateBin: recipes/privatebin.md - - Restic: recipes/restic.md - - Swarmprom: recipes/swarmprom.md - - Recipes: + - End: recipes/autopirate/end.md - Archive Box: recipes/archivebox.md - Bitwarden: recipes/bitwarden.md - Bookstack: recipes/bookstack.md @@ -107,30 +89,47 @@ nav: - CyberChef: recipes/cyberchef.md - Duplicati: recipes/duplicati.md - ElkarBackup: recipes/elkarbackup.md + - Emby: recipes/emby.md + - Home Assistant: recipes/homeassistant.md + - Huginn: recipes/huginn.md - Funkwhale: recipes/funkwhale.md - Ghost: recipes/ghost.md - GitLab: recipes/gitlab.md - GitLab Runner: recipes/gitlab-runner.md - Gollum: recipes/gollum.md - InstaPy: recipes/instapy.md + - Jellyfin: recipes/jellyfin.md - Keycloak: - recipes/keycloak/index.md - OIDC Provider: recipes/keycloak/setup-oidc-provider.md - - OpenLDAP: recipes/keycloak/authenticate-against-openldap.md - - Komga: recipes/komga.md + - OpenLDAP: recipes/keycloak/authenticate-against-openldap.md + - Kanboard: recipes/kanboard.md - Kavita: recipes/kavita.md + - Keycloak: + - recipes/keycloak/index.md + - OIDC Provider: recipes/keycloak/setup-oidc-provider.md + - OpenLDAP: recipes/keycloak/authenticate-against-openldap.md + - Komga: recipes/komga.md - Linx: recipes/linx.md + - Mastodon: recipes/mastodon.md - Mealie: recipes/mealie.md + - Miniflux: recipes/miniflux.md - Minio: recipes/minio.md + - Munin: recipes/munin.md + - NextCloud: recipes/nextcloud.md - NightScout: recipes/nightscout.md - OpenLDAP: recipes/openldap.md - OwnTracks: recipes/owntracks.md - - Photoprism: recipes/photoprism.md - Paperless NG: recipes/paperless-ng.md - - Portainer: recipes/portainer.md + - Photoprism: recipes/photoprism.md + - Plex: recipes/plex.md + - phpIPAM: recipes/phpipam.md + - Portainer: recipes/portainer.md + - PrivateBin: recipes/privatebin.md - Realms: recipes/realms.md - Restic: recipes/restic.md - - RSS Bridge: recipes/rss-bridge.md + - RSS Bridge: recipes/rss-bridge.md + - Swarmprom: recipes/swarmprom.md - Tiny Tiny RSS: recipes/tiny-tiny-rss.md - Traefik: docker-swarm/traefik.md - Traefik Forward Auth: @@ -227,7 +226,7 @@ nav: # - Istio: recipes/kubernetes/wip.md # - Jaeger: kubernetes/wip.md # - Kiali: kubernetes/wip.md - # - Minio: kubernetes/wip.md + # - Mastodon: recipes/kubernetes/mastodon.md # - NGINX Ingress: kubernetes/wip.md # - Polaris: kubernetes/wip.md # - Portainer: kubernetes/wip.md @@ -315,6 +314,8 @@ theme: extra: social: + - icon: 'fontawesome/brands/mastodon' + link: 'https://so.funkypenguin.co.nz/' - icon: 'fontawesome/brands/github' link: 'https://github.com/funkypenguin' - icon: 'fontawesome/brands/twitter'