diff --git a/manuscript/ha-docker-swarm/traefik.md b/manuscript/ha-docker-swarm/traefik.md index 1a958d4..bcd1b6e 100644 --- a/manuscript/ha-docker-swarm/traefik.md +++ b/manuscript/ha-docker-swarm/traefik.md @@ -26,30 +26,11 @@ To deal with these gaps, we need a front-end load-balancer, and in this design, ## Preparation -### Prepare the host - -The traefik container is aware of the __other__ docker containers in the swarm, because it has access to the docker socket at **/var/run/docker.sock**. This allows traefik to dynamically configure itself based on the labels found on containers in the swarm, which is hugely useful. To make this functionality work on a SELinux-enabled CentOS7 host, we need to add custom SELinux policy. - -!!! tip - The following is only necessary if you're using SELinux! - -Run the following to build and activate policy to permit containers to access docker.sock: - -``` -mkdir ~/dockersock -cd ~/dockersock -curl -O https://raw.githubusercontent.com/dpw/\ -selinux-dockersock/master/Makefile -curl -O https://raw.githubusercontent.com/dpw/\ -selinux-dockersock/master/dockersock.te -make && semodule -i dockersock.pp -``` - ### Prepare traefik.toml While it's possible to configure traefik via docker command arguments, I prefer to create a config file (`traefik.toml`). This allows me to change traefik's behaviour by simply changing the file, and keeps my docker config simple. -Create `/var/data/traefik/traefik.toml` as follows: +Create `/var/data/traefikv1/traefik.toml` as follows: ``` checkNewVersion = true diff --git a/manuscript/premix/ansible/design.md b/manuscript/premix/ansible/design.md new file mode 100644 index 0000000..4861f8d --- /dev/null +++ b/manuscript/premix/ansible/design.md @@ -0,0 +1,46 @@ +# Premix via Ansible + +!!! warning "This section is under construction :hammer:" + This section is a serious work-in-progress, and reflects the current development on the [sponsors](https://github.com/sponsors/funkypenguin)'s "premix" repository + So... There may be errors and inaccuracies. Jump into [Discord](http://chat.funkypenguin.co.nz) in the #dev channel if you're encountering issues 😁 + +## Design + +The ansible playbooks / roles in premix are intended to automate the deployment of an entire stack, along with selected recipes. The following design decisions influenced how the playbook is written: + +1. Users should be able to pull down updates to the repo without encountering conflicts in config files etc which they've changed. +2. Secrets should be stored securely +3. Configuration should be centralized (i.e., one place to manage changes) +4. Duplication should be avoided +5. The user is running in a self-managed, isolated environment, and secret storage is non-critical + +## Details + +**Duplication should be avoided** + +This means that ansible will use the same source files which we use to deploy swarm stacks manually (*i.e., /kanboard/*). This has some implications: + +1. Whenever a recipe requires more than just a .yml file, we provide "sample" files. The intention of sample files is to give the user direction on what to customize in order to deploy the stack. The sample files are named for their "real" counterparts, with `-sample` suffixed. For example, the sample file for `traefikv1/traefik.toml` is `traefikv1/traefik.toml-sample`. During ansible deployment, if the "real" version of the file doesn't exist, it'll be created from a copy of the sample file. However, if the user has already created teh "real" file, it'll remain untouched. + +!!! question "Why do we do this?" + + In an ansible-based deployment, we **don't** clone the premix repo to /var/data/config. Instead, we clone it somewhere local, and then use the playbook to launch the stack, including the creation of ceph shared storage at /var/data/config. The necessary files are then **copied** from the cloned repo into `/var/data/config`, so that they can be altered by the user, backed up, etc. This separation of code from config makes it easier for users to pull down updates to the premix repo, without having to worry about merge conflicts etc for the files they've manually changed during deployment. + +**Configuration should be centralized** + +What we _don't_ want, is to manually be editing `/.env` files all over, and tracking changes to all of these. To this end, there's a `config` dictionary defined, which includes a subsection for each recipe. Here's an example: + +```yaml +config: + traefik: + dns_provider: route53 + env: + # if you're using cloudflare + # cloudflare_email: + # cloudflare_api_key: + + # if you're using route53 + AWS_ACCESS_KEY_ID: "{{ vault_config.traefik.aws_access_key_id }}" + AWS_SECRET_ACCESS_KEY: "{{ vault_config.traefik.aws_secret_access_key }}" + AWS_REGION: "" +``` \ No newline at end of file diff --git a/manuscript/premix/ansible/operation.md b/manuscript/premix/ansible/operation.md new file mode 100644 index 0000000..fb6ff2c --- /dev/null +++ b/manuscript/premix/ansible/operation.md @@ -0,0 +1,133 @@ +# Operation + +!!! warning "This section is under construction :hammer:" + This section is a serious work-in-progress, and reflects the current development on the [sponsors](https://github.com/sponsors/funkypenguin)' "premix" repository + So... There may be errors and inaccuracies. Jump into [Discord](http://chat.funkypenguin.co.nz) in the #dev channel if you're encountering issues 😁 + +The design section details **why** the ansible playbook was designed the way it is. This section outlines how to **operate** the playbook! + +## Preparation + +Clone the repo locally, onto whichever host you plan to deploy the playbook from. You'll need an up-to-date installation of Ansible. + +Now we'll be creating 3 files.. + +### Hosts + +Create a new file at `ansible/hosts`, containing a variation on this: + +``` +[your-username:children] +proxmox_servers +proxmox_vms +swarm_nodes +k3s_masters +k3s_workers + +[proxmox_servers] +splinter ansible_host=192.168.29.3 ansible_user=root + +# Declare your desired proxmox VMs here. Note that the MAC address "lines up" with_ +# the IP address - this makes troubleshooting L2 issues easier under some circumstances, +# and declaring the MAC to proxmox avoids proxmox / terraform force-restarting the VMs +# when re-running the playbook. + +[proxmox_vms] +raphael ansible_host=192.168.38.101 mac=52:54:00:38:01:01 +donatello ansible_host=192.168.38.102 mac=52:54:00:38:01:02 +leonardo ansible_host=192.168.38.103 mac=52:54:00:38:01:03 +shredder ansible_host=192.168.38.201 mac=52:54:00:38:02:01 +rocksteady ansible_host=192.168.38.202 mac=52:54:00:38:02:02 +bebop ansible_host=192.168.38.203 mac=52:54:00:38:02:03 + +[swarm_nodes] +raphael ansible_host=192.168.38.101 keepalived_priority=101 +donatello ansible_host=192.168.38.102 keepalived_priority=102 +leonardo ansible_host=192.168.38.103 keepalived_priority=103 + +[k3s_masters] +shredder ansible_host=192.168.38.201 + +[k3s_workers] +rocksteady ansible_host=192.168.38.202 +bebop ansible_host=192.168.38.203 +``` + +!!! note + + 1. Replace `your-username` in line \#1. This line makes all subsequent groups "children" of a master group based on your username, which we'll use in the following step to let you keep your configs/secrets separate from the main repo, with minimal friction. + 2. If you don't populate a section, it won't get applied. I.e., if you don't care about k8s hosts, don't create any k8s groups, and all the k8s steps in the playbook will be ignored. The same is true for swarm_nodes. + +### Config + +The variables used in the playbook are defined in `ansible/group_vars/all/main.yml`. **Your** variables are going to be defined in a group_vars file based on your username, so that they're [treated with a higher preference](https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable) than the default values. + +Create a folder under `ansible/group_vars/` to match the group name you inserted in line \#1 of your hosts file, and copy `ansible/group_vars/all/main.yml` into this folder. Any values found in this file will override any values specified in `ansible/group_vars/all/main.yml`, but any values _not_ found in your file will be inherited from `ansible/group_vars/all/main.yml`. + +!!! tip "Go to town with your delete key" + To keep your version clean, edit `ansible/group_vars//main.yml` and delete anything you don't care to change. This will keep your "override" file nice and clean. + +!!! tip "Commit `ansible/group_vars//` to your own repo" + For extra geek-fu, you could commit the contents of ``ansible/group_vars//` to your own repo, so that you can version/track your own config! + +### Secrets + +Wait, what about secrets? How are we going to store sensitive information, like API keys etc? + +We'll always need to store some secrets, like your proxmox admin credentials. We want to do this in a way which is safe from accidental git commits, as well as convenient for repeated iterations, without having to pass secrets as variables on the command-line. + +Enter [Ansible Vault](https://docs.ansible.com/ansible/latest/user_guide/vault.html#creating-encrypted-files), a handy solution for encrypting secrets in a painless way. + +Create a password file, containing a vault password (*just generate one yourself*), and store it _outside_ of the repo: + +``` +echo mysecretpassword > ~/.ansible/vault-password-geek-cookbook-premix +``` + +Create an ansible-vault encrypted file in the `group_vars//vault.yml` using this password file: + +``` +ansible-vault create --encrypt-vault-id geek-cookbook-premix vars/vault.yml +``` + +Insert your secret values into this file (*refer to `group_vars/all/01_fake_vault.yml` for placeholders*), using a prefix of `vault_`, like this: + +``` +vault_proxmox_host_password: mysekritpassword +``` + +(You can always re-edit the file by running `ansible-vault edit vars/vault.yml`) + +The vault file is encrypted using a secret you store outside the repo, and now you can safely check in and version `group_vars//vault.yml` without worrying about exposing secrets in cleartext! + +!!! tip "Editing ansible-vault files with VSCode" + If you prefer to edit your vault file using VSCode (*with all its YAML syntax checking*) to nasty-ol' CLI editors, you can set your EDITOR ENV variable by running ` export EDITOR="code --wait"`. + +## Serving + +### Deploy (on autopilot) + +To deploy the playbook, run `ansible-playbook -i host deploy.yml`. This will deploy _everything_ on autopilot, including attempting to create VMs using Proxmox, if you've the necessary hosts. + +### Deploy (selectively) + +To run the playbook selectively (i.e., maybe just deploy traefik), add the name of the role(s) to the `-t` value. This leverages ansible tags to only run tasks which match these tags (*in our case, there's a 1:1 relationship between tags and roles*). + +I.e., to deploy only ceph: + +``` +ansible-playbook -i host deploy.yml -t ceph +``` + +To deploy traefik (overlay), traefikv1, and traefik-forward-auth: + +``` +ansible-playbook -i host deploy.yml -t traefik,traefikv1,traefik-forward-auth +``` + +### Deploy (with debugging) + +If something went wrong, append `-vv` to your deploy command, for extra-verbose output :thumbsup: + + + diff --git a/manuscript/premix/kubernetes.md b/manuscript/premix/kubernetes.md new file mode 100644 index 0000000..a398fd1 --- /dev/null +++ b/manuscript/premix/kubernetes.md @@ -0,0 +1,3 @@ +!!! warning "This section is under construction :hammer:" + This section is a serious work-in-progress, and reflects the current development on the [sponsors](https://github.com/sponsors/funkypenguin)'s "premix" repository + So... There may be errors and inaccuracies. Jump into [Discord](http://chat.funkypenguin.co.nz) in the #dev channel if you're encountering issues 😁 diff --git a/manuscript/premix/start.md b/manuscript/premix/start.md new file mode 100644 index 0000000..b5189ec --- /dev/null +++ b/manuscript/premix/start.md @@ -0,0 +1,610 @@ +# Premix Repository + + "Premix" is a private git repository available to [GitHub sponsors](https://github.com/sponsors/funkypenguin), which includes: + + 1. Necessary docker-compose and env files for all published recipes + 2. Ansible playbook for deploying the cookbook stack, as well as individual recipes + 3. Helm charts for deploying deploying recipes into Kubernetes + +The intention of Premix is that sponsors can launch any recipe with just a `git pull` followed by `ansible-playbook ...` (*Docker Swarm _or_ Kubernetes*), `docker stack deploy ...` (*Docker Swarm*), or `helm install ...` (*Kubernetes*). + +## Data Layout + +Generally, each recipe with necessary files is contained within its own folder. The intention is that a sponsor could run `git clone git@github.com:funkypenguin/geek-cookbook-premix.git /var/data/config`, and the recipes would be laid out per the [data layout](/reference/data_layout/). + +Here's a sample of the directory structure: + +??? "What will I find in the pre-mix?" + ``` + . + ├── README.md + ├── ansible + │   ├── README.md + │   ├── ansible.cfg + │   ├── carefully_destroy.yml + │   ├── deploy.yml + │   ├── deploy_swarm.yml + │   ├── group_vars + │   │   └── all + │   │   ├── 01_fake_vault.yml + │   │   ├── main.yml + │   │   └── vault.yml + │   ├── hosts.example + │   └── roles + │   ├── ceph + │   │   ├── tasks + │   │   │   ├── docker-swarm.yml + │   │   │   ├── kubernetes.yml + │   │   │   └── main.yml + │   │   └── templates + │   │   ├── cluster.yaml.j2 + │   │   ├── storageclass.yaml.j2 + │   │   └── toolbox.yaml.j2 + │   ├── destroy-proxmox + │   │   ├── defaults + │   │   │   └── main.yml + │   │   ├── tasks + │   │   │   └── main.yml + │   │   └── templates + │   │   ├── main.tf.j2 + │   │   └── swarm_node.tf.j2 + │   ├── docker-stack + │   │   ├── defaults + │   │   │   └── main.yml + │   │   ├── tasks + │   │   │   └── main.yml + │   │   └── vars + │   │   └── default.yaml + │   ├── docker-swarm + │   │   └── tasks + │   │   └── main.yml + │   ├── helm + │   │   └── tasks + │   │   └── main.yml + │   ├── helm-chart + │   │   ├── defaults + │   │   │   └── main.yml + │   │   ├── tasks + │   │   │   └── main.yml + │   │   └── vars + │   │   └── default.yaml + │   ├── k3s-master + │   │   ├── README.md + │   │   ├── defaults + │   │   │   └── main.yml + │   │   ├── handlers + │   │   │   └── main.yml + │   │   ├── meta + │   │   │   └── main.yml + │   │   ├── tasks + │   │   │   └── main.yml + │   │   ├── tests + │   │   │   ├── inventory + │   │   │   └── test.yml + │   │   └── vars + │   │   └── main.yml + │   ├── k3s-worker + │   │   ├── README.md + │   │   ├── defaults + │   │   │   └── main.yml + │   │   ├── handlers + │   │   │   └── main.yml + │   │   ├── meta + │   │   │   └── main.yml + │   │   ├── tasks + │   │   │   └── main.yml + │   │   ├── tests + │   │   │   ├── inventory + │   │   │   └── test.yml + │   │   └── vars + │   │   └── main.yml + │   ├── keepalived + │   │   └── tasks + │   │   └── main.yml + │   ├── proxmox + │   │   ├── defaults + │   │   │   └── main.yml + │   │   ├── tasks + │   │   │   └── main.yml + │   │   └── templates + │   │   ├── main.tf.j2 + │   │   └── swarm_node.tf.j2 + │   ├── traefik + │   │   └── tasks + │   │   └── main.yml + │   ├── traefik-forward-auth + │   │   └── tasks + │   │   └── main.yml + │   └── traefikv1 + │   └── tasks + │   └── main.yml + ├── autopirate + │   ├── authenticated-emails.txt-sample + │   ├── autopirate.yml + │   ├── bazarr.env-sample + │   ├── headphones.env-sample + │   ├── heimdall.env-sample + │   ├── lazylibrarian.env-sample + │   ├── lidarr.env-sample + │   ├── mylar.env-sample + │   ├── nzbget.env-sample + │   ├── nzbhydra.env-sample + │   ├── ombi.env-sample + │   ├── oscarr.env-sample + │   ├── radarr.env-sample + │   ├── sabnzbd.env-sample + │   └── sonarr.env-sample + ├── bitwarden + │   ├── bitwarden.yml + │   └── readme.md + ├── bookstack + │   ├── authenticated-emails-sample.txt + │   ├── bookstack.env-sample + │   └── bookstack.yml + ├── calibre-web + │   ├── authenticated-emails-sample.txt + │   ├── calibre-web.env-sample + │   └── calibre-web.yml + ├── ceph + ├── charts + │   ├── autopirate + │   │   ├── Chart.yaml + │   │   ├── README.MD + │   │   ├── charts + │   │   │   └── funkycore-1.0.0.tgz + │   │   ├── templates + │   │   │   ├── NOTES.txt + │   │   │   ├── _helpers.tpl + │   │   │   ├── apps + │   │   │   │   ├── bazarr + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── headphones + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── lazylibrarian + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── lidarr + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── mylar + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── nzbget + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── configmap.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── nzbhydra + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── ombi + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── radarr + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── rtorrent + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   ├── sabnzbd + │   │   │   │   │   ├── config-pvc.yaml + │   │   │   │   │   ├── configmap.yaml + │   │   │   │   │   ├── deployment.yaml + │   │   │   │   │   └── service.yaml + │   │   │   │   └── sonarr + │   │   │   │   ├── config-pvc.yaml + │   │   │   │   ├── deployment.yaml + │   │   │   │   └── service.yaml + │   │   │   ├── download-pvc.yaml + │   │   │   ├── ingress + │   │   │   │   ├── kube.yaml + │   │   │   │   └── traefik.yaml + │   │   │   └── media-pvc.yaml + │   │   └── values.yaml + │   ├── autopirate-storage + │   │   └── Chart.lock + │   ├── funkycore + │   │   ├── Chart.yaml + │   │   ├── templates + │   │   │   └── _helpers.tpl + │   │   └── values.yaml + │   ├── heimdall + │   │   └── Chart.lock + │   ├── huginn + │   │   ├── Chart.yaml + │   │   ├── README.MD + │   │   ├── charts + │   │   │   └── postgresql-8.3.0.tgz + │   │   ├── myvalues.yaml + │   │   ├── templates + │   │   │   ├── _helpers.tpl + │   │   │   ├── deployment.yaml + │   │   │   ├── ingress + │   │   │   │   ├── kube.yaml + │   │   │   │   └── traefik.yaml + │   │   │   ├── secret.yaml + │   │   │   └── service.yaml + │   │   └── values.yaml + │   ├── lidarr + │   │   └── Chart.lock + │   ├── rtorrent + │   │   └── Chart.lock + │   └── wash-hands + │   ├── Chart.yaml + │   ├── charts + │   │   ├── cert-manager-v0.13.0.tgz + │   │   ├── goldilocks-2.1.0.tgz + │   │   ├── grafana-1.2.5.tgz + │   │   ├── kube-eagle-1.1.5.tgz + │   │   ├── kured-1.4.2.tgz + │   │   ├── loki-0.25.0.tgz + │   │   ├── nginx-ingress-1.30.1.tgz + │   │   ├── prometheus-operator-0.11.1.tgz + │   │   └── promtail-0.18.0.tgz + │   ├── manifests + │   │   └── wash-hands + │   │   ├── charts + │   │   │   ├── cert-manager + │   │   │   │   └── templates + │   │   │   │   ├── cainjector-deployment.yaml + │   │   │   │   ├── cainjector-rbac.yaml + │   │   │   │   ├── cainjector-serviceaccount.yaml + │   │   │   │   ├── deployment.yaml + │   │   │   │   ├── rbac.yaml + │   │   │   │   ├── service.yaml + │   │   │   │   ├── serviceaccount.yaml + │   │   │   │   ├── webhook-deployment.yaml + │   │   │   │   ├── webhook-mutating-webhook.yaml + │   │   │   │   ├── webhook-rbac.yaml + │   │   │   │   ├── webhook-service.yaml + │   │   │   │   ├── webhook-serviceaccount.yaml + │   │   │   │   └── webhook-validating-webhook.yaml + │   │   │   ├── goldilocks + │   │   │   │   └── templates + │   │   │   │   ├── controller-clusterrole.yaml + │   │   │   │   ├── controller-clusterrolebinding.yaml + │   │   │   │   ├── controller-deployment.yaml + │   │   │   │   ├── controller-serviceaccount.yaml + │   │   │   │   ├── dashboard-clusterrole.yaml + │   │   │   │   ├── dashboard-clusterrolebinding.yaml + │   │   │   │   ├── dashboard-deployment.yaml + │   │   │   │   ├── dashboard-service.yaml + │   │   │   │   └── dashboard-serviceaccount.yaml + │   │   │   ├── kured + │   │   │   │   └── templates + │   │   │   │   ├── clusterrole.yaml + │   │   │   │   ├── clusterrolebinding.yaml + │   │   │   │   ├── daemonset.yaml + │   │   │   │   ├── role.yaml + │   │   │   │   ├── rolebinding.yaml + │   │   │   │   └── serviceaccount.yaml + │   │   │   └── nginx-ingress + │   │   │   └── templates + │   │   │   ├── clusterrole.yaml + │   │   │   ├── clusterrolebinding.yaml + │   │   │   ├── controller-deployment.yaml + │   │   │   ├── controller-role.yaml + │   │   │   ├── controller-rolebinding.yaml + │   │   │   ├── controller-service.yaml + │   │   │   ├── controller-serviceaccount.yaml + │   │   │   ├── default-backend-deployment.yaml + │   │   │   ├── default-backend-service.yaml + │   │   │   └── default-backend-serviceaccount.yaml + │   │   └── templates + │   │   ├── issuer-letsencrypt-staging-cloudflare.yaml + │   │   ├── issuer-letsencrypt-staging.yaml + │   │   ├── secret.yaml + │   │   └── test.yaml + │   ├── myvalues.yaml + │   ├── templates + │   │   ├── issuer-letsencrypt-prod.yaml + │   │   ├── issuer-letsencrypt-staging.yaml + │   │   └── secret.yaml + │   └── values.yaml + ├── cryptominer + │   ├── monitor-gpu.sh + │   └── stats-to-influxdb.sh + ├── dex + │   ├── README.md + │   ├── config.yml.example + │   └── dex.yml + ├── diskover + │   ├── diskover.env-sample + │   ├── diskover.yml + │   └── diskoverdash.env-sample + ├── docker-cleanup + │   ├── docker-cleanup.env-sample + │   └── docker-cleanup.yml + ├── dozzle + │   ├── authenticated-emails.txt-sample + │   ├── dozzle.env-sample + │   └── dozzle.yml + ├── duplicacy + │   ├── authenticated-emails.txt-sample + │   ├── duplicacy.env + │   └── duplicacy.yml + ├── duplicity + │   ├── duplicity.env-sample + │   └── duplicity.yml + ├── elkarbackup + │   ├── elkarbackup.env-sample + │   ├── elkarbackup.yml + │   └── elkarbackup.yml.proxy + ├── emby + │   ├── emby.env-sample + │   └── emby.yml + ├── filebrowser + │   ├── README.md + │   ├── config.json + │   ├── filebrowser.env.sample + │   ├── filebrowser.yml + │   ├── hostname + │   ├── hosts + │   └── resolv.conf + ├── ghost + │   └── ghost.yml + ├── gitlab + │   └── gitlab.yml + ├── gollum + │   ├── authenticated-emails-sample.txt + │   ├── gollum.env-sample + │   └── gollum.yml + ├── hackmd + │   ├── authenticated-emails-sample.txt + │   ├── hackmd-backup.env + │   ├── hackmd.env + │   └── hackmd.yml + ├── homeassistant + │   ├── README.md + │   ├── grafana.env-sample + │   ├── homeassistant.env-sample + │   └── homeassistant.yml + ├── huginn + │   ├── huginn.env-sample + │   ├── huginn.yml + │   └── kubernetes + │   ├── app.yml + │   ├── db-persistent-volumeclaim.yml + │   ├── db.yml + │   ├── ingress.yml + │   └── namespace.yml + ├── instapy + │   └── instapy.yml + ├── jellyfin + │   ├── jellyfin.env-sample + │   ├── jellyfin.yml + │   └── readme.md + ├── kanboard + │   ├── authenticated-emails-sample.txt + │   ├── kanboard.env-sample + │   ├── kanboard.yml + │   └── kubernetes + │   ├── app-persistent-volumeclaim.yml + │   ├── app.yml + │   ├── config.php + │   ├── ingress.yml + │   └── namespace.yml + ├── keycloak + │   ├── keycloak-backup.env-sample + │   ├── keycloak.env-sample + │   └── keycloak.yml + ├── mailserver + │   ├── mailserver.env-sample + │   └── mailserver.yml + ├── mastodon + │   └── mastodon.yml + ├── mattermost + │   ├── mattermost-backup.env-sample + │   ├── mattermost.env-sample + │   └── mattermost.yml + ├── mayan + │   ├── authenticated-emails-sample.txt + │   ├── mayan-backup.env + │   ├── mayan.env + │   └── mayan.yml + ├── miniflux + │   ├── kubernetes + │   │   ├── app.yml + │   │   ├── db-persistent-volumeclaim.yml + │   │   ├── db.yml + │   │   ├── ingress.yml + │   │   └── namespace.yml + │   ├── miniflux-backup.env-sample + │   ├── miniflux.env-sample + │   └── miniflux.yml + ├── minio + │   ├── minio.env-sample + │   └── minio.yml + ├── munin + │   ├── munin.env-sample + │   └── munin.yml + ├── nextcloud + │   ├── kubernetes + │   │   ├── app-persistent-volumeclaim.yml + │   │   ├── app.yml + │   │   ├── db-persistent-volumeclaim.yml + │   │   ├── db.yml + │   │   ├── ingress.yml + │   │   └── namespace.yml + │   ├── nextcloud.env-sample + │   └── nextcloud.yml + ├── owntracks + │   ├── owntracks.env + │   └── owntracks.yml + ├── phpipam + │   ├── nginx.conf-sample + │   ├── phpipam-backup.env-sample + │   ├── phpipam.env-sample + │   └── phpipam.yml + ├── piwik + │   └── piwik.yml + ├── plex + │   ├── nowshowing.env-sample + │   ├── plex.env-sample + │   ├── plex.yml + │   └── tautulli.env-sample + ├── portainer + │   ├── portainer-agent-stack.yml + │   ├── portainer.env-sample + │   ├── portainer.yml + │   └── portainer_with_oauth.yml + ├── privatebin + │   ├── authenticated-emails.txt-sample + │   ├── privatebin.env-sample + │   └── privatebin.yml + ├── realms + │   ├── authenticated-emails-sample.txt + │   ├── bookstack_authenticated-emails-sample.txt + │   ├── realms.env + │   └── realms.yml + ├── registry + │   ├── registry-mirror-config.yml + │   └── registry.yml + ├── shaarli + │   ├── authenticated-emails-sample.txt + │   ├── shaarli.env + │   ├── shaarli.env-sample + │   └── shaarli.yml + ├── shepherd + │   ├── shepherd.env-sample + │   └── shepherd.yml + ├── swarmprom + │   ├── Caddyfile + │   ├── alertmanager.env-sample + │   ├── grafana.env-sample + │   ├── prometheus.env-sample + │   ├── swarm_node.rules.yml + │   ├── swarm_task.rules.yml + │   ├── swarmprom.yml + │   └── unsee.env-sample + ├── tools + │   ├── README.MD.TEMPLATE + │   ├── aliases.sh + │   ├── chart.sh + │   ├── helm-boilerplate + │   │   ├── Chart.yaml + │   │   ├── templates + │   │   │   ├── NOTES.TXT + │   │   │   ├── _helpers.tpl + │   │   │   ├── deployment.yaml + │   │   │   ├── ingress + │   │   │   │   ├── kube.yaml + │   │   │   │   └── traefik.yaml + │   │   │   ├── secret.yaml + │   │   │   └── service.yaml + │   │   └── values.yaml + │   └── mkreadme.py + ├── traefik + │   ├── traefik.env-sample + │   └── traefik.yml + ├── traefik-forward-auth + │   ├── README-traefik-with-non-swarm-backends.txt + │   ├── traefik-forward-auth.env-sample + │   └── traefik-forward-auth.yml + ├── traefikv1 + │   ├── README-traefik-with-non-swarm-backends.txt + │   ├── authenticated-emails.txt-sample + │   ├── traefik.toml-sample + │   ├── traefikv1.env-sample + │   └── traefikv1.yml + ├── ttrss + │   ├── ttrss.env-sample + │   └── ttrss.yml + ├── turtle-pool + │   └── kubernetes + │   ├── README.md + │   ├── config.js + │   ├── custom.css + │   ├── daemon-persistent-volumeclaim.yml + │   ├── daemon.yml + │   ├── namespace.yml + │   ├── pool-ingress.yml + │   ├── pool-persistent-volumeclaim.yml + │   ├── pool-service-nodeport.yml + │   ├── pool-service.yml + │   ├── pool.yml + │   ├── redis-persistent-volumeclaim.yml + │   ├── redis.conf + │   ├── redis.yml + │   ├── trtl.json-example + │   ├── wallet-persistent-volumeclaim.yml + │   ├── wallet.conf-example + │   ├── wallet.yml + │   └── webhook_token.secret-example + ├── unifi + │   ├── authenticated-emails-sample.txt + │   ├── kubernetes + │   │   ├── authenticated-emails-sample.txt + │   │   ├── controller-persistent-volumeclaim.yml + │   │   ├── ingress.yml + │   │   ├── namespace.yml + │   │   ├── proxy.yaml + │   │   ├── service-controller-external.yml + │   │   └── unifi.yaml + │   ├── unifi.yml + │   └── unifi_with_proxy.yml + ├── wallabag + │   ├── authenticated-emails.txt-sample + │   ├── wallabag-backup.env-sample + │   ├── wallabag.env-sample + │   └── wallabag.yml + ├── wash-hands + │   ├── README.md + │   ├── azure-pipelines.yml + │   ├── manifests + │   │   └── wash-hands + │   │   └── charts + │   │   ├── goldilocks + │   │   │   └── templates + │   │   │   ├── controller-clusterrole.yaml + │   │   │   ├── controller-clusterrolebinding.yaml + │   │   │   ├── controller-deployment.yaml + │   │   │   ├── controller-serviceaccount.yaml + │   │   │   ├── dashboard-clusterrole.yaml + │   │   │   ├── dashboard-clusterrolebinding.yaml + │   │   │   ├── dashboard-deployment.yaml + │   │   │   ├── dashboard-service.yaml + │   │   │   └── dashboard-serviceaccount.yaml + │   │   ├── kured + │   │   │   └── templates + │   │   │   ├── clusterrole.yaml + │   │   │   ├── clusterrolebinding.yaml + │   │   │   ├── daemonset.yaml + │   │   │   ├── role.yaml + │   │   │   ├── rolebinding.yaml + │   │   │   └── serviceaccount.yaml + │   │   └── nginx-ingress + │   │   └── templates + │   │   ├── clusterrole.yaml + │   │   ├── clusterrolebinding.yaml + │   │   ├── controller-deployment.yaml + │   │   ├── controller-role.yaml + │   │   ├── controller-rolebinding.yaml + │   │   ├── controller-service.yaml + │   │   ├── controller-serviceaccount.yaml + │   │   ├── default-backend-deployment.yaml + │   │   ├── default-backend-service.yaml + │   │   └── default-backend-serviceaccount.yaml + │   └── scripts + │   └── local-ci.sh + ├── wekan + │   ├── authenticated-emails-sample.txt + │   ├── wekan.env-sample + │   └── wekan.yml + └── wetty + ├── wetty.env-sample + └── wetty.yml + + 166 directories, 422 files + ``` \ No newline at end of file diff --git a/manuscript/premix/swarm.md b/manuscript/premix/swarm.md new file mode 100644 index 0000000..a398fd1 --- /dev/null +++ b/manuscript/premix/swarm.md @@ -0,0 +1,3 @@ +!!! warning "This section is under construction :hammer:" + This section is a serious work-in-progress, and reflects the current development on the [sponsors](https://github.com/sponsors/funkypenguin)'s "premix" repository + So... There may be errors and inaccuracies. Jump into [Discord](http://chat.funkypenguin.co.nz) in the #dev channel if you're encountering issues 😁 diff --git a/manuscript/recipes/mattermost.md b/manuscript/recipes/mattermost.md index a6ab044..7739985 100644 --- a/manuscript/recipes/mattermost.md +++ b/manuscript/recipes/mattermost.md @@ -20,7 +20,7 @@ We'll need several directories to bind-mount into our container, so create them ``` mkdir -p /var/data/mattermost/{cert,config,data,logs,plugins,database-dump} -mkdir -p /var/data/realtime/mattermost/database +mkdir -p /var/data/runtime/mattermost/database ``` ### Prepare environment @@ -61,7 +61,7 @@ services: image: mattermost/mattermost-prod-db env_file: /var/data/config/mattermost/mattermost.env volumes: - - /var/data/realtime/mattermost/database:/var/lib/postgresql/data + - /var/data/runtime/mattermost/database:/var/lib/postgresql/data networks: - internal diff --git a/manuscript/reference/data_layout.md b/manuscript/reference/data_layout.md index 8439947..f8c6252 100644 --- a/manuscript/reference/data_layout.md +++ b/manuscript/reference/data_layout.md @@ -6,12 +6,12 @@ So that we can confidently backup all our data, I've setup a data layout as foll ## Configuration data -Configuration data goes into /var/data/config/[recipe name], and is typically only a docker-compose .yml, and a .env file +Configuration data goes into `/var/data/config/[recipe name]`, and is typically only a docker-compose .yml, and a .env file ## Runtime data -Realtime data (typically database files or files-in-use) are stored in /var/data/realtime/[recipe-name], and are **excluded** from backup (_They change constantly, and cannot be safely restored_). +Realtime data (*typically database files or files-in-use*) are stored in `/var/data/runtime/[recipe-name]`, and are **excluded** from backup (_They change constantly, and cannot be safely restored_). ## Static data -Static data goes into /var/data/[recipe name], and includes anything that can be safely backed up while a container is running. This includes database exports of the runtime data above. \ No newline at end of file +Static data goes into `/var/data/[recipe name]`, and includes anything that can be safely backed up while a container is running. This includes database exports of the runtime data above. \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index b2887c1..6d4d6bb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -159,6 +159,13 @@ nav: # - NextCloud: recipes/kubernetes/nextcloud.md # - phpIPAM: recipes/kubernetes/phpipam.md # - PrivateBin: recipes/kubernetes/privatebin.md + - Premix: + - Start: premix/start.md + - Ansible: + - Operation: premix/ansible/operation.md + - Design: premix/ansible/design.md + - Swarm: premix/swarm.md + - Kubernetes: premix/kubernetes.md - CHANGELOG: - CHANGELOG: CHANGELOG.md - Support: