mirror of
https://github.com/funkypenguin/geek-cookbook/
synced 2025-12-13 01:36:23 +00:00
Avoid commiting staging directory
This commit is contained in:
David Young
3
.gitignore
vendored
3
.gitignore
vendored
@@ -1,6 +1,9 @@
|
||||
# Don't include built site
|
||||
site/
|
||||
|
||||
# Don't include staging area for publishing
|
||||
publish/
|
||||
|
||||
# Don't include random notes
|
||||
notes/
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ whoami.md
|
||||
|
||||
sections/ha-docker-swarm.md
|
||||
ha-docker-swarm/design.md
|
||||
ha-docker-swarm/vms.md
|
||||
ha-docker-swarm/nodes.md
|
||||
ha-docker-swarm/shared-storage-ceph.md
|
||||
ha-docker-swarm/shared-storage-gluster.md
|
||||
ha-docker-swarm/keepalived.md
|
||||
|
||||
@@ -5,14 +5,34 @@ For truly highly-available services with Docker containers, we need an orchestra
|
||||
## Ingredients
|
||||
|
||||
!!! summary
|
||||
* [X] 3 x modern linux hosts (*bare-metal or VMs*). A reasonable minimum would be:
|
||||
* 2 x vCPU
|
||||
* 2GB RAM
|
||||
* 20GB HDD
|
||||
* [X] Hosts must be within the same subnet, and connected on a low-latency link (*i.e., no WAN links*)
|
||||
Existing
|
||||
|
||||
* [X] 3 x nodes (*bare-metal or VMs*), each with:
|
||||
* A mainstream Linux OS (*tested on either [CentOS](https://www.centos.org) 7+ or [Ubuntu](http://releases.ubuntu.com) 16.04+*)
|
||||
* At least 2GB RAM
|
||||
* At least 20GB disk space (_but it'll be tight_)
|
||||
* [X] Connectivity to each other within the same subnet, and on a low-latency link (_i.e., no WAN links_)
|
||||
|
||||
## Preparation
|
||||
|
||||
### Bash auto-completion
|
||||
|
||||
Add some handy bash auto-completion for docker. Without this, you'll get annoyed that you can't autocomplete ```docker stack deploy <blah> -c <blah.yml>``` commands.
|
||||
|
||||
```
|
||||
cd /etc/bash_completion.d/
|
||||
curl -O https://raw.githubusercontent.com/docker/cli/b75596e1e4d5295ac69b9934d1bd8aff691a0de8/contrib/completion/bash/docker
|
||||
```
|
||||
|
||||
Install some useful bash aliases on each host
|
||||
```
|
||||
cd ~
|
||||
curl -O https://raw.githubusercontent.com/funkypenguin/geek-cookbook/master/examples/scripts/gcb-aliases.sh
|
||||
echo 'source ~/gcb-aliases.sh' >> ~/.bash_profile
|
||||
```
|
||||
|
||||
## Serving
|
||||
|
||||
### Release the swarm!
|
||||
|
||||
Now, to launch a swarm. Pick a target node, and run `docker swarm init`
|
||||
@@ -145,22 +165,13 @@ services:
|
||||
|
||||
Launch shepherd by running ```docker stack deploy shepherd -c /var/data/config/shepherd/shepherd.yml```, and then just forget about it, comfortable in the knowledge that every day, Shepherd will check that your images are the latest available, and if not, will destroy and recreate the container on the latest available image.
|
||||
|
||||
### Summary
|
||||
|
||||
### Tweaks
|
||||
!!! summary
|
||||
Created
|
||||
|
||||
* [X] [Docker swarm cluster](/ha-docker-swarm/design/)
|
||||
|
||||
Add some handy bash auto-completion for docker. Without this, you'll get annoyed that you can't autocomplete ```docker stack deploy <blah> -c <blah.yml>``` commands.
|
||||
|
||||
```
|
||||
cd /etc/bash_completion.d/
|
||||
curl -O https://raw.githubusercontent.com/docker/cli/b75596e1e4d5295ac69b9934d1bd8aff691a0de8/contrib/completion/bash/docker
|
||||
```
|
||||
|
||||
Install some useful bash aliases on each host
|
||||
```
|
||||
cd ~
|
||||
curl -O https://raw.githubusercontent.com/funkypenguin/geek-cookbook/master/examples/scripts/gcb-aliases.sh
|
||||
echo 'source ~/gcb-aliases.sh' >> ~/.bash_profile
|
||||
```
|
||||
|
||||
## Chef's Notes
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Virtual Machines
|
||||
# Nodes
|
||||
|
||||
Let's start building our cluster. You can use either bare-metal machines or virtual machines - the configuration would be the same. Given that most readers (myself included) will be using virtual infrastructure, from now on I'll be referring strictly to VMs.
|
||||
Let's start building our cluster. You can use either bare-metal machines or virtual machines - the configuration would be the same. To avoid confusion, I'll be referring to these as "nodes" from now on.
|
||||
|
||||
!!! note
|
||||
In 2017, I **initially** chose the "[Atomic](https://www.projectatomic.io/)" CentOS/Fedora image for the swarm hosts, but later found its outdated version of Docker to be problematic with advanced features like GPU transcoding (in [Plex](/recipes/plex/)), [Swarmprom](/recipes/swarmprom/), etc. In the end, I went mainstream and simply preferred a modern Ubuntu installation.
|
||||
@@ -8,25 +8,18 @@ Let's start building our cluster. You can use either bare-metal machines or virt
|
||||
## Ingredients
|
||||
|
||||
!!! summary "Ingredients"
|
||||
3 x Virtual Machines, each with:
|
||||
New in this recipe:
|
||||
|
||||
* [ ] A mainstream Linux OS (*tested on either [CentOS](https://www.centos.org) 7+ or [Ubuntu](http://releases.ubuntu.com) 16.04+*)
|
||||
* [ ] At least 2GB RAM
|
||||
* [ ] At least 20GB disk space (_but it'll be tight_)
|
||||
* [ ] 3 x nodes (*bare-metal or VMs*), each with:
|
||||
* A mainstream Linux OS (*tested on either [CentOS](https://www.centos.org) 7+ or [Ubuntu](http://releases.ubuntu.com) 16.04+*)
|
||||
* At least 2GB RAM
|
||||
* At least 20GB disk space (_but it'll be tight_)
|
||||
* [ ] Connectivity to each other within the same subnet, and on a low-latency link (_i.e., no WAN links_)
|
||||
|
||||
|
||||
## Preparation
|
||||
|
||||
### Install Virtual machines
|
||||
|
||||
1. Install / launch virtual machines.
|
||||
2. The default username on CentOS atomic is "centos", and you'll have needed to supply your SSH key during the build process.
|
||||
|
||||
!!! tip
|
||||
If you're not using a platform with cloud-init support (i.e., you're building a VM manually, not provisioning it through a cloud provider), you'll need to refer to [trick #1](https://spinningmatt.wordpress.com/2014/01/08/a-recipe-for-starting-cloud-images-with-virt-install/) and [trick #2](http://blog.oddbit.com/2015/03/10/booting-cloud-images-with-libvirt/) for a means to override the automated setup, apply a manual password to the CentOS account, and enable SSH password logins.
|
||||
|
||||
### Permit connectivity between hosts
|
||||
### Permit connectivity
|
||||
|
||||
Most modern Linux distributions include firewall rules which only only permit minimal required incoming connections (like SSH). We'll want to allow all traffic between our nodes. The steps to achieve this in CentOS/Ubuntu are a little different...
|
||||
|
||||
@@ -52,7 +45,7 @@ Install the (*non-default*) persistent iptables tools, by running `apt-get insta
|
||||
|
||||
And refresh your running iptables rules with `iptables-restore < /etc/iptables/rules.v4`
|
||||
|
||||
### Enable host resolution
|
||||
### Enable hostname resolution
|
||||
|
||||
Depending on your hosting environment, you may have DNS automatically setup for your VMs. If not, it's useful to set up static entries in /etc/hosts for the nodes. For example, I setup the following:
|
||||
|
||||
@@ -74,9 +67,14 @@ ln -sf /usr/share/zoneinfo/<your timezone> /etc/localtime
|
||||
|
||||
After completing the above, you should have:
|
||||
|
||||
```
|
||||
[X] 3 x fresh linux instances, ready to become swarm nodes
|
||||
```
|
||||
!!! summary "Summary"
|
||||
Deployed in this recipe:
|
||||
|
||||
* [X] 3 x nodes (*bare-metal or VMs*), each with:
|
||||
* A mainstream Linux OS (*tested on either [CentOS](https://www.centos.org) 7+ or [Ubuntu](http://releases.ubuntu.com) 16.04+*)
|
||||
* At least 2GB RAM
|
||||
* At least 20GB disk space (_but it'll be tight_)
|
||||
* [X] Connectivity to each other within the same subnet, and on a low-latency link (_i.e., no WAN links_)
|
||||
|
||||
## Chef's Notes
|
||||
|
||||
@@ -42,14 +42,14 @@ COOKIE_DOMAINS=example.com
|
||||
|
||||
### Prepare the docker service config
|
||||
|
||||
This is a small container, you can simply add the following content to the existing `traefik.yml` deployed in the previous [Traefik](/recipes/traefik/) recipe:
|
||||
This is a small container, you can simply add the following content to the existing `traefik-app.yml` deployed in the previous [Traefik](/recipes/traefik/) recipe:
|
||||
|
||||
```
|
||||
traefik-forward-auth:
|
||||
image: thomseddon/traefik-forward-auth
|
||||
env_file: /var/data/config/keycloak/traefik-forward-auth.env
|
||||
networks:
|
||||
- public
|
||||
- traefik_public
|
||||
# Uncomment these lines if you're using auth host mode
|
||||
#deploy:
|
||||
# labels:
|
||||
@@ -66,7 +66,7 @@ If you're not confident that forward authentication is working, add a simple "wh
|
||||
whoami:
|
||||
image: containous/whoami
|
||||
networks:
|
||||
- public
|
||||
- traefik_public
|
||||
deploy:
|
||||
labels:
|
||||
- traefik.frontend.rule=Host:whoami.example.com
|
||||
@@ -85,7 +85,7 @@ If you're not confident that forward authentication is working, add a simple "wh
|
||||
|
||||
### Launch
|
||||
|
||||
Redeploy traefik with ```docker stack deploy traefik -c /var/data/traefik/docker-compose.yml```, to launch the traefik-forward-auth container.
|
||||
Redeploy traefik with ```docker stack deploy traefik-app -c /var/data/traefik/traeifk-app.yml```, to launch the traefik-forward-auth container.
|
||||
|
||||
### Test
|
||||
|
||||
|
||||
@@ -16,11 +16,11 @@ To deal with these gaps, we need a front-end load-balancer, and in this design,
|
||||
## Ingredients
|
||||
|
||||
!!! summary "You'll need"
|
||||
Already deployed:
|
||||
Existing
|
||||
|
||||
* [X] [Docker swarm cluster](/ha-docker-swarm/design/) with [persistent shared storage](/ha-docker-swarm/shared-storage-ceph)
|
||||
|
||||
New to this recipe:
|
||||
New
|
||||
|
||||
* [ ] Access to update your DNS records for manual/automated [LetsEncrypt](https://letsencrypt.org/docs/challenge-types/) DNS-01 validation, or ingress HTTP/HTTPS for HTTP-01 validation
|
||||
|
||||
|
||||
@@ -22,9 +22,12 @@ Bitwarden is a free and open source password management solution for individuals
|
||||
|
||||
## Ingredients
|
||||
|
||||
1. [Docker swarm cluster](/ha-docker-swarm/design/) with [persistent shared storage](/ha-docker-swarm/shared-storage-ceph.md)
|
||||
2. [Traefik](/ha-docker-swarm/traefik_public) configured per design
|
||||
3. DNS entry for the hostname you intend to use, pointed to your [keepalived](ha-docker-swarm/keepalived/) IP
|
||||
!!! summary "Ingredients"
|
||||
Existing:
|
||||
|
||||
1. [X] [Docker swarm cluster](/ha-docker-swarm/design/) with [persistent shared storage](/ha-docker-swarm/shared-storage-ceph.md)
|
||||
2. [X] [Traefik](/ha-docker-swarm/traefik_public) configured per design
|
||||
3. [X] DNS entry for the hostname you intend to use, pointed to your [keepalived](ha-docker-swarm/keepalived/) IP
|
||||
|
||||
## Preparation
|
||||
|
||||
@@ -35,6 +38,12 @@ We'll need to create a directory to bind-mount into our container, so create `/v
|
||||
```
|
||||
mkdir /var/data/bitwarden
|
||||
```
|
||||
### Setup environment
|
||||
|
||||
Create `/var/data/config/bitwarden/bitwarden.env`, and **leave it empty for now**.
|
||||
|
||||
!!! question
|
||||
What, why an empty env file? Well, the container supports lots of customizations via environment variables, for things like toggling self-registration, 2FA, etc. These are too complex to go into for this recipe, but readers are recommended to review the [dani-garcia/bitwarden_rs wiki](https://github.com/dani-garcia/bitwarden_rs), and customize their installation to suite.
|
||||
|
||||
### Setup Docker Swarm
|
||||
|
||||
@@ -88,10 +97,7 @@ Once you've created your account, jump over to https://bitwarden.com/#download a
|
||||
## Chef's Notes
|
||||
|
||||
1. You'll notice we're not using the *official* container images (*[all 6 of them required](https://help.bitwarden.com/article/install-on-premise/#install-bitwarden)!)*, but rather a [more lightweight version ideal for self-hosting](https://hub.docker.com/r/mprasil/bitwarden). All of the elements are contained within a single container, and SQLite is used for the database backend.
|
||||
2. The inclusion of Bitwarden was due to the efforts of @gkoerk in our [Discord server](http://chat.funkypenguin.co.nz)- Thanks Gerry!
|
||||
2. As mentioned above, readers should refer to the [dani-garcia/bitwarden_rs wiki](https://github.com/dani-garcia/bitwarden_rs) for details on customizing the behaviour of Bitwarden.
|
||||
3. The inclusion of Bitwarden was due to the efforts of @gkoerk in our [Discord server](http://chat.funkypenguin.co.nz)- Thanks Gerry!
|
||||
|
||||
### Tip your waiter (support me) 👏
|
||||
|
||||
Did you receive excellent service? Want to make your waiter happy? (_..and support development of current and future recipes!_) See the [support](/support/) page for (_free or paid)_ ways to say thank you! 👏
|
||||
|
||||
### Your comments? 💬
|
||||
|
||||
27
manuscript/stylesheets/mailchimp.css
Normal file
27
manuscript/stylesheets/mailchimp.css
Normal file
@@ -0,0 +1,27 @@
|
||||
#mailchimp {
|
||||
background: #f8f8f8;
|
||||
color: #3D6392;
|
||||
padding: 20px 15px;
|
||||
}
|
||||
#mailchimp input {
|
||||
border: medium none;
|
||||
color: gray;
|
||||
font-size: .8rem;
|
||||
margin-bottom: 10px;
|
||||
padding: 8px 10px;
|
||||
width: 300px;
|
||||
}
|
||||
#mailchimp input.email { background: #fff }
|
||||
#mailchimp input.name { background: #fff}
|
||||
#mailchimp input[type="submit"] {
|
||||
background: #795548;
|
||||
color: #fff;
|
||||
cursor: pointer;
|
||||
width: 35%;
|
||||
padding: 8px 0;
|
||||
font-size: .8rem;
|
||||
transition: background-color .25s cubic-bezier(.1,.7,.1,1),color .25s cubic-bezier(.1,.7,.1,1);
|
||||
}
|
||||
#mailchimp input[type="seeubmit"]:hover {
|
||||
color: #F2C94C;
|
||||
}
|
||||
@@ -15,7 +15,7 @@ repo_url: 'https://github.com/funkypenguin/geek-cookbook'
|
||||
docs_dir: 'manuscript'
|
||||
|
||||
# Copyright
|
||||
copyright: 'Copyright © 2016 - 2019 David Young'
|
||||
copyright: 'Copyright © 2016 - 2019 David Young, Funky Penguin Limited'
|
||||
|
||||
|
||||
#theme_dir: mkdocs-material
|
||||
@@ -27,7 +27,7 @@ nav:
|
||||
- whoami: whoami.md
|
||||
- Docker Swarm:
|
||||
- Design: ha-docker-swarm/design.md
|
||||
- VMs: ha-docker-swarm/vms.md
|
||||
- Nodes: ha-docker-swarm/nodes.md
|
||||
- Shared Storage (Ceph): ha-docker-swarm/shared-storage-ceph.md
|
||||
- Shared Storage (GlusterFS): ha-docker-swarm/shared-storage-gluster.md
|
||||
- Keepalived: ha-docker-swarm/keepalived.md
|
||||
@@ -170,6 +170,9 @@ google_analytics:
|
||||
- 'UA-139253-18'
|
||||
- 'auto'
|
||||
|
||||
extra_css:
|
||||
- 'stylesheets/mailchimp.css'
|
||||
|
||||
#extra_javascript:
|
||||
# - 'extras/javascript/discord.js'
|
||||
|
||||
|
||||
28
scripts/build.sh
Executable file
28
scripts/build.sh
Executable file
@@ -0,0 +1,28 @@
|
||||
#!/bin/bash
|
||||
# This script prepares mkdocs for a build (there are some adjustments to be made to the recipes before publishing)
|
||||
|
||||
# Copy the contents of "manuscript" to a new "publish" folder
|
||||
mkdir -p publish
|
||||
mkdir -p publish/overrides
|
||||
cp -r manuscript publish/
|
||||
cp mkdocs.yml publish/
|
||||
|
||||
# Append a common footer to all recipes
|
||||
for i in `find publish/manuscript -name "*.md"`
|
||||
do
|
||||
# Does this recipe already have a "tip your waiter" section?
|
||||
grep -q "Tip your waiter" $i
|
||||
if [ $? -eq 1 ]
|
||||
then
|
||||
echo -e "\n" >> $i
|
||||
cat scripts/recipe-footer.md >> $i
|
||||
else
|
||||
echo "WARNING - hard-coded footer exists in $i"
|
||||
fi
|
||||
done
|
||||
|
||||
# Now build the docs for publishing
|
||||
mkdocs build -f publish/mkdocs.yml
|
||||
|
||||
# Setup any necessary netlify redirects
|
||||
cp netlify_redirects.txt publish/site/_redirects
|
||||
28
scripts/recipe-footer.md
Normal file
28
scripts/recipe-footer.md
Normal file
@@ -0,0 +1,28 @@
|
||||
### Tip your waiter (support me) 👏
|
||||
|
||||
Did you receive excellent service? Want to make your waiter happy? (_..and support development of current and future recipes!_) See the [support](/support/) page for (_free or paid)_ ways to say thank you! 👏
|
||||
|
||||
### Flirt with your waiter (subscribe to updates) :kissing_heart::love_letter:
|
||||
|
||||
Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the [RSS feed](https://mastodon.social/@geekcookbook_changes.atom), or leave your email address below, and we'll keep you updated. (*double-opt-in, no monkey business, no spam either - check the [archive](https://us16.campaign-archive.com/home/?u=a1d9cee4402be76497a2baf49&id=10e284530a) for proof!*)
|
||||
|
||||
<!-- Begin Mailchimp Signup Form -->
|
||||
<div id="mailchimp">
|
||||
<form action="https://funkypenguin.us16.list-manage.com/subscribe/post?u=a1d9cee4402be76497a2baf49&id=10e284530a" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate>
|
||||
<div id="mc_embed_signup_scroll">
|
||||
|
||||
<div class="mc-field-group">
|
||||
<input type="email" value="your-email-address-here 💋" name="EMAIL" class="required email" id="mce-EMAIL" onfocus="if(this.value==this.defaultValue)this.value='';" onblur="if(this.value=='')this.value=this.defaultValue;"> <span align="right"><input type="submit" value="Subscribe me, sexy!" name="subscribe" id="mc-embedded-subscribe" class="button"></span>
|
||||
</div>
|
||||
<div id="mce-responses" class="clear">
|
||||
<div class="response" id="mce-error-response" style="display:none"></div>
|
||||
<div class="response" id="mce-success-response" style="display:none"></div>
|
||||
</div> <!-- real people should not fill this in and expect good things - do not remove this or risk form bot signups-->
|
||||
<div style="position: absolute; left: -5000px;" aria-hidden="true"><input type="text" name="b_a1d9cee4402be76497a2baf49_10e284530a" tabindex="-1" value=""></div>
|
||||
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
<!--End mc_embed_signup-->
|
||||
|
||||
### Your comments? 💬
|
||||
Reference in New Issue
Block a user