public/geek-cookbook
1
0
Fork 0
mirror of https://github.com/funkypenguin/geek-cookbook/ synced 2025-12-13 01:36:23 +00:00
Code Issues Releases Wiki Activity

Add post on proxmox MTU vs VLAN

Browse Source 
Signed-off-by: David Young <davidy@funkypenguin.co.nz>
This commit is contained in:
David Young
2023-02-22 22:26:43 +13:00
parent 29c92e4694
commit 57f7268b9f
9 changed files with 981 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
_snippets/channels.md Normal file
View File

@@ -0,0 +1,24 @@
## Channels
### 📔 Information
| Channel Name | Channel Use |
|--------------------|------------------------------------------------------------|
| #announcements | Used for important announcements |
| #changelog | Used for major changes to the cookbook (to be deprecated) |
| #cookbook-updates | Updates on all pushes to the master branch of the cookbook |
| #premix-updates | Updates on all pushes to the master branch of the premix |
### 💬 Discussion
| Channel Name | Channel Use |
|----------------|----------------------------------------------------------|
| #introductions | New? Pop in here and say hi :) |
| #general | General chat - anything goes |
| #cookbook | Discussions specifically around the cookbook and recipes |
| #kubernetes | Discussions about Kubernetes |
| #docker-swarm | Discussions about Docker Swarm |
| #today-i-learned | Post tips/tricks you've stumbled across
| #jobs | For seeking / advertising jobs, bounties, projects, etc |
| #advertisements | In here you can advertise your stream, services or websites, at a limit of 2 posts per day |
| #dev | Used for collaboration around current development. |

1
_snippets/common-links.md
View File

@@ -29,6 +29,7 @@
[jellyfin]: /recipes/jellyfin/
[k8s/invidious]: /recipes/kubernetes/invidious/
[k8s/mastodon]: /recipes/kubernetes/mastodon/
[k8s/matrix]: /recipes/kubernetes/matrix/
[metallb]: /kubernetes/loadbalancer/metallb/
[kavita]: /recipes/kavita/
[keycloak]: /recipes/keycloak/

175
docs/blog/posts/notes/proxmox-7-3-enforces-mtu-breaks-vault-raft.md Normal file
View File

@@ -0,0 +1,175 @@
---
date: 2023-02-22
categories:
- note
tags:
- proxmox
title: Proxmox 7.3 enforces 1500 MTU, breaks previously-working jumbo-framed VMs
description: Since upgrading to Proxmox 7.3, I discovered that my vault cluster was failing to sync. Turns out, a new setting enforcing a default MTU per-VM was the culprit!
---
# That time when a Proxmox upgrade silently capped my MTU
I feed and water several Proxmox clusters, one of which was recently upgraded to PVE 7.3. This cluster runs VMs used to build a CI instance of a bare-metal Kubernetes cluster I support. Every day the CI cluster is automatically destroyed and rebuilt, to give assurance that our recent changes haven't introduced a failure which would prevent a re-install.
Since the PVE 7.3 upgrade, the CI cluster has been failing to build, because the out-of-cluster Vault instance we use to secure etcd secrets, failed to sync. After much debugging, I'd like to present a variation of a [famous haiku](https://www.cyberciti.biz/humour/a-haiku-about-dns/)[^1] to summarize the problem:
> It's not MTU! <br>
> There's no way it's MTU! <br>
> It was MTU.
Here's how it went down...
<!-- more -->
## Vault fails to sync
We're using Hashicorp vault in HA mode with [integrated (raft) storage](https://developer.hashicorp.com/vault/docs/concepts/integrated-storage), and [AWSKMS auto-unsealing](https://developer.hashicorp.com/vault/docs/configuration/seal/awskms). All you have to do is initialize vault on your first node, and then include something like this in other nodes:
```
storage "raft" {
path = "/var/lib/vault"
node_id = "grumpy"
retry_join {
leader_api_addr = "https://192.168.37.11:8200"
leader_ca_cert_file = "/etc/kubernetes/pki/vault/ca.pem"
}
retry_join {
leader_api_addr = "https://192.168.37.12:8200"
leader_ca_cert_file = "/etc/kubernetes/pki/vault/ca.pem"
}
retry_join {
leader_api_addr = "https://192.168.37.13:8200"
leader_ca_cert_file = "/etc/kubernetes/pki/vault/ca.pem"
}
}
```
When vault starts up, it'll look for the leaders in the `retry_join` config, attempt to connect to them, and use raft magic to unseal themselves and join the raft.
On our victim cluster, instead of happily joining the raft, the other nodes were logging messages like this:
```bash
Feb 22 00:38:04 plum vault[32791]: 2023-02-22T00:38:04.126Z [INFO] core: stored unseal keys supported, attempting fetch
Feb 22 00:38:04 plum vault[32791]: 2023-02-22T00:38:04.126Z [WARN] failed to unseal core: error="stored unseal keys are supported, but none were found"
Feb 22 00:38:04 plum vault[32791]: 2023-02-22T00:38:04.648Z [ERROR] core: failed to retry join raft cluster: retry=2s err="failed to send answer to raft leader node: context deadline exceeded"
Feb 22 00:38:06 plum vault[32791]: 2023-02-22T00:38:06.648Z [INFO] core: security barrier not initialized
Feb 22 00:38:06 plum vault[32791]: 2023-02-22T00:38:06.655Z [INFO] core: attempting to join possible raft leader node: leader_addr=https://192.168.20.11:8200
Feb 22 00:38:06 plum vault[32791]: 2023-02-22T00:38:06.655Z [INFO] core: attempting to join possible raft leader node: leader_addr=https://192.168.20.13:8200
Feb 22 00:38:06 plum vault[32791]: 2023-02-22T00:38:06.655Z [INFO] core: attempting to join possible raft leader node: leader_addr=https://192.168.20.12:8200
Feb 22 00:38:06 plum vault[32791]: 2023-02-22T00:38:06.657Z [ERROR] core: failed to get raft challenge: leader_addr=https://192.168.20.13:8200
Feb 22 00:38:06 plum vault[32791]: error=
Feb 22 00:38:06 plum vault[32791]: | error during raft bootstrap init call: Error making API request.
Feb 22 00:38:06 plum vault[32791]: |
Feb 22 00:38:06 plum vault[32791]: | URL: PUT https://192.168.20.13:8200/v1/sys/storage/raft/bootstrap/challenge
Feb 22 00:38:06 plum vault[32791]: | Code: 503. Errors:
Feb 22 00:38:06 plum vault[32791]: |
Feb 22 00:38:06 plum vault[32791]: | * Vault is sealed
Feb 22 00:38:06 plum vault[32791]:
```
!!! note
In hindsight, the `context deadline exceeded` was a clue, but it was hidden in the noise of multiple nodes failing to join each other.
I discovered that an identical CI cluster on a non-upgrading proxmox didn't exhibit the error.
After exhausting all the conventional possibilites (*vault cli version mismatch, SSL issues, DNS*), I decided to check whether MTU was still working (*this cluster had worked fine until recently*).
Using `ping -M do 4000 <target>`, I was surprised to discover that my CI VMs could **not** ping each other with unfragmented, large payloads. I checked the working cluster - in that environment, I **could** pass large ping payloads.
## It's MTU, right?
> "Ha! The VMs MTUs must be set wrong!" - me
Nope. Checked that:
```yaml
network:
version: 2
ethernets:
ens18:
dhcp4: no
optional: true
mtu: 8894
ens19:
dhcp4: no
optional: true
mtu: 8894
bonds:
cluster:
mtu: 8894
<snip the irrelevant stuff>
```
## Proxmox upgrade broke MTU?
> "OK, so maybe the proxmox upgrade has removed the MTU we set on the bridge." - also me
Nope. Still good:
```bash
root@proxmox:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
iface eno1np0 inet manual
auto vmbr0
iface vmbr0 inet static
address 10.0.1.215
netmask 255.255.255.0
gateway 10.0.1.1
bridge_ports eno1np0
bridge_stp off
bridge_fd 0
mtu 9000
iface eno2np1 inet manual
root@proxmox:~#
```
> "Huh." - me, final state
OK, so every NIC on this proxmox host **should** be at MTU 9000, right?
Well, not exactly...
```bash
root@proxmox:~# ip link list | grep mtu
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
2: eno1np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
3: eno2np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000
<snip>
13: tap100i1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0v180 state UNKNOWN mode DEFAULT group default qlen 1000
14: tap100i2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0v42 state UNKNOWN mode DEFAULT group default qlen 1000
```
## MTU is broken on Proxmox :(
Aha, what is going on here? Every interface seems to have been set to an MTU of 1500.
In PVE 7.3, we now have the option to set the MTU of each network interface. This defaults to `1500`, but by setting to the magic number of `1`, the interface MTU will align with the MTU of its bridge.
So we should be able to just set each interface's MTU to `9000`, right?
Well no. Not if we're using VLANs (*which, of course we are, in a multi-networked replica of a real cluster, complete with multi-homed virtual firewalls!*)
It seems that the addition of MTU settings to Proxmox virtio NICs via the UI has broken the way that MTU is set on the tap interfaces on the proxmox hypervisor.
If you use a VLAN tag, your MTU is fixed at `1500`, regardless of what you set. If you **don't** use a VLAN tag, your MTU is set to the MTU of your bridge (*the old behaviour*), regardless of what you set.
So I created a [bug report](https://bugzilla.proxmox.com/show_bug.cgi?id=4547).
## Workaround for Proxmox MTU issue
I was hoping to be able to post a workaround here, a method to allow us to continue to use jumbo frames in our VLAN-aware CI cluster environment. Unfortunately, I've not been able to find a way to make this work, so until the bug is fixed, I've had to revert my CI clusters to a `1500` MTU, which now represents a minor deviation from our production clusters :(
## Summary
MTU issues cause mysterious failures in mysterious ways. Always test your MTU using `ping -M do -s <a big number> <target>`, to ensure that you actually **can** pass larger-than-normal packets!
[^1]: If it's not DNS, it's probably MTU.
--8<-- "blog-footer.md"

35
docs/community/discord.md
View File

@@ -38,13 +38,18 @@ Your report message will immediately be deleted from the channel, and an alert r
### 📔 Information
The following channels are automated, and provide updates for issues like new PRs, subreddit posts, etc:
| Channel Name | Channel Use |
|--------------------|------------------------------------------------------------|
| #announcements | Used for important announcements |
| #changelog | Used for major changes to the cookbook (to be deprecated) |
| #cookbook-updates | Updates on all pushes to the master branch of the cookbook |
| #premix-updates | Updates on all pushes to the master branch of the premix |
| #discourse-updates | Updates to Discourse topics |
| #forum-threads | Updates to Discourse topics |
| #subreddit-posts | New topics in r/funkypenguin |
### 💬 Discussion
@@ -57,30 +62,15 @@ Your report message will immediately be deleted from the channel, and an alert r
| #docker-swarm | Discussions about Docker Swarm |
| #today-i-learned | Post tips/tricks you've stumbled across
| #jobs | For seeking / advertising jobs, bounties, projects, etc |
| #advertisements | In here you can advertise your stream, services or websites, at a limit of 2 posts per day |
| #promotion | In here you can advertise your stream, services or websites, at a limit of 2 posts per day |
| #dev | Used for collaboratio around current development. |
### Suggestions
| Channel Name | Channel Use |
|--------------|-------------------------------------|
| #in-flight | A list of all suggestions in-flight |
| #completed | A list of completed suggestions |
### Music
| Channel Name | Channel Use |
|------------------|-----------------------------------|
| #music | DJs go here to control music |
| #listen-to-music | Jump in here to rock out to music |
## How to get help.
If you need assistance at any time there are a few commands that you can run in order to get help.
`!help` Shows help content.
`!faq` Shows frequently asked questions.
* `!help` Shows help content.
* `!faq` Shows frequently asked questions.
## Spread the love (inviting others)
@@ -95,10 +85,3 @@ Discord supports minimal message formatting using [markdown](https://support.dis
!!! tip "Editing your most recent message"
You can edit your most-recent message by pushing the up arrow, make your edits, and then push `Enter` to save!
## How do I suggest something?
1. Find the #completed channel (*under the **Suggestions** category*), and confirm that your suggestion hasn't already been voted on.
2. Find the #in-flight channel (*also under **Suggestions***), and confirm that your suggestion isn't already in-flight (*but not completed yet*)
3. In any channel, type `!suggest [your suggestion goes here]`. A post will be created in #in-flight for other users to vote on your suggestion. Suggestions change color as more users vote on them.
4. When your suggestion is completed (*or a decision has been made*), you'll receive a DM from carl-bot

102
docs/community/matrix.md Normal file
View File

@@ -0,0 +1,102 @@
---
title: Geek out with Funky Penguin's Matrix Server
description: Geek out in the Fediverse with our Matrix instance!
icon: simple/matrix
status: new
---
# How to geek out in [m]atrix
If you're hesitant to dip your toes into the wall-garden of our [Discord server](http://chat.funkypenguin.co.nz), then Matrix is the place for you!
!!! question "Eh? What's this federated chat thing? Who own my data? (/puts on tin-foil hat) !!"
Yeah, I know. I also thought Discord was just for the gamer kids, but it turns out it's great for a geeky community. Why? [Let me elucidate ya.](https://www.youtube.com/watch?v=1qHoSWxVqtE)..
1. Native markdown for code blocks
2. Drag-drop screenshots
3. Costs nothing, no ads
4. Mobile notifications are reliable, individual channels mutable, etc
## How do I join the Matrix server?
1. If you already have a matrix account, you can use it to access our server (the magic of federation). See the channels list below for details
2. If you don't have an account yet, [create one](https://m.fnky.nz), using your email address, or just login with GitHub
3. Use the WebUI, or get one of the [many available](https://matrix.org/clients/) mobile / desktop clients
4. Join the channels you're interested in (*below*)! and say hi!
## How do I search?
Maybe it's just my years of using Discord, but I expected to be able to enter any search string in the main "search" field in my Matrix client, and have the relevant messages surfaced. Turns out, that's not how it works! The primary "search" field in the Elements UI, for example, searches **room names** (*i.e., show me all the rooms matching the string "chat"*). To search **within** a room, it's necessary to first navigate to that room, and then "room search" is a contextual action you can take **within** the room. I've not yet found out how do a partial string match though (*"chatgpt" matches, but "chatgp" does no*t).
## Code of Conduct
With the goal of creating a safe and inclusive community, we've adopted the [Contributor Covenant Code of Conduct](https://www.contributor-covenant.org/), as described [here](/community/code-of-conduct/), and use of the Matrix instance is subject to this code of conduct.
## Channels
### 📔 Information
| Channel Name | Channel Use |
|--------------------|------------------------------------------------------------|
| [#announcements](https://matrix.to/#/#announcements:m.fnky.nz) | Used for important announcements |
| #cookbook-updates | Updates on all pushes to the master branch of the cookbook |
| #premix-updates | Updates on all pushes to the master branch of the premix |
| #discourse-updates | Updates to Discourse topics |
### 💬 Discussion
| Channel Name | Channel Use |
|----------------|----------------------------------------------------------|
| #introductions | New? Pop in here and say hi :) |
| [#general](https://matrix.to/#/#general:m.fnky.nz) | General chat - anything goes |
| [#cookbook](https://matrix.to/#/#cookbook:m.fnky.nz) | Discussions specifically around the cookbook and recipes |
| [#kubernetes](https://matrix.to/#/#kubernetes:m.fnky.nz) | Discussions about Kubernetes |
| [#docker-swarm](https://matrix.to/#/#docker-swarm:m.fnky.nz) | Discussions about Docker Swarm |
| #today-i-learned | Post tips/tricks you've stumbled across
| [#jobs](https://matrix.to/#/#jobs:m.fnky.nz) | For seeking / advertising jobs, bounties, projects, etc |
| [#advertisements](https://matrix.to/#/#advertisments:m.fnky.nz) | In here you can advertise your stream, services or websites, at a limit of 2 posts per day |
| [#dev](https://matrix.to/#/#dev:m.fnky.nz) | Used for collaboratio around current development. |
### Suggestions
| Channel Name | Channel Use |
|--------------|-------------------------------------|
| #in-flight | A list of all suggestions in-flight |
| #completed | A list of completed suggestions |
### Music
| Channel Name | Channel Use |
|------------------|-----------------------------------|
| #music | DJs go here to control music |
| #listen-to-music | Jump in here to rock out to music |
## How to get help.
If you need assistance at any time there are a few commands that you can run in order to get help.
`!help` Shows help content.
`!faq` Shows frequently asked questions.
## Spread the love (inviting others)
Invite your co-geeks to Discord by:
1. Sending them a link to <http://chat.funkypenguin.co.nz>, or
2. Right-click on the Discord server name and click "Invite People"
## Formatting your message
Discord supports minimal message formatting using [markdown](https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline-).
!!! tip "Editing your most recent message"
You can edit your most-recent message by pushing the up arrow, make your edits, and then push `Enter` to save!
## How do I suggest something?
1. Find the #completed channel (*under the **Suggestions** category*), and confirm that your suggestion hasn't already been voted on.
2. Find the #in-flight channel (*also under **Suggestions***), and confirm that your suggestion isn't already in-flight (*but not completed yet*)
3. In any channel, type `!suggest [your suggestion goes here]`. A post will be created in #in-flight for other users to vote on your suggestion. Suggestions change color as more users vote on them.
4. When your suggestion is completed (*or a decision has been made*), you'll receive a DM from carl-bot

48
docs/community/slack.md Normal file
View File

@@ -0,0 +1,48 @@
---
title: Engage with the Geek Cookbook in GitHub
description: How to use GitHub to interact with the Geek Cookbook community
icon: fontawesome/brands/slack
status: new
---
# How to geek on Slack
## Get yer invite!
Joing a Slack server requires an invite. To get yours, go [here](https://communityinviter.com/apps/funkypenguin/geek-with-us), or use the form below:
<div id="CommunityInviter"></div>
<script>
  window.CommunityInviterAsyncInit = function () {
    CommunityInviter.init({
      app_url:'geek-with-us',
      team_id:'funkypenguin'
   })
  };
  (function(d, s, id){
    var js, fjs = d.getElementsByTagName(s)[0];
    if (d.getElementById(id)) {return;}
    js = d.createElement(s); js.id = id;
    js.src = "https://communityinviter.com/js/communityinviter.js";
    fjs.parentNode.insertBefore(js, fjs);
  }(document, 'script', 'Community_Inviter'));
</script>
## Code of Conduct
With the goal of creating a safe and inclusive community, we've adopted the [Contributor Covenant Code of Conduct](https://www.contributor-covenant.org/), as described [here](/community/code-of-conduct/). Use of the Slack server is subject to this code of conduct.
## Spread the love (inviting others)
Invite your co-geeks to Slack by sending them a link to this page, or directly to the [invite generator](https://communityinviter.com/apps/funkypenguin/geek-with-us)
## Formatting your message
See Slack's help page on formatting messages, [here](https://slack.com/help/articles/202288908-Format-your-messages).
!!! tip "Editing your most recent message"
You can edit your most-recent message by pushing the up arrow, make your edits, and then push `Enter` to save!
--8<-- "common-links.md"

BIN
docs/images/blog/proxmox-set-mtu-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

612
docs/recipes/kubernetes/matrix.md Normal file
View File

@@ -0,0 +1,612 @@
---
title: Install Matrix in Kubernetes
description: How to install your own Matrix instance using Kubernetes
status: new
---
# Install Invidious in Kubernetes
# gotchas
Create signing key first. Else you'll ban yourself from the federation!
```bash
~ ❯ docker run --rm -it ananace/matrix-synapse generate_signing_key ed25519 a_cQgt sOaAmEl7a9s2S0RCr7FT9nzuSjEjYVRrNNzwIKsutzA
~ ❯
```
Thanks to [Sealed Secrets](/kubernetes/sealed-secrets/), we have a safe way of committing secrets into our repository, so to create this cloudflare secret, you'd run something like this:
```bash
kubectl create secret generic matrix-synapse-signingkey \
--namespace matrix \
--dry-run=client \
--from-literal=signing.key=YOURSIGNINGKEYGOESHERE -o json \
| kubeseal --cert <path to public cert> \
> <path to repo>/matrix/sealedsecret-matrix-synapse-signingkey.yaml
```
echo -e ed25519 a_oDEG IiFDnZsM4TaROTdnFmcfa15Ee/srcF/J2bQ9VP5o0Pk
Why not Dendrite?
AFAIK, it woen't yet work with SSO (login with GitHub), and requires nats for messaging, which will consume more PVCs on my limited DO cluster!
### Create matrix_media_repo database
```bash
kubectl exec -n matrix matrix-synapse-postgresql-0 -it -- \
/bin/bash -c PGPASSWORD=$POSTGRES_PASSWORD createdb matrix_media_repo -U synapse
```
### Create mautrix-discord database
```bash
kubectl exec -n matrix matrix-synapse-postgresql-0 -it -- \
/bin/bash -c PGPASSWORD=$POSTGRES_PASSWORD createdb matrix_discord -U synapse # (1)!
```
1. No hyphens allowed in database names, apparently!
### Register admin user
kubectl -n matrix exec -it $(kubectl -n matrix get pod -l "app.kubernetes.io/name=matrix-synapse" -o jsonpath='{.items[0].metadata.name}') /bin/bash
```bash
root@matrix-synapse-5d7cf8579-zjk7c:/# register_new_matrix_user -k '<MY REGISTRATION KEY>' https://matrix.funkypenguin.co.nz
New user localpart [root]: root
Password:
Confirm password:
Make admin [no]: yes
Sending registration request...
Success!
root@matrix-synapse-5d7cf8579-zjk7c:/#
```
YouTube is ubiquitious now. Almost every video I'm sent, takes me to YouTube. Worse, every YouTube video I watch feeds Google's profile about me, so shortly after enjoying the latest Marvel movie trailers, I find myself seeing related adverts on **unrelated** websites.
Creepy :bug:!
As the connection between the videos I watch and the adverts I see has become move obvious, I've become more discerning re which videos I choose to watch, since I don't necessarily **want** algorithmically-related videos popping up next time I load the YouTube app on my TV, or Marvel merchandise advertised to me on every second news site I visit.
This is a PITA since it means I have to "self-censor" which links I'll even click on, knowing that once I *do* click the video link, it's forever associated with my Google account :facepalm:
After playing around with [some of the available public instances](https://docs.invidious.io/instances/) for a while, today I finally deployed my own instance of [Invidious](https://invidious.io/) - an open source alternative front-end to YouTube.
![Invidious Screenshot](/images/invidious.png){ loading=lazy }
Here's an example from my public instance (*yes, running on Kubernetes*):
<iframe id='ivplayer' width='640' height='360' src='https://in.fnky.nz/embed/o-YBDTqX_ZU?t=3' style='border:none;'></iframe>
## Invidious requirements
!!! summary "Ingredients"
Already deployed:
* [x] A [Kubernetes cluster](/kubernetes/cluster/) (*not running Kubernetes? Use the [Docker Swarm recipe instead][invidious]*)
* [x] [Flux deployment process](/kubernetes/deployment/flux/) bootstrapped
* [x] An [Ingress](/kubernetes/ingress/) to route incoming traffic to services
* [x] [Persistent storage](/kubernetes/persistence/) to store persistent stuff
* [x] [External DNS](/kubernetes/external-dns/) to create an DNS entry
New:
* [ ] Chosen DNS FQDN for your instance
## Preparation
### GitRepository
The Invidious project doesn't currently publish a versioned helm chart - there's just a [helm chart stored in the repository](https://github.com/invidious/invidious/tree/main/chart) (*I plan to submit a PR to address this*). For now, we use a GitRepository instead of a HelmRepository as the source of a HelmRelease.
```yaml title="/bootstrap/gitrepositories/gitepository-invidious.yaml"
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: invidious
namespace: flux-system
spec:
interval: 1h0s
ref:
branch: master
url: https://github.com/iv-org/invidious
```
### Namespace
We need a namespace to deploy our HelmRelease and associated ConfigMaps into. Per the [flux design](/kubernetes/deployment/flux/), I create this example yaml in my flux repo at `/bootstrap/namespaces/namespace-invidious.yaml`:
```yaml title="/bootstrap/namespaces/namespace-invidious.yaml"
apiVersion: v1
kind: Namespace
metadata:
name: invidious
```
### Kustomization
Now that the "global" elements of this deployment (*just the GitRepository in this case*) have been defined, we do some "flux-ception", and go one layer deeper, adding another Kustomization, telling flux to deploy any YAMLs found in the repo at `/invidious`. I create this example Kustomization in my flux repo:
```yaml title="/bootstrap/kustomizations/kustomization-invidious.yaml"
apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
metadata:
name: invidious
namespace: flux-system
spec:
interval: 15m
path: invidious
prune: true # remove any elements later removed from the above path
timeout: 2m # if not set, this defaults to interval duration, which is 1h
sourceRef:
kind: GitRepository
name: flux-system
validation: server
healthChecks:
- apiVersion: apps/v1
kind: Deployment
name: invidious-invidious # (1)!
namespace: invidious
- apiVersion: apps/v1
kind: StatefulSet
name: invidious-postgresql
namespace: invidious
```
1. No, that's not a typo, just another pecularity of the helm chart!
### ConfigMap
Now we're into the invidious-specific YAMLs. First, we create a ConfigMap, containing the entire contents of the helm chart's [values.yaml](https://github.com/iv-org/invidious/blob/master/kubernetes/values.yaml). Paste the values into a `values.yaml` key as illustrated below, indented 4 spaces (*since they're "encapsulated" within the ConfigMap YAML*). I create this example yaml in my flux repo:
```yaml title="invidious/configmap-invidious-helm-chart-value-overrides.yaml"
apiVersion: v1
kind: ConfigMap
metadata:
name: invidious-helm-chart-value-overrides
namespace: invidious
data:
values.yaml: |- # (1)!
# <upstream values go here>
```
1. Paste in the contents of the upstream `values.yaml` here, intended 4 spaces, and then change the values you need as illustrated below.
Values I change from the default are:
```yaml
postgresql:
image:
tag: 14
auth:
username: invidious
password: <redacted>
database: invidious
primary:
initdb:
username: invidious
password: <redacted>
scriptsConfigMap: invidious-postgresql-init
persistence:
size: 1Gi # (1)!
podAnnotations: # (2)!
backup.velero.io/backup-volumes: backup
pre.hook.backup.velero.io/command: '["/bin/bash", "-c", "PGPASSWORD=$POSTGRES_PASSWORD pg_dump -U postgres -d $POSTGRES_DB -h 127.0.0.1 > /scratch/backup.sql"]'
pre.hook.backup.velero.io/timeout: 3m
post.hook.restore.velero.io/command: '["/bin/bash", "-c", "[ -f \"/scratch/backup.sql\" ] && PGPASSWORD=$POSTGRES_PASSWORD psql -U postgres -h 127.0.0.1 -d $POSTGRES_DB -f /scratch/backup.sql && rm -f /scratch/backup.sql;"]'
extraVolumes:
- name: backup
emptyDir:
sizeLimit: 1Gi
extraVolumeMounts:
- name: backup
mountPath: /scratch
resources:
requests:
cpu: "10m"
memory: 32Mi
# Adapted from ../config/config.yml
config:
channel_threads: 1
feed_threads: 1
db:
user: invidious
password: <redacted>
host: invidious-postgresql
port: 5432
dbname: invidious
full_refresh: false
https_only: true
domain: in.fnky.nz # (3)!
external_port: 443 # (4)!
banner: ⚠️ Note - This public Invidious instance is sponsored ❤️ by <A HREF='https://geek-cookbook.funkypenguin.co.nz'>Funky Penguin's Geek Cookbook</A>. It's intended to support the published <A HREF='https://geek-cookbook.funkypenguin.co.nz/recipes/invidious/'>Docker Swarm recipes</A>, but may be removed at any time without notice. # (5)!
default_user_preferences: # (6)!
quality: dash # (7)! auto-adapts or lets you choose > 720P
```
1. 1Gi is fine for the database for now
2. These annotations / extra Volumes / volumeMounts support automated backup using Velero
3. Invidious needs this to generate external links for sharing / embedding
4. Invidious needs this too, to generate external links for sharing / embedding
5. It's handy to tell people what's special about your instance
6. Check out the [official config docs](https://github.com/iv-org/invidious/blob/master/config/config.example.yml) for comprehensive details on how to configure / tweak your instance!
7. Default all users to DASH (*adaptive*) quality, rather than limiting to 720P (*the default*)
### HelmRelease
Finally, having set the scene above, we define the HelmRelease which will actually deploy the invidious into the cluster. I save this in my flux repo:
```yaml title="/invidious/helmrelease-invidious.yaml"
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: invidious
namespace: invidious
spec:
chart:
spec:
chart: ./charts/invidious
sourceRef:
kind: GitRepository
name: invidious
namespace: flux-system
interval: 15m
timeout: 5m
releaseName: invidious
valuesFrom:
- kind: ConfigMap
name: invidious-helm-chart-value-overrides
valuesKey: values.yaml # (1)!
```
1. This is the default, but best to be explicit for clarity
### Ingress / IngressRoute
Oddly, the upstream chart doesn't include any Ingress resource. We have to manually create our Ingress as below (*note that it's also possible to use a Traefik IngressRoute directly*)
```yaml title="/invidious/ingress-invidious.yaml"
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: invidious
namespace: invidious
spec:
ingressClassName: nginx
rules:
- host: in.fnky.nz
http:
paths:
- backend:
service:
name: invidious
port:
number: 3000
path: /
pathType: ImplementationSpecific
```
An alternative implementation using an `IngressRoute` could look like this:
```yaml title="/invidious/ingressroute-invidious.yaml"
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: in.fnky.nz
namespace: invidious
spec:
routes:
- match: Host(`in.fnky.nz`)
kind: Rule
services:
- name: invidious-invidious
kind: Service
port: 3000
```
### Create postgres-init ConfigMap
Another pecularity of the Invidious helm chart is that you have to create your own ConfigMap containing the PostgreSQL data structure. I suspect that the helm chart has received minimal attention in the past 3+ years, and this could probably easily be turned into a job as a pre-install helm hook (*perhaps a future PR?*).
In the meantime, you'll need to create ConfigMap manually per the [repo instructions](https://github.com/iv-org/invidious/tree/master/kubernetes#installing-helm-chart), or cheat, and copy the one I paste below:
??? example "Configmap (click to expand)"
```yaml title="/invidious/configmap-invidious-postgresql-init.yaml"
apiVersion: v1
kind: ConfigMap
metadata:
name: invidious-postgresql-init
namespace: invidious
data:
annotations.sql: |
-- Table: public.annotations
-- DROP TABLE public.annotations;
CREATE TABLE IF NOT EXISTS public.annotations
(
id text NOT NULL,
annotations xml,
CONSTRAINT annotations_id_key UNIQUE (id)
);
GRANT ALL ON TABLE public.annotations TO current_user;
channel_videos.sql: |+
-- Table: public.channel_videos
-- DROP TABLE public.channel_videos;
CREATE TABLE IF NOT EXISTS public.channel_videos
(
id text NOT NULL,
title text,
published timestamp with time zone,
updated timestamp with time zone,
ucid text,
author text,
length_seconds integer,
live_now boolean,
premiere_timestamp timestamp with time zone,
views bigint,
CONSTRAINT channel_videos_id_key UNIQUE (id)
);
GRANT ALL ON TABLE public.channel_videos TO current_user;
-- Index: public.channel_videos_ucid_idx
-- DROP INDEX public.channel_videos_ucid_idx;
CREATE INDEX IF NOT EXISTS channel_videos_ucid_idx
ON public.channel_videos
USING btree
(ucid COLLATE pg_catalog."default");
channels.sql: |+
-- Table: public.channels
-- DROP TABLE public.channels;
CREATE TABLE IF NOT EXISTS public.channels
(
id text NOT NULL,
author text,
updated timestamp with time zone,
deleted boolean,
subscribed timestamp with time zone,
CONSTRAINT channels_id_key UNIQUE (id)
);
GRANT ALL ON TABLE public.channels TO current_user;
-- Index: public.channels_id_idx
-- DROP INDEX public.channels_id_idx;
CREATE INDEX IF NOT EXISTS channels_id_idx
ON public.channels
USING btree
(id COLLATE pg_catalog."default");
nonces.sql: |+
-- Table: public.nonces
-- DROP TABLE public.nonces;
CREATE TABLE IF NOT EXISTS public.nonces
(
nonce text,
expire timestamp with time zone,
CONSTRAINT nonces_id_key UNIQUE (nonce)
);
GRANT ALL ON TABLE public.nonces TO current_user;
-- Index: public.nonces_nonce_idx
-- DROP INDEX public.nonces_nonce_idx;
CREATE INDEX IF NOT EXISTS nonces_nonce_idx
ON public.nonces
USING btree
(nonce COLLATE pg_catalog."default");
playlist_videos.sql: |
-- Table: public.playlist_videos
-- DROP TABLE public.playlist_videos;
CREATE TABLE IF NOT EXISTS public.playlist_videos
(
title text,
id text,
author text,
ucid text,
length_seconds integer,
published timestamptz,
plid text references playlists(id),
index int8,
live_now boolean,
PRIMARY KEY (index,plid)
);
GRANT ALL ON TABLE public.playlist_videos TO current_user;
playlists.sql: |
-- Type: public.privacy
-- DROP TYPE public.privacy;
CREATE TYPE public.privacy AS ENUM
(
'Public',
'Unlisted',
'Private'
);
-- Table: public.playlists
-- DROP TABLE public.playlists;
CREATE TABLE IF NOT EXISTS public.playlists
(
title text,
id text primary key,
author text,
description text,
video_count integer,
created timestamptz,
updated timestamptz,
privacy privacy,
index int8[]
);
GRANT ALL ON public.playlists TO current_user;
session_ids.sql: |+
-- Table: public.session_ids
-- DROP TABLE public.session_ids;
CREATE TABLE IF NOT EXISTS public.session_ids
(
id text NOT NULL,
email text,
issued timestamp with time zone,
CONSTRAINT session_ids_pkey PRIMARY KEY (id)
);
GRANT ALL ON TABLE public.session_ids TO current_user;
-- Index: public.session_ids_id_idx
-- DROP INDEX public.session_ids_id_idx;
CREATE INDEX IF NOT EXISTS session_ids_id_idx
ON public.session_ids
USING btree
(id COLLATE pg_catalog."default");
users.sql: |+
-- Table: public.users
-- DROP TABLE public.users;
CREATE TABLE IF NOT EXISTS public.users
(
updated timestamp with time zone,
notifications text[],
subscriptions text[],
email text NOT NULL,
preferences text,
password text,
token text,
watched text[],
feed_needs_update boolean,
CONSTRAINT users_email_key UNIQUE (email)
);
GRANT ALL ON TABLE public.users TO current_user;
-- Index: public.email_unique_idx
-- DROP INDEX public.email_unique_idx;
CREATE UNIQUE INDEX IF NOT EXISTS email_unique_idx
ON public.users
USING btree
(lower(email) COLLATE pg_catalog."default");
videos.sql: |+
-- Table: public.videos
-- DROP TABLE public.videos;
CREATE UNLOGGED TABLE IF NOT EXISTS public.videos
(
id text NOT NULL,
info text,
updated timestamp with time zone,
CONSTRAINT videos_pkey PRIMARY KEY (id)
);
GRANT ALL ON TABLE public.videos TO current_user;
-- Index: public.id_idx
-- DROP INDEX public.id_idx;
CREATE UNIQUE INDEX IF NOT EXISTS id_idx
ON public.videos
USING btree
(id COLLATE pg_catalog."default");
```
## :octicons-video-16: Install Invidious!
Commit the changes to your flux repository, and either wait for the reconciliation interval, or force a reconcilliation[^1] using `flux reconcile source git flux-system`. You should see the kustomization appear...
```bash
~ ❯ flux get kustomizations | grep invidious
invidious main/d34779f False True Applied revision: main/d34779f
~ ❯
```
The helmrelease should be reconciled...
```bash
~ ❯ flux get helmreleases -n invidious
NAME REVISION SUSPENDED READY MESSAGE
invidious 1.1.1 False True Release reconciliation succeeded
~ ❯
```
And you should have happy Invidious pods:
```bash
~ ❯ k get pods -n invidious
NAME READY STATUS RESTARTS AGE
invidious-invidious-64f4fb8d75-kr4tw 1/1 Running 0 77m
invidious-postgresql-0 1/1 Running 0 11h
~ ❯
```
... and finally check that the ingress was created as desired:
```bash
~ ❯ k get ingress -n invidious
NAME CLASS HOSTS ADDRESS PORTS AGE
invidious <none> in.fnky.nz 80, 443 19h
~ ❯
```
Or in the case of an ingressRoute:
```bash
~ ❯ k get ingressroute -n invidious
NAME AGE
in.fnky.nz 19h
```
Now hit the URL you defined in your config, you'll see the basic search screen. Enter a search phrase (*"marvel movie trailer"*) to see the YouTube video results, or paste in a YouTube URL such as `https://www.youtube.com/watch?v=bxqLsrlakK8`, change the domain name from `www.youtube.com` to your instance's FQDN, and watch the fun [^2]!
You can also install a range of browser add-ons to automatically redirect you from youtube.com to your Invidious instance. I'm testing "[libredirect](https://addons.mozilla.org/en-US/firefox/addon/libredirect/)" currently, which seems to work as advertised!
## Summary
What have we achieved? We have an HTTPS-protected private YouTube frontend - we can now watch whatever videos we please, without feeding Google's profile on us. We can also subscribe to channels without requiring a Google account, and we can share individual videos directly via our instance (*by generating links*).
!!! summary "Summary"
Created:
* [X] We are free of the creepy tracking attached to YouTube videos!
--8<-- "recipe-footer.md"
[^1]: There is also a 3rd option, using the Flux webhook receiver to trigger a reconcilliation - to be covered in a future recipe!
[^2]: Gotcha!

9
mkdocs.yml
View File

@@ -259,6 +259,7 @@ nav:
# - Kiali: kubernetes/wip.md
- Invidious: recipes/kubernetes/invidious.md
- Mastodon: recipes/kubernetes/mastodon.md
# - Matrix: recipes/kubernetes/matrix.md
# - NGINX Ingress: kubernetes/wip.md
# - Polaris: kubernetes/wip.md
# - Portainer: kubernetes/wip.md
@@ -293,6 +294,8 @@ nav:
- Contribute: community/contribute.md
- Code of Conduct: community/code-of-conduct.md
- Discord: community/discord.md
# - Slack: community/slack.md
# - Matrix: community/matrix.md
- Reddit: community/reddit.md
- Mastodon: community/mastodon.md
- Forum: community/discourse.md
@@ -364,6 +367,12 @@ extra:
link: 'https://so.fnky.nz/@funkypenguin'
- icon: 'fontawesome/brands/github'
link: 'https://github.com/funkypenguin'
- icon: 'fontawesome/brands/discord'
link: 'http://chat.funkypenguin.co.nz'
- icon: 'simple/matrix'
link: 'http://m.fnky.nz'
- icon: 'fontawesome/brands/slack'
link: 'https://communityinviter.com/apps/funkypenguin/geek-with-us'
- icon: 'fontawesome/brands/stack-overflow'
link: 'https://stackoverflow.com/cv/funkypenguin'
- icon: 'fontawesome/brands/linkedin'