---
date: 2023-06-11
categories:
  - note
tags:
  - elfhosted
title: Kubernetes on Hetzner dedicated server
description: How to setup and secure a bare-metal Kubernetes infrastructure on Hetzner dedicated servers
draft: true
---

# Kubernetes (K3s) on Hetzner

In this post, we continue our adventure setting up an app hosting platform running on Kubernetes.


My two physical servers were "delivered" (to my inbox), along with instructions re SSHing to the "rescueimage" environment, which looks like this:



<!-- more -->

--8<-- "what-is-elfhosted.md"


## Secure nodes

Per the K3s docs, there are some local firewall requirements for K3s server/worker nodes:

https://docs.k3s.io/installation/requirements#inbound-rules-for-k3s-server-nodes



It's aliiive!

```
root@fairy01 ~ # kubectl get nodes
NAME      STATUS   ROLES                       AGE   VERSION
elf01     Ready    <none>                      15s   v1.26.5+k3s1
fairy01   Ready    control-plane,etcd,master   96s   v1.26.5+k3s1
root@fairy01 ~ #
```

Now install flux, according to this documentedb bootstrap process...


https://metallb.org/configuration/k3s/


Prepare for Longhorn's [NFS schenanigans](https://longhorn.io/docs/1.4.2/deploy/install/#installing-nfsv4-client):

```
apt-get -y install nfs-common tuned
```

Performance mode!

`tuned-adm profile throughput-performance`

Taint the master(s)

```
kubectl taint node fairy01 node-role.kubernetes.io/control-plane=true:NoSchedule
```


```
increase max pods:
https://stackoverflow.com/questions/65894616/how-do-you-increase-maximum-pods-per-node-in-k3s

https://gist.github.com/rosskirkpat/57aa392a4b44cca3d48dfe58b5716954

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --kubelet-arg=config=/etc/rancher/k3s/kubelet-server.config --disable traefik  --disable servicelb --flannel-backend=wireguard-native --flannel-iface=enp0s31f6.4000 --kube-controller-manager-arg=node-cidr-mask-size=22 --kubelet-arg=max-pods=500 --node-taint node-role.kubernetes.io/control-plane --prefer-bundled-bin" sh -
```

create secondary masters:

```
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --kubelet-arg=config=/etc/rancher/k3s/kubelet-server.config --disable traefik  --disable servicelb --flannel-backend=wireguard-native --flannel-iface=enp0s31f6.4000 --kube-controller-manager-arg=node-cidr-mask-size=22 --kubelet-arg=max-pods=500 --node-taint node-role.kubernetes.io/control-plane --prefer-bundled-bin" sh -

```


```
mkdir -p /etc/rancher/k3s/
cat << EOF >> /etc/rancher/k3s/kubelet-server.config
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 500
EOF
```




and on the worker


Ensure that `/etc/rancher/k3s` exists, to hold our kubelet custom configuration file:

```bash
mkdir -p /etc/rancher/k3s/
cat << EOF >> /etc/rancher/k3s/kubelet-server.config
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 500
EOF
```

Get [token](https://docs.k3s.io/cli/token) from `/var/lib/rancher/k3s/server/token` on the server, and prepare the environment like this:
```bash
export K3S_TOKEN=<token from master>
export K3S_URL=https://<ip of master>:6443
```

Now join the worker using

```
curl -sfL https://get.k3s.io |  INSTALL_K3S_EXEC="agent --flannel-iface=eno1.4000 --kubelet-arg=config=/etc/rancher/k3s/kubelet-server.config --prefer-bundled-bin" sh -

```


```
flux bootstrap github \
  --owner=geek-cookbook \ 
  --repository=geek-cookbook/elfhosted-flux \
  --path bootstrap
  ```

```
root@fairy01:~# kubectl -n sealed-secrets create secret tls elfhosted-expires-june-2033 \
  --cert=mytls.crt --key=mytls.key
secret/elfhosted-expires-june-2033 created
root@fairy01:~# kubectl kubectl -n sealed-secrets label secret^C
root@fairy01:~# kubectl -n sealed-secrets label secret elfhosted-expires-june-2033 sealedsecrets.bitnami.com/sealed-secrets-key=active
secret/elfhosted-expires-june-2033 labeled
root@fairy01:~# kubectl rollout restart -n sealed-secrets deployment sealed-secrets
deployment.apps/sealed-secrets restarted
```

increase watchers (jellyfin)
echo fs.inotify.max_user_watches=2097152 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

echo 512 > /proc/sys/fs/inotify/max_user_instances

on dwarves

k taint node dwarf01.elfhosted.com  node-role.elfhosted.com/node=storage:NoSchedule