4.2 KiB
date, categories, tags, title, description, draft
| date | categories | tags | title | description | draft | ||
|---|---|---|---|---|---|---|---|
| 2023-06-11 |
|
|
Kubernetes on Hetzner dedicated server | How to setup and secure a bare-metal Kubernetes infrastructure on Hetzner dedicated servers | true |
Kubernetes (K3s) on Hetzner
In this post, we continue our adventure setting up an app hosting platform running on Kubernetes.
My two physical servers were "delivered" (to my inbox), along with instructions re SSHing to the "rescueimage" environment, which looks like this:
--8<-- "what-is-elfhosted.md"
Secure nodes
Per the K3s docs, there are some local firewall requirements for K3s server/worker nodes:
https://docs.k3s.io/installation/requirements#inbound-rules-for-k3s-server-nodes
It's aliiive!
root@fairy01 ~ # kubectl get nodes
NAME STATUS ROLES AGE VERSION
elf01 Ready <none> 15s v1.26.5+k3s1
fairy01 Ready control-plane,etcd,master 96s v1.26.5+k3s1
root@fairy01 ~ #
Now install flux, according to this documentedb bootstrap process...
https://metallb.org/configuration/k3s/
Prepare for Longhorn's NFS schenanigans:
apt-get -y install nfs-common tuned
Performance mode!
tuned-adm profile throughput-performance
Taint the master(s)
kubectl taint node fairy01 node-role.kubernetes.io/control-plane=true:NoSchedule
increase max pods:
https://stackoverflow.com/questions/65894616/how-do-you-increase-maximum-pods-per-node-in-k3s
https://gist.github.com/rosskirkpat/57aa392a4b44cca3d48dfe58b5716954
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --kubelet-arg=config=/etc/rancher/k3s/kubelet-server.config --disable traefik --disable servicelb --flannel-backend=wireguard-native --flannel-iface=enp0s31f6.4000 --kube-controller-manager-arg=node-cidr-mask-size=22 --kubelet-arg=max-pods=500 --node-taint node-role.kubernetes.io/control-plane --prefer-bundled-bin" sh -
create secondary masters:
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --kubelet-arg=config=/etc/rancher/k3s/kubelet-server.config --disable traefik --disable servicelb --flannel-backend=wireguard-native --flannel-iface=enp0s31f6.4000 --kube-controller-manager-arg=node-cidr-mask-size=22 --kubelet-arg=max-pods=500 --node-taint node-role.kubernetes.io/control-plane --prefer-bundled-bin" sh -
mkdir -p /etc/rancher/k3s/
cat << EOF >> /etc/rancher/k3s/kubelet-server.config
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 500
EOF
and on the worker
Ensure that /etc/rancher/k3s exists, to hold our kubelet custom configuration file:
mkdir -p /etc/rancher/k3s/
cat << EOF >> /etc/rancher/k3s/kubelet-server.config
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 500
EOF
Get token from /var/lib/rancher/k3s/server/token on the server, and prepare the environment like this:
export K3S_TOKEN=<token from master>
export K3S_URL=https://<ip of master>:6443
Now join the worker using
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent --flannel-iface=eno1.4000 --kubelet-arg=config=/etc/rancher/k3s/kubelet-server.config --prefer-bundled-bin" sh -
flux bootstrap github \
--owner=geek-cookbook \
--repository=geek-cookbook/elfhosted-flux \
--path bootstrap
root@fairy01:~# kubectl -n sealed-secrets create secret tls elfhosted-expires-june-2033 \
--cert=mytls.crt --key=mytls.key
secret/elfhosted-expires-june-2033 created
root@fairy01:~# kubectl kubectl -n sealed-secrets label secret^C
root@fairy01:~# kubectl -n sealed-secrets label secret elfhosted-expires-june-2033 sealedsecrets.bitnami.com/sealed-secrets-key=active
secret/elfhosted-expires-june-2033 labeled
root@fairy01:~# kubectl rollout restart -n sealed-secrets deployment sealed-secrets
deployment.apps/sealed-secrets restarted
increase watchers (jellyfin) echo fs.inotify.max_user_watches=2097152 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
echo 512 > /proc/sys/fs/inotify/max_user_instances
on dwarves
k taint node dwarf01.elfhosted.com node-role.elfhosted.com/node=storage:NoSchedule