From 5697e6d311277bf61e8f966cc948270c779b4f7b Mon Sep 17 00:00:00 2001 From: MegaShinySnivy Date: Sun, 26 Mar 2023 12:08:07 -0500 Subject: [PATCH 1/7] Adding provider --- .../persistence/nfs-subdirectory-provider.md | 180 ++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100644 docs/kubernetes/persistence/nfs-subdirectory-provider.md diff --git a/docs/kubernetes/persistence/nfs-subdirectory-provider.md b/docs/kubernetes/persistence/nfs-subdirectory-provider.md new file mode 100644 index 0000000..b27b4d4 --- /dev/null +++ b/docs/kubernetes/persistence/nfs-subdirectory-provider.md @@ -0,0 +1,180 @@ +--- +description: How to install the NSF-Subdir provider +recipe: NFS-Subdirectory Provider +title: Use an NFS server for your storage in Kubernetes +image: /images/.png +--- + +# {{ page.meta.recipe }} on Kubernetes + +This storage provider allows you to use an NFS server like a native K8s storage provider, allowing you to use non-duplicated mass storage for things like media or other large files + +## {{ page.meta.recipe }} requirements + +!!! summary "Ingredients" + + Already deployed: + + * [x] A [Kubernetes cluster](/kubernetes/cluster/) (*not running Kubernetes? Use the [Docker Swarm recipe instead][invidious]*) + * [x] [Flux deployment process](/kubernetes/deployment/flux/) bootstrapped + + New: + + * [ ] An already existing NFS server + +## Preparation + +!!! warning + +This recpie assumes you have an NFS server ready to go with a username and a password. Setting this up is outside the current scope of this recipe. This provider is also not to be used for persisting SQLite databases, as storing them on NFS will cause the database to corrupt. + +### HelmRepository + +We're going to install a helm chart from the NFS Subdirectory External Provisioner chart repository, so I create the following in my flux repo: + + +```yaml title="/bootstrap/helmrepositories/nfs-subdir-provider.yaml" +apiVersion: source.toolkit.fluxcd.io/v1beta1 +kind: HelmRepository +metadata: + name: nfs-subdir + namespace: flux-system +spec: + interval: 15m + url: https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/ +``` + +Note that I have shortened the name to nfs-subdir, a theme you will find running throughout.[^1] + +### Namespace + +We need a namespace to deploy our HelmRelease and associated ConfigMaps into. Per the [flux design](/kubernetes/deployment/flux/), I create this example yaml in my flux repo at `/bootstrap/namespaces/namespace-invidious.yaml`: + +```yaml title="/bootstrap/namespaces/namespace-nfs-subdir.yaml" +apiVersion: v1 +kind: Namespace +metadata: + name: nfs-subdir +``` + +### Kustomization + +Now that the "global" elements of this deployment have been defined, we do some "flux-ception", and go one layer deeper, adding another Kustomization, telling flux to deploy any YAMLs found in the repo at `/nfs-subdir`. I create this example Kustomization in my flux repo: + +```yaml title="/bootstrap/kustomizations/kustomization-nfs-subdir.yaml" +apiVersion: kustomize.toolkit.fluxcd.io/v1beta1 +kind: Kustomization +metadata: + name: nfs-subdir + namespace: flux-system +spec: + interval: 15m + path: nfs-subdir + prune: true # remove any elements later removed from the above path + timeout: 2m # if not set, this defaults to interval duration, which is 1h + sourceRef: + kind: GitRepository + name: flux-system + validation: server +``` +!!! warning + +Needs some sort of health check! + +### ConfigMap + +Now we're into the invidious-specific YAMLs. First, we create a ConfigMap, containing the entire contents of the helm chart's [values.yaml](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/charts/nfs-subdir-external-provisioner/values.yaml). Paste the values into a `values.yaml` key as illustrated below, indented 4 spaces (*since they're "encapsulated" within the ConfigMap YAML*). I create this example yaml in my flux repo: + +```yaml title="invidious/configmap-invidious-helm-chart-value-overrides.yaml" +apiVersion: v1 +kind: ConfigMap +metadata: + name: nfs-subdir-helm-chart-value-overrides + namespace: nfs-subdir +data: + values.yaml: |- # (1)! + # +``` + +1. Paste in the contents of the upstream `values.yaml` here, intended 4 spaces, and then change the values you need as illustrated below. + +Values you will want to change from the default are: + +```yaml + nfs: + server: #insert server IP or DNS name here + path: #Insert mount path here + mountOptions: #Set things like your user or specific versions here +``` + + +### HelmRelease + +Finally, having set the scene above, we define the HelmRelease which will actually deploy the invidious into the cluster. I save this in my flux repo: + +```yaml title="/invidious/helmrelease-nfs-subdir.yaml" +apiVersion: helm.toolkit.fluxcd.io/v2beta1 +kind: HelmRelease +metadata: + name: nfs-subdir + namespace: nfs-subdir +spec: + chart: + spec: + chart: nfs-subdir-external-provisioner + version: 4.X.X + sourceRef: + kind: HelmRepository + name: nfs-subdir + namespace: flux-system + interval: 15m + timeout: 5m + releaseName: nfs-subdir-external-provisioner + valuesFrom: + - kind: ConfigMap + name: nfs-subdir-helm-chart-value-overrides + valuesKey: values.yaml # This is the default, but best to be explicit for clarity +``` + +## :octicons-video-16: Install the provider. + +Commit the changes to your flux repository, and either wait for the reconciliation interval, or force a reconcilliation using `flux reconcile source git flux-system`. You should see the kustomization appear... + +```bash +~ ❯ flux get kustomizations | grep nfs-subdir +nfs-subdir main@sha1:f1b8c5ad False True Applied revision: main@sha1:f1b8c5ad +~ ❯ +``` + +The helmrelease should be reconciled... + +```bash +~ ❯ $ flux get helmreleases -n nfs-subdir +NAME REVISION SUSPENDED READY MESSAGE +nfs-subdir 4.0.18 False True Release reconciliation succeeded +~ ❯ +``` + +And you should have a happy NFS-Subdirectory pod: + +```bash +~ ❯ kubectl get pods -n nfs-subdir +NAME READY STATUS RESTARTS AGE +nfs-subdir-external-provisioner-9cf9d78b5-6zd7r 1/1 Running 22 (4d11h ago) 105d +~ ❯ +``` + +You can now use this new provider to use an extrernal NFS server for storage. Why would this be useful? Things that you don't want to be replicated, for example, media or large data such as game servers! Of course, this does add a singe point of failure, but a lot less expensive than replicaing data out to many nodes. + +## Summary + +What have we achieved? We have a storage provider that can use an NFS server as it's storage backend, useful for large files, such as media for the autopirate recipe! + +!!! summary "Summary" + Created: + + * [X] We have a new storage provider + +--8<-- "recipe-footer.md" + +[^1]: The reason I shortened it is so I didn't have to type nfs-subdirectory-provider each time. If you want that sort of pain in your life, feel free to change it! From 610bc1120ec7a8ad6391ba898eab05d2e05adb0f Mon Sep 17 00:00:00 2001 From: MegaShinySnivy Date: Sun, 26 Mar 2023 12:29:58 -0500 Subject: [PATCH 2/7] Update Mkdocs.yml --- mkdocs.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/mkdocs.yml b/mkdocs.yml index 7e673ef..75c2da7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -185,6 +185,7 @@ nav: # - Dashboard: kubernetes/ingress/traefik/dashboard.md - Nginx: kubernetes/ingress/nginx.md - Persistence: + - NFS-Subdirectory-Provider: kubernetes/persistence/nfs-subdirectory-provider.md - kubernetes/persistence/index.md - Local Path Provisioner: kubernetes/persistence/local-path-provisioner.md - TopoLVM: kubernetes/persistence/topolvm.md From 232a85e75dcff8ffe3e87ac6374ba3c3b2ae6fcf Mon Sep 17 00:00:00 2001 From: MegaShinySnivy Date: Sun, 26 Mar 2023 20:01:43 -0500 Subject: [PATCH 3/7] Removing all references to invidious --- .../persistence/nfs-subdirectory-provider.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/kubernetes/persistence/nfs-subdirectory-provider.md b/docs/kubernetes/persistence/nfs-subdirectory-provider.md index b27b4d4..860aaa3 100644 --- a/docs/kubernetes/persistence/nfs-subdirectory-provider.md +++ b/docs/kubernetes/persistence/nfs-subdirectory-provider.md @@ -15,7 +15,7 @@ This storage provider allows you to use an NFS server like a native K8s storage Already deployed: - * [x] A [Kubernetes cluster](/kubernetes/cluster/) (*not running Kubernetes? Use the [Docker Swarm recipe instead][invidious]*) + * [x] A [Kubernetes cluster](/kubernetes/cluster/) * [x] [Flux deployment process](/kubernetes/deployment/flux/) bootstrapped New: @@ -48,7 +48,7 @@ Note that I have shortened the name to nfs-subdir, a theme you will find running ### Namespace -We need a namespace to deploy our HelmRelease and associated ConfigMaps into. Per the [flux design](/kubernetes/deployment/flux/), I create this example yaml in my flux repo at `/bootstrap/namespaces/namespace-invidious.yaml`: +We need a namespace to deploy our HelmRelease and associated ConfigMaps into. Per the [flux design](/kubernetes/deployment/flux/), I create this example yaml in my flux repo at `/bootstrap/namespaces/namespace-nfs-subdir.yaml`: ```yaml title="/bootstrap/namespaces/namespace-nfs-subdir.yaml" apiVersion: v1 @@ -83,9 +83,9 @@ Needs some sort of health check! ### ConfigMap -Now we're into the invidious-specific YAMLs. First, we create a ConfigMap, containing the entire contents of the helm chart's [values.yaml](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/charts/nfs-subdir-external-provisioner/values.yaml). Paste the values into a `values.yaml` key as illustrated below, indented 4 spaces (*since they're "encapsulated" within the ConfigMap YAML*). I create this example yaml in my flux repo: +Now we're into the nfs-subdir-specific YAMLs. First, we create a ConfigMap, containing the entire contents of the helm chart's [values.yaml](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/charts/nfs-subdir-external-provisioner/values.yaml). Paste the values into a `values.yaml` key as illustrated below, indented 4 spaces (*since they're "encapsulated" within the ConfigMap YAML*). I create this example yaml in my flux repo: -```yaml title="invidious/configmap-invidious-helm-chart-value-overrides.yaml" +```yaml title="nfs-subdir/configmap-nfs-subdir-helm-chart-value-overrides.yaml" apiVersion: v1 kind: ConfigMap metadata: @@ -110,9 +110,9 @@ Values you will want to change from the default are: ### HelmRelease -Finally, having set the scene above, we define the HelmRelease which will actually deploy the invidious into the cluster. I save this in my flux repo: +Finally, having set the scene above, we define the HelmRelease which will actually deploy the provider into the cluster. I save this in my flux repo: -```yaml title="/invidious/helmrelease-nfs-subdir.yaml" +```yaml title="/nfs-subdir/helmrelease-nfs-subdir.yaml" apiVersion: helm.toolkit.fluxcd.io/v2beta1 kind: HelmRelease metadata: From eb6196e0c587785c1172d1668cfddd729baa5593 Mon Sep 17 00:00:00 2001 From: MegaShinySnivy Date: Sun, 26 Mar 2023 20:10:13 -0500 Subject: [PATCH 4/7] Adding a health check and removing mentions of invidious --- docs/kubernetes/persistence/nfs-subdirectory-provider.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/kubernetes/persistence/nfs-subdirectory-provider.md b/docs/kubernetes/persistence/nfs-subdirectory-provider.md index 860aaa3..2776fc9 100644 --- a/docs/kubernetes/persistence/nfs-subdirectory-provider.md +++ b/docs/kubernetes/persistence/nfs-subdirectory-provider.md @@ -24,7 +24,7 @@ This storage provider allows you to use an NFS server like a native K8s storage ## Preparation -!!! warning +!!! warning "SQLite hates NFS" This recpie assumes you have an NFS server ready to go with a username and a password. Setting this up is outside the current scope of this recipe. This provider is also not to be used for persisting SQLite databases, as storing them on NFS will cause the database to corrupt. @@ -76,10 +76,12 @@ spec: kind: GitRepository name: flux-system validation: server + healthChecks: + - apiVersion: apps/v1 + kind: Deployment + name: nfs-subdir-external-provisioner ``` -!!! warning -Needs some sort of health check! ### ConfigMap From fdb7c10a355eb1dac1f497d46904e89018f7c61a Mon Sep 17 00:00:00 2001 From: MegaShinySnivy Date: Sun, 26 Mar 2023 21:19:30 -0500 Subject: [PATCH 5/7] fixing typo and rendering --- docs/kubernetes/persistence/nfs-subdirectory-provider.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/kubernetes/persistence/nfs-subdirectory-provider.md b/docs/kubernetes/persistence/nfs-subdirectory-provider.md index 2776fc9..57d232c 100644 --- a/docs/kubernetes/persistence/nfs-subdirectory-provider.md +++ b/docs/kubernetes/persistence/nfs-subdirectory-provider.md @@ -24,9 +24,8 @@ This storage provider allows you to use an NFS server like a native K8s storage ## Preparation -!!! warning "SQLite hates NFS" - -This recpie assumes you have an NFS server ready to go with a username and a password. Setting this up is outside the current scope of this recipe. This provider is also not to be used for persisting SQLite databases, as storing them on NFS will cause the database to corrupt. +!!! warning "SQLite hates NFS" + This recpie assumes you have an NFS server ready to go with a username and a password. Setting this up is outside the current scope of this recipe. This provider is also not to be used for persisting SQLite databases, as storing them on NFS will cause the database to corrupt. ### HelmRepository From 89f5562c652bbf657a96d55a0d1efa2607b36fcc Mon Sep 17 00:00:00 2001 From: MegaShinySnivy Date: Sun, 16 Apr 2023 19:22:22 -0500 Subject: [PATCH 6/7] More edits --- .../persistence/nfs-subdirectory-provider.md | 82 +++++++++++++++++-- 1 file changed, 76 insertions(+), 6 deletions(-) diff --git a/docs/kubernetes/persistence/nfs-subdirectory-provider.md b/docs/kubernetes/persistence/nfs-subdirectory-provider.md index 57d232c..948b05a 100644 --- a/docs/kubernetes/persistence/nfs-subdirectory-provider.md +++ b/docs/kubernetes/persistence/nfs-subdirectory-provider.md @@ -7,7 +7,7 @@ image: /images/.png # {{ page.meta.recipe }} on Kubernetes -This storage provider allows you to use an NFS server like a native K8s storage provider, allowing you to use non-duplicated mass storage for things like media or other large files +This storage provider allows you to use an NFS server like a native K8s storage provider, letting you to use mass storage for things like media or other large files. Why would this be useful? Things that you don't want to be replicated, for example, media (replicating 4.5TB can get expensive quick) or large data such as game servers! Of course, this does add a singe point of failure, but a lot less expensive than replicating data out to many nodes. ## {{ page.meta.recipe }} requirements @@ -79,6 +79,7 @@ spec: - apiVersion: apps/v1 kind: Deployment name: nfs-subdir-external-provisioner + namespace: nfs-suibdir ``` @@ -103,11 +104,14 @@ Values you will want to change from the default are: ```yaml nfs: - server: #insert server IP or DNS name here - path: #Insert mount path here - mountOptions: #Set things like your user or specific versions here + server: # (1)! + path: # (2)! + mountOptions: # (3)! ``` +1. Insert server IP or DNS name +2. Insert mount path here +3. Set things like your user or specific versions here ### HelmRelease @@ -137,7 +141,7 @@ spec: valuesKey: values.yaml # This is the default, but best to be explicit for clarity ``` -## :octicons-video-16: Install the provider. +## Install the provider. Commit the changes to your flux repository, and either wait for the reconciliation interval, or force a reconcilliation using `flux reconcile source git flux-system`. You should see the kustomization appear... @@ -165,7 +169,73 @@ nfs-subdir-external-provisioner-9cf9d78b5-6zd7r 1/1 Running 22 (4d11h ag ~ ❯ ``` -You can now use this new provider to use an extrernal NFS server for storage. Why would this be useful? Things that you don't want to be replicated, for example, media or large data such as game servers! Of course, this does add a singe point of failure, but a lot less expensive than replicaing data out to many nodes. +You can now use this new provider to use an external NFS server for storage. + +### How do I know it's working? + +So the provisioner is running, but how do we know we can actually provision volumes? + +#### Create PVC + +Create a PVC, by running: + +```bash +cat < Date: Tue, 18 Apr 2023 13:34:40 -0500 Subject: [PATCH 7/7] More edits --- docs/kubernetes/persistence/nfs-subdirectory-provider.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/kubernetes/persistence/nfs-subdirectory-provider.md b/docs/kubernetes/persistence/nfs-subdirectory-provider.md index 948b05a..d501324 100644 --- a/docs/kubernetes/persistence/nfs-subdirectory-provider.md +++ b/docs/kubernetes/persistence/nfs-subdirectory-provider.md @@ -5,7 +5,7 @@ title: Use an NFS server for your storage in Kubernetes image: /images/.png --- -# {{ page.meta.recipe }} on Kubernetes +# {{ page.meta.recipe }} on Kubernetes This storage provider allows you to use an NFS server like a native K8s storage provider, letting you to use mass storage for things like media or other large files. Why would this be useful? Things that you don't want to be replicated, for example, media (replicating 4.5TB can get expensive quick) or large data such as game servers! Of course, this does add a singe point of failure, but a lot less expensive than replicating data out to many nodes. @@ -24,14 +24,13 @@ This storage provider allows you to use an NFS server like a native K8s storage ## Preparation -!!! warning "SQLite hates NFS" +!!! warning "SQLite hates NFS" This recpie assumes you have an NFS server ready to go with a username and a password. Setting this up is outside the current scope of this recipe. This provider is also not to be used for persisting SQLite databases, as storing them on NFS will cause the database to corrupt. ### HelmRepository We're going to install a helm chart from the NFS Subdirectory External Provisioner chart repository, so I create the following in my flux repo: - ```yaml title="/bootstrap/helmrepositories/nfs-subdir-provider.yaml" apiVersion: source.toolkit.fluxcd.io/v1beta1 kind: HelmRepository @@ -82,7 +81,6 @@ spec: namespace: nfs-suibdir ``` - ### ConfigMap Now we're into the nfs-subdir-specific YAMLs. First, we create a ConfigMap, containing the entire contents of the helm chart's [values.yaml](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/charts/nfs-subdir-external-provisioner/values.yaml). Paste the values into a `values.yaml` key as illustrated below, indented 4 spaces (*since they're "encapsulated" within the ConfigMap YAML*). I create this example yaml in my flux repo: