Skip to main content

Local Cluster

In this guide we'll set up a local k3d cluster to apply and explore the configuration described in our other guides. After completing this guide you'll have a standard Kubernetes API server with proper DNS and TLS certificates. You'll be able to easily reset the cluster to a known good state to iterate on your own Platform.

The Concepts page defines capitalized terms such as Platform and Component.

Reset the Cluster​

If you've already followed this guide, reset the cluster by running the following commands. Skip this section if you're creating a cluster for the first time.

First, delete the cluster.

k3d cluster delete workload

Then create the cluster again.

k3d cluster create workload \
  --registry-use k3d-registry.holos.localhost:5100 \
  --port "443:443@loadbalancer" \
  --k3s-arg "--disable=traefik@server:0"

Finally, add your trusted certificate authority.

kubectl apply --server-side=true -f "$(mkcert -CAROOT)/namespace.yaml"
kubectl apply --server-side=true -n cert-manager -f "$(mkcert -CAROOT)/local-ca.yaml"

You're back to the same state as the first time you completed this guide.

What you'll need​

You'll need the following tools installed to complete this guide.

  1. holos - to build the platform.
  2. helm - to render Holos components that wrap upstream Helm charts.
  3. k3d - to provide a k8s api server.
  4. OrbStack or Docker - to use k3d.
  5. kubectl - to interact with the k8s api server.
  6. mkcert - to make trusted TLS certificates.
  7. jq - to fiddle with JSON output.

Configure DNS​

Configure your machine to resolve *.holos.localhost to your loopback interface. This is necessary for requests to reach the workload cluster. Save this script to a file and execute it.

#! /bin/bash
#

set -euo pipefail

tmpdir="$(mktemp -d)"
finish() {
  [[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"

brew install dnsmasq

cat <<EOF >"$(brew --prefix)/etc/dnsmasq.d/holos.localhost.conf"
# Refer to https://holos.run/docs/tutorial/local/k3d/
address=/holos.localhost/127.0.0.1
EOF

if [[ -r /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist ]]; then
  echo "dnsmasq already configured"
else
  sudo cp "$(brew list dnsmasq | grep 'dnsmasq.plist$')" \
    /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
  sudo launchctl unload /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
  sudo launchctl load /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
  dscacheutil -flushcache
  echo "dnsmasq configured"
fi

sudo mkdir -p /etc/resolver
sudo tee /etc/resolver/holos.localhost <<EOF
domain holos.localhost
nameserver 127.0.0.1
EOF
sudo killall -HUP mDNSResponder

echo "all done."

Create the Cluster​

The Workload Cluster is where your applications and services will be deployed. In production this is usually an EKS, GKE, or AKS cluster.

tip

Holos supports all compliant Kubernetes clusters. Holos was developed and tested on GKE, EKS, Talos, k3s, and Kubeadm clusters.

Create a local registry to speed up image builds and pulls.

k3d registry create registry.holos.localhost --port 5100

Create the workload cluster configured to use the local registry.

k3d cluster create workload \
  --registry-use k3d-registry.holos.localhost:5100 \
  --port "443:443@loadbalancer" \
  --k3s-arg "--disable=traefik@server:0"

Traefik is disabled because Istio provides the same functionality.

Setup Root CA​

Platforms most often use cert-manager to issue tls certificates. The browser and tools we're using need to trust these certificates to work together. Generate a local, trusted root certificate authority with the following script.

Admin access is necessary for mkcert to manage the certificate into your trust stores.

sudo -v

Manage the local CA and copy the CA key to the workload cluster so that cert manager can manage trusted certificates.

Save this script to a file and execute it to configure a trusted certificate authority.

#! /bin/bash
#

set -euo pipefail

mkcert --install

tmpdir="$(mktemp -d)"
finish() {
  [[ -d "$tmpdir" ]] && rm -rf "$tmpdir"
}
trap finish EXIT
cd "$tmpdir"

# Create the local CA Secret with ca.crt, tls.crt, tls.key

mkdir local-ca
cd local-ca
CAROOT="$(mkcert -CAROOT)"
cp -p "${CAROOT}/rootCA.pem" ca.crt
cp -p "${CAROOT}/rootCA.pem" tls.crt
cp -p "${CAROOT}/rootCA-key.pem" tls.key
kubectl create secret generic --from-file=.  --dry-run=client -o yaml local-ca > ../local-ca.yaml
echo 'type: kubernetes.io/tls' >> ../local-ca.yaml

cd ..

cat <<EOF > namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  labels:
    kubernetes.io/metadata.name: cert-manager
  name: cert-manager
spec:
  finalizers:
  - kubernetes
EOF
kubectl apply --server-side=true -f namespace.yaml
kubectl apply -n cert-manager --server-side=true -f local-ca.yaml

# Save the Secret to easily reset the cluster later.
install -m 0644 namespace.yaml "${CAROOT}/namespace.yaml"
install -m 0600 local-ca.yaml "${CAROOT}/local-ca.yaml"
warning

Take care to run the local-ca script each time you create the workload cluster so that Certificates are issued correctly.

Clean Up​

If you'd like to clean up the resources you created in this guide, remove them with:

k3d cluster delete workload

Next Steps​

Now that you have a real cluster, apply and explore the manifests Holos renders in the Quickstart guide.