Installing k3s and MetalLB in a Raspberry Pi Cluster

December 19, 2022 · 3 minutes

After a long time, I create courageous to start my first post in English, here we go.
Please be nice and comment if you found anything to help me to improve.

Some days ago, I back to study more about Kubernetes and decide to create my own cluster, previously running with Docker Swarm, and I’ve started to move all applications to Kubernetes.

Current cluster configuration:

Raspberry Pi 4 4GB (Control Plane)
- 240GB SSD Connected over USB (Cluster volume)
Raspberry Pi 3B+
Raspberry Pi 3B+

I decided to install k3s since it’s recommended way to run Kubernetes in devices like Raspberry Pi’s. Here the steps that I followed to install/configure a k3s cluster with MetalLB in my Raspberry Pi Cluster.

Preparing Raspberry Pi

Connect with all nodes and edit /boot/cmdline.txt file and the configuration at the end of line:

cgroup_memory=1 cgroup_enable=memory

Installing k3s

Let’s connect with control node and generate a secret and install k3s, here we are disabling servicelb and traekif due we will use MetalLB as a Load Balancer service, and I will install Traefik manually latter:

$ TOKEN=$(python3 -c "import secrets; print(secrets.token_hex(32))")
$ curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644 --disable servicelb --disable traefik --token $TOKEN --bind-address <control_node_ip>

Run the followed command in worker nodes:

$ curl -sfL https://get.k3s.io | K3S_URL=https://<control_node_ip>:6443 K3S_TOKEN=<token> sh -

When the installation is completed, let’s label the worker nodes:

$ kubectl label nodes <node_name> kubernetes.io/role=worker

Installing MetalLB

Obs: The followed commands need to be executed in Control Node.

Now, connect throught ssh with control node and install MetalLB:

$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml

Define the IP addresses that will be used by MetalLB, for that we need to create a config.yml file and apply with kubectl:

$ cat << EOF >> config.yml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: config
  namespace: metallb-system
spec:
  addresses:
  - <first_node_ip_in_cluster>-<last_node_ip_in_cluster>
EOF

Let’s apply the configuration using kubectl:

$ kubectl apply -f config.yml

Deploy Nginx

To testing these configurations, let’s deploy the a Nginx service, let’s create a file called deployment.yml:

$ cat << EOF >> deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
EOF

Create a file with service configuration:

$ cat << EOF >> service.yml
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80
  type: LoadBalancer
EOF

Now we can appply these configurations using kubectl:

$ kubectl apply -f deployment.yml
$ kubectl apply -f service.yml

To check if the deployment and services are running as expected, run the followed commands:

$ kubectl get pods
$ kubectl get services

The output will be similar to this one, note the External IP address for Nginx service provided by MetalLB:

pi@raspberrypi:~/k8s-nginx $ kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-965685897-bddf7   1/1     Running   0          19h
nginx-965685897-cfx4z   1/1     Running   0          19h
nginx-965685897-n274m   1/1     Running   0          19h
nginx-965685897-wxblb   1/1     Running   0          19h
nginx-965685897-t2zhr   1/1     Running   0          19h
pi@raspberrypi:~/k8s-nginx $ kubectl get services
NAME         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP      10.43.0.1       <none>        443/TCP        19h
nginx        LoadBalancer   10.43.134.193   10.0.0.10     80:30341/TCP   19h

The nginx service will be avaliable on External IP http://10.0.0.10, to access in your browser.

Removing k3s

If you need to remove the k3s installation just run the followed commands:

Control Plane: $ /usr/local/bin/k3s-uninstall.sh
Workers: $ /usr/local/bin/k3s-agent-uninstall.sh

Useful links related to MetalLB configurations:

Reload IP configuration
IP Range change workflow