Installing metallb on OpenShift with Kustomize
Out of the box, OpenShift (4.x) on bare metal doesn’t come with any integrated load balancer support (when installed in a cloud environment, OpenShift typically makes use of the load balancing features available from the cloud provider). Fortunately, there are third party solutions available that are designed to work in bare metal environments. MetalLB is a popular choice, but requires some minor fiddling to get it to run properly on OpenShift.
If you read through the installation instructions, you will see this note about installation on OpenShift:
To run MetalLB on Openshift, two changes are required: changing the pod UIDs, and granting MetalLB additional networking privileges.
Pods get UIDs automatically assigned based on an OpenShift-managed UID range, so you have to remove the hardcoded unprivileged UID from the MetalLB manifests. You can do this by removing the spec.template.spec.securityContext.runAsUser field from both the controller Deployment and the speaker DaemonSet.
Additionally, you have to grant the speaker DaemonSet elevated privileges, so that it can do the raw networking required to make LoadBalancers work. You can do this with:
The docs here suggest some manual changes you can make, but it’s possible to get everything installed correctly using Kustomize (which makes sense especially given that the MetalLB docs already include instructions on using Kustomize).
A vanilla installation of MetalLB with Kustomize uses a
file that looks like this:
namespace: metallb-system resources: - github.com/metallb/metallb//manifests?ref=v0.9.3 - configmap.yml - secret.yml
secret.yml are files you create locally
containing, respectively, the MetalLB configuration and a secret used to
authenticate cluster members.)
Fixing the security context⌗
In order to remove the
runAsUser directive form the template
securityContext setting, we can use the patchesStrategicMerge
feature. In our
kustomization.yml file we add:
patches: - |- apiVersion: apps/v1 kind: Deployment metadata: name: controller namespace: metallb-system spec: template: spec: securityContext: $patch: replace runAsNonRoot: true
kustomize to replace the contents of the
key with the value included in the patch (without the
directive, the default behavior is to merge the contents, which in this
situation would effectively be a no-op).
We can accomplish the same thing using jsonpatch syntax. In this case, we would write:
patches: - target: kind: Deployment name: controller namespace: metallb-system patch: |- - op: remove path: /spec/template/spec/securityContext/runAsUser
With either solution, the final output includes a
that looks like this:
spec: template: spec: securityContext: runAsNonRoot: true
Granting elevated privileges⌗
The MetaLB docs suggest running:
oc adm policy add-scc-to-user privileged -n metallb-system -z speaker
But we can configure the same privilege level by setting up an appropriate role binding as part of our Kustomize manifests.
First, we create an
allow-privileged cluster role by adding the following
--- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: allow-privileged rules: - apiGroups: - security.openshift.io resourceNames: - privileged resources: - securitycontextconstraints verbs: - use
Then we bind the
speaker service account to the
by adding a
--- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: metallb-allow-privileged roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: allow-privileged subjects: - kind: ServiceAccount name: speaker namespace: metallb-system
You will need to add these new manifests to your
should now look like:
namespace: metallb-system resources: - github.com/metallb/metallb//manifests?ref=v0.9.3 - configmap.yml - secret.yml - clusterole.yml - rolebinding.yml patches: - target: kind: Deployment name: controller namespace: metallb-system patch: |- - op: remove path: /spec/template/spec/securityContext/runAsUser
The changes described here will result in a successful MetalLB deployment into your OpenShift environment.