In this post, we take a look at how to apply custom Nginx configuration directives when you’re using the NGINX Gateway Fabric.

What’s the NGINX Gateway Fabric?

The NGINX Gateway Fabric is an implementation of the Kubernetes Gateway API.

What’s the Gateway API?

The Gateway API is an evolution of the Ingress API; it aims to provide a flexible mechanism for managing north/south network traffic (that is, traffic entering or exiting your Kubernetes cluster), with additional work to support east/west traffic (traffic between pods in your cluster).

What’s this about custom configuration?

I’ve deployed a local development cluster, and I wanted to be able to push images into an image registry hosted on the cluster. This requires (a) running a registry, which is easy, and (b) somehow exposing that registry outside the cluster, which is also easy unless you decide to make it more complex.

In this case, I decided that rather than running an Ingress provider I was going to start familiarizing myself with the Gateway API, so I deployed NGINX Gateway Fabric. My first attempt at pushing an image into the registry looked like this:

$ podman push --tls-verify=false example
Getting image source signatures
Copying blob b9fe5313d237 done   |
Copying blob cc2447e1835a done   |
Copying blob cb8b0886acfb done   |
Copying blob c4219a5645ea [===>----------------------------------] 9.3MiB / 80.2MiB | 372.7 MiB/s
Copying blob c6e5c62d1726 done   |
Copying blob 9ee7eb11f876 done   |
Copying blob f064c46326cb done   |
Copying blob 9c45ffa2a02a done   |
Copying blob 9a6c9897f309 done   |
Copying blob 27a0dbb2828e done   |
Error: writing blob: uploading layer chunked: StatusCode: 413, <html>
<head><title>413 Request Entity Too Large<...

Nginx, by default, restricts the maximum size of a request body to 1m, which is to say, 1 megabyte. You can increase (or remove) this limit by setting the client_max_body_size parameter…but how do you do this in the context of a managed deployment like the NGINX Gateway Fabric?

Via the API?

As of this writing, there is no mechanism to apply custom configuration options via the API (although there is ongoing work to provide this, see issue #1258).

What about dropping a config file into conf.d?

My first thought was that I could mount a custom configuration file into /etc/nginx/conf.d, along the lines of:

  - name: nginx
      - name: nginx-extra-conf
        mountPath: /etc/nginx/conf.d/client_max_body_size.conf
        subPath: client_max_body_size
  - name: nginx-extra-conf
      name: nginx-extra-conf

…but this fails because the Nginx controller explicitly cleans out that directory on startup and is unhappy if it is unable to delete a file.

Replacing nginx.conf

Right now, the solution is to replace /etc/nginx/nginx.conf. This is a relatively simple operation using kustomize to apply a patch to the deployment manifests.

Grab the original configuration

First, we need to retrieve the original nginx.conf:

mkdir configs
podman run --rm --entrypoint cat \ /etc/nginx/nginx.conf > configs/nginx.conf

Modify configs/nginx.conf as necessary; in my case, I added the following line to the http section:

client_max_body_size 0;

Patch the deployment

We can deploy the stock NGINX Gateway Fabric with a kustomization.yaml file like this:

kind: Kustomization
  nginxGatewayVersion: v1.0.0


To patch the Deployment resource, we extend the kustomization.yaml with the following patch:

  - patch: |
      apiVersion: apps/v1
      kind: Deployment
        name: nginx-gateway
        namespace: nginx-gateway
              - name: nginx
                  - mountPath: /etc/nginx/nginx.conf
                    name: nginx-conf-override
                    subPath: nginx.conf
              - name: nginx-conf-override
                  name: nginx-conf-override

And then we add a confdigMapGenerator to generate the nginx-conf-override ConfigMap:

  - name: nginx-conf-override
    namespace: nginx-gateway
      disableNameSuffixHash: true
      - configs/nginx.conf

Now when we deploy from this directory…

kubectl apply -k . --server-side

…the deployment includes our patched nginx.conf and we are able to successfully push images into the cluster registry.

I’ve included the complete kustomization.yaml alongside this post.