Why you need a Kubernetes admission controller

Unless you have experience as a Kubernetes operator or administrator, admission controllers may be a new feature for you. These controllers work mostly in the background and many are available as compiled-in plugins, but they can powerfully contribute to a deployment’s security.

Admission controllers intercept API requests before they pass to the API server and can prohibit or modify them. This applies to most types of Kubernetes requests except for pure read requests, which aren’t passed by the controllers. Admission controllers handle requests after they’ve been properly authenticated and authorized.

This post discusses the reasoning behind including Kubernetes admission controllers. It also underscores the advantages of possessing greater levels of familiarity with their functionality.

Several admission controllers are enabled by default because most normal Kubernetes operations rely upon them. Most of these controllers comprise some of the Kubernetes source tree and are compiled as plugins. However, it’s also possible to code and deploy third-party admission controllers. Some illustrative examples will address this later on. For a more detailed look at the specifics of implementing admission controllers, refer to the Kubernetes documentation.

Default admission controllers

Kubernetes features multiple built-in admission controllers. One simple example, DefaultIngressClass, applies the default ingress class to ingress objects that don’t yet have a specified class. Similarly, DefaultStorageClass applies the default storage class to PersistentVolumeClaims that don’t yet have one. This controller must be enabled to allow for dynamic storage provisioning based on storage class.

Admission controllers can be extremely helpful in maintaining security. For example, they can mitigate denial of service (DoS) attacks on multitenant clusters. Consider the LimitRanger plugin, which — as the name suggests — enforces limit ranges. Limit ranges define mandatory ranges of resource consumption on a per-namespace basis. This prevents tenants from depleting one another’s resources.

Another concern arises with so-called event flooding, where the cluster becomes inundated with events and can’t adequately handle other legitimate requests. The EventRateLimit controller is a powerful mitigating tool for such scenarios. Its design enables it to limit the rate of events either per namespace or per user.

Additionally, there are two significant controllers that allow developers to run their admission plugins as webhooks to be configured at runtime. MutatingAdmissionWebhook enables webhooks to modify submitted resources and is typically used to enforce custom defaults. Meanwhile, the ValidatingAdmissionWebhook controller enables registered webhooks to decide whether an API-validated resource in its final state will continue through the chain or be discarded entirely.

Purpose of controllers

The original way to execute multiple services on a physical machine was to have virtual machines sharing the same host with a hypervisor separating their operating systems. A complicated system of cloud configurations — for example, those defined by AWS — kept systems separate and ensured that tenants could not accidentally or deliberately harm one another.

Kubernetes was initially designed as a cooperative system that a single organization or user could use. Additionally, it relied far more on mutual consideration than other cloud systems. However, as Kubernetes grows in both the variety of available deployments and in its ability to handle larger cluster sizes, it is increasingly important to enact policies that ensure that a single user cannot interfere with a system’s operation.

To automate this process, organizations need a policy system. Kubernetes features some built-in support, but it does not have the capacity of a full-featured, dedicated policy engine.

External policy engines

There are two leading open source policy engines for Kubernetes: Open Policy Agent (OPA) Gatekeeper and Kyverno.

Both engines are donations to the Cloud Native Computing Foundation (CNCF), which is dedicated to the standardization and proliferation of cloud native technologies. It operates under its parent organization, the Linux Foundation. Notably, Kubernetes is a CNCF project.

Kyverno’s primary advantage is that it doesn’t require learning an additional language. All of its policies are defined as Kubernetes resources. In contrast, Gatekeeper leverages OPA’s declarative language, Rego. Gatekeeper is part of the larger OPA system, while Kyverno is a standalone project for Kubernetes. Summarily, Gatekeeper is the more mature project, but Kyverno features a smaller learning curve.

Your custom admission controller

You can use webhooks to code custom admission controller logic using any language that can handle HTTP requests and return Javascript Object Notation (JSON). Go, Python, or Ruby, for example, are all valid options.

The example below demonstrates how to set up a webhook for a custom admission controller. It’s similar to the LimitRanger introduced above, which rejects requests for pods that exceed the namespace limits for resources. Note that this example doesn’t include the entirety of the controller source code, but you can consult the Kubernetes documentation on admission webhook servers to dive deep into the process.

First, register the webhook with a config object:

1apiVersion: admissionregistration.k8s.io/v1beta1
2kind: ValidatingWebhookConfiguration
3metadata:
4  name: mywebhook
5webhooks:
6  - name: mywebhook
7    clientConfig:
8      service:
9        name: mywebhook
10        namespace: project1
11        path: "/hook"
12    rules:
13      - operations: ["CREATE"]
14        apiVersions: ["v1"]
15        apiGroups: [""]
16        resources: ["pods"]

This tells the ValidatingWebhookController about the webhook. It also specifies which service to access and which path to probe on the container running the server. It also identifies which rules to apply when deciding whether to invoke the webhook. This example focuses on the creation of new pods.

Realistically, creating this resource on the cluster would occur last — after creating a deployment for the webhook server. The deployment includes a service matching the definition in the file above:

1apiVersion: v1
2kind: Service
3metadata:
4  name: mywebhook
5  namespace: project1
6spec:
7  - ports:
8      name: mywebhook
9      port: 80
10      targetPort: 8000

Below is a sample deployment of a webhook, no different than a regular app. We won’t explore securing communications with transport layer security (TLS), although this is strongly recommended.

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: mywebhook
5  namespace: project1
6spec:
7  selector:
8    matchLabels:
9      app: mywebhook
10  template:
11    metadata:
12      labels:
13        app: mywebhook
14    spec:
15      containers:
16        - image: webhook1:latest
17          name: mywebhook

After a new pod request is submitted to Kubernetes and passed by the ValidatingWebhook, the relevant information is sent as a POST request to the configured URL path and contains a JSON object for the webhook to process. To validate the request in the affirmative, return a response similar to the following, along with a 200 OK status code:

1{
2  "apiVersion": "admission.k8s.io/v1",
3  "kind": "AdmissionReview",
4  "response": {
5    "uid": "6ce7a33c-ea67-40e5-9cc8-f710d31985dc",
6    "allowed": true
7  }
8}

The uid field is taken from the request to match requests to responses. To prevent the pod from being created, the allowed field must be set to false and an HTTP error code must be signaled.

A custom admission controller can be as simple as this example or considerably more complex. For a more thorough overview, consult the information in the admission webhook documentation.

Conclusion

A Kubernetes admission controller can modify or reject requests to the API server before the object is persisted. There are numerous versatile built-in Kubernetes controllers, including several pre-compiled plugins and two types of admission control webhooks. However, coding unique webhooks and controllers can provide greater flexibility and finely tuned control.

Admission controllers and webhooks enable projects to provide policy engines like OPA Gatekeeper and Kyverno. Furthermore, defining a custom admission system through HTTP-enabled webhooks is easy to implement in any language that can serve HTTP responses with a JSON payload.

Overall, admission controllers are only one component in a comprehensive Kubernetes security strategy. There are risks and vulnerabilities at each stage of the software development lifecycle, but mitigating these risks doesn’t have to disrupt the entire workflow. When seeking the right type of expertise, a security-focused organization can provide valuable insight.

Chris Laux has developed software for more than 20 years in a range of languages.