Why you need a Kubernetes admission controller
25 avril 2022
0 minutes de lectureUnless you have experience as a Kubernetes operator or administrator, admission controllers may be a new feature for you. These controllers work mostly in the background and many are available as compiled-in plugins, but they can powerfully contribute to a deployment’s security.
Admission controllers intercept API requests before they pass to the API server and can prohibit or modify them. This applies to most types of Kubernetes requests except for pure read requests, which aren’t passed by the controllers. Admission controllers handle requests after they’ve been properly authenticated and authorized.
This post discusses the reasoning behind including Kubernetes admission controllers. It also underscores the advantages of possessing greater levels of familiarity with their functionality.
Several admission controllers are enabled by default because most normal Kubernetes operations rely upon them. Most of these controllers comprise some of the Kubernetes source tree and are compiled as plugins. However, it’s also possible to code and deploy third-party admission controllers. Some illustrative examples will address this later on. For a more detailed look at the specifics of implementing admission controllers, refer to the Kubernetes documentation.
Default admission controllers
Kubernetes features multiple built-in admission controllers. One simple example, DefaultIngressClass
, applies the default ingress class to ingress objects that don’t yet have a specified class. Similarly, DefaultStorageClass
applies the default storage class to PersistentVolumeClaims
that don’t yet have one. This controller must be enabled to allow for dynamic storage provisioning based on storage class.
Admission controllers can be extremely helpful in maintaining security. For example, they can mitigate denial of service (DoS) attacks on multitenant clusters. Consider the LimitRanger
plugin, which — as the name suggests — enforces limit ranges. Limit ranges define mandatory ranges of resource consumption on a per-namespace basis. This prevents tenants from depleting one another’s resources.
Another concern arises with so-called event flooding, where the cluster becomes inundated with events and can’t adequately handle other legitimate requests. The EventRateLimit
controller is a powerful mitigating tool for such scenarios. Its design enables it to limit the rate of events either per namespace or per user.
Additionally, there are two significant controllers that allow developers to run their admission plugins as webhooks to be configured at runtime. MutatingAdmissionWebhook
enables webhooks to modify submitted resources and is typically used to enforce custom defaults. Meanwhile, the ValidatingAdmissionWebhook
controller enables registered webhooks to decide whether an API-validated resource in its final state will continue through the chain or be discarded entirely.
Purpose of controllers
The original way to execute multiple services on a physical machine was to have virtual machines sharing the same host with a hypervisor separating their operating systems. A complicated system of cloud configurations — for example, those defined by AWS — kept systems separate and ensured that tenants could not accidentally or deliberately harm one another.
Kubernetes was initially designed as a cooperative system that a single organization or user could use. Additionally, it relied far more on mutual consideration than other cloud systems. However, as Kubernetes grows in both the variety of available deployments and in its ability to handle larger cluster sizes, it is increasingly important to enact policies that ensure that a single user cannot interfere with a system’s operation.
To automate this process, organizations need a policy system. Kubernetes features some built-in support, but it does not have the capacity of a full-featured, dedicated policy engine.
External policy engines
There are two leading open source policy engines for Kubernetes: Open Policy Agent (OPA) Gatekeeper and Kyverno.
Both engines are donations to the Cloud Native Computing Foundation (CNCF), which is dedicated to the standardization and proliferation of cloud native technologies. It operates under its parent organization, the Linux Foundation. Notably, Kubernetes is a CNCF project.
Kyverno’s primary advantage is that it doesn’t require learning an additional language. All of its policies are defined as Kubernetes resources. In contrast, Gatekeeper leverages OPA’s declarative language, Rego. Gatekeeper is part of the larger OPA system, while Kyverno is a standalone project for Kubernetes. Summarily, Gatekeeper is the more mature project, but Kyverno features a smaller learning curve.
Your custom admission controller
You can use webhooks to code custom admission controller logic using any language that can handle HTTP requests and return Javascript Object Notation (JSON). Go, Python, or Ruby, for example, are all valid options.
The example below demonstrates how to set up a webhook for a custom admission controller. It’s similar to the LimitRanger introduced above, which rejects requests for pods that exceed the namespace limits for resources. Note that this example doesn’t include the entirety of the controller source code, but you can consult the Kubernetes documentation on admission webhook servers to dive deep into the process.
First, register the webhook with a config object:
1apiVersion: admissionregistration.k8s.io/v1beta1
2kind: ValidatingWebhookConfiguration
3metadata:
4 name: mywebhook
5webhooks:
6 - name: mywebhook
7 clientConfig:
8 service:
9 name: mywebhook
10 namespace: project1
11 path: "/hook"
12 rules:
13 - operations: ["CREATE"]
14 apiVersions: ["v1"]
15 apiGroups: [""]
16 resources: ["pods"]
This tells the ValidatingWebhookController
about the webhook. It also specifies which service to access and which path to probe on the container running the server. It also identifies which rules to apply when deciding whether to invoke the webhook. This example focuses on the creation of new pods.
Realistically, creating this resource on the cluster would occur last — after creating a deployment for the webhook server. The deployment includes a service matching the definition in the file above:
1apiVersion: v1
2kind: Service
3metadata:
4 name: mywebhook
5 namespace: project1
6spec:
7 - ports:
8 name: mywebhook
9 port: 80
10 targetPort: 8000
Below is a sample deployment of a webhook, no different than a regular app. We won’t explore securing communications with transport layer security (TLS), although this is strongly recommended.
1apiVersion: apps/v1
2kind: Deployment
3metadata:
4 name: mywebhook
5 namespace: project1
6spec:
7 selector:
8 matchLabels:
9 app: mywebhook
10 template:
11 metadata:
12 labels:
13 app: mywebhook
14 spec:
15 containers:
16 - image: webhook1:latest
17 name: mywebhook
After a new pod request is submitted to Kubernetes and passed by the ValidatingWebhook
, the relevant information is sent as a POST
request to the configured URL path and contains a JSON object for the webhook to process. To validate the request in the affirmative, return a response similar to the following, along with a 200 OK
status code:
1{
2 "apiVersion": "admission.k8s.io/v1",
3 "kind": "AdmissionReview",
4 "response": {
5 "uid": "6ce7a33c-ea67-40e5-9cc8-f710d31985dc",
6 "allowed": true
7 }
8}
The uid
field is taken from the request to match requests to responses. To prevent the pod from being created, the allowed
field must be set to false
and an HTTP error code must be signaled.
A custom admission controller can be as simple as this example or considerably more complex. For a more thorough overview, consult the information in the admission webhook documentation.
Conclusion
A Kubernetes admission controller can modify or reject requests to the API server before the object is persisted. There are numerous versatile built-in Kubernetes controllers, including several pre-compiled plugins and two types of admission control webhooks. However, coding unique webhooks and controllers can provide greater flexibility and finely tuned control.
Admission controllers and webhooks enable projects to provide policy engines like OPA Gatekeeper and Kyverno. Furthermore, defining a custom admission system through HTTP-enabled webhooks is easy to implement in any language that can serve HTTP responses with a JSON payload.
Overall, admission controllers are only one component in a comprehensive Kubernetes security strategy. There are risks and vulnerabilities at each stage of the software development lifecycle, but mitigating these risks doesn’t have to disrupt the entire workflow. When seeking the right type of expertise, a security-focused organization can provide valuable insight.
Sécurité IaC conçue pour les développeurs
Snyk assure la sécurité de votre infrastructure en tant que code du cycle du développement logiciel à son exécution dans le cloud avec un moteur de politique en tant que code unifié pour permettre à chaque équipe de développer, déployer et faire fonctionner les solutions en toute sécurité dans le cloud.
Chris Laux has developed software for more than 20 years in a range of languages.