Kubernetes open sourced their security audit. What can we learn?

Earlier this week, on 6th August, the Cloud Native Computing Foundation (CNCF) published a blog post detailing their recent Kubernetes Security Audit. Last year, the CNCF started their security audit program with three projects: CoreDNS, Envoy, and Prometheus. Since this pilot program was successful, the CNCF is rolling it out to other projects in their ecosystem.

Kubernetes is an open source container orchestration engine for automating deployment, scaling, and management of containerized applications, and is the largest project in the CNCF ecosystem. It is awesome that the CNCF is prioritizing security for its flagship project.

Quick takeaways

The full findings of the report are beyond the scope of this post, but we wanted to give you a general sense of the findings anyway. Many of the recommendations in the report involved cleaning up the code base, adding further testing and documentation, and making defaults more security conscious. These basic recommendations would make it easier to patch and resolve problems when they are found. It is important to note that there were five “high severity” findings that included problems with access control, authentication, timing, and data validation. These issues are, briefly:

• An access control bypass of PodSecurityPolicy

• K8s does not facilitate certificate revocation

• HTTPS connections are not authenticated

• Time of check, time of use problem with moving PID

• Improperly patched directory traversal in kubectl cp

The report is definitely worth a read and can be found here.

Security breadth vs. depth

The audit was scoped in such a way that these five control families for potential problems were considered: networking, cryptography, authentication and authorization, secrets management, and multi-tenancy isolation. Kubernetes is a large project, and the report favors breadth rather than depth with regards to security issues. The audit is a birds-eye view of the security health of the project.

Although there are areas of the codebase that could benefit from a deeper review, I think this breadth-first approach was a really good choice. As an illustration, if you go to the doctor and get a random blood test or two, you might find out something interesting about your health, but you are not likely to have a full picture of how you are doing. Conversely, if you consider your health on a holistic scale, you’ll have a better picture of your general health and your doctor is likely to then be able to pinpoint issues for further investigation.

This report was about more than pinpointing an issue or two (although there are some noted), but rather getting a general sense of the health of Kubernetes, so priorities and goals can be chosen in a way to benefit the security and sustainability of the project as a whole. Short and long term goals were recommended for each of the five control categories and provided us with a glimpse into what might be the future of Kubernetes.

Benefits to maintainers and communities

It is great to see such an important and widely used project undergo a security audit. It sets a good example for other projects on the scale of Kubernetes and benefits both the project maintainers and the community at large.

The CNCF and the maintainers of Kubernetes made a thoughtful investment in a security audit. A security audit means the maintainers are potentially a step ahead of malicious actors and can take a proactive security stance, instead of simply a reactive one. It is always preferable to find your own vulnerabilities before others do. There is also a lot to be said for addressing problems found in a report that you planned for, instead of finding out about a problem at a random and potentially inopportune time. Fixes can be thoughtfully prioritized and addressed, and you aren’t working against the clock in the same way you would if vulnerabilities were disclosed through other means. In conducting the audit, and open sourcing the findings, the CNCF and Kubernetes maintainers have established themselves as examples in this field and will inspire other projects to follow their lead in conducting and releasing security audits.

Additionally through open sourcing the findings, community members not only benefit when the short and long term goals are achieved, but they also benefit immediately. This benefit is derived in a few ways. First, users can elect to use Kubernetes, confident in the fact that the CNCF and the maintainers of the project take security seriously. Second, users can review the findings and make more informed choices for their individual projects. And finally, they can avail themselves of all the resources included in the audit, including the white paper and the threat model which are interesting reads for people looking to up their Kubernetes or security skills.

Conclusion

If you're interested in seeing how security audits like this work, or even getting involved in future audits, you can join the CNCF special interest group (or SIG) focused on security. You can learn more about it here: https://github.com/cncf/sig-security.

Congratulations and kudos to CNCF for undertaking such an important effort. We hope you see lots of benefits from the audit. We join you in saying that we hope that through open sourcing your security audits and process, CNCF inspires other projects to follow your lead.