This article is more than one year old. Older articles may contain outdated content. Check that the information in the page has not become incorrect since its publication.
Flux Security Audit has concluded
As Flux is an Incubation project within the Cloud Native Computing Foundation, we were graciously granted a sponsored audit. The primary aim was to assess Flux’s fundamental security posture and to identify next steps in its security story. The audit was commissioned by the CNCF, and facilitated by OSTIF (the Open Source Technology Improvement Fund). ADA Logics was quickly brought into the picture, and spent a month on the audit.
The Flux maintainers and community are very grateful for the work put into this by everyone and the opportunity to grow and improve as a project.
Our first CVE in Flux
Let’s start with what will likely interest you as a Flux user. The engagement uncovered a privilege escalation vulnerability in Flux that could enable users to gain cluster admin privileges. The issue has been fixed and is assigned CVE 2021-41254, and the full disclosure advisory is available at the following link:
CVE-2021-41254: Privilege escalation to cluster admin on multi-tenant Flux.
Description:
Users that can create Kubernetes Secrets, Service Accounts and Flux
Kustomization objects, could execute commands inside the
kustomize-controller container by embedding a shell script in a
Kubernetes Secret. This can be used to run kubectl commands under the
Service Account of kustomize-controller, thus allowing an authenticated
Kubernetes user to gain cluster admin privileges.
Impact:
Multi-tenant environments where non-admin users have permissions to
create Flux Kustomization objects are affected by this issue.
Fix:
This vulnerability was fixed in kustomize-controller v0.15.0 (included
in Flux v0.18.0) released on 2021-10-08. Starting with v0.15, the
kustomize-controller no longer executes shell commands on the container
OS and the kubectl binary has been removed from the container image.
Audit report with full details
We are thankful for the great attention to detail by the team at ADA Logics. The whole report can be found here. To benefit from the analysis in all its detail, we created a project board in GitHub. If you take a look at it closely, you will see that we have fixed some of the most immediate issues already.
Broadly speaking, the issues fall into three categories:
- Enabling Fuzzing for the Flux project
- Documentation issues
- Concrete issues discovered in the Flux code
Flux coming to OSS-Fuzz
The team at ADA Logics didn’t stop at reviewing Flux code. We were pleasantly surprised to receive actual PRs by the team, who set down and helped us integrate with the OSS-Fuzz project. Some of this work still needs to be integrated into all of the Flux controllers, but we are very pleased that a start has been made! OSS-Fuzz is a service for running fuzzers continuously on important open source projects, and the goal is to use sophisticated dynamic analysis to uncover security and reliability issues. There are already numerous other CNCF projects integrated, e.g. Kubernetes, Envoy and Fluent-bit, and we’re excited to be a part of that.
Our documentation from an outside perspective
One very important piece of feedback was that our documentation is mostly geared towards end users, who need very concrete advice on how to integrate Flux into their setups. We provide lots of examples, which are helpful if you want Flux to behave the right way. What is missing to date is an architectural overview and documentation which focuses on the security-related aspects of Flux.
What transpired during the code review
The team at ADA Logics found 22 individual issues, some of which were results from the fuzzers. 1 high severity (that’s the above mentioned CVE), 3 medium severity, 13 low severity and 5 informational.
We appreciate the attention to detail by the team at ADA Logics. The issues range from dependency upgrades to oversights in the code (files which aren’t closed during an operation, unhandled errors) to misleading documentation.
At the time of writing, 43% of the issues were still TODO, 21% WIP and 36% DONE.
The Road Ahead
We are very happy we were given the opportunity to work with and have our assumptions and code reviewed and tested by security experts. Early on we decided that we want to benefit from the findings as much as possible. That’s why we created a project board and added a review of it as a standing agenda item in our weekly dev meetings.
“The Flux team also created a public and easy to track dashboard showing all of the work we've done together and is a fantastic example of good issue-tracking and remediation.”
-- Derek Zimmer, President and Executive Director, OSTIF
Growing the team
If you are interested in contributing to this, we are very much looking forward to working with you. We welcome contributions in helping resolve issues of the road, additional comments on our security posture and also welcome contributions in the form of extending our fuzzing infrastructure. Finally, if you have any additional security feedback, please come and talk to us.
We are working full steam on the Flux Roadmap, just recently got more maintainers involved and continue to listen to feedback.
Again we would like to thank the Cloud Native Computing Foundation for sponsoring the audit, the Open Source Technology Improvement Fund for the coordination and ADA Logics for the careful review and advice during the audit period.
We are happy and proud to be part of this community!