Join our community of software engineering leaders and aspirational developers. Always
stay in-the-know by getting the most important news and exclusive content delivered
fresh to your inbox to learn more about at-scale software development.
REQUIRED
It seems that you've previously unsubscribed from our newsletter
in the past. Click the button below to open the re-subscribe form
in a new tab. When you're done, simply close that tab and continue
with this form to complete your subscription.
The New Stack does not sell your information or share it with
unaffiliated third parties. By continuing, you agree to our
Terms of Use and
Privacy Policy.
Welcome and thank you for joining The New Stack community!
Please answer a few simple questions to help us deliver the news and resources you are interested in.
REQUIRED
REQUIRED
REQUIRED
REQUIRED
REQUIRED
Great to meet you!
Tell us a bit about your job so we can cover the topics you find most relevant.
REQUIRED
REQUIRED
REQUIRED
REQUIRED
REQUIRED
Welcome!
We’re so glad you’re here. You can expect all the best TNS content to arrive
Monday through Friday to keep you on top of the news and at the top of your game.
What’s next?
Check your inbox for a confirmation email where you can adjust your preferences
and even join additional groups.
Follow TNS on your favorite social media networks.
Kubernetes has become one of the most popular container orchestrators for deploying applications at scale. And building the cluster securely has always been a matter of concern. Many tools can scan your cluster and list its security threats. But implementing them can be challenging as one has to write and manage custom webhooks for each of them.
So, policy as a code has been introduced to help Kuberentes admins and make their life easier. Policy means a set of rules that must be met to impose security and govern IT operations easily. Policy as a code is an approach for policy management using high-level languages such as YAML or Rego.
There are different policy engines, such as Kyverno and OPA, to impose policies as code to secure the Kubernetes cluster.
In this blog, we will learn more about Kyverno, its policies, its use in the Kubernetes cluster and how to set up a Kyverno project locally. Along with this, we’ll look at how to work with the Kyverno command line interface (CLI), write test cases for validating and mutating policies, and test them with the Kyverno CLI, which has also been my Linux Foundation mentorship work.
Let’s first understand what the admission controller in Kubernetes is as Kyverno uses it.
The Linux Foundation is the world’s leading home for collaboration on open source software, hardware, standards, and data. Linux Foundation projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, ONAP, Hyperledger Foundation, PyTorch, RISC-V, and more.
Learn More
The latest from Linux Foundation Training
Admission Controller in Kubernetes
An admission controller in Kubernetes is a piece of code that intercepts the incoming object request to the Kuberentes API server before persisting it into the ETCD. But this is done only after the request is authenticated and authorized. These controllers validate or mutate the incoming request based on the validating and mutating admission webhooks, generally known as dynamic admission controllers.
The following diagram illustrates its working inside the Kubernetes cluster.
Kyverno inside the cluster works in place of these webhooks, making the work easier through policies.
What Is Kyverno?
Kyverno, a Greek word for “govern,”is a policy engine natively designed for Kubernetes. Currently, it is an incubating project under the Cloud Native Computing Foundation (CNCF). Kyverno policies are managed as Kubernetes resources and are written in YAML. So, no new language is required to write Kyverno policies.
Features of Kyverno
Various features of Kyverno make it unique from other policy engines, such as:
Policies are managed as Kubernetes resources
Policies are manageable with kubectl, git and Kustomize tools
Validate, mutate, generate or even remove any resources
Helps in container image signing and verification for software supply chain security
Match resources using label selectors and wildcards
Enable background scans on existing Kubernetes resources to ensure best practices
Block resources and report policy violations
Generate policy reports
Test policies and validate resources with Kyverno CLI in CI/CD pipeline before using them in a cluster
How Kyverno Works
Kyverno is used as a dynamic admission controller inside the Kuberentes cluster. It receives validating and mutating admission webhook HTTP callbacks from the kube-apiserver and applies matching policies to return results that enforce admission policies or reject requests.
Kyverno policies can match resources with the help of resource kind, name, label selectors, user, and much more. Policy enforcement is recorded through Kubernetes events, and it also reports violations of existing resources through background scans.
Policy Types and Their Behaviors
Two types of policies are permitted with Kyverno:
ClusterPolicy: The policy scope is cluster-wide.
Policy: The policy scope is limited to the namespace in which it is applied.
The different behaviors found in Kyverno policies for resources are:
Validate: This uses overlay-style syntax with pattern-matching support and conditional (if-then-else) processing. If the Kubernetes resource matches the pattern specified, then the resource is allowed to be created, otherwise, the creation will be blocked.
Mutate: This uses RFC 6902 JSONPatch format, which is used to modify the matching resources.
Generate: This is used when there is a need to create supporting resources based on a new resource.
Verify Images: With the help of Sigstore’s sub-project Cosign, container image signatures are checked.
Kyverno in the Kubernetes Cluster
Kyverno Installation
To work with Kyverno policies in the Kubernetes cluster, first, install Kyverno custom resource definitions (CRDs) either via Helm or kubectl.
Verify the pods in the Kyverno namespace are running.
kubectl get pods -n kyverno
For now, we will see Kyverno policies with validate and mutate behavior on Kubernetes resources.
Validating Policy in Kyverno
To ensure pods are allowed to mount `hostPath` volumes in `readOnly` mode. This validating policy checks all the containers of pods for any 1hostPath1 volumes and ensures that all of them are only in `readOnly` mode.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: ensure-readonly-hostpath
annotations:
policies.kyverno.io/title: Ensure Read Only hostPath
policies.kyverno.io/category: Other
policies.kyverno.io/severity: medium
policies.kyverno.io/minversion: 1.6.0
kyverno.io/kyverno-version: 1.6.2
kyverno.io/kubernetes-version: "1.23"
policies.kyverno.io/subject: Pod
policies.kyverno.io/description: >-
Pods which are allowed to mount hostPath volumes in read/write mode pose a security risk
even if confined to a "safe" file system on the host and may escape those confines (see
https://blog.aquasec.com/kubernetes-security-pod-escape-log-mounts). The only true way
to ensure safety is to enforce that all Pods mounting hostPath volumes do so in read only
mode. This policy checks all containers for any hostPath volumes and ensures they are
explicitly mounted in readOnly mode.
spec:
background: false
validationFailureAction: Enforce
rules:
- name: ensure-hostpaths-readonly
match:
any:
- resources:
kinds:
- Pod
preconditions:
all:
- key: "{{ request.operation || 'BACKGROUND' }}"
operator: AnyIn
value:
- CREATE
- UPDATE
validate:
message: All hostPath volumes must be mounted as readOnly.
foreach:
- list: "request.object.spec.volumes[?hostPath][]"
deny:
conditions:
any:
- key: "{{ request.object.spec.[containers, initContainers, ephemeralContainers][].volumeMounts[?name == '{{element.name}}'][] | length(@) }}"
operator: NotEquals
value: "{{ request.object.spec.[containers, initContainers, ephemeralContainers][].volumeMounts[?name == '{{element.name}}' && readOnly] [] | length(@) }}"
When the policy attribute `validationFailureAction` is set to “Audit”(default) mode, it will report policy violations by resources but will not block them from being created. On the other hand, “Enforce” mode will block them as well.
kubectl apply -f ensure-readonly-hostpath.yaml
kubectl get clusterpolicies
Create new pod resources with `hostPath` volume to test policy.
On creating the above pod resources, one which has `hostPath` mounted as `readOnly` will be created and the other one will be blocked from being created.
Mutating Policy in Kyverno
It is always a good practice to work with fully resolved container images that have digest in them rather than the tag. This mutate policy changes the pod container images and sets them as fully resolved.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: resolve-image-to-digest
annotations:
policies.kyverno.io/title: Resolve Image to Digest
policies.kyverno.io/category: Other
policies.kyverno.io/severity: medium
kyverno.io/kyverno-version: 1.6.0
policies.kyverno.io/minversion: 1.6.0
kyverno.io/kubernetes-version: "1.23"
policies.kyverno.io/subject: Pod
policies.kyverno.io/description: >-
Image tags are mutable and the change of an image can result in the same tag.
This policy resolves the image digest of each image in a container and replaces
the image with the fully resolved reference which includes the digest rather than tag.
spec:
background: false
rules:
- name: resolve-to-digest
match:
any:
- resources:
kinds:
- Pod
preconditions:
all:
- key: "{{request.operation || 'BACKGROUND'}}"
operator: NotEquals
value: DELETE
mutate:
foreach:
- list: "request.object.spec.containers"
context:
- name: resolvedRef
imageRegistry:
reference: "{{ element.image }}"
jmesPath: "resolvedImage"
patchStrategicMerge:
spec:
containers:
- name: "{{ element.name }}"
image: "{{ resolvedRef }}"
kubectl apply -f resolve-image-to-digest.yaml
kubectl get clusterpolicies
From the above output, it can be clearly seen that the NGINX image has been fully resolved along with the image digest.
Setting Up Kyverno Locally
To set up the Kyverno project locally, you must have Golang installed on your machine. Ensure that the `GOPATH` environment variable is set up correctly.
Now clone the Kyverno repository locally on your system under a subdirectory of your `$GOPATH` location, as shown below.
You can also check out the Kyverno project wiki for further guidance You can also create a local cluster with Kind/Minikube and build a local Kyverno image and use them in your local cluster. Find out more from these training videos.
Install the Kyverno CLI. One can find more information about different versions of it on the release page.
wget https://github.com/kyverno/kyverno/releases/download/v1.9.2/kyverno-cli_v1.9.2_linux_x86_64.tar.gz
tar -xvzf kyverno-cli_v1.9.2_linux_x86_64.tar.gz
mv kyverno /usr/local/bin
Verify the Kyverno CLI installation by checking its version.
kyverno version
What Is Kyverno CLI?
The Kyverno CLI is designed to apply and test policies outside the cluster. This is mainly used to validate and test the policy behavior on resources before adding them to the cluster. This CLI can be used with kubectl as a plugin, CI/CD pipelines or as a standalone.
Different Kyverno CLI Commands
Different subcommands can be used with the Kyverno CLI, such as:
Kyverno Apply Command
The `kyverno apply` command is used to try out the policies with given resources. For validating policies, it gives out the result as pass/fail/skip depending on the resource given against the policy. In the case of mutate policies, it generates the mutated resource as an output.
The syntax of the command is as follows:
The `-f` flag is used to pass variables that are needed to test the policy against the resources. Find out more about it here.
Kyverno Test Command
The `kyverno test` command is used to test the resources against policies for desired results. For this, a separate (standard) `kyverno-test.yaml` file is created to mention the tests.
The syntax of the command is as follows:
kyverno test /path/to/testyamlfile
The `-f` flag is used to set a custom file name, which includes test cases. By default, it will search for a file called `kyverno-test.yaml`. Find out more about it here.
Now that we learned about the Kyverno CLI, we can discuss how to write the test cases for resources to be tested against the policies.
Writing Test Cases for Kyverno Policies and Testing Them with Kyverno CLI
To write test cases one by one, we will use the same validating and mutating policies we used above.
Tests for Validating Policy
This validating policy will work on pod resources that have a precondition and rule for matching `hostPath` volume. This precondition is used to have more control over rules by defining variables that will be defined in the `values.yaml` file.
It’s vital to have negative and positive cases to ensure when the policy is being passed and when it is being failed.
Below is the `values.yaml` and`kyverno-test.yaml` file for the validating policy.
This mutating policy will work on a pod resource with a mutate rule to change image tag to digest and a precondition with it. The `kyverno apply` command will generate a mutate (`patchedResource.yaml`) file for the resource on which the policy is applied.
Below is the `values.yaml` , `kyverno-test.yaml` and `patchedResource.yaml` file for the mutating policy.
So, this is how to write test cases for Kyverno policies and test them locally.
The Linux Foundation is the world’s leading home for collaboration on open source software, hardware, standards, and data. Linux Foundation projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, ONAP, Hyperledger Foundation, PyTorch, RISC-V, and more.