Kyverno (kyverno-infra)
Optional. The platform runs without it. What you lose is enforcement: the imageVerification, imageRegistry and podPrivileges modes in the tenant values become labels that nothing reads.
kyverno-infra is not part of the OLIVE platformIt is provided as-is to make setup easier — not maintained, patched or supported as part of the platform release, and not covered by OLIVE support. Kyverno is third-party software with its own licence, release cycle and security advisories; tracking and applying those is yours regardless of how you installed it.
What OLIVE does stand behind is the contract below. Meet it however you like.
You do not need this chart to use the policies. OLIVE Kyverno Policies lists each one — what it does, why it exists, and the manifest in a form you can apply and adapt with your own Kyverno.
There are three policy families, each rendered twice (once for audit, once for enforce), so a full installation is up to six policy objects.
Why OLIVE offers it
OLIVE signs the container images it builds with cosign and pins every image by digest. Kyverno is what makes that verifiable at admission: it checks the signature before a pod is created, and refuses anything that does not verify or that comes from an unapproved registry.
Without an admission engine, digest pinning still guarantees you run the artefact the manifest named — but nothing checks that the artefact the manifest named is the one OLIVE built.
What the chart enforces
| Policy family | Values block | Enrolment label | Default |
|---|---|---|---|
| Image signatures — keyless cosign signature on images from the OLIVE registry | imageVerification | olive/image-verification | enabled, audit only |
| Registry allow-list — container images must come from an approved registry prefix | imageRegistryAllowList | olive/image-registry | enabled, audit only |
Pod privileges — no privileged mode, no privilege escalation, no hostPath, no host namespaces | podSecurityRestrictions | olive/pod-privileges | disabled |
The three are deliberately independent, because they cost different amounts to comply with. Pod privileges is usually the first a tenant can enforce — the pod spec only has to stop asking for host access. The registry allow-list is next; nothing has to be rebuilt. Signatures are last, because they need a signing pipeline for everything the namespace pulls.
What the platform administrator specifies
The policies are the platform administrator's: which identity is allowed to have signed, which registries are on the allow-list, which label keys enrol a namespace, and which of the three controls a given namespace is held to. Those are listed in Infrastructure Prerequisites → Kyverno, and the policies themselves in OLIVE Kyverno Policies.
This page covers installing the engine and rendering those policies into the cluster — all of which needs cluster rights.
Using the kyverno-infra chart
Wrapper chart
apiVersion: v2
name: kyverno-infra-wrapper
description: Helm chart for managing Kyverno
type: application
version: 1.0.0
appVersion: '1.0'
dependencies:
- name: kyverno-infra
version: 1.0.10
repository: oci://registry.gitlab.com/olive6224315/helm-charts/charts
condition: services.global.kyverno-infra
Values
services:
global:
kyverno-infra: true
global:
clusterName: 'develop'
domain: 'olive-team.com'
imageVerification:
labelKey: 'olive/image-verification'
imageRegistry:
labelKey: 'olive/image-registry'
podPrivileges:
labelKey: 'olive/pod-privileges'
serviceMesh:
mode: none
kyverno-infra:
namespace:
create: true
name: '' # follows the Application's destination namespace
syncOptions: 'Prune=false'
imagePinning:
enforce: false # until the kyverno image tags are digest-pinned
imageVerification:
enabled: true
imageReferences:
- 'registry.gitlab.com/olive6224315/docker-images/*'
keyless:
subject: 'https://gitlab.com/olive6224315/docker-images//.gitlab-ci.yml@refs/heads/main'
issuer: 'https://gitlab.com'
rekorUrl: 'https://rekor.sigstore.dev'
# Must exist in the KYVERNO namespace, not the tenant's. Created out of band.
imagePullSecrets:
- 'olive-regcred'
excludeNamespaces:
- 'kube-system'
- 'kube-public'
- 'kube-node-lease'
- 'oli-kyverno'
- 'argocd'
modes:
audit:
enabled: true
enforce:
enabled: false
# ---- registry allow-list --------------------------------------------
imageRegistryAllowList:
enabled: true
# Every entry is a PREFIX, matched up to the ':tag' or '@sha256:' boundary.
# Docker Hub short names are expanded by the chart to all five equivalent
# spellings; write anything else fully qualified.
allowed:
- 'registry.gitlab.com/olive6224315/docker-images'
- 'gcr.io/distroless/nodejs24-debian13'
- 'gcr.io/distroless/python3-debian13'
- 'node'
- 'quay.io/keycloak/keycloak'
- 'quay.io/oauth2-proxy/oauth2-proxy'
- 'hashicorp/vault'
- 'hashicorp/vault-k8s'
- 'hashicorp/vault-secrets-operator'
- 'couchdb'
- 'redis'
- 'chrislusf/seaweedfs'
- 'temporalio/server'
- 'temporalio/admin-tools'
- 'temporalio/ui'
- 'alpine'
- 'alpine/curl'
- 'alpine/k8s'
- 'docker.io/busybox'
# Needed as soon as the mesh is on in sidecar mode — a blocked sidecar
# image means no pod in an enrolled namespace can start.
- 'docker.io/istio/proxyv2'
# Appended to `allowed` rather than replacing it, so a cluster override
# cannot silently drop the platform's own images.
extraAllowed: []
excludeNamespaces:
- 'kube-system'
- 'kube-public'
- 'kube-node-lease'
- 'oli-kyverno'
- 'argocd'
# Pure spec inspection, no network — so it also evaluates what is already
# running. Leave it on: enrolling a namespace produces a full inventory
# within a minute, without restarting a single workload.
background: true
modes:
audit:
enabled: true
enforce:
enabled: false
# ---- pod privilege restrictions -------------------------------------
podSecurityRestrictions:
enabled: false # off by default — switch on per cluster
# The four rules are individually switchable. hostPath is the one that
# blocks legitimate-looking workloads (log shippers, node exporters, CSI
# helpers) and the one most likely to need turning off for a while.
# At least one must stay on.
rules:
privileged: true
privilegeEscalation: true
hostNamespaces: true
hostPath: true
excludeNamespaces:
- 'kube-system'
- 'kube-public'
- 'kube-node-lease'
- 'oli-kyverno'
- 'argocd'
background: true
modes:
audit:
enabled: true
enforce:
enabled: false
kyverno:
enabled: true
crds:
install: true
# Argo CD applies client-side by default, which stuffs the whole manifest
# into last-applied-configuration. Kyverno's CRDs blow past the 262144-byte
# annotation limit, so those specific objects are applied server-side.
annotations:
argocd.argoproj.io/sync-options: ServerSideApply=true
# THE SECOND HALF OF THE SAME FIX. Kyverno 3.9 splits its CRDs across two
# subcharts with two separate annotation keys — see the warning below.
kyverno-api:
annotations:
argocd.argoproj.io/sync-options: ServerSideApply=true
config:
preserve: false
admissionController:
replicas: 2
The Kyverno chart 3.9 ships its CRDs from two subcharts: crds (11 CRDs across three groups — 7 kyverno.io, 2 reports.kyverno.io, 2 wgpolicyk8s.io) and kyverno-api (the 11 policies.kyverno.io types OLIVE's policies use, from a separate chart with its own top-level annotations value).
Setting only kyverno.crds.annotations gets the admission controller healthy — its startup check wants the legacy CRDs — while leaving exactly 11 CRDs permanently OutOfSync with error when patching ... annotations: Too long.
Set both. On the Argo CD Application itself, also set ServerSideApply=true and ServerSideDiff=true, and do not use Replace=true — Kyverno's own installation notes warn against it for existing resources.
Prune=false on the namespace is load-bearingDeleting this namespace removes the admission controller while its cluster-scoped ValidatingWebhookConfiguration lingers — a webhook with no backend, which either blocks pod creation cluster-wide or silently stops enforcing every policy. The second is worse, because nothing reports it.
Registry credentials, in Kyverno's own namespace
The cosign signature lives in the registry as an ordinary tag beside the image. The OLIVE registry is private, so Kyverno needs its own pull credentials — the tenant's imagePullSecret is not consulted.
kubectl -n oli-kyverno create secret docker-registry olive-regcred \
--docker-server=registry.gitlab.com \
--docker-username=<user> --docker-password=<token>
The error is "no signatures found"-shaped. It reads like a signing failure and is in fact a credentials failure. This is the single most common reason a working signing pipeline appears broken.
Namespaces to exclude
System and platform namespaces should not be evaluated: kube-system, kube-public, kube-node-lease, the Kyverno namespace itself, and argocd.
This matters most for the pod-privilege policy: the CNI, CoreDNS and every CSI driver legitimately run privileged, with hostPath and host namespaces. An enforcing version of that policy reaching kube-system is a cluster that cannot recover a node restart.
The platform's own namespaces enrol themselves
vault-infra and keycloak-infra now label their own namespaces with all three keys at audit by default (namespace.labels and namespace.extraLabels respectively), because tenant-setup does not manage those namespaces. Remove a label line there to opt that namespace out of that one policy.
hashicorp/vault-secrets-operator must be on imageRegistryAllowList.allowed before the Vault namespace can move past audit.
Service mesh interaction
Leave the Kyverno namespace out of the mesh, or apply a PERMISSIVE exemption for its webhook service first. The admission webhook is called by the kube-apiserver, which is never in the mesh and has no SPIFFE identity — a mesh-wide STRICT policy rejects those calls, and then every pod creation in every enrolled namespace fails.
What the Argo CD project must permit
This is the heaviest of the five in cluster-scoped terms — it installs the policy engine and the policies, and both are cluster-scoped:
clusterResourceWhitelist:
- group: ''
kind: Namespace # namespace.create: true, and the subchart's own
- group: 'apiextensions.k8s.io'
kind: CustomResourceDefinition # 24 of them, see the note below
- group: 'rbac.authorization.k8s.io'
kind: ClusterRole
- group: 'rbac.authorization.k8s.io'
kind: ClusterRoleBinding
# The policies themselves - one entry per Kind the chart actually renders
- group: 'policies.kyverno.io'
kind: ImageValidatingPolicy # imageVerification.enabled: true
- group: 'policies.kyverno.io'
kind: ValidatingPolicy # imageRegistryAllowList.enabled: true,
# and podSecurityRestrictions, which ships false
Nothing in this chart renders a ValidatingWebhookConfiguration or MutatingWebhookConfiguration. Kyverno's admission controller registers its own webhooks at runtime, from the policies it finds, and rewrites them as policies come and go.
Two consequences. The project does not need those Kinds permitted — granting them achieves nothing here. And Argo CD never sees the webhooks, so they are outside the Application: they are not pruned when a policy is removed and not reported as drift when Kyverno changes them. An unexpected webhook is a question for Kyverno's logs, not for the sync status.
skipCrds does not reach themThe 24 CRDs come from the chart's crds and kyverno-api subcharts, rendered as normal templates rather than from a crds/ directory. helm.skipCrds therefore has no effect on them, and the project must permit CustomResourceDefinition — there is no install-without-CRDs variant of this chart.
They also have to be applied server-side. Argo CD applies client-side by default, which writes the whole manifest into the last-applied-configuration annotation; Kyverno's CRDs are far too large for that, and the same size is why they need a server-side diff as well — see below.
Argo CD adaptations
Two adjustments keep Argo CD and Kyverno from fighting each other. Add to the argocd-cm ConfigMap:
# Kyverno creates ClusterRoles with an aggregationRule; the API server fills
# in /rules, which Argo CD reads as drift and selfHeal keeps trying to revert.
resource.customizations.ignoreDifferences.rbac.authorization.k8s.io_ClusterRole: |
jsonPointers:
- /rules
# Kyverno churns report CRs constantly. Without this they flood the Argo CD
# resource tree and make the application view unusable.
resource.exclusions: |
- apiGroups: ["kyverno.io", "reports.kyverno.io"]
kinds:
- AdmissionReport
- BackgroundScanReport
- ClusterAdmissionReport
- ClusterBackgroundScanReport
clusters: ["*"]
In the ApplicationSet, the Kyverno application also needs a server-side diff, because its CRDs are too large for a client-side diff to converge — otherwise the CRD resources report OutOfSync forever even when they are synced:
{{- if eq .path.basename "kyverno-infra" }}
argocd.argoproj.io/compare-options: ServerSideDiff=true
{{- end }}
Rollout
Start every namespace at audit, on all three labels. The registry allow-list and the pod-privilege policy go quiet within a minute, because they are pure spec inspection and evaluate pods that are already running. Signature verification takes longer, because it has to make a registry call and therefore only evaluates at pod creation.
The usual order of reaching enforce, cheapest first:
olive/pod-privileges— complying needs no rebuild and no signature, only a pod spec that stops asking for host access.olive/image-registry— nothing has to be rebuilt; an unlisted image has to be replaced or the prefix added.olive/image-verification— needs every OLIVE image in the namespace to be signed by the pipeline.
enforce fails lateAdmission control acts on pod creation. Switching a namespace to enforce evicts nothing and everything looks fine — until the next rollout, scale-up, eviction or node drain. A namespace can sit in a broken state for weeks and discover it during a node reboot at 2 a.m. A crash-looping container does not recreate the pod object and is therefore not re-admitted, which is why the failure is so delayed.
Move to enforce only after the audit report has been empty across a full deployment cycle.
Related
- OLIVE Kyverno Policies — the policies themselves, in copy-and-adapt form
- Security Configuration → Admission Policies — the per-tenant mode values
- Cluster Operator → Admission Control and Image Integrity