Skip to main content

Production Readiness Checklist

Everything on this page is documented in detail elsewhere. What this page adds is that it is one list you can sign off, in the order a reviewer asks about it — which is what a go-live review, an internal audit and a security questionnaire all want, and what no individual page in this manual provides.

Use it twice: once before an environment carries real data, and again after any change to the cluster, the ingress, the mesh, the CNI or the identity provider.

Most of these fail silently

That is the reason for a checklist rather than a values review. A values file that says a control is on is not evidence that it is: a network policy peer that matches nothing is zero peers, a Kyverno policy that selects no namespace reports nothing, a mesh mode that disagrees with the cluster renders policies for the wrong traffic shape. Each row below says how to verify, not how to configure.


1. The deployment is what you think it is​

  • global.deploymentMode is multi-shared for the pool and multi-tenant for each tenant. No production deployment uses single-no-auth — it has no login and no per-user authorization
  • Exactly one shared runtime pool per cluster, and one deployment per tenant
  • global.tenant is short, lowercase, ≤ 20 characters, and final — it builds the DNS subdomain, appears in log records, and names the Vault and Temporal accounts
  • The cluster-wide values (networkPolicy, serviceMesh, nonRoutableCidrs, keycloakIpBlocks, vault, ingress.className, storageClassName, appArmor, seccomp) live in one shared file every deployment includes
  • Volume sizes and global.volumeLabels were set before the first deployment — neither can be changed afterwards
  • No opensearch key, no common-assets deployment, no control-plane namespace — all three are gone in 1.3

→ Chart Values Reference · Dev & Prod Profiles

2. Identity and access​

  • Keycloak is reached over its public HTTPS address with a publicly trusted certificate, and global.keycloakHostnameStrict is true
  • global.jwtIssuer.rootPath matches that address exactly — issuers are compared with no wildcards and no suffix matching
  • MFA is enforced for every account holding the admin role, verified in the realm rather than in a values file
  • Brute-force detection is enabled on the realm
  • Session lifetimes have an absolute maximum, and removing a user actually terminates their sessions — tested, not assumed
  • No account holds both admin and editor. The shipped /admins group grants both; a production tenant assigns the roles separately
  • Admin accounts are not used for everyday work or for executing connectivity
  • The tenant realm's client secrets are treated as administrative credentials and have a rotation owner
  • Keycloak user events are enabled on each tenant realm — on by default in newly onboarded realms, but not in one created before that default changed, and realm import does not revisit an existing realm
  • Keycloak events are emitted as JSON with global.audit.successLevel: info, so they reach stdout rather than only the Keycloak database
  • Keycloak admin events are on — OLIVE's onboarding enforces this on every sync, so verify rather than configure

→ Identity and Secrets · Keycloak prerequisites

3. Workload identity and secrets​

  • global.serviceAccounts.mode is split, not legacy — in legacy the Vault policy separation is nominal rather than enforced
  • The Vault KV mount is KV v2 and named secret
  • The Vault bootstrap policy is the documented one, with sudo on sys/auth/jwt-* and the name prefixes intact — not widened to sys/policies/acl/*
  • Exactly one Vault Secrets Operator in the cluster
  • secret/shared-secrets/olive-regcred exists and its token has a known expiry and an owner
  • You hold an inventory of which Kubernetes Secrets are authoritative, mirrored, or unreconciled — in particular that vault-init-keys and vault-root-token are not reconciled by anything
  • Vault's unseal or recovery material is stored outside the cluster, and not beside the snapshots it unlocks
  • Each infrastructure secret has a rotation procedure and an owner

→ Identity and Secrets · Secrets Rotation

4. Network​

  • global.networkPolicy.enabled: true on the pool and every tenant
  • The CNI enforces NetworkPolicy — confirmed with the cluster operator, not inferred. On EKS this means the VPC CNI add-on with enableNetworkPolicy=true, or Calico or Cilium
  • networkPolicy.ingressController matches real Traefik pods: kubectl -n <ns> get pods -l <label> returns something
  • global.keycloakIpBlocks matches how Keycloak is reached: empty for a public address, the VPC CIDR for an internal load balancer. Verified by resolving the host from inside the cluster, not from a workstation
  • If a range wider than a /32 is pinned, it has been checked by hand against the cluster's pod and Service CIDRs — a wide range other than 0.0.0.0/0 is rendered without the carve-out
  • If a public load-balancer address is pinned, that address is monitored for change — a stale value costs every tenant its logins at once
  • global.nonRoutableCidrs is the full wide list, and any legitimate private destination is added back under the chart's extraEgress
  • Traefik is the ingress controller; NGINX Ingress is not supported in 1.3
  • Tested: a pod in one tenant's namespace cannot reach a service in another's

→ Network Policies · Networking

5. Encryption in transit and at rest​

  • global.serviceMesh.mode matches what is actually installed on the cluster — the two meshed modes need opposite policy changes, and a mismatch produces timeouts rather than errors
  • vaultMeshed is true only once the Vault namespace is genuinely enrolled — over-declaring it is tolerated silently and leaves Vault traffic in plaintext
  • The Vault Agent Injector is deployed (vault.injector.enabled: true — it ships off) and a CouchDB pod reports agent-inject-status: injected
  • The mesh exemptions are in place: the vault-agent-injector webhook, Kyverno's webhook callers, the ingress controller, the storage AuthorizationPolicy, and health probes from 169.254.7.127/32
  • Encryption at rest for volumes, backups, object storage and etcd is confirmed with the cluster operator — OLIVE provides none of it
  • TLS terminates at the ingress with a certificate covering all five tenant hostnames

→ Service Mesh and mTLS · Ingress Requirements

6. Supply chain and admission​

  • Pod Security Admission labels are on the namespace, and kubectl get events -n <ns> --field-selector reason=FailedCreate is quiet
  • tenant-setup.imageRegistry.mode is enforce — no network call, no rebuild, evaluates what is already running. The recommended steady state
  • tenant-setup.podPrivileges.mode is enforce, and distinguished from podSecurity (Pod Security Admission) in your own notes
  • tenant-setup.imageVerification.mode is audit — the recommended steady state, because enforce requires failurePolicy: Fail and so makes pod creation depend on Fulcio and Rekor being reachable. If you have chosen enforce, record the reason, confirm the Kyverno pods' egress to both, monitor it, and run at least two Kyverno replicas
  • Verified, not assumed: kubectl get polr -n <namespace> returns rows. A labelled namespace with no rows is enrolled and not being evaluated
  • The cluster operator has confirmed the policies exist and compiled — a policy whose CEL failed to compile is listed and completely inert
  • The Kyverno admission controller runs at least two replicas if any policy uses failurePolicy: Fail
  • global.appArmor and global.seccomp are true, or appArmor is false with the host's lack of AppArmor recorded as the reason

→ Admission Policies · Workload Hardening

7. Logging, monitoring and detection​

  • Logs are collected, and the log_type streams (app, security, audit_ai) are routed separately
  • Retention is set per stream, deliberately, with the reasoning recorded — OLIVE prescribes no period
  • Read access to the security stream is restricted to the people who need it; it carries user names and source IP addresses
  • The developer log stream is not enabled
  • Client IP propagation has been re-verified since the last ingress or load-balancer change — several security events depend on it
  • Alert rules exist on the event codes, not on message text. At minimum: OLIAUTHN4 (cross-tenant access), OLIFETCH1 (blocked internal egress), repeated OLIAUTHZ2/OLIAUTHZ3, OLIORCH1, and OLIMISC1/OLIMISC2 outside a change window
  • A dead-man's-switch alert per tenant on the absence of OLIALIVE1 — this is the one signal that says a tenant has stopped working end to end
  • Keycloak's event log is ingested, since OLIVE emits no code for a successful login — and both capture and emission are confirmed: eventsEnabled on the realm, and global.audit.successLevel: info on the Keycloak chart
  • GLOBAL_DIAGNOSTICS_ENABLED is not set, and the Temporal Web UI is not enabled in the shared runtime pool

→ Security Logging · Log Events and Automated Reactions

8. Backup and recovery​

  • Each tenant's CouchDB and SeaweedFS volumes are backed up as one set, taken as close together as the storage system allows — the content references the files
  • Vault and Keycloak backups are no older than the tenant data they have to unlock
  • Vault raft snapshots are scheduled, and the unseal or KMS material is stored separately from them
  • OLIVE Connectivity Bundle exports are taken in addition to volume backups — they are the only internally consistent copy and the only one restorable into another deployment
  • You know that volume snapshots of a running deployment are crash-consistent: OLIVE has no hook that pauses writes and flushes state before a snapshot, so recovery relies on each data store's own crash handling
  • A restore has been rehearsed, in the documented order — Vault, Keycloak, tenant volumes, deployment, Temporal — and verified by a tenant emitting OLIALIVE1 again
  • RPO and RTO targets are stated and tested against that rehearsal

→ What to Back Up · Storage Requirements

9. Operations and ownership​

  • The split rows in Shared Responsibility have an owner on each side: backup execution and restore verification, alerting tooling and OLIVE-level rules, encryption at rest, secret rotation, and the monitor objects
  • The Handover Checklist has been worked through, and every value either side needs is recorded
  • The deployment is in Git, and the values files are the only source of truth for what is deployed
  • Argo CD has the ignoreDifferences entries — without them selfHeal reverts working state
  • The platform is on the current release; there is no backport commitment for earlier ones
  • Node maintenance is planned as a window per tenant: OLIVE does not yet document what a tenant experiences during a node drain, and creates no PodDisruptionBudget
  • Growing a tenant's volume is understood to be disruptive and needs the cluster operator
  • Your own Impressum and privacy notices are published and wired into both the login page and the application
  • The foundational OLIVE Connectivity Bundle is imported into every tenant, and the five-minute liveness check is running
  • Connectivity authored by Customizers is reviewed before it is enabled — it executes with the platform's privileges and inside the user's browser session

→ Verify the Deployment · Legal Notices and Links


What OLIVE does not provide, and you therefore must​

Worth stating on the same page, because a reviewer will ask and the answer is not "it is covered":

Not provided by OLIVEWhose it is
An observability stack — collection, storage, Grafana, alert routingCluster operator
OLIVE dashboards and alert rules on top of itYou — they depend on knowing what OLIVE's metrics mean
Detection logic — anomaly detection, per-tenant baselines, impossible-travel analysisYou, on top of the event catalogue
Incident response, escalation, containment, statutory reportingYou
Cluster and node hardening, CNI choice, cloud IAM, perimeter firewallsCluster operator
Encryption at rest for volumes, backups, object storage, etcdCluster operator
Backup execution, scheduling, isolation, restore testingCluster operator, verified by you
Malware scanning of uploaded filesYou. File type, extension, MIME and size validation are in place; no virus scanner is applied
AutoscalingNobody — replicas and resources are what you set in the values

Compliance mapping​

OLIVE does not publish a mapping of its controls to a specific framework — CIS, SOC 2, ISO 27001, NIS2 or otherwise. What it publishes is the material such a mapping is built from: the security capabilities split into always-on and switchable, the responsibility matrix, the security event catalogue as an audit trail with actor, tenant, target, outcome and time, and this checklist as the evidence that the switchable controls are actually in effect.

If you need a formal mapping for a certification or a customer questionnaire, raise it with the OLIVE team at olive-support@boc-group.com with the framework and the control list — that is a joint exercise, not something either side writes alone.