diff --git a/docs/architecture/concepts/external-key-management.md b/docs/architecture/concepts/external-key-management.md new file mode 100644 index 00000000..d81c78d4 --- /dev/null +++ b/docs/architecture/concepts/external-key-management.md @@ -0,0 +1,39 @@ +--- +title: "External Key Management" +description: "An external KMS keeps the keys that protect simplyblock volume encryption material outside the storage cluster, enabling separation of duty, rotation, and audit." +weight: 30220 +--- + +Volume encryption protects data at rest by ciphering every block written to a logical volume. The cipher itself +needs a key; the question of *where that key lives and who controls it* is the responsibility of the key management +layer. + +By default, simplyblock manages encryption keys internally. For environments where the team operating the storage +cluster should not also be in possession of the long-lived key material — common in regulated environments and any +deployment that separates storage and security duties — the key-encryption keys can be offloaded to an external Key +Management Service (KMS). Simplyblock supports [Hashicorp Vault](https://www.vaultproject.io/){:target="_blank" rel="noopener"} +and [Openbao](https://openbao.org/){:target="_blank" rel="noopener"} as KMS backends. + +## Two-Layer Key Model + +When an external KMS is configured, simplyblock applies a two-layer model: + +- **Data Encryption Keys (DEKs)** are generated per volume and used to encrypt the blocks of that volume. They are + short-lived in cluster memory and never stored in plaintext at rest. +- **Key Encryption Keys (KEKs)** live in the KMS. The cluster asks the KMS to wrap each DEK on creation and to unwrap + it when the volume is brought online. The KEKs never leave the KMS. + +Compromise of the cluster does not compromise the KEKs; compromise of the KMS does not directly expose any specific +volume's data, because DEKs are wrapped against the KEK rather than stored alongside it. + +## Authentication and Trust + +The KMS authenticates simplyblock components using a client certificate issued by the +`simplyblock-certificate-authority-issuer` ClusterIssuer, which the operator creates as part of its mTLS setup. +Because the KMS depends on this CA, [control-plane mTLS](../../deployments/kubernetes/security.md#transport-layer-security-mtls) +must be configured before an external KMS can be wired up. + +Operationally, this means the KMS team and the storage team share only the CA bundle and an agreed-upon DNS name for +the simplyblock client — no static passwords or long-lived tokens are exchanged. + +For the setup steps, see [Securing the Control Plane: External KMS](../../deployments/kubernetes/security.md#external-key-management-kms). diff --git a/docs/deployments/kubernetes/k8s-control-plane.md b/docs/deployments/kubernetes/k8s-control-plane.md index 772f2eb9..7a842287 100644 --- a/docs/deployments/kubernetes/k8s-control-plane.md +++ b/docs/deployments/kubernetes/k8s-control-plane.md @@ -32,17 +32,13 @@ helm upgrade --install simplyblock -n simplyblock simplyblock/spdk-csi \ ``` !!! important "TLS Encryption" - {{experimental}} + {{ experimental }} - Simplyblock has just added support for TLS encryption for all internal communication. At the moment, it's an - experimental feature only available when installed into OpenShift clusters. + All internal control-plane traffic can be encrypted with TLS. On OpenShift, the cluster's built-in certificate + manager is used out of the box. Mutual TLS (mTLS), where components additionally authenticate each other with + client certificates, is supported only with cert-manager — on any Kubernetes distribution, including OpenShift. - It generally is a good idea to install the operator with TLS support enabled on OpenShift to ensure that all - internal communication is encrypted and secure. - - To enable TLS, add the `--set tls.enabled=true` flag to the `helm install` command. - - In the future, the support will be extended to further Kubernetes distributions by enabling Cert-Manager support. + See [Securing the Control Plane](security.md#transport-layer-security-mtls) for configuration. After installation, verify the operator is running: diff --git a/docs/deployments/kubernetes/k8s-storage-plane.md b/docs/deployments/kubernetes/k8s-storage-plane.md index c76367fc..d05a7b32 100644 --- a/docs/deployments/kubernetes/k8s-storage-plane.md +++ b/docs/deployments/kubernetes/k8s-storage-plane.md @@ -100,6 +100,12 @@ been registered, it has no storage nodes yet. Those are added in the next step. as NVMe-oF transport security, backup configuration, capacity thresholds, and more, are available at [Cluster Deployment Options](../cluster-deployment-options.md). +!!! tip "External KMS" + If volumes in this cluster should offload their encryption keys to an external KMS, set + `spec.hashicorpVaultSettings.base_url` on the `StorageCluster` now. The setting can also be added later, but + configuring it up front means encrypted volumes use the external KMS from day one. See + [Securing the Control Plane: External KMS](security.md#external-key-management-kms). + ## Add Storage Nodes Now, Kubernetes worker nodes will be transformed into simplyblock storage nodes. To initiate the process, a diff --git a/docs/deployments/kubernetes/security.md b/docs/deployments/kubernetes/security.md new file mode 100644 index 00000000..63688f81 --- /dev/null +++ b/docs/deployments/kubernetes/security.md @@ -0,0 +1,211 @@ +--- +title: "Securing the Control Plane" +description: "Configure mTLS for simplyblock control plane communication and offload at-rest encryption keys to an external KMS (Hashicorp Vault or Openbao)." +weight: 30050 +--- + +This page covers two security features for simplyblock on Kubernetes: transport-layer encryption and mutual +authentication for the control plane (mTLS), and offloading volume encryption keys to an external Key Management +Service (KMS). + +mTLS must be configured before an external KMS can be wired up: the KMS authenticates simplyblock components using a +certificate issued by the operator-managed certificate authority, which is only provisioned when mTLS is active. + +## Transport Layer Security (mTLS) + +{{ experimental }} + +Internal control-plane traffic between the webappapi, the operator, and the storage-node handlers can be encrypted +with TLS. When mutual TLS is additionally enabled, every component must present a valid client certificate, which +means components authenticate each other rather than relying on network position alone. + +**mTLS is only supported with the cert-manager provider.** On OpenShift, the cluster's built-in certificate manager +provides one-way TLS (server certificates) but does not issue the client certificates required for mutual +authentication; running mTLS on OpenShift therefore also requires installing +[cert-manager](https://cert-manager.io/){:target="_blank" rel="noopener"} and switching the provider over. + +### Prerequisites + +- cert-manager installed in the cluster. +- A `ClusterIssuer` (or namespaced `Issuer`) that cert-manager can use to mint certificates. Most installations point + this at an internal corporate CA or at the cluster-local self-signed issuer; any issuer that simplyblock components + can trust is acceptable. + +### Enabling mTLS + +mTLS is configured at Helm install time by setting four values on the operator chart: + +```yaml title="Helm values for mTLS" +tls: + enabled: true + mutual_enabled: true + provider: cert-manager + cert-manager: + issuer: my-cluster-issuer +``` + +Apply the values during the operator install (see [Install Simplyblock Operator](k8s-control-plane.md)): + +```bash title="Install the operator with mTLS" +helm upgrade --install simplyblock -n simplyblock simplyblock/spdk-csi \ + --create-namespace \ + --set operator.enabled=true \ + --set tls.enabled=true \ + --set tls.mutual_enabled=true \ + --set tls.provider=cert-manager \ + --set tls.cert-manager.issuer=my-cluster-issuer +``` + +Replace `my-cluster-issuer` with the name of the `ClusterIssuer` the operator should use to obtain its certificates. + +### What the Operator Provisions + +When mTLS is enabled, the operator creates a dedicated `ClusterIssuer` named +`simplyblock-certificate-authority-issuer` and issues all internal component certificates from it. The same issuer can +be used to mint certificates for other workloads that need to talk to simplyblock — most notably an external KMS, as +described in the next section. + +!!! note "OpenShift" + On OpenShift, setting `tls.enabled=true` with the default `tls.provider=openshift` activates one-way TLS using + OpenShift-managed certificates. Mutual TLS is **not** available with this provider — `tls.mutual_enabled=true` + requires `tls.provider=cert-manager` regardless of the underlying Kubernetes distribution. + +## External Key Management (KMS) + +{{ experimental }} + +By default, simplyblock manages volume encryption keys internally. For environments that require stricter key handling +— separation of duty between storage administrators and key custodians, regular rotation, or audit trails — the +cluster can be configured to keep the key-encryption material in an external KMS. Both +[Hashicorp Vault](https://www.vaultproject.io/){:target="_blank" rel="noopener"} and +[Openbao](https://openbao.org/){:target="_blank" rel="noopener"} are supported; the configuration is identical for +either. + +### Prerequisites + +- [mTLS configured](#transport-layer-security-mtls) — required, because the vault is authenticated to the cluster via + a certificate issued by the operator's `simplyblock-certificate-authority-issuer`. +- A Vault or Openbao instance reachable from the simplyblock namespace. The instance must be initialized and unsealed + before configuring authentication. + +### Step 1 — Issue a TLS certificate for the vault + +Create a cert-manager `Certificate` resource that uses the operator-managed issuer. The resulting Secret holds the +TLS material the vault serves to clients, and is trusted by the simplyblock components because it chains to the same +CA. + +```yaml title="vault-tls.yaml" +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: vault-tls + namespace: vault +spec: + secretName: vault-tls + issuerRef: + name: simplyblock-certificate-authority-issuer + kind: ClusterIssuer + commonName: vault + dnsNames: + - vault + - vault.vault + - vault.vault.svc + - vault.vault.svc.cluster.local +``` + +Mount the resulting `vault-tls` Secret into the vault deployment as its serving certificate. Mount the issuer's CA +bundle (typically `ca.crt`) at a path the vault can read — the example below assumes `/vault/tls/ca.crt` for Vault +and `/bao/tls/ca.crt` for Openbao. + +### Step 2 — Deploy the vault + +Install Vault or Openbao using the upstream Helm chart and expose it inside the cluster. For the rest of this guide +the in-cluster Service is assumed to be `vault.vault:8200`; adjust the URL to match the actual deployment. + +### Step 3 — Configure auth, policy, and secret engines + +Configure the vault with a policy that grants simplyblock access to the `transit` and `kv` backends, enable +certificate authentication bound to the simplyblock CA, and enable the required secret engines. The script below +works for both Vault (`vault`) and Openbao (`bao`) — assign the appropriate CLI to `$CLI`. + +```bash title="Configure the vault for simplyblock" +CLI=vault # or: CLI=bao + +# Policy granting access to the transit and kv backends +$CLI policy write webappapi-policy - <` | + +For details, see [Securing the Control Plane](../../deployments/kubernetes/security.md). + ## Storage Node Parameters | Parameter | Description | Default | diff --git a/docs/reference/operator.md b/docs/reference/operator.md index 5ed9e2f1..b9451a6e 100644 --- a/docs/reference/operator.md +++ b/docs/reference/operator.md @@ -78,6 +78,7 @@ spec: | `warningThreshold.provisionedCapacity` | int | Provisioned capacity warning threshold (percent). | | `criticalThreshold.provisionedCapacity` | int | Provisioned capacity critical threshold (percent). | | `action` | string | Lifecycle action: `activate` or `expand`. | +| `hashicorpVaultSettings.base_url` | string | Base URL of an external Hashicorp Vault or Openbao instance used to manage volume encryption keys (e.g., `https://vault.vault:8200/`). See [Securing the Control Plane: External KMS](../deployments/kubernetes/security.md#external-key-management-kms). | | `backup.credentialsSecretRef.name` | string | Name of the Secret (in the same namespace) holding `access_key_id` and `secret_access_key`. **Required when `backup` is set**. | | `backup.localEndpoint` | string | S3-compatible endpoint URL for backup storage. | | `backup.snapshotBackups` | bool | Enable snapshot-based backups. | diff --git a/docs/release-notes/26-2-pre.md b/docs/release-notes/26-2-pre.md index 625d89ae..ebe5a1c3 100644 --- a/docs/release-notes/26-2-pre.md +++ b/docs/release-notes/26-2-pre.md @@ -15,6 +15,8 @@ Simplyblock is happy to release the general availability release of Simplyblock - Storage Plane: Added support for FTT2 (failure to tolerate) which enables up to two failover paths. When enabled, simplyblock creates three communication paths between the initiators and the NVMe targets (primary, secondary, and tertiary) with identical subsystems able to process IO. This allows the loss or maintenance of any two nodes in the cluster, regardless of the combination, at a point in time. - Storage Plane: Added support for full multi-pathing within the storage cluster and between initiators and NVMe targets. This means that clients can have four (FTT=1) or six (FTT=2) connections via separate VLANs using separate networking paths. Each endpoint-combination uses two separate connections through the available network paths. It has advantages over simple bonding and multi-chassis link aggregation groups (MLAG). - Kubernetes: Added support for OpenShift-managed TLS certificates for communication within the control plane and cluster endpoints. +- Kubernetes: Added support for mutual TLS (mTLS) on the control plane, using cert-manager-issued certificates. +- Kubernetes: Added support for offloading volume encryption keys to an external KMS (Hashicorp Vault or Openbao). - Kubernetes: Added support for namespaced volumes, which auto-create subsystems and namespaces within those subsystems in a pre-defined ratio (e.g., 8 or 16 namespaces per subsystem) in Kubernetes-based environments. The ratio can be configured via the operator. - Kubernetes: Added a new, experimental Kubernetes operator to simplify the deployment of Simplyblock control planes, storage planes, and the CSI driver in Kubernetes-based environments, such as OpenShift and Talos. - General: All container images are now based on RHEL 10 base images and use Python 3.12. This solves multiple open CVEs and additional vulnerabilities.