Create a MySQL Cluster With mTLS on KubeBlocks

This guide explains how to configure a MySQL cluster with mutual TLS (mTLS) encryption using KubeBlocks. mTLS ensures both the server and client authenticate each other during a connection, providing enhanced security for your database infrastructure. This guide covers certificate generation, cluster deployment, user configuration for mTLS, and secure connection verification.

What is mTLS?

Mutual TLS (mTLS) is an enhanced security protocol that ensures both the server and the client authenticate each other during a connection. Unlike traditional TLS, where only the client verifies the server's identity, mTLS adds an extra layer of security by requiring both sides to present valid certificates issued by a trusted Certificate Authority (CA).

Prerequisites

Before proceeding, ensure the following:

• Environment Setup:
  • A Kubernetes cluster is up and running.
  • The kubectl CLI tool is configured to communicate with your cluster.
  • KubeBlocks CLI and KubeBlocks Operator are installed. Follow the installation instructions here.
• Namespace Preparation: To keep resources isolated, create a dedicated namespace for this tutorial:

kubectl create ns demo
namespace/demo created

Generating Certificates

To enable TLS encryption, you will need to provide a Certificate Authority (CA), a server certificate, and a private key. Follow these steps to generate these using OpenSSL:

1. Generate the Root Certificate (CA)

# Create the CA private key (password optional)
openssl genrsa -aes256 -out ca-key.pem 4096

# Generate a self-signed root certificate (valid for 10 years)
openssl req -x509 -new -nodes -key ca-key.pem -sha256 -days 3650 -out ca.pem
# Enter the required information (e.g., Common Name can be "MySQL Root CA")

2. Generate the Server Certificate & Key

# Generate the server private key
openssl genrsa -out server-key.pem 4096

# Create a Certificate Signing Request (CSR)
openssl req -new -key server-key.pem -out server-req.pem
# Enter server identification details, such as:
# Common Name (CN) = Server domain name or IP (must match the MySQL server address!)

# Sign the server certificate with the CA (valid for 10 years)
openssl x509 -req -in server-req.pem -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -days 3650 -sha256

3. Generate the Client Certificate & Key

# Generate the client private key
openssl genrsa -out client-key.pem 4096

# Create a Certificate Signing Request (CSR)
openssl req -new -key client-key.pem -out client-req.pem
# Enter client identification details, such as:
# Common Name (CN) = Client username (e.g., "mysql_client_1")

# Sign the client certificate with the CA (valid for 1 year)
openssl x509 -req -in client-req.pem -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -days 365 -sha256

4. Verify the Certificates Verify that the server certificate is valid and signed by the CA:

# Verify the server certificate
openssl verify -CAfile ca.pem server-cert.pem

Expected Output:

server-cert.pem: OK

# Verify the client certificate
openssl verify -CAfile ca.pem client-cert.pem

Expected Output:

client-cert.pem: OK

Create Kubernetes Secrets

Store the generated certificates and keys in a Kubernetes Secret to make them accessible to your MySQL cluster.

kubectl create secret generic mysql-tls-secret \
  --namespace=demo \
  --from-file=ca.crt=ca.pem \
  --from-file=tls.crt=server-cert.pem \
  --from-file=tls.key=server-key.pem \
  --type=kubernetes.io/tls

This secret contains the CA, server certificate, and private key required to enable mTLS on the MySQL cluster.

Deploying the MySQL Semi-Synchronous Cluster

KubeBlocks uses a declarative approach for managing MySQL clusters. Below is an example configuration for deploying a MySQL cluster with 2 nodes (1 primary, 1 replicas) in semi-synchronous mode with user-provided TLS certificates:

kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: example-mysql-cluster
  namespace: demo
spec:
  clusterDef: mysql
  topology: semisync
  terminationPolicy: Delete
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.35
      replicas: 2
      tls: true
      issuer:
        name: UserProvided
        secretRef:
          name: mysql-tls-secret
          namespace: demo
          ca: ca.crt
          cert: tls.crt
          key: tls.key
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
EOF

Verifying the Deployment

Check cluster status until it reaches Running state:

kubectl get cluster -n demo

Expected Output:

NAME                    CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Running   11m

Configure an mTLS User in MySQL

Retrieve MySQL Root Credentials

KubeBlocks automatically creates a secret containing the MySQL root credentials. Retrieve the credentials with the following commands:

1. Retrieve the root username:

kubectl get secrets -n demo example-mysql-cluster-mysql-account-root -o jsonpath='{.data.username}' | base64 -d

Expected Output:

root

2. Retrieve the root password:

kubectl get secrets -n demo example-mysql-cluster-mysql-account-root -o jsonpath='{.data.password}' | base64 -d

Expected Output:

D0o5P43S8G

Connect Using the Root Account

kubectl exec -it -n demo example-mysql-cluster-mysql-0 -c mysql -- mysql -h example-mysql-cluster-mysql.demo.svc.cluster.local -uroot -pD0o5P43S8G

Create an mTLS User

Run the following commands in the MySQL shell to create a user that requires client certificate authentication:

mysql> CREATE USER 'mtls_user'@'%' IDENTIFIED BY 'kni676X2W1' REQUIRE X509;
Query OK, 0 rows affected (0.01 sec)

mysql> GRANT ALL PRIVILEGES ON *.* TO 'mtls_user'@'%';
Query OK, 0 rows affected (0.01 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)

Connect to MySQL Cluster with mTLS

use the kubectl port-forward command to map port 3306 of the primary replica of the MySQL Cluster to port 3306 on your local machine:

kubectl port-forward svc/mysql-cluster-mysql 3306:3306 -n default
Forwarding from 127.0.0.1:3306 -> 3306
Forwarding from [::1]:3306 -> 3306

Then, open another shell and use the mysql command-line tool to connect to the local port 3306.

If connecting without client certificates, you will see an error:

mysql -h 127.0.0.1 -umtls_user -pkni676X2W1 --ssl-mode=REQUIRED

Expected Output:

ERROR 1045 (28000): Access denied for user 'mtls_user'@'127.0.0.1' (using password: YES)

To connect successfully, provide the client certificate and key:

mysql -h 127.0.0.1 -umtls_user -pkni676X2W1 --ssl-mode=REQUIRED --ssl-ca=/path/to/ca.pem --ssl-cert=/path/to/client-cert.pem --ssl-key=/path/to/client-key.pem

Verify TLS connection status in MySQL shell:

mysql> STATUS;
--------------

SSL:			Cipher in use is TLS_AES_256_GCM_SHA384

Cleanup

Remove all resources created during this tutorial:

kubectl delete cluster example-mysql-cluster -n demo
kubectl delete secret mysql-tls-secret -n demo
kubectl delete ns demo

Summary

In this guide, you learned how to:

1. Generate a self-signed CA, server, and client certificates using OpenSSL.
2. Deploy a MySQL cluster with mTLS enabled on KubeBlocks.
3. Configure an mTLS user and verify secure connections.

mTLS provides an additional layer of trust and security by ensuring both client and server authentication. By following this guide, you can securely deploy and manage MySQL clusters with mTLS on Kubernetes using KubeBlocks.