Azure Kubernetes Service (AKS) - A greenfield implementation
Azure Kubernetes Service (AKS) is a fully managed Kubernetes offering from Azure, simplifying the deployment and management of Kubernetes clusters. A greenfield implementation of AKS involves building the cluster from scratch and configuring authentication, authorization, networking, load balancing, and storage. This guide will walk through setting up an AKS cluster, along with recommendations for service accounts, authentication, authorization, networking, and storage.
1. Initial Setup: Create an AKS Cluster
Step 1: Prerequisites
- Azure CLI installed (v2.0+): [Install CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli).
- An active Azure subscription.
- Create a resource group:
```bash
az group create --name myResourceGroup --location eastus
```
Step 2: Create the AKS Cluster
The following command will create a basic AKS cluster. Use the --enable-managed-identity flag to enable the use of managed identities, which is more secure than service principals.
```bash
az aks create \
--resource-group myResourceGroup \
--name myAKSCluster \
--node-count 3 \
--enable-managed-identity \
--generate-ssh-keys
```
This will:
- Deploy the AKS control plane and nodes.
- Enable managed identities for better security.
Recommendation: Start with at least 3 nodes for high availability. Adjust node sizes based on workload needs (e.g., Standard_DS2_v2).
2. Authentication and Authorization
Step 3: Integrating Azure Active Directory (AAD) for Authentication
Integrating AKS with Azure Active Directory (AAD) allows you to authenticate Kubernetes users using their AAD credentials, improving security.
```bash
az aks create \
--resource-group myResourceGroup \
--name myAKSCluster \
--enable-aad \
--aad-admin-group-object-ids <AAD-group-object-id> \
--enable-managed-identity
```
This command integrates AKS with AAD, allowing you to map AAD groups to Kubernetes RBAC (Role-Based Access Control) roles. Replace <AAD-group-object-id> with the Object ID of the AAD group for admins.
Recommendation: Map different AAD groups to Kubernetes roles, following the principle of least privilege.
Step 4: Set Up Role-Based Access Control (RBAC)
AKS supports Kubernetes RBAC to restrict permissions for users. Define Roles and RoleBindings:
- Role: Defines permissions (e.g., read, write).
- RoleBinding: Assigns Roles to users or groups.
Create a ClusterRole for read-only access:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: read-only
rules:
- apiGroups: [""]
resources: ["pods", "services"]
verbs: ["get", "list"]
```
Create a ClusterRoleBinding to assign the role to an AAD user or group:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: read-only-binding
subjects:
- kind: User
name: <aad-user-name> # Replace with AAD user
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: read-only
apiGroup: rbac.authorization.k8s.io
```
Recommendation: Use ClusterRoles for permissions across the entire cluster and Roles for namespace-specific permissions. Create separate roles for developers, operators, and administrators.
3. Networking
Step 5: Choose Networking Model: Azure CNI vs Kubenet
- Azure CNI: Pods get IPs from the same VNET as the nodes.
- Kubenet: Pods get IPs from an internal network, and nodes get IPs from the VNET.
```bash
az aks create \
--resource-group myResourceGroup \
--name myAKSCluster \
--network-plugin azure \
--vnet-subnet-id <subnet-id>
```
Recommendation: Use Azure CNI for production workloads requiring advanced networking like VNET peering and private endpoints. Kubenet is suited for smaller or less complex deployments.
Step 6: Configure Network Policies
Network policies allow control over which Pods can communicate with each other. Use Azure's native network policies or Calico for more granular control.
Example of a network policy allowing only communication from Pods with the label app=frontend to Pods labeled app=backend:
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-frontend
namespace: default
spec:
podSelector:
matchLabels:
app: backend
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
```
Recommendation: Define network policies to secure internal traffic and isolate sensitive components (e.g., databases).
4. Load Balancers
Step 7: Configuring Load Balancers
AKS provides native support for Azure Load Balancer to expose services externally.
Example of a Service using an external load balancer:
```yaml
apiVersion: v1
kind: Service
metadata:
name: my-loadbalancer-service
spec:
type: LoadBalancer
selector:
app: my-app
ports:
- protocol: TCP
port: 80
targetPort: 8080
```
Recommendation: Use LoadBalancer services for external access and ClusterIP for internal services. For more advanced traffic management, consider Ingress Controllers (e.g., NGINX Ingress).
Step 8: Ingress Controllers for Advanced Load Balancing
Ingress Controllers offer Layer 7 (HTTP/HTTPS) load balancing and routing. Set up an NGINX Ingress Controller for more advanced traffic management:
```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml
```
Define an Ingress resource for routing:
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
spec:
rules:
- host: myapp.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-app
port:
number: 8080
```
5. Storage
Step 9: Persistent Storage with Persistent Volumes (PV) and Persistent Volume Claims (PVC)
Persistent Volumes (PVs) in AKS can be backed by Azure Disk or Azure File. Example of creating a Persistent Volume (PV) and a Persistent Volume Claim (PVC):
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: azurefile
```
Recommendation: Use Azure Disk for single Pod write access (ReadWriteOnce) and Azure File for multiple Pod write access (ReadWriteMany). Make sure to define storage classes for dynamic provisioning.
6. Service Accounts, Secrets, and ConfigMaps
Step 10: Managing Service Accounts and Secrets
Use Service Accounts to provide Pods with limited access to the Kubernetes API. Secrets are used to store sensitive data (e.g., credentials, tokens).
Create a Service Account:
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-service-account
```
Assign it to a Pod:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
serviceAccountName: my-service-account
containers:
- name: my-container
image: nginx
```
Create a Secret:
```yaml
kubectl create secret generic my-secret --from-literal=username=my-app --from-literal=password=my-password
```
Use the Secret in a Pod:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: my-secret-pod
spec:
containers:
- name: my-container
image: nginx
env:
- name: USERNAME
valueFrom:
secretKeyRef:
name: my-secret
key: username
- name: PASSWORD
valueFrom:
secretKeyRef:
name: my-secret
key: password
```
Recommendation: Rotate service account tokens and secrets regularly, and use Azure Key Vault to manage sensitive data securely.
Conclusion: Best Practices
1. Authentication & Authorization: Integrate with Azure AD and use RBAC for secure access control.
2. Networking: Use Azure CNI for production clusters and configure network policies for traffic isolation.
3. Load Balancers: Use Ingress Controllers for advanced routing, especially for multi-service deployments.
4. Storage: Choose the appropriate storage backend based on your application’s needs (Azure Disk vs Azure File).
5. Service Accounts and Secrets: Limit the access of service accounts and use Secrets and ConfigMaps for sensitive configuration.
By following these steps, you can set up a robust AKS cluster with proper authentication, authorization, networking, and storage strategies for a greenfield implementation.