Exposing your AKS workloads using External DNS and App Gateway

Introduction

We’re going to follow on from the last article https://adrianhynes.medium.com/exposing-your-aks-workloads-using-external-dns-and-nginx-ingress-controller-434482ea153b where we deployed the kubernetes community addons External DNS and the Nginx Ingress Controller community edition to expose our AKS Cluster workloads over custom URL’s.

In this article we’re going to add App Gateway as an Ingress option to our AKS cluster.

Simple Architecture

In this simple Architecture we’re going to deploy

  1. Public or Private AKS Cluster

The only difference here from the previous article is we are creating an App Gateway in a new subnet in the vnet.

Steps

Ansible Playbook

Apply the Ansible Playbook from here: https://github.com/aido123/ansible/blob/main/azure_ansible_appgateway.yaml subbing in your own values as required

ansible-playbook azure_ansible_appgateway.yaml --extra-vars "resource_group_name=myrsg subscription_id=ABC123 tenant_id=DEF456"

App Gateway Helm Chart

Pull down the latest App Gateway Helm Chart

helm repo add application-gateway-kubernetes-ingress https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/helm pull application-gateway-kubernetes-ingress/ingress-azuretar -xvf ingress-azure-<version>.tgz

Add the env’s AZURE_TENANT_ID and AZURE_SUBSCRIPTION_ID with your own tenant and subscription to the configmap located in ingress-azure/templates/configmap.yaml

...
{{- if eq .Values.armAuth.type "aadPodIdentity"}}
AZURE_CLIENT_ID: "{{ .Values.armAuth.identityClientID }}"
AZURE_TENANT_ID: "ABCD-1234"
AZURE_SUBSCRIPTION_ID: "WXYZ-9876"

...

Remove the files ingress-azure/templates/aadpodidbinding.yaml and ingress-azure/templates/aadpodidentity.yaml. These are use for configuring pod identity. We’re just attaching our user managed identity to the vmscaleset for demo purposes

Apply the helm chart plugging in your own config

helm install ingress-azure \
ingress-azure \
--namespace default \
--debug \
--set appgw.resourceGroup=<resource group name> \
--set appgw.environment=AZUREPUBLICCLOUD \
--set appgw.subscriptionId=<WXYZ-9876> \
--set appgw.shared=false \
--set appgw.name=myApplicationGateway \
--set appgw.usePrivateIP=true \
--set armAuth.type=aadPodIdentity \
--set armAuth.identityClientID=<User Managed Identity Client ID> \
--set rbac.enabled=true \
--set verbosityLevel=3 \
--set aksClusterConfiguration.apiServerAddress=<guid>.hcp.<location>.azmk8s.io

Nginx Ingress Controller

Apply the Nginx Ingress Controller as per my other Article https://adrianhynes.medium.com/exposing-your-aks-workloads-using-external-dns-and-nginx-ingress-controller-434482ea153b

Externdal DNS

Apply External DNS as per my other article https://adrianhynes.medium.com/exposing-your-aks-workloads-using-external-dns-and-nginx-ingress-controller-434482ea153b

App Gateway Ingress Example

Apply the App Gateway Ingress example as per below. Notice the different ingress annotations compared to our previous Nginx example.

kubectl apply -f example-app-gateway.yaml#example-app-gateway.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: aks-helloworld-two
spec:
replicas: 1
selector:
matchLabels:
app: aks-helloworld-two
template:
metadata:
labels:
app: aks-helloworld-two
spec:
containers:
- name: aks-helloworld-two
image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
ports:
- containerPort: 80
env:
- name: TITLE
value: "AKS Ingress Demo"
---
apiVersion: v1
kind: Service
metadata:
name: aks-helloworld-two
spec:
ports:
- protocol: TCP
port: 80
targetPort: 80
selector:
app: aks-helloworld-two
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
rules:
- host: aido.hynes.pri
http:
paths:
- backend:
serviceName: aks-helloworld-two
servicePort: 80
path: /

Nginx Ingress Example

We’ll also apply the following Nginx Ingress example to show them side by side.

kubectl apply -f example-nginx.yaml#example-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: aks-helloworld-two
spec:
replicas: 1
selector:
matchLabels:
app: aks-helloworld-two
template:
metadata:
labels:
app: aks-helloworld-two
spec:
containers:
- name: aks-helloworld-two
image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
ports:
- containerPort: 80
env:
- name: TITLE
value: "AKS Ingress Demo"
---
apiVersion: v1
kind: Service
metadata:
name: aks-helloworld-three
spec:
type: ClusterIP
ports:
- port: 80
selector:
app: aks-helloworld-two
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress-1
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: bido.hynes.pri
http:
paths:
- backend:
serviceName: aks-helloworld-three
servicePort: 80
path: /(.*)

Test

RDP into the Custom VM and open a terminal. Perform an nslookup of both aido.hynes.pri and bido.hynes.pri.

aido.hynes.pri will resolve to App Gateway’s IP Address

bido.hynes.pri will resolve to the Standard Load Balancer IP Address (created via Nginx Ingress Controller Loadbalancer Service)

Type http://aido.hynes.pri into a browser on the Widnows VM and you will get the following

Type http://bido.hynes.pri into a browser and you will get the following

Conclusion

I hope you found this article useful in setting up an Application Gateway (Via Helm Charts) for your AKS Cluster.

Note, it’s possible to setup App Gateway in your AKS clusters via the az aks cli, which I have not looked at here.

Cloud Platform Architect. Opinions and articles on medium are my own.