Installation Internal Nginx Ingress for a Private AKS Cluster

4 min readJun 18, 2022

Hi everyone, in this article, I will talk about how to install Internal Nginx Ingress Controller for a Private AKS Cluster using terraform and helm.

Before continuing, you can take a look at my article about ingress controllers.

Let’s get started!

Prerequisites

Azure Subscription
Private AKS Cluster
Kubernetes Admin Service Account
Azure CLI
Terraform

Kubernetes Service Account

Role Based Access Control is comprised of four layers:

ClusterRole - permissions assigned to a role that apply to an entire cluster
ClusterRoleBinding - binding a ClusterRole to a specific account
Role - permissions assigned to a role that apply to a specific namespace
RoleBinding - binding a Role to a specific account

In order for RBAC to be applied to an ingress-nginx-controller, that controller should be assigned to a ServiceAccount. That ServiceAccount should be bound to the Roles and ClusterRoles defined for the ingress-nginx-controller.

Create Service Account with Terraform

Terraform needs to Kubernetes and Azure terraform provider for RBAC creation.

Configurations in the serviceaccountmain.tf file are parametric.

You can access the variables and provider versions from this github link.

Internal Nginx-Ingress Controller

You can access the service account ouputs to be used for installations from local state files or remote state files. Below are examples of both configurations. If you want to access from remote state, you need to open the backend configurations in the serviceaccountmain.tf file.

Instead of the configs used in the helm charts, it should be configured as follows to be able to use its own values.The file to be referenced here should be values.yaml in the relevant helm chart. The required parameters in the values.yaml file are given our own values in accordance with their usage styles.

By default, an NGINX ingress controller is created with a dynamic public IP address assignment. A common configuration requirement is to use an internal, private network and IP address. This approach allows you to restrict access to your services to internal users, with no external access.

For internal ingress, the service.beta.kubernetes.io/azure-load-balancer-internalannotation should be used.

Here, the ingress will be created in the system node pool. If there is more than one subnet, it is necessary to get an ip from the subnet used by the node pool where ingress will be created for the internal ingress load balancer ip. Otherwise, it will get an error because it cannot get a rope from the appropriate place and ingress will not work. The service.beta.kubernetes.io/azure-load-balancer-internal-subnetannotation must be added for this configuration.

Tips: LoadBalancerIP can be left empty. If left blank, an unused ip from the relevant subnet will be assigned. If it is not desired to be left blank, one of the unused threads in the relevant subnet should be checked and given to the loadbalancer ip parameter.

Provide your own internal IP address for use with the ingress controller. Make sure that this IP address isn’t already in use within your virtual network. Also, if you’re using an existing virtual network and subnet, you must configure your AKS cluster with the correct permissions to manage the virtual network and subnet. For more information, see Use kubenet networking with your own IP address ranges in Azure Kubernetes Service (AKS) or Configure Azure CNI networking in Azure Kubernetes Service (AKS).

You can access the variables and provider versions from this github link.

Terraform Execute

Don’t forget to review this document to login to Azure from Terraform.

az login
az account set --subscription "<subscription_id_or_name>#If a new provider is added to the module, it should be run by adding -upgrade.
terraform init#Optional. It can be used if it is desired to look at the changes before installation.
terraform plan 
#If you want changes to be applied without requiring an additional confirmation, it can be run by adding -auto-approve.
terraform apply#If you want to undo the changes made, this command can be run. But when this is run all processes are destroyed. At the same time, auto approve can be used as in apple.
terraform destroy