aws-nodegroup

Creates an additional nodegroup for the primary EKS cluster.

This module creates an additional nodegroup for the primary EKS cluster. Note that the aws-eks module creates a default nodegroup so this should only be used when you want one more.

IAM Permissions given to the Nodegroup

Along with the nodegroup, Opta creates a AWS IAM role that is attached to each EC2 in the pool and handles all of the machine’s (and Kubernetes actions done by the kubelet in the machine like, for example, downloading an ecr image) IAM permissions. Opta gives this service account the following policies:

  • AmazonEKSWorkerNodePolicy
  • AmazonEKS_CNI_Policy
  • AmazonEC2ContainerRegistryReadOnly

The first 2 are needed for the EC2 to function as a k8s node properly and the last ensures we can read ecr images from this account. If you need more permissions, feel free to attach extra policies to the iam role via the awscli or AWS web ui console– assuming you do not destroy/modify the existing policies attached there should be no problem.

THIS IAM ROLE IS NOT THE ONE USED BY YOUR CONTAINERS RUNNING IN THE CLUSTER– Opta handles creating appropriate IAM roles for each K8s service, but for any non-opta managed workloads in the cluster, please refer to this AWS documentation (the OIDC provider is created by Opta).

Taints

Opta gives you the option of adding taints to the nodes created in this nodepool. The official documentation gives an excellent detailed summary, but in short one can use taints to stop workloads from running in said nodes unless they have a matching toleration. Simply provide a list of such taints as inputs like so:

  - type: aws-nodegroup
    name: nodepool1
    min_nodes: 1
    max_nodes: 3
    taints:
      - key: instancetype
        value: memoryoptimized
        effect: "NoExecute"
      - key: team
        value: booking
        # Tolerates for default effect of NoSchedule
      - key: highpriority
        # Tolerates for default value of opta

For most cases, simply specifying the key should do.

Fields

Name Description Default Required
labels labels for the kubernetes nodes {} False
max_nodes Max number of nodes to allow via autoscaling 15 False
min_nodes Min number of nodes to allow via autoscaling 3 False
node_disk_size The size of disk to give the nodes' ec2s in GB. 20 False
node_instance_type The ec2 instance type for the nodes. t3.medium False
spot_instances A boolean specifying whether to use spot instances for the default nodegroup or not. The spot instances will be configured to have the max price equal to the on-demand price (so no danger of overcharging). WARNING: By using spot instances you must accept the real risk of frequent abrupt node terminations and possibly (although extremely rarely) even full blackouts (all nodes die). The former is a small risk as containers of Opta services will be automatically restarted on surviving nodes. So just make sure to specify a minimum of more than 1 containers – Opta by default attempts to spread them out amongst many nodes. The former is a graver concern which can be addressed by having multiple node groups of different instance types (see aws nodegroup module) and ideally at least one non-spot. False False
taints Taints to add to the nodes in this nodegroup. [] False
use_gpu Should we expect and use the gpus present in the ec2? Note: This input would be deprecated in the coming releases. Please switch to using ami_type. Usage: If using, use_gpu: false, just remove it. If using use_gpu: true replace it with ami_type: AL2_x86_64_GPU False False
ami_type The AMI type to use for the nodes. For more information about this, please visit here Note: Currently, “CUSTOM” ami type is not supported. AL2_x86_64 False

Last modified May 23, 2022 : Docs for new release (#191) (ba6432d)