gcp-nodepool

Creates an additional nodepool for the primary GKE cluster.

This module creates an additional nodepool for the primary GKE cluster. Note that the gcp-gke module creates a default nodepool so this should only be used when you want one more.

IAM Permissions given to the Nodepool

Along with the nodepool, Opta creates a GCP IAM service account that is attached to each VM in the pool and handles all of the machine’s (and Kubernetes actions done by the kubelet in the machine like, for example, downloading a gcr image) IAM permissions. Opta gives this service account the following roles:

  • logging.logWriter
  • monitoring.metricWriter
  • monitoring.viewer
  • stackdriver.resourceMetadata.writer
  • storage.objectViewer on the project’s gcr bucket

The first 4 roles are the default roles/permissions required by GKE while the last ensures that each VM can pull docker images stored in your project’s gcr bucket. If you need more permissions, feel free to add them via the gcloud cli or gcp web ui console– assuming you do not destroy/modify the existing roles attached there should be no problem.

THIS SERVICE ACCOUNT IS NOT THE ONE USED BY YOUR CONTAINERS RUNNING IN THE CLUSTER– Opta handles creating appropriate service accounts for each K8s service, but for any non-opta managed workloads in the cluster, please refer to this GCP documentation.

Taints

Opta gives you the option of adding taints to the nodes created in this nodepool. The official documentation gives an excellent detailed summary, but in short one can use taints to stop workloads from running in said nodes unless they have a matching toleration. Simply provide a list of such taints as inputs like so:

  - type: gcp-nodepool
    name: nodepool1
    min_nodes: 1
    max_nodes: 3
    taints:
      - key: instancetype
        value: memoryoptimized
        effect: "NoExecute"
      - key: team
        value: booking
        # Tolerates for default effect of NoSchedule
      - key: highpriority
        # Tolerates for default value of opta

For most cases, simply specifying the key should do.

Fields

Name Description Default Required
max_nodes The maximum number of nodes to be set by the autoscaler in for the current nodegroup PER AVAILABILITY ZONE (there’s almost always 3). 5 False
min_nodes The minimum number of nodes to be set by the autoscaler in for the current nodegroup PER AVAILABILITY ZONE (there’s almost always 3). 1 False
node_disk_size The size of disk to give the nodes' vms in GB. 20 False
node_instance_type The gcloud machine type for the nodes. n2-highcpu-4 False
gke_channel The GKE K8s release channel to bind the cluster too. Gives you automatic K8s version management for the cluster and node pools. REGULAR False
preemptible A boolean specifying whether to use preemptible instances for the default nodegroup or not. The preemptible instances will be configured to have the max price equal to the on-demand price (so no danger of overcharging). WARNING: By using preemptible instances you must accept the real risk of frequent abrupt node terminations and possibly (although extremely rarely) even full blackouts (all nodes die). The former is a small risk as containers of Opta services will be automatically restarted on surviving nodes. So just make sure to specify a minimum of more than 1 containers – Opta by default attempts to spread them out amongst many nodes. False False
taints Taints to add to the nodes in this nodegroup. [] False

Last modified May 23, 2022 : Docs for new release (#191) (ba6432d)