This module creates an additional nodepool for the primary GKE cluster. Note that the
gcp-gke module creates a default nodepool so this should only be used when
you want one more.
IAM Permissions given to the Nodepool
Along with the nodepool, Opta creates a GCP IAM service account that is attached to each VM in the pool and handles all of the machine’s (and Kubernetes actions done by the kubelet in the machine like, for example, downloading a gcr image) IAM permissions. Opta gives this service account the following roles:
- storage.objectViewer on the project’s gcr bucket
The first 4 roles are the default roles/permissions required by GKE
while the last ensures that each VM can pull docker images stored in your project’s gcr bucket. If you need more
permissions, feel free to add them via the
gcloud cli or gcp web ui console– assuming you do not destroy/modify the
existing roles attached there should be no problem.
THIS SERVICE ACCOUNT IS NOT THE ONE USED BY YOUR CONTAINERS RUNNING IN THE CLUSTER– Opta handles creating appropriate service accounts for each K8s service, but for any non-opta managed workloads in the cluster, please refer to this GCP documentation.
Opta gives you the option of adding taints to the nodes created in this nodepool. The official documentation gives an excellent detailed summary, but in short one can use taints to stop workloads from running in said nodes unless they have a matching toleration. Simply provide a list of such taints as inputs like so:
- type: gcp-nodepool name: nodepool1 min_nodes: 1 max_nodes: 3 taints: - key: instancetype value: memoryoptimized effect: "NoExecute" - key: team value: booking # Tolerates for default effect of NoSchedule - key: highpriority # Tolerates for default value of opta
For most cases, simply specifying the
key should do.
WarningAdding taints to nodes also forbids most daemonsets from running in said node. This can be a problem with security/monitoring solutions (e.g. Datadog) which typiclly use daemonsets to run their agents in each node, so please be careful and read their instructions on how to add tolerations
||The maximum number of nodes to be set by the autoscaler in for the current nodegroup PER AVAILABILITY ZONE (there’s almost always 3).||
||The minimum number of nodes to be set by the autoscaler in for the current nodegroup PER AVAILABILITY ZONE (there’s almost always 3).||
||The size of disk to give the nodes' vms in GB.||
||The gcloud machine type for the nodes.||
||The GKE K8s release channel to bind the cluster too. Gives you automatic K8s version management for the cluster and node pools.||
||A boolean specifying whether to use preemptible instances for the default nodegroup or not. The preemptible instances will be configured to have the max price equal to the on-demand price (so no danger of overcharging). WARNING: By using preemptible instances you must accept the real risk of frequent abrupt node terminations and possibly (although extremely rarely) even full blackouts (all nodes die). The former is a small risk as containers of Opta services will be automatically restarted on surviving nodes. So just make sure to specify a minimum of more than 1 containers – Opta by default attempts to spread them out amongst many nodes.||
||Taints to add to the nodes in this nodegroup.||
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.