k8s-service
The most important module for deploying apps, k8s-service
deploys a kubernetes app on Azure.
It deploys your service as a rolling update securely and with simple autoscaling right off the bat– you
can even expose it to the world, complete with load balancing both internally and externally.
Features
Set custom environment variables
Opta allows you to pass in custom environment variables to your k8s-service.
Just use the env_vars
field:
name: hello
environments:
- name: staging
path: "opta.yaml"
modules:
- name: hello
type: k8s-service
port:
http: 80
image: ghcr.io/run-x/hello-opta/hello-opta:main
healthcheck_path: "/"
public_uri: "/hello"
env_vars:
- name: "API_KEY"
value: "value"
With this configuration, your container will get an env var named API_KEY
with
the value value
!
You can also use Opta’s interpolation variables features to refer to other values.
External/Internal Image
This module supports deploying from an “external” image repository (currently only public ones supported)
by setting the image
field to the repo (e.g. “kennethreitz/httpbin” in the examples). If you set the value to “AUTO” however,
it will automatically create a secure container repository with ECR on your account. You can then use the Opta push
command to push to it!
Healthcheck Probe
One of the benefits of K8s is that it comes with built-in checks for the responsiveness of your server. These are called liveness and readiness probes.
tl;dr An (optional) liveness probe determines whether your server should be restarted, and an (optional) readiness probe determines if traffic should be sent to a replica or be temporarily rerouted to other replicas. Essentially smart healthchecks. For websockets Opta just checks the tcp connection on the given port.
Opta supports 2 types of possible health checks (please be advised that while they’re not required, they’re highly recommended):
Option 1: Health Check HTTP Ping
Quite straightforward, K8s does regular http get requests to your server under a specific given path.
Any code greater than or equal to 200 and less than 400 indicates success.
Any other code indicates failure. You can specify this by setting the healthcheck_path
input, or alternatively the
liveness_probe_path
and/or the readiness_probe_path
if you want different behavior for the liveness and readiness
checks.
Option 2: Health Check Command
Also simple to understand, K8s regularly execute a shell command of your choosing and considers the
server healthy if it has and exit code of 0. Commands should be passed in as a list of the different
parts (e.g. [“echo”, “hello”]). You can specify this by setting the healthcheck_command
input, or alternatively the
liveness_probe_command
and/or the readiness_probe_command
if you want different behavior for the liveness and
readiness checks.
Autoscaling
As mentioned, autoscaling is available out of the box. We currently only support autoscaling based on the pod’s cpu and memory usage, but we hope to soon offer the ability to use 3rd party metrics like datadog to scale. As mentioned in the k8s docs, the horizontal pod autoscaler (which is what we use) assumes a linear relationship between # of replicas and cpu (twice the replicas means half expected cpu usage), which works well assuming low overhead. The autoscaler then uses this logic to try and balance the cpu/memory usage at the percentage of request. So, for example, if the target memory is 80% and we requested 100mb, then it’ll try to keep the memory usage at 80mb. If it finds that the average memory usage was 160mb, it would logically try to double the number of replicas.
Resource Requests
One of the other benefits of kubernetes is that a user can have fine control over the resources used by each of their containers. A user can control the cpu, memory and disk usage with which scheduling is made. With Opta, we expose such settings to the user, while keeping sensible defaults.
Resource Limits
While the Container Resources specify the “planned” amount of resource allocation (and thereby which pods are deployed to which nodes given the available CPU and memory), a pod can go over their requested CPU/memory if there is more available on its running node. The resource limits, however, specify the max amount of resources allocated to a pod at any time. These limits exist for (obvious) safety measures (e.g. preventing one app from starving the whole cluster). By default, this limit is set in Opta to twice the resource request value, but is exposed to users for further configuration.
Please refer to the official kubernetes resource management page for more info.
NOTE: Based on reasons outlined here Opta does not add CPU limits, only memory.
Ingress
You can control if and how you want to expose your app to the world! Check out the Ingress docs for more details.
Persistent Storage
A user can now specify persistent storage to be provisioned for your servers and kept intact over your different
deployments + scaling. This will take form of a list of entries holding a size
(size of the storage to create in
gigabytes) and path
(path to put it on your server’s container) field, like so:
.
.
.
modules:
- name: app
type: k8s-service
image: kennethreitz/httpbin
min_containers: 2
max_containers: "{vars.max_containers}"
liveness_probe_path: "/get"
readiness_probe_path: "/get"
port:
http: 80
public_uri: "subdomain1.{parent.domain}"
persistent_storage:
- path: "/DESIRED/PATH1"
size: 20 # 20 GB
- path: "/DESIRED/PATH2"
size: 30 # 30 GB
.
.
.
Under the hood, an Azure StandardSSD is being created to house your data for each coexisting server container of your app.
WARNING Switching between having the persistent_storage field set or not will lead to some minor downtime as the underlying resource kind is being switched.
NOTE because of the nature of these disks, they will not be cleaned up automatically unless during a service destruction. If you wish to release the StandardSSD disks for whatever reason you will need to manually do so by deleting the kubernetes persistent volume claims.
Fields
Name | Description | Default | Required |
---|---|---|---|
image |
Set to AUTO to create a private repo for your own images. Otherwises attempts to pull image from public dockerhub | AUTO |
True |
port |
Specifies what port your app was made to be listened to. Currently it must be a map of the form http: [PORT_NUMBER_HERE] or tcp: [PORT_NUMBER_HERE] . Use http if you just have a vanilla http server and tcp for websockets. |
none | False |
min_containers |
The minimum number of replicas your app can autoscale to. | 1 |
False |
max_containers |
The maximum number of replicas your app can autoscale to. | 3 |
False |
autoscaling_target_cpu_percentage |
See the autoscaling section. | 80 |
False |
autoscaling_target_mem_percentage |
See the autoscaling section. | 80 |
False |
secrets |
Deprecated, see secrets instructions. | [] |
False |
env_vars |
A map of key values to add to the container as environment variables (key is name, value is value). yaml env_vars: FLAG: "true" |
[] |
False |
healthcheck_path |
See the See the healthcheck probe section. Default null (i.e., no user-specified healthchecks) |
None |
False |
healthcheck_command |
See the See the healthcheck probe section. Default [] (i.e., no user-specified healthchecks) |
[] |
False |
liveness_probe_command |
Use if using shell command liveness checks and liveness probe != readiness probe | [] |
False |
readiness_probe_command |
Use if using shell command readiness checks and liveness probe != readiness probe | [] |
False |
liveness_probe_path |
Use if using http ping liveness checks and liveness probe != readiness probe | None |
False |
readiness_probe_path |
Use if using http ping readiness checks and liveness probe != readiness probe | None |
False |
initial_liveness_delay |
Use if the initial delay needs to be changed. | 30 |
False |
initial_readiness_delay |
Use if the initial delay needs to be changed. | 30 |
False |
consistent_hash |
Use consistent hashing | None |
False |
sticky_session |
Use sticky sessions via cookies for your service (first request will send you a cookie called opta_cookie which you should add on future requests). | False |
False |
sticky_session_max_age |
If the sticky session is enabled, how long should the cookie last? | 86400 |
False |
resource_request |
See the container resources section. CPU is given in millicores, and Memory is in megabytes. | {'cpu': 100, 'memory': 128} |
False |
resource_limits |
See the container resources section. Memory is in megabytes.. | None |
False |
public_uri |
The full domain to expose your app under as well as path prefix. Must be the full parent domain or a subdomain referencing the parent as such: “dummy.{parent[domain]}/my/path/prefix” | [] |
False |
keep_path_prefix |
Should we keep the prefix path which you set in the public uri when forwarding requests to your service? | False |
False |
links |
A list of extra IAM role policies not captured by Opta which you wish to give to your service. | [] |
False |
persistent_storage |
A list persistent storages to add to each instance of your service (need to give a size which is the size in GB for the storage volume to be, and path which is the path in the filesystem of each instance to place it under) |
[] |
False |
ingress_extra_annotations |
These are extra annotations to add to ingress objects | {} |
False |
pod_annotations |
These are extra annotations to add to k8s-service pod objects / replace defaults | {} |
False |
pod_labels |
These are extra labels to add to k8s-service pod objects / replace defaults | {} |
False |
timeout |
Time in seconds to wait for deployment. | 300 |
False |
max_history |
The max amount of helm revisions to keep track of (0 for infinite) | 25 |
False |
Outputs
Name | Description |
---|---|
docker_repo_url |
Url to the docker repository created for images to be deployed in this env |
current_image |
The full name plus tag of the docker imge used by the deployment. |
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.