(Note: This is part one of a two-part series on Amazon EKS; to see a more in-depth review and recommendations, see Part II: “Figuring out if Kubernetes on AWS (EKS) is right for you” .)
Kubernetes is the open-source automation system originally designed by Google (now maintained under the CNCF, the Cloud Native Computing Foundation) for management, scaling, and deployment of containerized applications. A recent survey by the CNCF indicates that AWS has seen the largest increase in Kubernetes clusters in production deployment; ~69% of respondents report they deploy there, leading all the other Cloud providers as well as all on-premise deployments, reported as of late 2017.
Recently Amazon launched its Amazon Elastic Container Service (EKS) for Kubernetes, which has become generally available as of June 5th. We’ve recently completed our testing and review of Amazon EKS and want to give you the benefit of some additional background and what you need to know before you dive in.
An intro overview of EKS
Simply put, EKS is Amazon’s ‘Kubernetes as a service’. It simplifies the process of spinning up and maintaining highly-available Kubernetes clusters in the AWS cloud.
The typical EKS cluster consists of two main components, each running on its own VPC – The Control Plane and Worker Nodes.
Optimize cloud economics and drive Business Goals
Proven best practices that help both finance & engineering teams
- Control Plane: Consists of three Kubernetes Master nodes running in three different AZs for high availability. Incoming traffic to Kubernetes API comes through the AWS network load balancer (NLB). The Control Plane runs on the Amazon-controlled VPC that cannot be managed by the customer and is fully managed by Amazon.
- Worker Nodes: Run on usual Amazon EC2 instances in the customer-controlled VPC. Any AWS instance type can be used as a worker node. Worker nodes can be accessed via SSH or provisioned with any existing automation.
EKS gives you a lot of flexibility in terms of layout. You can deploy one EKS (read ‘Kubernetes’) cluster per environment, system or application. You can also use a single Amazon EKS cluster to run multiple applications by taking advantage of Kubernetes namespaces and IAM security policies.
Below are the official visuals from AWS that better describe how the typical EKS cluster appears in AWS and how it works.
EKS usage algorithm. Source: Amazon AWS
EKS architecture schemas. Source: Amazon AWS
10 things to know about EKS on AWS
Now that you have a picture from an architecture perspective, let's look at ten things you'll find useful to think about in using Amazon EKS.
1. What instance types are supported by EKS?
EKS supports many EC2 instance types, such as t2, m3, m4, m5, c4, c5, i3, r3, r4, x1, p2 and p3 instances. The smallest recommended instance type is t2.small where ~1.1Gb of 2Gb memory is available for the Kubernetes pods after ‘warming up’ Docker, Kubelet, and OS under some load.
2. What versions of OS, Docker, and Kubernetes are used by EKS?
As of this blog post, AWS EKS supports only Kubernetes version 1.10.3.
By default, Amazon EKS provides AWS CloudFormation templates to spin up your worker nodes with the Amazon EKS-optimized AMI. This AMI is built on top of Amazon Linux 2. The AMI is configured to work with Amazon EKS out of the box and it includes Docker 17.06.2-ce (with overlay2 as a Docker storage driver), Kubelet 1.10.3, and the AWS authenticator. The AMI also launches with specialized Amazon EC2 user data that allows it to discover and connect to your cluster's control plane automatically. The example AMI for us-east-1 region is ami-dea4d5a1.
It is possible, however, to build your own AMI using a different OS, kernel, or Docker version as well as different settings that still integrate with Amazon EKS.
3. What is used as a network driver for EKS and how well does the network perform?
The AWS VPC container network interface (CNI) plugin is responsible for providing pod networking in Kubernetes using Elastic Network Interfaces on AWS. Amazon EKS works with Calico by Tigera to integrate with the CNI plugin to provide fine grained networking policies.
Our tests have shown that the CNI plugin itself does not produce a network bottleneck when running worker instances with a 10GB/s network – which we can’t say about some other popular network drivers for Kubernetes, such as Flannel and Opencontrail.
4. Can I automatically add/remove nodes on a per-workload basis?
Yes. You can launch an Auto Scaling group of worker nodes and register it with your EKS cluster. The recommended AWS CloudFormation templates for EKS already come with an autoscaling group that launches your on-demand worker instances.
5. Can I SSH to worker nodes and manage them?
Yes. Worker nodes are really just EC2 instances running in your VPC.
6. Can EKS use reserved or spot instances?
Yes. You can use reserved or spot instances for EKS worker nodes in the same way as your other EC2 instances. As of this writing, there are no reservation plans for the EKS control plane aka Master nodes, but such plans are likely in the future.
7. What existing Kubernetes add-ons can be used with Amazon EKS?
As of this writing, we’ve tested the DNS and Kubernetes Dashboard addons with EKS, and they both worked well. For example, the DNS add-on comes enabled out-the-box and runs the DNS pod in 3 replicas; in our experience, it's quite enough for production usage. As we test logs- and metrics-aggregation add-ons in production workloads, we will provide more updates.
8. How do you migrate the existing Kubernetes cluster to EKS?
If your system is already running in Kubernetes the migration will not require changing the code of your applications. However, for legacy applications that are not designed to run in Docker and Kubernetes, or that don’t support running more than one replica of the same app, some code and design changes may be required.
Some AWS partner companies already have quite a bit of experience in moving legacy applications to the cloud (including containerization and refactoring workloads for better scaling). But, it’s easier said than done; only some of these companies (CloudGeometry included) have mastered Amazon EKS and added it to the list of officially supported cloud services for migration :) As such, we encourage you to contact us for complicated cloud migration cases, especially ones that involve Amazon EKS.
9. I already have the kubectl installed. What else do I need to connect to EKS?
If you already have kubectl version 1.10+, then in order to use Amazon EKS you will only need to install heptio-authenticator-aws. See this guide.
10. Do all kubectl commands work with EKS?
Yes, thanks to the fact that Amazon EKS provisions elastic network interfaces in your VPC subnets to provide connectivity from the control plane instances to the worker nodes. Kubectl port-forwarding, log-tailing, exec-ing into the pods – all these work reliably and fast.
If you'd like to learn more about Kubernetes and how it simplifies autoscaling and HA with well-designed cloud apps, or could use some consultation on the topic of migration, please don't hesitate to contact us. And, for a more thorough review of Amazon EKS with professional recommendations on its deployment, don't forget to check out part II of this post: Figuring out if Kubernetes on AWS (EKS) is right for you.
