Earlier this year I wrote about
container computing and enumerated
some of the benefits that you get when you use it as the basis for a distributed application
platform: consistency & fidelity, development efficiency, and operational efficiency. Because
containers are lighter in weight and have less memory and computational overhead than virtual machines,
they make it easy to support applications that consist of hundreds or thousands of small, isolated “moving parts.”
A properly containerized application is easy to scale and maintain, and makes efficient use of
available system resources.
Introducing Amazon EC2 Container Service
In order to help you to realize these benefits, we are announcing a preview of our new
container management service, EC2 Container Service (or ECS for short).
This service will make it easy for you for run any number of Docker
containers across a managed cluster of Amazon Elastic Compute Cloud (EC2) instances using powerful APIs and other tools. You do not
have to install cluster management software, purchase and maintain the cluster hardware, or match your hardware
inventory to your software needs when you use ECS. You simply launch some instances in
a cluster, define some tasks, and start them. ECS is built around a scalable,
multi-tenant base that takes care of all of the details of cluster management on your
By the way, don’t let the word “cluster” scare you off! A cluster is simply a pool of compute, storage, and
networking resources that serves as a host for one or more containerized applications. In fact,
your cluster can even consist of a single t2.micro instance. In general, a single
mid-sized EC2 instance has sufficient resources to be used productively as a starter cluster.
EC2 Container Service Benefits
Here’s how this service will help you to build, run, and scale Docker-based applications:
- Easy Cluster Management –
ECS sets up and manages clusters made up of Docker containers. It launches
and terminates the containers and maintains complete information about the state of your cluster. It can scale
to clusters that encompass tens of thousands of containers across multiple Availability Zones.
- High Performance – You can use the
containers as application building
blocks. You can start, stop, and manage thousands of containers in seconds.
- Flexible Scheduling –
ECS includes a built-in scheduler that strives to spread your
containers out across your cluster to balance availability and
utilization. Because ECS provides you with access to complete state
information, you can also build your
own scheduler or adapt an existing open source scheduler to use the service’s APIs.
- Extensible & Portable –
ECS runs the same Docker daemon that you would run on-premises. You can easily
move your on-premises workloads to the AWS cloud, and back.
- Resource Efficiency –
A containerized application can make very efficient use of resources. You
can choose to run multiple, unrelated containers on the same EC2 instance in order to
make good use of all available resources. You could, for example, to run a mix
of short-term image processing jobs and long-running web services on the same instance.
- AWS Integration –
Your applications can make use of AWS features such as Elastic IP addresses, resource tags,
and Virtual Private Cloud (VPC). The containers are, in effect, a new
base-level building block in the same vein as EC2 and S3.
- Secure –
Your tasks run on EC2 instances within an Amazon Virtual Private Cloud. The tasks
can take advantage of IAM roles, security groups, and other
AWS security features. Containers run in a multi-tenant environment and can communicate
with each other only across defined interfaces. The contained are launched on
EC2 instances that you own and control.
Using EC2 Container Service
ECS was designed to be easy to set up and to use!
You can launch an ECS-enabled AMI and your instances will be
automatically checked into your default cluster. If you want to launch
into a different cluster you can specify it by modifying the
configuration file in the image, or passing in User Data on launch.
To ECS-enable a Linux AMI, you simply install the ECS Agent and
the Docker daemon.
ECS will add the newly launched instance to its capacity pool and run containers
on it as directed by the scheduler. In other words, you can add capacity to any of your
clusters by simply launching additional EC2 instances in them!
The ECS Agent will be available in open source form under an
Apache license. You can install it on any of your existing Linux AMIs and call
registerContainerInstances to add them to your cluster.
Here are a few vocabulary items to help you to get familiar with the terminology used by ECS:
- Cluster –
A cluster is a pool of EC2 instances in a particular AWS Region, all managed
by ECS. One cluster can contain multiple instance types and
sizes, and can reside within one or more Availability Zones.
- Scheduler –
A scheduler is associated with each cluster. The scheduler is responsible for
making good use of the resources in the cluster by assigning containers
to instances in a way that respects any placement constraints and
simultaneously drives as much parallelism as possible, while also
aiming for high availability.
- Container –
A container is a packaged (or “Dockerized,” as the cool kids
like to say) application component. Each EC2 instance in a
cluster can serve as a host to one or more containers.
- Task Definition –
A JSON file that defines a Task as a set of containers. Fields
in the file define the image for each container, convey memory
and CPU requirements, and also specify the port mappings that
are needed for the containers in the task to communicate with
- Task – A task is
an instantiation of a Task Definition consisting of one or more
containers, defined by the work that they do and their
relationship to each other.
- ECS-Enabled AMI –
An Amazon Machine Image (AMI)
that runs the ECS Agent and dockerd. We plan to ECS-enable
the Amazon Linux AMI and are working with our partners to similarly
enable their AMIs.
EC2 Container Service includes a set of APIs that are both simple and powerful. You
can create, describe, and destroy clusters and you can register EC2 instances therein.
You can create task definitions and initiate and manage tasks.
Here is the basic set of steps that you will follow in order to run your application
on ECS. I am making the assumption that you have already Dockerized your
application by breaking it down in to fine-grained components, each described by a
Dockerfile and each running nicely on your existing infrastructure. There
are plenty of good resources online to help you with this process. Many popular
applications application components have already been Dockerized and can be found on
Docker Hub. You can use ECS with any
public or private Docker repository that you can acccess. Ok, so here are the steps:
- Create a cluster, or decide to use the default one
for your account in the target Region.
- Create your task definitions and register them with the cluster.
- Launch some EC2 instances and register them with the cluster.
- Start the desired number of copies of each task.
Monitor the overall utilization of the cluster and the overall throughput
of your application, and make adjustments as desired. For example, you can
launch and then register additional EC2 instances in order to expand the
cluster’s pool of available resources.
EC2 Container Service Pricing and Availability
The service is launch today in preview form. If you are interested in signing up,
click here to join
the waiting list.
There is no extra charge for ECS. As usual, you pay only for the resources that you use.