Amazon #EC2 Container Service (#ECS) – Container Management for the #AWS Cloud

Earlier this year I wrote about
container computing and enumerated
some of the benefits that you get when you use it as the basis for a distributed application
platform: consistency &amp fidelity, development efficiency, and operational efficiency. Because
containers are lighter in weight and have less memory and computational overhead than virtual machines,
they make it easy to support applications that consist of hundreds or thousands of small, isolated “moving parts.”
A properly containerized application is easy to scale and maintain, and makes efficient use of
available system resources.

Introducing Amazon EC2 Container Service

In order to help you to realize these benefits, we are announcing a preview of our new
container management service, Container Service (or for short).
This service will make it easy for you for run any number of Docker
containers across a managed cluster of Amazon Elastic Compute Cloud (EC2) instances using powerful APIs and other tools. You do not
have to install cluster management software, purchase and maintain the cluster hardware, or match your hardware
inventory to your software needs when you use ECS. You simply launch some instances in
a cluster, define some tasks, and start them. ECS is built around a scalable,
fault-tolerant,
multi-tenant base that takes care of all of the details of cluster management on your
behalf.

By the way, don’t let the word “cluster” scare you off! A cluster is simply a pool of compute, storage, and
networking resources that serves as a host for one or more containerized applications. In fact,
your cluster can even consist of a single t2.micro instance. In general, a single
mid-sized EC2 instance has sufficient resources to be used productively as a starter cluster.

EC2 Container Service Benefits
Here’s how this service will help you to build, run, and scale Docker-based applications:

  • Easy Cluster Management
    ECS sets up and manages clusters made up of Docker containers. It launches
    and terminates the containers and maintains complete information about the state of your cluster. It can scale
    to clusters that encompass tens of thousands of containers across multiple Availability Zones.
  • High Performance – You can use the
    containers as application building
    blocks. You can start, stop, and manage thousands of containers in seconds.
  • Flexible Scheduling
    ECS includes a built-in scheduler that strives to spread your
    containers out across your cluster to balance availability and
    utilization. Because ECS provides you with access to complete state
    information, you can also build your
    own scheduler or adapt an existing open source scheduler to use the service’s APIs.
  • Extensible & Portable
    ECS runs the same Docker daemon that you would run on-premises. You can easily
    move your on-premises workloads to the cloud, and back.
  • Resource Efficiency
    A containerized application can make very efficient use of resources. You
    can choose to run multiple, unrelated containers on the same EC2 instance in order to
    make good use of all available resources. You could, for example, to run a mix
    of short-term image processing jobs and long-running web services on the same instance.
  • AWS Integration
    Your applications can make use of AWS features such as Elastic IP addresses, resource tags,
    and Virtual Private Cloud (VPC). The containers are, in effect, a new
    base-level building block in the same vein as EC2 and S3.
  • Secure
    Your tasks run on EC2 instances within an Amazon Virtual Private Cloud. The tasks
    can take advantage of IAM roles, security groups, and other
    AWS security features. Containers run in a multi-tenant environment and can communicate
    with each other only across defined interfaces. The contained are launched on
    EC2 instances that you own and control.

Using EC2 Container Service
ECS was designed to be easy to set up and to use!

You can launch an ECS-enabled AMI and your instances will be
automatically checked into your default cluster. If you want to launch
into a different cluster you can specify it by modifying the
configuration file in the image, or passing in User Data on launch.
To ECS-enable a Linux AMI, you simply install the ECS Agent and
the Docker daemon.

ECS will add the newly launched instance to its capacity pool and run containers
on it as directed by the scheduler. In other words, you can add capacity to any of your
clusters by simply launching additional EC2 instances in them!

The ECS Agent will be available in open source form under an
Apache license. You can install it on any of your existing Linux AMIs and call
registerContainerInstances to add them to your cluster.

Here are a few vocabulary items to help you to get familiar with the terminology used by ECS:

  • Cluster
    A cluster is a pool of EC2 instances in a particular AWS Region, all managed
    by ECS. One cluster can contain multiple instance types and
    sizes, and can reside within one or more Availability Zones.
  • Scheduler
    A scheduler is associated with each cluster. The scheduler is responsible for
    making good use of the resources in the cluster by assigning containers
    to instances in a way that respects any placement constraints and
    simultaneously drives as much parallelism as possible, while also
    aiming for high availability.
  • Container
    A container is a packaged (or “Dockerized,” as the cool kids
    like to say) application component. Each EC2 instance in a
    cluster can serve as a host to one or more containers.
  • Task Definition
    A JSON file that defines a Task as a set of containers. Fields
    in the file define the image for each container, convey memory
    and CPU requirements, and also specify the port mappings that
    are needed for the containers in the task to communicate with
    each other.
  • Task – A task is
    an instantiation of a Task Definition consisting of one or more
    containers, defined by the work that they do and their
    relationship to each other.
  • ECS-Enabled AMI
    An Amazon Machine Image (AMI)
    that runs the ECS Agent and dockerd. We plan to ECS-enable
    the Amazon Linux AMI and are working with our partners to similarly
    enable their AMIs.

EC2 Container Service includes a set of APIs that are both simple and powerful. You
can create, describe, and destroy clusters and you can register EC2 instances therein.
You can create task definitions and initiate and manage tasks.

Here is the basic set of steps that you will follow in order to run your application
on ECS. I am making the assumption that you have already Dockerized your
application by breaking it down in to fine-grained components, each described by a
Dockerfile and each running nicely on your existing infrastructure. There
are plenty of good resources online to help you with this process. Many popular
applications application components have already been Dockerized and can be found on
Docker Hub. You can use ECS with any
public or private Docker repository that you can acccess. Ok, so here are the steps:

  1. Create a cluster, or decide to use the default one
    for your account in the target Region.
  2. Create your task definitions and register them with the cluster.
  3. Launch some EC2 instances and register them with the cluster.
  4. Start the desired number of copies of each task.
  5. Monitor the overall utilization of the cluster and the overall throughput
    of your application, and make adjustments as desired. For example, you can
    launch and then register additional EC2 instances in order to expand the
    cluster’s pool of available resources.

EC2 Container Service Pricing and Availability
The service is launch today in preview form. If you are interested in signing up,
click here to join
the waiting list.

There is no extra charge for ECS. As usual, you pay only for the resources that you use.

Jeff;

Related posts