Introduction to Container Orchestration

As organizations continue their journey towards microservices and distributed applications deployed as containers, the need for tools to manage the life cycle of hundreds or even thousands of containers has become apparent. Container orchestration platforms provide this orchestration layer to automate deployment, scaling, networking, availability and more across clusters of hosts running Docker and Kubernetes.

According to Gartner, more than 75% of global organizations will be running containerized applications in production by 2022, up from less than 30% today. The right container orchestration platform is critical for reaping the benefits of containerization like portability, resource efficiency and developer velocity.

This article provides an in-depth look at the most widely used open source and commercial container orchestration tools suitable for diverse use cases.

Why is Container Orchestration Important?

Containers package applications with their entire runtime environment to run uniformly and reliably across diverse infrastructure. But running containers at scale requires automating routine tasks like:

• Deployment and placement
• Health monitoring and healing
• Horizontal scaling
• Networking between containers across hosts
• Service discovery
• Zero-downtime updates
• Securing access
• Monitoring resource usage
• Logging

Container orchestration handles all these tasks through a control plane that continually works to maintain the desired state of containers and services defined through declarative configurations.

For example, if a container fails on a host, the orchestrator will automatically redeploy it elsewhere. Requests to scale out a microservice can trigger the orchestrator to deploy more container instances.

By providing this automation layer, orchestrators allow developers and site reliability engineers to focus on building and running applications rather than infrastructure management.

Key Capabilities of Container Orchestrators

While implementations differ across tools, most container orchestrators share some common capabilities:

Cluster management: Automatically configure and monitor Kubernetes or Docker hosts pooled together into a cluster to provide infrastructure for containerized services. Handle node additions and failures.

Declarative service deployment: Define the desired state of containers and other application resources through yaml or json manifests rather than imperative commands. The orchestrator works constantly to maintain this state.

Health checks: Monitor health of container instances and restart or reschedule them if they fail.

Scaling: Horizontally scale stateless services by increasing container replicas. Some auto-scaling rules can also scale up/down based on utilization metrics.

Load balancing and networking: Allocate each service a DNS name or IP address that balances requests across healthy container endpoints. Handle container-to-container communications.

Rolling updates: Deploy new software versions through controlled, zero-downtime rolling updates that replace containers on each host without dropping user requests. Roll back failed deployments.

Storage orchestration: Manage external storage volumes that can persist data beyond the lifetime of containers.

Security: Securely authenticate users, apply resource constraints, policies and data encryption across clusters.

Beyond these basics, enterprise tools offer added capabilities like graphical UIs, monitoring, logging, tracing, CI/CD integration, GitOps workflows, autoscaling and more.

With this background, let’s survey the landscape of popular orchestrators.

Kubernetes

Kubernetes has become the industry standard for container orchestration with wide adoption. Created by Google based on their internal Borg system and now maintained by the Cloud Native Computing Foundation, Kubernetes excels at automating deployment, scaling and operations of containerized applications.

Image by Kubernetes.io

The Kubernetes control plane continually works toward the desired state for containers and services defined in YAML or JSON manifests. The kubectl CLI tool communicates with the API server to create, update, or delete these resource configurations.

Kubernetes runs applications as Pods – one or more containers sharing resources and namespaces – that can be horizontally scaled using ReplicaSets. Pods get their own internal and external IP addresses and can discover each other via DNS.

The basic resources used to describe Kubernetes applications include:

• Pods
• ReplicaSets
• Deployments
• Services

Additional resources exist for storage, config, automation, networking, security and more.

A powerful feature called operators extends Kubernetes to natively manage stateful applications like databases through Custom Resource Definitions tailored to each app.

Kubernetes supports rich policy-based access control, resource quotas and isolation through namespaces. Enterprise features include multi-tenancy, security hardening and cluster federation.

Prominent users running Kubernetes in production include Reddit, Spotify, Github, The New York Times and many more corporations.

Main Features

• Automatic binpacking
• Horizontal pod autoscaler
• Batch execution
• Service load balancing
• Secret management
• Storage orchestration
• Automatic rollouts and rollbacks
• Self healing
• Service discovery and DNS management

Docker Swarm

The native Docker Swarm orchestrator offers a simple solution for clusters running the Docker engine. It is well suited to smaller deployments where Kubernetes may be overkill.

A Docker Swarm cluster comprises manager and worker nodes where the managers handle orchestration and scheduling through the Swarm API. Workers run containerized tasks assigned by managers.

Image by Docker

Managers use Raft consensus to maintain consistency in case of failures. The system automatically elects a new leader manager.

You define services to deploy containers using Docker Compose yaml files or equivalent docker service commands. These describe desired state in terms of number of replicas, storage and networks.

Managers constantly monitor the cluster state and reschedule containers if nodes fail or replicas diverge from desired count. This provides self-healing behavior.

Load balancing, overlay networking and secure secrets distribution among nodes are included features.

Overall, Docker Swarm offers an easy on-road to container orchestration compatible with existing Docker environments. But it lags behind Kubernetes in features and community support.

Main Features

• Declarative service deployment
• Scaling and rescheduling
• Multi-host networking
• Load balancing
• Service discovery
• Encrypted secrets
• Rolling updates

Apache Mesos

Developed at UC Berkeley around 2009, Apache Mesos provides an abstraction layer for pooling compute resources across data centers – physical or virtual machines. Frameworks can run on top of Mesos to enable specific capabilities like container orchestration.

Mesos clusters rely on ZooKeeper for leader election and state management. Agents run on cluster nodes to provide resource offers to the elected Mesos master scheduler. The master then negotiates resource allocations between frameworks.

Prominent Mesos frameworks include:

Marathon – Designed for long running services and container orchestration using Docker or rkt runtimes. Allows health checking, auto scaling, service discovery, constraints, etc.

Aurora – Aimed at scheduling jobs/tasks rather than services. Used at companies like Twitter.

Chronos – Schedules cron jobs.

So Mesos delegates orchestration capabilities to specialized frameworks like Marathon while providing the resource pooling and allocation engine itself. This plugin architecture allows mixing multiple orchestrators like Kubernetes and batch schedulers together.

Notable Mesos users include Twitter, Airbnb, Yahoo and CERN‘s large hadron collider.

Main Features

• Dynamic partitioning of cluster resources
• Fault tolerant with ZooKeeper
• Multi-framework support
• Multi-tenant capabilities
• Centralized cluster management

Hashicorp Nomad

Nomad by HashiCorp takes a flexible, lightweight approach to workload orchestration on bare metal or virtualized infrastructure. It can manage containers, legacy apps, microservices and batch jobs.

The lean Nomad server handles scheduling and clients run on hosts to execute tasks. Server nodes form a consensus cluster manage state.

Image by Nomad

Task drivers support Docker, Qemu, Java, exec and custom logic. Placing constraints control task placement.

Nomad handles failures by redeploying allocations and rescheduling batch jobs. Jobs describe tasks, constraints, dependencies and require resources.

Service networking connects containers across Nomad clients using Consul or Vault. Load balancers automatically attach to services.

The main advantage over Kubernetes is easy deployment on existing infrastructure plus scheduling non-containerized apps. Suitable for apps not needing complex operational logic.

Used by companies like Samsung, Roblox, Citadel Securities.

Main Features

• Supports container and non container workloads
• Easy to deploy and operate
• Flexible constraint-based scheduling
• Scales to clusters of thousands of nodes
• Service networking and load balancing
• Batch scheduling
• Resource monitoring

OpenShift

RedHat OpenShift adds enterprise capabilities and support on top of Kubernetes for production environments. Customers run OpenShift clusters on their infrastructure.

It enhances Kubernetes with security, governance, compliance, and automation features appealing to regulated industries. For example, OpenShift extends role-based access control with delineation of duties to mitigate insider threats.

Management and cluster operations are facilitated through various operator plugins as well as Prometheus and Grafana monitoring builtin. Multiple log aggregation sources, including EF fluentd Kubernetes metadata, get collected in Elasticsearch to enable analysis.

Image by Openshift

Source-to-image capability allows building container images from source code repositories while hooks enable policy enforcement during the build process.

To summarize, OpenShift hardens Kubernetes for the enterprise environment while adding devops capabilities like standardized CD pipelines.

Customers running OpenShift span financial services, telecom, government and healthcare verticals including Keybank, Santander, British Telecom and more.

Main Features

• Kubernetes and container image governance
• Multi-tenancy and Cluster policies
• Security and compliance
• Automated install
• Integrated developer experience
• Red Hat support

Cloud Native Container Orchestration Engines

While you can self-manage Kubernetes on your own infrastructure, all major cloud providers offer fully managed Kubernetes services that reduce operational complexity:

Amazon EKS
Azure Kubernetes Service (AKS)
Google Kubernetes Engine (GKE)

These hosted Kubernetes platforms streamline cluster deployment, upgrades, scaling, logging, and monitoring.

For example, Google GKE automates mundane tasks like health checking node status, spinning up replacement nodes on failures, patching master servers, or encrypted communication between components.

They facilitate integration natively with other cloud services:

• Load balancing requests across containers
• Autoscaling based on utilization
• Cloud provider storage for persistent data
• IAM for access management
• Cloud Logging and Monitoring

Additionally, most public cloud Kubernetes offerings allow running serverless, FaaS workloads on top of Kubernetes leveraging scalability of the cloud provider without capacity planning.

So if you are already using AWS, Azure or Google Cloud, hosted Kubernetes makes getting started easy yet allows room to customize clusters as usage advances.

Fully Managed Containers-as-a-Service

Lastly, I would be remiss without mentioning a few prominent Containers-as-a-Service (CaaS) platforms that remove the need to manage infrastructure completely:

Amazon ECS – AWS-native container orchestration service with tight integration across other AWS services

Google Cloud Run – Serverless containers launched from Docker images with autoscaling, billing per use and HTTPS load balancing built-in

Azure Container Instances – Quickly run containers without provisioning servers or a cluster. Usage metered by the second.

These fully managed services can be easiest way to get started for lightweight applications requiring little customization.

Criteria for Evaluating Solutions

With the diversity of container orchestrators available, your needs and environment guide which solutions merit deeper investigation. Consider:

Application Requirements
Do your workloads require handling state or stateless processing? Specific OS characteristics — Linux, Windows? Any preference for locality, hardware accelerators? This filters compatible orchestrators.

Environment
Is your deployment targeting on-prem infrastructure or the public cloud? Do you operate solely Ubuntu or centOS Linux or have a heterogenous environment? Can dictate solution.

In-house Expertise
Assess skill sets around Docker, Kubernetes and specific platforms like Mesos. Leverage existing knowledge or evaluate ramp up time. Open source options have abundant documentation.

Hybrid and Multi-Cloud
If operating across data centers, private and public cloud, evaluate hybrid management capabilities or emerging multi cluster management tools.

Security and Governance Mandates
Highly regulated environments may require security hardening, strict access controls, governance of Kubernetes objects, etc which would favor options like OpenShift.

Scalability Needs
The scale of your workload and infrastructure guides choices. Kubernetes shines for large clusters and complex deployments.

Evaluating orchestrators along these lines illuminates best fit solutions.

Adopting Container Orchestration

Adoption phases for harnessing container orchestrators often progress along the following:

Getting Started – Provision Kubernetes using K8s operators on cloud platforms or tools like Kind and Minikube. Define pod and service resources through yaml files, perhaps via Helm charts. Enhance skills through online courses, documentation and conferences.

Automation – Gradually automate application CI/CD pipelines leveraging GitOps workflows and Kubernetes operators or admission webhooks. Embrace infrastructure-as-code for environment consistency.

Optimization – Instrument applications and infrastructure to collect metrics on performance, availability and efficiency. Tune and customize based on usage patterns.

Advanced Operations – As administrators gain expertise, tap advanced policy controls for governance, security, quota management and multi-tenancy. Enable cluster federation.

Proceeding along these stepping stones unlocks immense value from container orchestration.

Container orchestrators have graduated from early adoption to become the de facto platform for empowering developers through cloud native technologies while giving operations teams guardrails. Leveraging orchestrators like Kubernetes unlocks order of magnitude productivity gains. The ecosystem continues rapid evolution – where will you steer it next?