Securing Docker for Production: An In-Depth Practical Guide

Hey there! Docker adoption has exploded as companies race to launch applications faster, but many overlook the security risks. Running containers in production requires hardening your environment properly.

In this hands-on guide, we‘ll walk through proven techniques to lock down Docker hosts, images, and workloads. I‘ll share tips from my experience securing containers at Acme Corp so you can avoid the pitfalls!

Let‘s start with the basics…

A Quick Docker Security Primer

Before we dive into hardening tactics, let‘s recap how Docker works:

Containers virtualize apps and dependencies into portable, isolated packages called containers. These provide standardized units for development, testing, and deployment.
Docker Engine manages containers on hosts via a daemon, REST API, and CLI client. It handles lifecycle events like starting and stopping containers.
Images are read-only templates used to launch container instances. Images are made up of layered filesystems with app code, libraries, dependencies, and OS packages.
Registries store and distribute images. Docker Hub is the main public registry, but enterprises often use internal ones.

With containers, security risks mainly stem from:

Vulnerabilities in container OS packages or app dependencies.
Insecure container-to-host access letting attackers escalate locally.
Unrestricted network access allowing lateral movement between containers.

Now let‘s look at tactics to address these risks across the Docker stack.

Step 1: Harden Your Docker Hosts

The Docker host is the foundation, so hardening it is critical. Start by choosing a secure OS like CentOS hardened via security best practices:

Use a minimal image with only absolutely essential packages.
Enable SELinux to impose restrictions on processes via security policies.
Configure the host firewall to only allow necessary ports. SSH should only be open to your jump server‘s IP for example.
Disable any unused services to reduce the attack surface.
Keep the OS and all software like Docker Engine updated. Sign up for security bulletins.
Send OS and application logs to a central SIEM to monitor for issues.
Setup file integrity monitoring to detect unauthorized changes to binaries.

Next restrict access to the Docker daemon which runs with root privileges by default. Here are some tips:

Keep the daemon bound to a local Unix socket instead of opening a TCP port for remote access. This prevents external attacks.
If you do require remote access, use TLS client certificates to authenticate and encrypt connections.
Create a Docker group, add users to it, and reconfigure the daemon to run as that group so it loses root access.

As an extra hardening step, install Docker Bench Security which checks your configuration against dozens of security best practices. Here‘s how to run it:

git clone https://github.com/docker/docker-bench-security.git
cd docker-bench-security
sudo sh docker-bench-security.sh

It outputs failed and warned checks for things like:

Using default cgroup usage limits
Checking test containers are not deployed to production
Disabling unneeded Docker socket connections

Research the failed ones and determine which hardening steps make sense for your environment.

Lastly, limit container access to host resources which could let a single container monopolize them with a denial of service. Set CPU shares and memory limits when running containers.

You can also create control groups to allocate guaranteed resource shares for containers. This ensures availability for critical workloads.

Step 2: Secure Docker Images

Images built internally or pulled from repositories like Docker Hub can contain vulnerabilities allowing containers to be compromised on start.

Here are some best practices to secure images:

Scan Images for Vulnerabilities

Tools like Anchore Engine and Snyk can scan Docker images and dependencies for known vulnerabilities using CVE databases.

For example, Acme Corp built scanning into our CI/CD pipeline using Anchore‘s API:

# Build image 
docker build -t acme/app:v1 .

# Scan image
curl -sXPOST http://anchore/v1/images -d ‘{"image_type":"docker", "image_name":"acme/app", "force_reload": true}‘

# Retrieve analysis 
curl http://anchore/v1/images/acme/app/v1/vulnerabilities

This scans each image before deployment and rejects any builds with high or critical vulnerabilities.

Sign Images with Notary

Enabling Docker Notary lets you cryptographically sign images and verify the signature before running them.

This ensures containers only run images from trusted publishers that have not been tampered with.

To try it out:

# Initialize Notary
docker run -d -p 4443:4443 --name notary -v notarycerts:/certs docker/notary-server:latest

# Trust the built-in Notary server self-signed certs 
docker trust key generate acme
yes | docker trust signer add --key acme acme

# Enable Docker Content Trust
export DOCKER_CONTENT_TRUST=1
export DOCKER_CONTENT_TRUST_SERVER=https://10.0.0.2:4443

# Sign an image
docker trust sign acme/app:v1

Now when running docker pull it will verify the image signature matches before starting the container.

Use Vetted Base Images

Avoid using random public images as base images – who knows what they could contain!

Instead build from vetted official images like Ubuntu, Nginx, and Redis that are scanned and maintained. Or build your own base images in-house to control the contents fully.

Step 3: Secure Container Workloads

Once you have hardened hosts and images, the final step is configuring secure workloads.

Restrict Network Access

By default, containers can make arbitrary connections to internal and external networks. This provides a large attack surface.

Instead, limit access by:

Using the --network flag when running containers to place them on non-default networks with firewall rules.
Configuring the host firewall to only allow containers access to required endpoints.
Placing restrictions directly in application config files regarding outbound connections.

For example, our Nginx containers are limited to communicating on port 80/443 to internal apps, and port 443 to pull updates from the Ubuntu repos. Nothing else.

Drop Privileged Access

Avoid running containers in privileged mode which lifts many security restrictions – it should only be used when absolutely necessary.

Also be aware that containers run as root by default. Here are some steps to limit this:

Use the --cap-drop=ALL flag to remove all Linux capabilities which grant privileged access.
Make containers run as a non-root user via the --user flag. Set up users in the container or reference a user on the host.
Define a custom security profile via AppArmor, SELinux, or seccomp to enforce restrictions.

Integrate Runtime Scanning

Monitor running containers for anomalous activity indicating threats:

Use Falco to detect suspicious events at the kernel and system call level.
Feed audit logs from the Docker daemon and containers into your SIEM. Look for unusual processes, network connections, mounts, etc.
Use Sysdig Falco to scan containers and audit Kubernetes as well.

Encrypt Secrets

Hardcoding secrets like passwords and API keys into image builds is an anti-pattern. Anyone with access to the image can extract them.

Instead, use a secrets management tool like HashiCorp Vault to inject secrets at container runtime. Storing them externally also allows rotating without rebuilding images.

Make sure secrets are encrypted both at rest and in transit. Vault‘s transit engine can handle this automatically.

Apply Least Privilege

Only grant containers the minimum permissions needed to operate. For example, our app containers have read-only mounts to config files, and write mounts limited to /tmp and logging directories.

Define non-root users in the image rather than using the root account. Avoid privileged mode. This limits the blast radius if a container does get compromised.

Final Thoughts

There are definitely challenges around securing Docker, but following security best practices at the host, image, and workload layers can reduce risk substantially.

Make sure to monitor container activity at runtime and feed audits into your SIEM for threat detection. Also integrate security into the development process early so it is not an afterthought!

Let me know if you have any other tips on locking down Docker in production environments! I‘m always looking to learn. Stay safe out there!