Docker Orchestration

Docker Orchestration

by Randall Smith
Released January 2017
Publisher(s): Packt Publishing
ISBN: 9781787122123

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

A concise, fast-paced guide to orchestrating and deploying scalable services with Docker

About This Book

  • Explore the new features added to the core Docker Engine to make multi-container orchestration easy
  • Leverage tools such as Docker Machine, Swarm, Compose, and third-party tools such as Kubernetes, Mesosphere, and CoreOS to orchestrate containers
  • Use Docker Compose with Swarm and apply rolling updates for zero downtime deployments

Who This Book Is For

This book is aimed at Sysadmins and DevOps engineers who know what Docker does and are now looking to manage multiple containers on multiple hosts using the orchestration feature.

What You Will Learn

  • Build scalable, reliable services with Docker
  • See how to manage a service in Docker using Docker Swarm, Kubernetes, and Mesosphere
  • Discover simpler orchestration tools such as CoreOS/Fleet and Rancher Cattle
  • Understand cluster-wide logging, system monitoring, and troubleshooting
  • Build, test, and deploy containers using Continuous Integration
  • Deploy cluster hosts on cloud services and automate your infrastructure

In Detail

Docker orchestration is what you need when transitioning from deploying containers individually on a single host to deploying complex multi-container apps on many machines.

This book covers the new orchestration features of Docker 1.12 and helps you efficiently build, test, and deploy your application using Docker. You will be shown how to build multi-container applications using Docker Compose. You will also be introduced to the building blocks for multi-host Docker clusters such as registry, overlay networks, and shared storage using practical examples.

This book gives an overview of core tools such as Docker Machine, Swarm, and Compose which will enhance your orchestration skills. You'll learn how to set up a swarm using the decentralized building block. Next, you'll be shown how to make the most out of the in-built orchestration feature of Docker engine and you'll use third-party tools such as Kubernetes, Mesosphere, and CoreOS to orchestrate your existing process. Finally, you will learn to deploy cluster hosts on cloud services and automate your infrastructure.

Style and approach

This comprehensive guide will take you through the orchestration feature of Docker. Using practical examples, you will discover various tools that can be used to manage multiple containers with ease.

Table of contents

  1. Docker Orchestration
    1. Docker Orchestration
    2. Credits
    3. About the Author
    4. About the Reviewer
    5. www.PacktPub.com
      1. Why subscribe?
    6. Customer Feedback
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Downloading the color images of this book
        3. Errata
        4. Piracy
        5. Questions
    8. 1. Getting Started with Docker Orchestration
      1. Installing Docker Engine
      2. Installing with Docker Machine
        1. Starting a host on AWS
        2. Starting a host on GCE
        3. Starting a host on Microsoft Azure
        4. Installing on a running Linux host
      3. Introducing Docker-specific distributions
        1. CoreOS
        2. RancherOS
        3. Project Atomic / RHEL Atomic
      4. Running single container applications
        1. What is running?
      5. Inspecting your container
      6. Summary
    9. 2. Building Multi-Container Applications with Docker Compose
      1. Building an image with Docker Engine
        1. Building from a Dockerfile
        2. Tagging the image
        3. Skipping the build cache
        4. Running the image
      2. Installing Docker Compose
      3. Writing a Docker Compose file
      4. Multi-container applications
        1. Using environment files
      5. Extending compose files
        1. Controlling start order
      6. Using Docker networks
      7. Keeping your data safe in volumes
      8. Summary
    10. 3. Cluster Building Blocks – Registry, Overlay Networks, and Shared Storage
      1. Creating a Docker Registry
        1. Using the Docker Hub
          1. Logging in
          2. Working with a repository
          3. Automating image builds with Docker Hub
          4. Integration with Docker Cloud
        2. Using the GitLab Container Registry
          1. Installing GitLab on Ubuntu
          2. Enabling Docker Registry on a project
        3. Introducing the Docker Trusted Registry and Docker Datacenter
        4. Building it yourself
      2. Connecting containers with overlay networks
        1. Using Docker native overlays
          1. Working with DNS-SD
          2. Publishing services with mesh routing mode
        2. Using Weave
          1. Installing Weave
          2. Connecting containers to Weave
        3. Using Flannel
          1. Configuring etcd and Flannel
          2. Starting a CoreOS cluster with Flannel
          3. Connecting to Flannel
      3. Using shared network storage for Docker volumes
        1. Introducing Ceph
        2. Using Ceph with Docker
          1. Creating a Docker volume
        3. Other shared storage plugins
      4. Summary
    11. 4. Orchestration with Docker Swarm
      1. Setting up a swarm
        1. Initializing a swarm
      2. Managing a swarm
        1. Adding a node
        2. Promoting and demoting nodes
        3. Changing node availability
          1. Pausing a node
          2. Draining a node
          3. Activating a node
        4. Removing nodes
      3. Recovering from a disaster
        1. Restarting the full cluster
        2. Backup and recovery
          1. Backing up the swarm
          2. Recovering a swarm
          3. Backing up services
        3. Grouping nodes with labels
      4. Managing services
        1. Running services
          1. Creating replicas
          2. Running global services
          3. Setting constraints
          4. Stopping services
          5. Upgrading a service with rolling updates
        2. Using Docker Compose with swarm
          1. Building images
          2. Starting an application
          3. Bundling an application
          4. Deploying a stack
      5. Introducing Docker Datacenter
      6. Summary
    12. 5. Deploying and Managing Services with Kubernetes
      1. Getting to know Kubernetes
        1. Command-line tools
        2. Master components
        3. Node components
      2. Installing Kubernetes
        1. Installing with kubeadm
        2. Installing with kube-up.sh
        3. Installing add-ons
      3. Managing resources
        1. Creating resources
        2. Deleting resources
      4. Running pods
        1. Defining a pod
          1. Viewing pods
          2. Environment
          3. Multi-container pods
          4. Using secrets
            1. Creating secrets
            2. Using secrets in environment variables
            3. Using secrets in files
          5. Logging into a container
          6. Deleting a pod
        2. Using deployments
          1. Creating a single container deployment
          2. Creating a multi-container deployment
          3. Scaling a deployment
          4. Rolling updates
            1. Updating a deployment
            2. Getting the rollout history
            3. Rolling back an update
          5. Deleting a deployment
        3. Running pods on every node with DaemonSets
        4. Introducing scheduled jobs
        5. Using labels
          1. Using selectors to find resources
        6. Using namespaces
          1. Viewing namespaces
          2. Creating namespaces
          3. Switching namespaces
          4. Deleting a namespace
      5. Networking
        1. Services
          1. Creating a service
          2. Listing services
        2. Introducing DNS-SD
      6. Using volumes
        1. Using plain volumes
        2. Storing long-lived data in persistent volumes
          1. Creating a PersistentVolume
          2. Creating the PersistentVolumeClaim resource
          3. Using a PersistentVolumeClaim as a volume
      7. Further reading
      8. Summary
    13. 6. Working with Mesosphere
      1. Getting started with DC/OS
        1. Logging into the web interface
        2. Installing the DC/OS client
      2. Managing applications
        1. Running a simple application
        2. Stopping an application
        3. Removing an application
        4. Forcing Marathon to pull images
        5. Using environment variables
        6. Scaling applications
        7. Checking application health
          1. Using TCP and HTTP checks
          2. Using command checks
        8. Deploying rolling updates
        9. Setting an upgrade strategy
          1. Testing updates with blue-green deployments
        10. Introducing pods
      3. Using network services and load balancing
        1. Discovering services with DNS
        2. Connecting to the overlay network
        3. Using virtual IPs
        4. Using the Marathon load balancer
      4. Providing persistent storage
        1. Using local volumes
        2. Using external volumes
      5. Using a private registry
      6. Summary
    14. 7. Using Simpler Orchestration Tools – Fleet and Cattle
      1. Using Fleet
        1. Starting Fleet
        2. Verifying the cluster
        3. Starting a container
        4. Monitoring services
        5. Viewing service logs
        6. Stopping a service
        7. Using environment variables
        8. Scheduling services with affinity
          1. Setting positive affinity
          2. Using negative affinity
          3. High availability
        9. Sidekick services
        10. Starting global units
      2. Using Rancher Cattle
        1. Installing Rancher
          1. Installing Rancher server
          2. Installing Rancher agents
          3. Installing Rancher Compose
          4. Introducing environments
        2. Managing services
          1. Starting a stack
          2. Stopping a stack
          3. Removing a stack
          4. Upgrading a stack
          5. Rolling back and confirming upgrades
          6. Checking service health
          7. Scheduling and affinity
        3. Using networking services
        4. Discovering services
        5. Load balancer
      3. Further reading
      4. Summary
    15. 8. Monitoring Your Cluster
      1. Logging with containers
        1. Using Docker logging plugins
        2. Logging containers that do not play by the rules
        3. Logging with the ELK stack
          1. Starting Elasticsearch
          2. Starting Logstash
          3. Viewing the logs with Kibana
        4. Remote logging in Docker Datacenter
        5. Logging from Kubernetes
        6. Logging from Mesosphere
        7. Logging your Docker hosts
      2. Collecting and graphing performance data
        1. Collecting data with collectd
          1. Running collectd
          2. Configuring collectd
            1. Storing performance data in Elasticsearch
            2. Storing performance data in InfluxDB
          3. Graphing data with Kibana
            1. Creating a new chart
            2. Adding charts to a dashboard
        2. Graphing data with Grafana
          1. Installing Grafana
          2. Adding data source
          3. Creating a new dashboard
          4. Creating a graph
        3. Using Heapster in Kubernetes
      3. Considerations for monitoring system health
        1. Monitor services
        2. Watching services in real time with Sysdig
      4. Summary
    16. 9. Using Continuous Integration to Build, Test, and Deploy Containers
      1. The importance of using CI
      2. Using GitLab for CI
        1. Setting up GitLab for CI
        2. Installing the GitLab Runner on Ubuntu
        3. Registering a runner
        4. Enabling a Docker Repository
        5. Adding the project to the repository
        6. Creating .gitlab-ci.yml
        7. Testing the image
        8. Cleanup jobs
        9. Releasing the image
        10. Deploying the image
        11. Deploying the image manually
        12. Deploying to Kubernetes
        13. Building DAB files
      3. Using Docker Cloud for CI
        1. Enabling autobuilds
        2. Testing the image
      4. Summary
    17. 10. Why Stop at Containers? Automating Your Infrastructure
      1. Configuring Docker hosts
        1. Using configuration management
        2. Configuring a server on boot
        3. Using a custom image
          1. Building a new image with Packer
            1. Preparing the Ubuntu Cloud Image
            2. Building the image
          2. Using Puppet with Packer
          3. Treating infrastructure like code
          4. Post-build configuration
      2. Automating host deployment
        1. Deploying hosts with Terraform
          1. Creating a Terraform configuration
          2. Running Terraform
          3. Showing the cluster state
        2. Deploying hosts with Docker Cloud
        3. Introducing Docker for AWS and Docker for Azure
      3. Scaling down nodes
      4. Summary

Product information

  • Title: Docker Orchestration
  • Author(s): Randall Smith
  • Release date: January 2017
  • Publisher(s): Packt Publishing
  • ISBN: 9781787122123