Snippets

klassify Docker training notes

Created by Miro Adamy
Day-one.md
Raw

Docker: Up and Running Training

O'Reilly Online Training

Title: Prerequisites

  • A recent computer and OS
    • Recent Linux, OS X, or Windows 10
    • root/admin rights
    • Sufficient resources to run one 2 CPU virtual machine (VM)
    • CPU Virtualization extensions MUST be enabled in your BIOS/EFI
      • This is likely only an issue if you have NEVER run a virtual machine on your system and are using a PC-based system.
  • Reliable and fast internet connectivity
    • We will be downloading a few 100MB of docker images, etc.
  • A text editor
  • Git client
  • Docker Community Edition
  • Basic comfort with the Unix command line will be helpful.
  • [optional] wget utility
  • [optional] curl utility
  • [optional] jq utility
  • [optional] SSH client

Title: A Note for Windows Users

This class was written from a largely Unix based perspective, but everything can be made to work in Windows with very little effort.

  • Unix Variables
export MY_VAR=test; echo ${MY_VAR}
  • Windows 10 Variables (powershell)
$env:my_var = "test"
Get-ChildItem Env:my_var

Title: A Note about Proxies

  • If required, you can configure a proxy in Docker: Community Edition via the preferences.

Title: Instructor Environment

Title: The Slow Rise Of Linux Containers

  • chroot system call (1979)
    • Version 7 Unix
  • Virtuozzo (2000)
  • jail (2000)
  • FreeBSD 4.0
  • Solaris Zones (2004)
  • Solaris 10
  • OpenVZ (2005)
  • Linux Containers - LXC (2008)
    • version 2.6.24 of the Linux kernel

Title: The Quick Rise of Docker

  • Docker Engine
    • Announced March 15, 2013
  • Provided:
    • Application packaging
    • App & dependencies packed together
    • Single deployment artifact
    • Abstract software from hardware without sacrificing resources

Title: Docker Engine isn’t a…

  • virtualization platform (VMware, KVM, etc.)
  • cloud platform (AWS, Azure, etc.)
  • configuration management tool (Chef, Puppet, etc.)
  • deployment framework (Capistrano, etc.)
  • workload management tool (Mesos, Kubernetes, etc.)
  • development environment (Vagrant, etc.)

Title: Docker in Translation

  • Docker client (Docker CE) - The docker command used to control most of the Docker workflow and talk to remote Docker servers.
  • Docker server (Docker CE) - The docker command run in daemon mode. This turns a Linux system into a Docker server that can have containers deployed, launched, and torn down via a remote client.
  • Virtual Machine (Docker CE) - In general, the docker server can be only directly run on Linux. Because of this, it is common to utilize a Linux virtual machine to run Docker on other development platforms. Docker Community Edition makes this very easy.
  • Docker images - Docker images consist of one or more filesystem layers and some important metadata that represent all the files required to run a Dockerized application. A single Docker image can be copied to numerous hosts. A container will typically have both a name and a tag. The tag is generally used to identify a particular release of an image.
  • Docker containers - A Docker container is a Linux container that has been instantiated from a Docker image. A specific container can only exist once; however, you can easily create multiple containers from the same image.

Title: Exploring the Docker Machine VM

  • Based on Alpine Linux (apk)
docker run -it --cap-add SYS_ADMIN --cap-add SYS_PTRACE --pid=host debian nsenter -t 1  -m -u -n -i sh

cat /etc/os-release
exit

Title: ChatOps

  • nrrd newrelic servers name chi-staging-docker-24
    • UP chi-staging-docker-24.nr-ops.net (377631) CPU:1.57% Mem:1.17% Disk:5%
  • nrrd who's on call for container-primary-oncall
  • nrrd time
    • Server time is: Thu Jun 23 2016 03:15:17 GMT+0000 (UTC)

Title: Launching your First Container

docker images
docker run --rm -ti fedora:22 /bin/bash

cat /etc/os-release
exit

docker images
docker ps
docker run -d fedora:22 sleep 20
docker ps
docker ps -a

Title: Exploring the Dockerfile

cd ${HOME}
mkdir docker-class
cd docker-class
git clone https://github.com/spkane/hubot-docker.git --config core.autocrlf=input
cd hubot-docker

Title: Registering with Docker Hub

docker login

cat ~/.docker/config.json

{
    "auths": {
        "https://index.docker.io/v1/": {
            "auth": "q378t348q7tb78bfs387b==",
            "email": "me@example.com"
        }
    }
}

Title: Create your Image Repository

  • Login to Docker Hub
  • Click: Create Repository+
  • Enter name: mybot
  • Set visibility: public
  • Click: Create

Title: Building Your First Image

export HUB_USER=${USER}
docker images
docker build -t ${HUB_USER}/mybot:latest .
docker images
docker push ${HUB_USER}/mybot:latest
docker search ${HUB_USER}
  • Docker Hub API Examples
curl -s -S "https://registry.hub.docker.com/v2/repositories/${HUB_USER}/mybot/tags/" | jq '."results"[]["name"]' |sort
curl -s -S "https://registry.hub.docker.com/v2/repositories/strongloop/node/tags/" | jq '."results"[]["name"]' |sort

Title: Configuration

cat bin/hubot-env

#!/bin/sh
set -e
npm install
export PATH="node_modules/.bin:node_modules/hubot/node_modules/.bin:$PATH"
exec node_modules/.bin/hubot --adapter ${HUBOT_ADAPTER} --alias ${HUBOT_ALIAS} --name ${HUBOT_NAME} “$@"

Title: Configuring Your Container

  • In the Slack Channel:

    • `0 sid get``

  • SLACK_TOKEN: Ask Instructor for Slack Token

cp compose.env .env
vi .env

*Edit the STUDENT_ID, SLACK_TOKEN, & HUB_USER variables.

Title: Composing a Docker Service

alias dc='docker-compose'

For Windows (cmd) try: doskey dc=docker-compose.exe $* For Windows (powershell) try: New-Alias dc docker-compose.exe

Title: Launching the Composition

docker-compose config
docker-compose build
docker-compose up
  • <control>-C
docker-compose up -d
docker ps

Title: Testing the Slack Bot

${STUDENT_ID} help
${STUDENT_ID} meme Brace yourselves Docker is here!

docker ps
  • http://${DOCKER_IP}:8080/time
    • ${STUDENT_ID}_bot
    • ${STUDENT_ID}_pw
docker-compose stop

${STUDENT_ID} help

docker ps -a

Title: What Have We Learned?

  • What Linux containers are.
  • What Docker is.
  • What a Dockerfile is.
  • How to create a Docker image
  • How to use Docker Hub
  • How to launch a Docker container
  • How to use environment variables
  • How to use Docker Compose

Title: Bonus Points

git clone https://github.com/spkane/wearebigchill.git --config core.autocrlf=input
cd wearebigchill
  • Read the README
  • Build it
  • Run it (hint: -p 8090:80)
  • Play the game
  • Kill the container
  • Use an env var to change the theme, and test it

Title: Tour the Code

  • README.md
  • What is this code?
  • Dockerfile
  • How is the container built?
  • httpd.conf
  • Apache httpd configuration
  • publish/html5/theme1/*
  • First set of images
  • publish/html5/theme2/*
  • Second set of images

Title: Binding to Ports

  • Exposing container ports to the outside world.
    • docker run --rm -p 8090:80 ${IMAGE}
  • This exposes port 80 in the container to port 8090 on the external docker host.
  • Note: Some web browsers cache parts of this app incorrectly. If you are having trouble seeing changes you have made, try testing your solution in another web browser.

Title: Image Credits

Title: Instructor

Sean P. Kane Twitter: @spkane

Day-two.md
Raw

Docker: Up and Running Training

O'Reilly Online Training

Title: The Solution

git clone https://github.com/spkane/wearebigchill.git --config core.autocrlf=input
cd wearebigchill
docker build .
docker run --rm -p 8090:80 ${IMAGE}
docker build .
docker run --rm -p 8090:80 -e "THEME=2" ${IMAGE}

Title: Keep it Small

  • Each and every layer’s size matters
    • Don’t install unnecessary files
docker run -d -p 8070:8080 adejonge/helloworld
docker ps
docker export ${CONTAINER} -o export.tar
tar -tvf export.tar

Title: Smart Layering

  • Each and every layer’s size matters
    • Clean up, inside of each step.
cd wearebigchill
docker build .
docker tag ${IMAGE_ID} size${#}
docker history size${#}

RUN dnf install -y httpd
RUN dnf clean all
RUN dnf install -y httpd && dnf clean all

Title: Order Matters

  • Keep commands that change towards the end of your Dockerfile.
cd wearebigchill
time docker build --no-cache .
time docker build .
vi boring_404.html
time docker build .
  • Add to the top of Dockerfile:
RUN mkdir -p /var/www/html
ADD boring_404.html /var/www/html/

time docker build --no-cache .
time docker build .
vi boring_404.html
time docker build .
  • Note: Windows user can try something like this for timing:
$t = Measure-Command { docker build --no-cache . }
Write-Host That command took $t.TotalSeconds to complete.

Title: Re-Launching Your Container

cd hubot-docker
docker-compose up -d
docker ps

${STUDENT_ID} help
${STUDENT_ID} meme dg Docker Training~Day Two

Title: Accessing STDOUT/STDERR

  • Log everything to STDOUT / STDERR
  • Avoid writing logs into file
docker ps
docker logs --help
docker logs —f ${CONTAINER}

${STUDENT_ID} help

docker logs -t ${CONTAINER}

Title: Statistics

docker stats --help
docker stats ${CONTAINER}
docker stats --no-stream ${CONTAINER}
docker top ${CONTAINER}

curl --no-buffer -XGET --unix-socket /var/run/docker.sock http:/containers/${CONTAINER}/stats

Title: Events

docker events --help

docker events --since 20160628

curl --no-buffer -XGET --unix-socket /var/run/docker.sock http:/events

Title: Debugging an Image

  • If your image has a shell installed you can access it using a command like this:
docker images
docker run --rm -ti ${IMAGE} /bin/bash

ls /data/app
cat /data/app/package.json
exit

Title: Debugging a Live Container

  • If your container has a shell installed you can access it using a command like this:
docker ps
docker exec -ti ${CONTAINER} /bin/bash

ps auxwww
exit

Title: Linux Namespaces

  • Mount (filesystem resources)
  • UTS (host & domain name)
  • IPC (shared memory, semaphores)
  • PID (process tree)
  • Network (network layer)
  • User (user and group IDs)

Title: Control Groups (cgroups)

  • Resource limiting
  • Prioritization
  • Accounting
  • Control

Title: Stressing the System

docker run --rm -ti spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 120s 
docker run -it --pid=host alpine sh

apk update
apk add htop
htop -p $(pgrep stress | tr '\n' ',')
exit

Title: CPU Quotas

docker run -d --cpu-shares=512 spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s

docker run -d --cpuset-cpus="0" spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s

docker run -d --cpus=".25" spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s

docker run -d spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s && docker run -d spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s

```docker run -d --cpus="1.5" spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s && docker run -d --cpu-shares=512 —cpus=".5" spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s

docker run -d --cpus="5" spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 60s

### Title: Memory Quotas

docker run --rm -ti --memory="512m" spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 10s docker run --rm -ti --memory="100m" spkane/train-os:latest stress -v --cpu 2 --io 1 --vm 2 --vm-bytes 128M --timeout 10s docker run -it --cap-add SYS_ADMIN --cap-add SYS_PTRACE --pid=host debian nsenter -t 1 -m -u -n -i sh

dmesg exit

### Title: I/O Quotas

docker pull spkane/train-os:latest time docker run -ti --rm spkane/train-os:latest bonnie++ -u 500:500 -d /tmp -r 1024 -s 2048 -x 1 time docker run -ti --rm --device-write-iops /dev/sda:256 spkane/train-os:latest bonnie++ -u 500:500 -d /tmp -r 1024 -s 2048 -x 1 time docker run -ti --rm --device-write-bps /dev/sda:5mb spkane/train-os:latest bonnie++ -u 500:500 -d /tmp -r 1024 -s 2048 -x 1

**Note**: Windows user can try something like this for timing:

$t = Measure-Command { docker run -ti --rm spkane/train-os:latest bonnie++ -u 500:500 -d /tmp -r 1024 -s 2048 -x 1 } Write-Host That command took $t.TotalSeconds to complete.

### Title: Container UID

docker run spkane/train-os:latest whoami docker run -u 500 spkane/train-os:latest whoami docker run -u 99 spkane/train-os:latest whoami docker run -v /etc:/mnt/etc spkane/train-os:latest cat /mnt/etc/docker/key.json docker run -v /etc:/mnt/etc -u 500 spkane/train-os:latest cat /mnt/etc/docker/key.json

* Further Exploration: `--userns-remap`
   * https://docs.docker.com/engine/security/userns-remap/

### Title: Privileged Containers

docker run --rm -ti spkane/train-os:latest /bin/bash

ip link ls ip link set eth0 address 02:0a:03:0b:04:0c exit

docker run --rm -ti --privileged=true spkane/train-os:latest /bin/bash

ip link ls ip link set eth0 address 02:0a:03:0b:04:0c ip link ls exit

### Title: Limited Privileges

docker run -ti --rm spkane/train-os:latest ntpdate 0.fedora.pool.ntp.org docker run -ti --rm --privileged=true spkane/train-os:latest ntpdate 0.fedora.pool.ntp.org docker run -ti --rm --privileged=true spkane/train-os:latest bash -c "mount /dev/sda1 /mnt && ls -F /mnt/docker/volumes | head -n 10" docker run -ti --rm --cap-add=SYS_TIME spkane/train-os:latest ntpdate 0.fedora.pool.ntp.org docker run -ti --rm --cap-add=SYS_TIME spkane/train-os:latest bash -c "mount /dev/sda1 /mnt && ls -F /mnt/docker/volumes | head -n 10" docker run -ti --rm spkane/train-os:latest tcpdump -i eth0 docker run -ti --rm --cap-drop=NET_RAW spkane/train-os:latest tcpdump -i eth0

* Capabilities list: 
   * http://man7.org/linux/man-pages/man7/capabilities.7.html

### Title: Secure Computing Mode

docker run -ti --rm spkane/train-os:latest strace whoami docker run -ti --rm --cap-add=SYS_PTRACE spkane/train-os:latest strace whoami

* https://docs.docker.com/engine/security/seccomp/

wget https://raw.githubusercontent.com/spkane/train-os/master/default.json wget https://raw.githubusercontent.com/spkane/train-os/master/strace.json diff -u default.json strace.json docker run -ti --rm --security-opt=seccomp:strace.json spkane/train-os:latest strace whoami ```

Title: Future Exploration

Title: What Have We Learned?

  • Advanced Dockerfile Techniques
  • Docker logging
  • Docker statistics
  • Docker events
  • Debugging Images & Containers
  • Linux Namespaces
  • Linux Control Groups
  • Resource Limits
  • Privileged Containers
  • Secure Computing Mode

Title: Additional Reading

Title: Additional Learning

Title: Image Credits

Title: Instructor

Sean P. Kane Twitter: @spkane

Title: Survey

  • Please take a moment to fill out the class survey linked to in the Slack channel.

  • O’Reilly and I value your comments about the class.

  • Thank you!

Instructor: Sean P. Kane

  • Twitter: @spkane
Setup
Raw 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#mkdir docker-class
#cd docker-class
docker pull debian:latest
docker pull fedora:22
docker pull redis:3.2.5
docker pull node:6.11.3
docker pull spkane/train-os:latest
docker pull adejonge/helloworld
git clone https://github.com/spkane/hubot-docker.git
git clone https://github.com/spkane/wearebigchill.git
wget https://raw.githubusercontent.com/spkane/train-os/master/default.json # optional
wget https://raw.githubusercontent.com/spkane/train-os/master/strace.json # optional

➜  docker-up-running-sean-kane pwd
/Users/miro/src/PLG/docker-up-running-sean-kane

Comments (0)

HTTPS SSH

You can clone a snippet to your computer for local editing. Learn more.