Docker & Git Essentials

The Docker Series: What Docker Is

By this point in the book you have trained models, but you have not yet shipped one. The hardest part of deploying machine learning is not the model itself; it is reproducing the exact environment where the model ran. A model that works on your laptop can fail instantly on a colleague's machine because a library version differs by one digit.

Docker solves this by packaging an application together with everything it needs to run: the code, the runtime, libraries, environment variables, and configuration. The resulting bundle is called a container. Containers are lightweight, isolated environments that run on any machine that has Docker installed, so a container built once runs identically on a laptop, a server, or in the cloud.

Memory hook: The old slogan "it works on my machine" becomes irrelevant. With Docker, your container is the machine, so "works" travels with the container.

Images vs Containers

Beginners often mix up the two central terms. An image is an immutable template; a container is a running instance of that template.

Think of an image as a recipe or a class definition.
Think of a container as a finished dish, or an object created from that class.
You can launch many containers from one image, and each container is independent. Stopping or deleting one container never touches the image.

You build images with a file called Dockerfile, then docker run starts a container from that image whenever you need it.

Docker vs Virtual Machines

Virtual machines (VMs) also isolate environments, but the two approaches work very differently under the hood:

A VM emulates an entire operating system on top of a hypervisor. Each VM carries its own full guest OS, which consumes gigabytes of disk and a lot of RAM.
A container shares the host operating system kernel. Containers start in seconds and use only the resources of the processes inside them, so a single server can run dozens of containers comfortably.

Neither tool replaces the other. Teams often run VMs for strict isolation and infrastructure boundaries, while containers become the default for packaging applications.

Installing Docker

Download Docker Desktop from the official Docker website for Windows or macOS. Linux users install the Docker Engine package through their package manager.
On Windows, Docker Desktop requires the Windows Subsystem for Linux (WSL 2). The installer guides you through enabling it.
After installation, open a terminal and verify the install: docker --version.
Confirm the engine is running with docker info, or by checking the Docker Desktop icon in the system tray.

Heads-up: Docker Desktop is a graphical application. Even if you run everything from the terminal, Docker Desktop must stay running in the background, otherwise commands return a "cannot connect to the Docker daemon" error.

Essential Docker Commands

The daily workflow of a Docker user revolves around a handful of commands:

docker pull python:3.11       # download an image
docker run python:3.11        # create and start a container
docker ps                     # list running containers
docker ps -a                  # list all containers, stopped or running
docker stop friendly_lewis    # stop a container by name or ID
docker build -t my-app .      # build an image from a Dockerfile
docker exec -it my-app bash   # open a shell inside a running container
docker images                 # list local images

docker pull downloads an image from a registry such as Docker Hub so it is available locally.
docker run creates a fresh container from an image. Add -it for interactive use and --rm to delete the container when it exits.
docker exec runs a new command inside an already running container, which is how you debug a live service.
Containers get random names like friendly_lewis unless you pass --name.

Creating a Docker Image with a Dockerfile

A Dockerfile is a plain text file of instructions that Docker executes top to bottom to build an image. A minimal machine learning example looks like this:

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "app.py"]

FROM names the base image you start from.
WORKDIR sets the working folder inside the container.
COPY moves files from your computer into the image.
RUN executes commands while the image is being built, such as installing packages.
CMD declares the command that runs when a container starts from the image.

Build the image from the same folder as the Dockerfile:

docker build -t my-app:latest .

Pushing an Image to Docker Hub

To share an image, upload it to a registry. Docker Hub is the default public registry.

Create a free account at hub.docker.com and log in from the terminal: docker login.
Tag the image with your username so the registry knows where it belongs: docker tag my-app username/my-app:v1.
Upload it: docker push username/my-app:v1.
Anyone can then pull it: docker pull username/my-app:v1.

Use descriptive tags such as v1.0 or latest; the tag is the version label of the image.

Docker Compose

Real projects rarely consist of a single container. A web app might need an API container, a database container, and a model-serving container together. Docker Compose lets you describe the whole stack in one file and start everything with a single command.

version: "3"
services:
  web:
    build: .
    ports:
      - "8000:8000"
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret

Run docker compose up to build and start all services, and docker compose down to stop them. Compose wires the services together on a shared network, so the web service can reach the database simply by its service name.

Heads-up: Older Docker versions used the command docker-compose (with a hyphen) as a separate binary. Modern versions prefer docker compose as a sub-command. The syntax is otherwise the same.

Git: Version Control for Everything

While Docker solves the "runs everywhere" problem, Git solves the "who changed what and when" problem. Git is a distributed version control system that records the complete history of a project. Every change becomes a commit, and you can always compare, revert, or branch the history.

For machine learning, Git matters because experiments multiply quickly. A model trained on a different data split, a refactored preprocessing step, a changed seed, and suddenly you cannot remember which version produced your best result. Git gives you a timestamped, labelled history of every step.

Installing Git

Download the Git installer from the official Git website and run it, accepting the defaults for a beginner.
Verify the installation in a new terminal: git --version.
Set your identity once, because every commit records who made it:

git config --global user.name "Your Name"
git config --global user.email "you@example.com"

First Steps: init, add, commit, status

Start versioning a folder by turning it into a repository:

git init                 # create a new repository in the current folder
git status               # show what has changed since the last commit
git add model.py         # stage a single file
git add .                # stage every changed file
git commit -m "add model script"

git init creates a hidden .git folder that stores all history.
git status is the command you will run the most. It lists files in three states: staged, modified, and untracked.
git add moves changes into the staging area, a draft tray of what the next commit will contain.
git commit permanently records the staged snapshot with a message describing the change.

Never commit secrets or large binary files. Add a .gitignore file listing patterns such as *.csv or .env so Git ignores them.

merge, push, checkout, and log

As projects grow you create branches: separate lines of development. The default branch is usually called main. The commands below cover the essentials:

git log --oneline         # show the commit history in compact form
git checkout feature       # switch to an existing branch
git checkout -b experiment # create and switch to a new branch
git merge feature          # fold the feature branch into the current branch
git push origin main       # upload local commits to a remote repository

git log shows history; --oneline prints one compact line per commit.
git checkout moves you between branches so you can work on different versions of the same code.
git merge combines the changes of another branch into your current one.
git push synchronises your local commits to a remote such as GitHub, which we use heavily in the next chapter.

Tip: A very common pattern is git checkout -b to start a branch, commit your work, then git checkout main and git merge the branch back. Branches are cheap, so create one for every idea.

Resolving a Merge Conflict

A merge conflict occurs when two branches changed the same lines of the same file in different ways, and Git cannot decide which version wins. Git does not panic; it pauses the merge, marks the conflicting file, and lets you decide.

The conflicted file contains markers that Git inserts. A snippet of a Python file might look like this:

def learning_rate():
<<<<<<< HEAD
    return 0.001
=======
    return 0.01
>>>>>>> tuning-branch

Resolving the conflict is a manual edit:

Open the file in your editor and read both versions.
Delete the conflict markers (<<<<<<<, =======, >>>>>>>) and keep the code you want, or combine both ideas.
Stage the resolved file: git add filename.py.
Finish the merge: git commit. Git supplies a default merge message you can keep or edit.

Conflicts look intimidating the first time, but they are a normal part of collaborative development. The golden rule is to read the two sides, choose deliberately, and never delete a colleague's work without saying so.

Exercise: Build a minimal Docker image that runs python -c "print('containerized!')", run it, and push it to Docker Hub under your username. Then create a Git repository for a small project, make two branches that edit the same line, merge them to provoke a conflict, and resolve it. Confirm with git log --oneline that both branches are now part of the history.