Dockerize a Flask, Celery, and Redis Application with Docker Compose
Learn how to install and use Docker to run a multi-service Flask, Celery and Redis application in development with Docker Compose.
Quick Jump: What Is Docker and Why Is It Useful? | Installing Docker | Creating the Flask Application | Dockerize the Flask Application | Run the Flask Application
After this tutorial, you’ll understand what the benefits of using Docker are and will be able to:
- Install Docker on all major platforms in 5 minutes or less
- Clone and run an example Flask app that uses Celery and Redis
- Know how to write a Dockerfile
- Run multiple Docker containers with Docker Compose
Also, there’s a free email course to learn a bit about Docker at the bottom of this post.
What Is Docker and Why Is It Useful?
Docker allows you to package up an application or service with all of its dependencies into a standardized unit. This unit is typically labeled as a Docker image.
Everything the application needs to run is included. The Docker image contains the code, runtime, system libraries and anything else you would install on a server to make it run if you weren’t using Docker.
To get a better idea of how Docker will affect you on a day to day basis as a software developer I highly recommend you read one of my previous blog posts which will save you from years of turmoil by using Docker.
There’s also another post I wrote which directly compares setting up a Python development environment with and without Docker. It focuses on developing a web app specifically.
Installing Docker
The code base we’ll be working with is compatible with all modern versions of Docker and Docker Compose. Feel free to install the latest stable release of Docker.
This guide expects you to have Docker already installed. If you’re at ground 0 then you may want to sign up for the free Docker email course at the bottom of this post, because it covers a myriad of ways to install Docker on Mac, Windows and Linux.
Otherwise, feel free to check out Docker’s documentation on installing Docker if you want to get going right now. To help you out, I’ve also written a comparison guide on Docker for Mac / Windows vs Docker Toolbox.
Ensure Docker and Docker Compose Are Working
Before continuing on you should see what I see, or something very similar:
docker --version
> Docker version 18.09.1, build 4c52b90
docker-compose --version
> docker-compose version 1.24.0, build 0aa59064
Creating the Flask Application
We’re going to be using the open source version of the application in my Build a SAAS App with Flask course.
The open source version only covers a tiny fraction of what the course covers, but it will be more than enough to exercise how to use Docker in development.
Clone the Project
git clone https://github.com/nickjj/build-a-saas-app-with-flask
# Make sure we're in the correct directory.
cd build-a-saas-app-with-flask
git checkout 8804f0d899c9908a6cc45756a00f284daba462f5
The reason we’re checking out that commit is because I’ve done many free updates to this course and the current release is way newer than this blog post.
So much has changed since then, and I wanted to make sure you can still follow along in this post. You can always upgrade afterwards.
Open the Project in Your Favorite Code Editor
# Copy the example env file since the real .env file is ignored by git
cp .env.example .env
# Open the project with your favorite editor (mine is Sublime)
subl .
Feel free to use whatever editor you want, but if you like Sublime Text 3 and you want to configure it for Python, Docker and more then check out my post on 25 Sublime Text 3 Packages for Polyglot Programmers.
Dockerize the Flask Application
There’s a few things we need to do to Dockerize the application.
Logging
In order for logs to function properly, Docker expects your application or process to log to STDOUT. Lucky for us, Flask does this by default.
Docker Specific Files
The root of the project has a few files that are related to Docker:
nick@oriath:/tmp/bsawf ((HEAD detached at 8804f0d)) ⚡ ls -la
-rwxrwxr-x 1 nick nick 643 Jun 10 12:57 docker-compose.yml
-rwxrwxr-x 1 nick nick 346 Jun 10 12:57 Dockerfile
-rw-rw-r-- 1 nick nick 19 Jun 10 12:57 .dockerignore
-rwxrwxr-x 1 nick nick 31 Jun 10 12:57 .env
The only file that’s necessary to add is the Dockerfile but you’ll find that most web applications that are Docker-enabled will have the others.
Dockerfile
Let’s start off with the Dockerfile because to talk about the other files will require having a little bit of knowledge about how Docker images get built.
You can think of this file as your Docker image blueprint or recipe. When you run the docker build command it will execute each line from top to bottom.
It’s going to run all of these commands in the context of the Docker image.
To get a better understanding of this file, then check out my shiny new Dive into Docker course (which is even more up to date than this article).
FROM python:3.7.4-slim-buster
LABEL maintainer="Nick Janetakis <nick.janetakis@gmail.com>"
WORKDIR /app
COPY requirements.txt requirements.txt
ENV BUILD_DEPS="build-essential" \
APP_DEPS="curl libpq-dev"
RUN apt-get update \
&& apt-get install -y ${BUILD_DEPS} ${APP_DEPS} --no-install-recommends \
&& pip install -r requirements.txt \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /usr/share/doc && rm -rf /usr/share/man \
&& apt-get purge -y --auto-remove ${BUILD_DEPS} \
&& apt-get clean
ARG FLASK_ENV="production"
ENV FLASK_ENV="${FLASK_ENV}" \
PYTHONUNBUFFERED="true"
COPY . .
RUN pip install --editable .
EXPOSE 8000
CMD ["gunicorn", "-c", "python:config.gunicorn", "snakeeyes.app:create_app()"]
At this point we could build the image and you’d be able to access the Flask app, but let’s avoid doing that for now.
.dockerignore
Let’s first look at the next file which is the .dockerignore file.
.git
.dockerignore
When we copied in all of the files from our current directory into the Docker image with the COPY . . command, it’s going to copy literally everything.
That’s not the best idea in the world because if your project is a git repo you’re going to have a TON of extra data. You should strive to have the smallest Docker images you can within reason.
The .dockerignore file is very similar to a .gitignore file. It lets you black list certain folders or files from being included.
In our case, we’re ignoring the git folder but we’re also excluding the .dockerignore file itself because it’s not part of our Flask application.
docker-compose.yml
Docker Compose is an official tool supplied by Docker. At its core, it’s a utility that lets you “compose” Docker commands and manage multiple containers in an easy way.
Let’s take a glance at the docker-compose.yml file:
version: "3.4"
services:
redis:
env_file:
- ".env"
image: "redis:5.0.4-stretch"
restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
stop_grace_period: "${DOCKER_STOP_GRACE_PERIOD:-3s}"
volumes:
- "redis:/data"
web:
build:
context: "."
args:
- "FLASK_ENV=${FLASK_ENV:-production}"
depends_on:
- "redis"
env_file:
- ".env"
healthcheck:
test: "${DOCKER_HEALTHCHECK_TEST:-curl localhost:8000/healthy}"
interval: "60s"
timeout: "3s"
start_period: "5s"
retries: 3
ports:
- "${DOCKER_WEB_PORT:-127.0.0.1:8000}:8000"
restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
stop_grace_period: "${DOCKER_STOP_GRACE_PERIOD:-3s}"
volumes:
- "${DOCKER_WEB_VOLUME:-./public:/app/public}"
worker:
build:
context: "."
args:
- "FLASK_ENV=${FLASK_ENV:-production}"
command: celery worker -B -l info -A snakeeyes.blueprints.contact.tasks
depends_on:
- "redis"
env_file:
- ".env"
restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
stop_grace_period: "${DOCKER_STOP_GRACE_PERIOD:-3s}"
volumes:
- "${DOCKER_WEB_VOLUME:-./public:/app/public}"
volumes:
redis: {}
It looks a lot more complicated than it is. All of those variables wrapped in ${} are coming in from the .env file which we’ll see below.
Dive Into Docker covers everything in great detail if you want to see how all of this ties together. Often times knowing the “why” is more important than seeing how it’s done. That is what enables you to apply things on your own.
.env
This file isn’t technically part of Docker, but it’s used by Docker Compose.
By default Docker Compose will look for an .env file in the same directory as your docker-compose.yml file.
We can set various environment variables here, and you can even add your custom environment variables here too if your application uses ENV variables.
COMPOSE_PROJECT_NAME=snakeeyes
# ...I omit the others since you can check them out in the .env file itself
By setting the COMPOSE_PROJECT_NAME to snakeeyes, Docker Compose will automatically prefix our Docker images, containers, volumes and networks with snakeeyes.
In addition to COMPOSE_PROJECT_NAME you’ll notice many other env variables. Each of them are commented, so feel free to check them out on your own.
Run the Flask Application
You can run everything by typing: docker-compose up --build. Docker Compose has many different sub-commands and flags. You’ll definitely want to check them out on your own.
After the up command finishes, open up a new terminal tab and check out what was created on your behalf.
Docker Images
Run docker images:
snakeeyes_worker latest ... 214 MB
snakeeyes_web latest ... 214 MB
redis 5.0.7-alpine ... 29.8 MB
python 3.7.4-slim-buster ... 179 MB
Docker Compose automatically pulled down Redis and Python for you, and then built the Flask (web) and Celery (worker) images for you.
Docker Containers
Run docker-compose ps:
Name State Ports
----------------------------------------------------------
snakeeyes_redis_1 ... Up 6379/tcp
snakeeyes_web_1 ... Up 0.0.0.0:8000->8000/tcp
snakeeyes_worker_1 ... Up 8000/tcp
Docker Compose automatically named the containers for you, and it appended a _1 because it’s running 1 instance of the Docker image. Docker Compose supports scaling but that goes beyond the scope of this tutorial.
We can also see which ports the services are using. Only the web service has port 8000 published in such a way that you can access it in a browser. The other ports you see for Redis and the worker are acting as documenting for which ports it could technically publish if you wanted to.
There’s a lot more to go over but the above is enough to get rolling.
Viewing the Site
If you’re using Docker Desktop (Mac / Windows) or native Linux you can visit http://localhost:8000 in your browser and you should see the home page.
At this point you have a Dockerized Flask application running. Congrats!
If you installed Docker through the Docker Toolbox then you’ll need to make 1 change to the .env file. Check out the SERVER_NAME=localhost:8000 value.
You will need to change localhost to your Docker Machine IP address instead. Chances are that will be 192.168.99.100 but if it’s not, you can find your Docker Machine IP by running docker-machine ip and then visit http://192.168.99.100:8000 in your browser instead of localhost.
By the way, if you’re using Docker Toolbox and you tried to submit the contact form and you received a CSRF token error then check out how to fix this problem. Spoiler alert: it’s a bug with Chrome.
Shutting Things Down
You’ll want to goto your Docker Compose terminal tab and press CTRL+C. Most of the time that works but sometimes you get an ABORT error. If that happens you can run docker-compose stop to stop everything.
Conclusion
Docker is awesome. Now you can run your projects on other platforms without having to worry about dependencies and platform specific gotchas.
You can even deploy your projects to production with minimal fuss.
You can learn much more about Flask by checking out the Build a SAAS App with Flask course. I keep the course updated regularly so it’s never out of date.
Or, if you’re ready to master Docker, then check out the Dive Into Docker course.