Updated on June 26, 2018 in #docker, #elixir, #javascript, #ruby-on-rails

Dealing with Lock Files When Using Ruby, Node and Elixir with Docker

You can use this method with any package manager that has the concept of a lock file to help with dependency management.

Most package managers support having a lock file. Here’s a few examples:

• Gemfile.lock with Ruby
• yarn.lock with Node (using yarn instead of npm)
• mix.lock with Elixir
• composer.lock with PHP

Lock files are useful because if one is present, it will guarantee that the next time you run your package manager’s install command you will get the exact version of each dependency that matches up with the lock file.

This prevents you from getting different versions of dependencies between builds, such as development and production, or having another developer build your app’s code locally.

Lock files are useful to have around and should be a part of your source code (and even be commit to version control).

# Using a Lock File without Docker

Using a lock file without Docker is really straight forward.

After installing your dependencies, your package manager will save these dependencies somewhere on your system and then your package manager will create a lock file in the same directory as your regular dependency file.

Now you can commit this lock file to version control and you’re all set.

# Understanding the Lock file + Docker Problem

First, let’s look at a snippet from a Dockerfile for dealing with your package manager’s dependency file as well as the lock file and copying in your code.

An example snippet from a Ruby based Dockerfile:

# Install your dependencies separately, so the Docker layer gets cached between builds.
COPY Gemfile* ./
RUN bundle install

# Copy the rest of your source code into the Docker image afterwards.
COPY . .

This copies in both your Gemfile and Gemfile.lock and then runs a bundle install.

This pattern could also be used for Node, Elixir, PHP or any other language that supports lock files. I’m sure you’re probably doing something similar already.

The Problem with the above Snippet in Development

The above snippet works great in production where you won’t have any volume bind mounts, but in development you’ll want a bind mount so your app code is mounted into the container so you can develop your code without rebuilding your Docker image.

You would do that by having something like this in your docker-compose.yml file:

  web:
    # ...
    volumes:
      - ".:/app"

Here lies the problem, because check out this work flow:

• You have an existing Gemfile in your app’s source code.
• You run docker-compose build and bundle installs the gems into your Docker image.
• bundle creates a Gemfile.lock file in the Docker image.
• You run docker-compose up and your volume bind mount overrides what’s in /app.

And that’s part of the problem. the Gemfile.lock was built into the image, but it wasn’t reflected back to your Docker host’s file system because there’s no concept of a volume mount at build time. That means there’s never a lock file to commit to version control.

That’s an easy problem to solve right? You can do this:

One way to solve that would be to run docker-compose run web bundle install and now since we’re dealing with a runtime operation, and we have a volume mount set, this will result in an initial Gemfile.lock file being written back to your Docker host.

Now you can commit it to version control and you’re good to go.

Not so fast, what about when you modify your dependencies:

The above also works when updating your dependencies too. You would be responsible for running docker-compose build to first install your new dependencies into the Docker image itself, but now you’ll need to run docker-compose run web bundle install again to get the updated Gemfile.lock back on your Docker host.

If you didn’t do that, then the Gemfile.lock file that you would have in version control and mounted into your container would be the old / outdated Gemfile.lock from the previous bundle install that happened when you built the image.

Certain package managers will freak out about this and won’t even run, and that’s good. It’s protecting us from using a lock file that doesn’t match with what you have defined as a dependency in your regular dependency file.

The above steps will work but it’s a stinky solution:

It stinks because you need to not only build your dependencies but you need to remember to build them again at runtime. This is going to eventually catch up with you and you’ll likely commit an old lock file by accident.

# Fixing the Lock File Problem Once and for All

The cool thing about the pattern we’re about to go over now is it will work exactly the same for Ruby, Node, Elixir, PHP or any other language.

Protect your lock file from getting clobbered by your mount:

RUN bundle install && cp Gemfile.lock /tmp

Here we just add && cp Gemfile.lock /tmp to bundle install which won’t add an extra layer since it’s part of the bundle install RUN instruction.

This allows us to store our built / up to date Gemfile.lock in a safe spot.

Do something with the protected lock file at runtime:

RUN chmod +x docker-entrypoint.sh
ENTRYPOINT ["/app/docker-entrypoint.sh"]

Next up we need to use that copied Gemfile.lock file at runtime which means we need to introduce an ENTRYPOINT into our Dockerfile.

By the way, you can find a complete version of this Dockerfile on GitHub.

Set up an ENTRYPOINT script:

You’ll want to create a docker-entrypoint.sh file along side your Dockerfile and then copy what’s below into the script:

#!/bin/sh

set -e

built_lock_file="/tmp/Gemfile.lock"
current_lock_file="Gemfile.lock"

function cp_built_lock_file() {
    cp "${built_lock_file}" "${current_lock_file}"
}

if [ -f "${current_lock_file}" ]; then
    diff="$(diff "${built_lock_file}" "${current_lock_file}")"
    if [ "${diff}" != "" 2>/dev/null ]; then
        cp_built_lock_file
    fi
else
    cp_built_lock_file
fi

exec "$@"

The basic idea here is we’ll compare the protected Gemfile.lock with the Gemfile.lock from the /app/Gemfile.lock location. This will either be the built Gemfile.lock or the volume mounted Gemfile.lock depending on if you use volumes or not.

First we check to make sure we have a current Gemfile.lock. If we don’t then we copy the protected / built Gemfile.lock over to /app/Gemfile.lock.

If we do have a current lock file then we do a diff on it vs the protected lock file. If they are different then we know we ran into a situation where the current Gemfile.lock file is out of date, so we copy over the protected / up to date Gemfile.lock (this updates the volume).

If none of those conditions are true then nothing happens, which is what would happen if we ran the container in production with no volume mounts.

Here we get the best of both worlds where our lock file is always up to date on our Docker host and we don’t need to worry about running bundle install at runtime a second time.

But There Is One Edge Case

I haven’t figured out how to get around this one but there is one potential edge case.

Let’s say you updated your Gemfile and added a new dependency and then ran docker-compose build. If you commit your code right now then your Gemfile.lock would still be the old lock file since the ENTRYPOINT didn’t run yet.

So the thing to remember here is you need to run your container at least once before committing your code.

This is an edge case for sure, but in practice this is extremely unlikely to ever happen. Before committing your code would always run your container at least once after changing your dependencies to make sure it works.

# Full Examples for Ruby, Node and Elixir

We talked about quite a bit of code in this article, and if you want to see a few fully working examples using Express, Phoenix and Rails you can find them below:

• Rails
• Node / Express
• Phoenix

The above repo was made to showcase a bunch of Dockerized web application examples.

What do you think about this approach? Have you thought of something better?