Updated on December 13, 2022 in #docker

Passing Exported Env Vars into a Docker Container without an .env File

This could be handy if you have encrypted secrets in your CI pipeline and want to pass a bunch of them into a container.

Prefer video? Here’s a recorded version of this post on YouTube that demos things in more detail.

There’s a couple of ways to insert environment variables into a container, such as:

• docker container run --env-file ".env" ...
• docker container run -e "NAME=nick" ...

The first option is nice, especially with Docker Compose when using the env_file property. This lets you keep all of your app’s env vars in a single file and easily reference it.

The second option is nice for quickly inserting 1 or maybe a handful of env vars into your container. It also works with Docker Compose using the environment property.

The second option supports passing in multiple env vars like this:

docker container run -e "NAME=nick" -e "HELLO=world" ...

But, recently I learned there’s another variation on this where you can do:

docker container run -e "NAME" -e "HELLO" ...

This works as long as NAME and HELLO are already exported from your shell where you’re running the Docker command. AKA., if you can run echo $NAME from the shell where you’re running the Docker command then NAME will be set in your container with that value.

Now you might be thinking “cool story, why not just use an .env file?” and I agree, in a lot of cases using an .env file is more convenient if you have a bunch of env vars to pass in but sometimes it’s not the right choice.

# Going Over a Real Example

Let’s say you’re writing a script that you want to run in CI and the script needs access to 10 env variables which are secrets that you have saved in your CI provider’s admin area for defining encrypted environment variables.

Most of the vars start with APP_ and a few start with COOL_.

Writing a little script to output env vars to an .env file so you can reference --env-file .env feels dirty. It feels like you’re fighting against the system to fulfill a requirement. The file isn’t really necessary and it’s 1 more spot where secrets get logged to disk.

Alternatively you can do:

env \
  | grep -E "^(APP|COOL)_.*$" \
  | cut -d "=" -f 1 \
  | sed "s/^/-e /"

• env lists all of the exported environment variables in your shell
• grep narrows down that list of env vars that start with our prefixes
• cut gives us the variable name, ie. NAME=nick after being cut is NAME
• sed converts NAME to -e NAME to prepare sending it to Docker

The idea here is you’ll build up a long string of output that may have -e NAME -e HELLO -e SOMETHING_ELSE. I’m ignoring the APP_ or COOL_ prefix here since it’s not important.

Since you’re probably doing this in a script you could put it all together like this:

env_var_flags="$(env \
  | grep -E "^(APP|COOL)_.*$" \
  | cut -d "=" -f 1 \
  | sed "s/^/-e /")"

# It's important not to quote this variable since we want each -e XXX item to
# be passed in as an individual flag, not one large string.
docker container run ${env_var_flags} ...

That’s it. Now whatever you’re doing inside of your container will have access to all of those variables and you didn’t have to hard code anything!

Sounds Tedious…

That looks like a lot of work, especially considering you could do env > .env to write an .env file or perhaps env | grep -E "^(APP|COOL)_.*$" > .env if you wanted the filtered list.

I won’t contest that and I did evaluate that as an option for something I was working on but I decided against it – at least for my specific use case. Here’s why.

I was working on a Dockerized Python script that:

• Uses the AWS SDK to get the latest production RDS (database) snapshot
• Creates a new development RDS instance based off that production snapshot
• Runs a SQL script to sanitize the data to remove personal information

The project itself is in a private repo for a client and they don’t want to store any secrets in the source code itself. Long story short there’s ~25 env vars that have very sensitive secrets. I also modified the SQL script to run it through envsubst to leverage the env vars.

This lets us commit all of the code and SQL to a code repo without committing secrets. There’s an .env.example file commit to the repo documenting each env var but none of the real values are there. There’s just dummy values.

Basically after the dev DB gets created it runs a mysql command to connect to the DB and imports that SQL after it has the env vars injected into it, an example of how that works is mysql ... < <(envsubst sanitize.sql). All of this logic happens inside of a Docker container.

The above solution let me quickly inject all of the CI defined env vars into the container without worrying about keeping around a sensitive .env file on disk. It’s 1 less potential spot where something can get accidentally logged.

Functionality wise writing the .env file would have worked just the same, even with a tiny bit less code (no cut or sed parsing), but in this specific case I preferred a slightly more secure approach. Plus I got to learn a new feature of Docker which is nice, even after using it since 2014. Today is a good day.

# Demo Video

Timestamps

• 0:15 – The env command
• 0:55 – A quick refresher on using env vars with Docker
• 3:20 – Taking a look at a script to build up a series of env flags
• 4:31 – Using grep to filter the results
• 5:03 – Using cut to only get the env var’s name by itself
• 5:31 – Turning the env var names into Docker e flags
• 6:05 – Using the script
• 6:53 – Real world use case of using secure env vars from a CI pipeline

What will you use this for? Let me know in the comments below!