Updated on June 13, 2023 in #deployment, #flask, #ruby-on-rails

Required vs Optional Environment Variables for Safer Deploys

It's important to differentiate these to protect your app from not starting up if a required env var is missing.

If a required environment variable is missing and your app throws an exception on start up that’s really good because you can protect yourself from unwanted downtime.

For example, on deploy you might only serve live traffic to the new version once your app passes a few health checks. If your app doesn’t start it will fail those health checks and never go live. Instead, the previous version will continue to run successfully.

Often times you git ignore an .env file which may contain secrets such as API tokens and other sensitive data. The process of deploying code and setting env vars are usually done at different times in different places.

For example, you may set an environment variable before you ship the code to use it. This way it’s available when the new version of your app goes live. This introduces potential human error where you can accidentally forget to set the env var.

# Failing Early

If you forget a required env var you want your app to fail as early as possible.

With a lot of web frameworks you can make this happen by only referencing env vars in your app’s config file(s). Then, throughout your application you can reference the value through its config option.

The basic idea here is when your app starts up it will read its config file(s) and if a required env var is missing an exception will be thrown. Since all of your envs exist in config files, they are all checked early in this 1 spot.

In my opinion this is much better than scattering around direct environment variable references throughout your app because this allows your app to boot with missing required env vars and you only get an exception much later at run-time when it’s being accessed on a page. That might happen days or weeks after a deploy.

Plus, I think it’s nice to have these env references in 1 area of your code. This is beneficial in 2 different ways:

• You know for sure your app doesn’t have any hard to detect hidden dependencies lingering around
• You’ve created a nice abstraction where your app doesn’t need to know about env vars, it only knows about config options

Practical Examples

With Flask, you can create a config file and define your config options there, and then within your app you can reference the config variables. You’d load this config in your create_app function which runs when your app starts.

With Rails, you can put your env var references in any file that exists in the config/ directory and you’re good to go. These files will be read on app startup. Then you can reference your config parameters or variables throughout your app.

I have example Docker apps using this pattern here:

• https://github.com/nickjj/docker-flask-example
• https://github.com/nickjj/docker-rails-example

There’s example apps for Django, Node and Phoenix too.

NOTE: The only exception is the example apps reference an env var to get the programming language’s run-time version on the home page. I’ve left that in these apps because the home page is meant to be changed by you. It’s throw away code and refactoring that env var into a config option will make it harder to remove it since it’ll be referenced in a few spots.

# Required vs Optional

In practice I think there’s 2 ways to define environment variables:

• Required: the env var must be defined with a value
• Optional: the env var might be defined and if not it has a default fallback

I’ve personally found that to cover most use cases I’ve encountered.

Technically you could have an optional env var with no default fallback (defaults to None, nil, null, etc.) but in that case depending on the use case you could consider any of these:

• Make it required
• Default to a boolean false or a relevant data type depending on what you’re doing
• Ask yourself if the value needs to be an environment variable
• If the none of the above makes sense then sure default to null

Here’s a few examples using Python, Ruby and Node.

Python:

• Required: use os.environ since it’ll fail if the env var isn’t set with a value
• Optional: use os.getenv with a default fallback

Here’s 3 examples (required + optional with and without defining the env var):

$ docker container run --rm python:3-slim-bullseye \
    python3 -c 'import os; os.environ["NAME"]'

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen os>", line 679, in __getitem__
KeyError: 'NAME'

$ docker container run --rm python:3-slim-bullseye \
    python3 -c 'import os; print(os.getenv("NAME", "nobody"))'

nobody

$ docker container run --rm -e NAME=nick python:3-slim-bullseye \
    python3 -c 'import os; print(os.getenv("NAME", "nobody"))'

nick

Ruby:

• Required: use ENV.fetch with no default since it’ll fail if the env isn’t set with a value
• Optional: use ENV.fetch with a default fallback

Here’s 3 examples (required + optional with and without defining the env var):

$ docker container run --rm ruby:3-slim-bullseye \
    ruby -e 'puts ENV.fetch("NAME")'

-e:1:in `fetch': key not found: "NAME" (KeyError)
        from -e:1:in `<main>'

$ docker container run --rm ruby:3-slim-bullseye \
    ruby -e 'puts ENV.fetch("NAME", "nobody")'

nobody

$ docker container run --rm -e NAME=nick ruby:3-slim-bullseye \
    ruby -e 'puts ENV.fetch("NAME", "nobody")'

nick

Node:

Node doesn’t have a built-in way to access env vars in a required vs optional way. If you reference process.env.NAME and it doesn’t exist then you’ll get undefined back but it won’t throw an exception.

The example below shows how you can make a helper function to check and then there’s an example of referencing an optional PORT env var and required SECRET_KEY variable:

function processEnvRequired(key) {
  const value = process.env[key];

  if (value === undefined || value === null || value === '') {
    throw Error(`KeyError: ${key} not found`);
  }

  return value;
}

config.express = {
  port: process.env.PORT || 8000,
  secret: processEnvRequired('SECRET_KEY'),
};

The video below covers what’s in this post but with more details going over the demo apps.

# Demo Video

Timestamps

• 0:53 – Reading all env vars when your app starts up is worth it
• 3:39 – Demo’ing what happens when we don’t set a SECRET_KEY
• 4:37 – Spending a few seconds looking at a Rails project
• 5:21 – Going over required vs optional in the example Flask app
• 7:30 – Python: Reading required vs optional env vars
• 9:18 – Ruby: Reading required vs optional env vars
• 12:06 – Node: Making our own required function

Are you using this pattern for managing env vars? Let me know below.