Learn Docker With My Newest Course

Dive into Docker takes you from "What is Docker?" to confidently applying Docker to your own projects. It's packed with best practices and examples. Start Learning Docker →

Using Diff, Process Substitution and Pipes to Solve a Real Problem


In this video, we'll use diff, find, sed, grep, cut and sort together to write a custom Jekyll tag checking Shell script.

Quick Jump: Going Over Everything

In this specific case I wanted to make sure that the 150+ tags I have referenced on my Running in Production podcast site all map back to valid tag files on disk. This script helped me find a few tags that would have thrown a 404 due to typos or had missing information.

After watching this video you’ll be able to apply the same strategies back to your own use cases and issues. That’s the beauty of using Unix pipes. You can combine a bunch of different independent tools to solve unique problems as you come across them.

Going Over Everything


  • 1:06 – It’s so easy to make a typo or forget a tag name
  • 2:51 – Creating a Shell script to automate looking for tag mismatches
  • 3:21 – The TL;DR on how the diff command works
  • 3:41 – Using Bash Process Substitution to diff the output of 2 commands
  • 5:27 – High level overview of the tools used to compare tags in 2 different ways
  • 6:03 – Using sed range patterns
  • 7:03 – Breaking down each piped tool on the command line (sed, grep, cut, sort)
  • 9:03 – Using basename to get the file name without extension for many files
  • 10:44 – The end result is having 2 lists of sorted tags to diff
  • 10:48 – Evaluating the exit code of a command to determine if the tags match
  • 11:52 – More Shell scripting to make sure each tag has every property filled out
  • 12:57 – Make it work, make it nice and then make it fast


We go over more than what’s included below, but here’s the snippet associated to diffing the output of 2 commands. The rest is on GitHub:

diff --color -u <(for file in $(find _posts -type f)
    sed -n "/^tags:$/,/^title:/p" "${file}" | grep "^  - " | cut -d '"' -f 2
done | sort -u) <(basename -s .md tags/*.md | sort -u)

What types of problems have you solved with these tools? Let me know below!

Never Miss a Tip, Trick or Tutorial

Like you, I'm super protective of my inbox, so don't worry about getting spammed. You can expect a few emails per month (at most), and you can 1-click unsubscribe at any time. See what else you'll get too.