Using Extended Globbing for Patterns in Bash
Sometimes you might want extra features like listing all files except for a specific one, we'll cover a few examples.
When filtering files on the command line, you may have encountered patterns like this:
cat *
to output all files but ignore files that start with.
cat *.bak
to output all files with a.bak
file extensioncat a*
to output all files that start witha
cat ??.txt
to output all text files that are 2 characters long
If those concepts are new to you, you can experiment with this:
echo "hello" >hello.bak &&
echo "hello" >hello.txt &&
echo "apple" >apple &&
echo "ab" >ab.txt
Now you can run the above commands and see what they output.
# Extended Globbing
What if you wanted to output all files except 1 specific file? You could write a custom script to loop over things and skip the one you want but it’s possible to do this out of the box.
You know how you can do set -o errexit
to force scripts to exit on the first
error? You can do something similar with shopt
options such as shopt -s extglob
. Keep in mind if you’re using zsh
you’ll want to open a bash
shell to allow that command to work.
To help remember the command think of it as “shell options” AKA. “shopt”.
Listing shopt Options
You can confirm it’s on by running shopt -s
to list all enabled options:
$ shopt -s
checkwinsize on
cmdhist on
complete_fullquote on
expand_aliases on
extglob on
extquote on
force_fignore on
globasciiranges on
hostcomplete on
interactive_comments on
progcomp on
promptvars on
sourcepath on
Running shopt -u
will list disabled options and shopt -p
will list all
options and their values.
All Files Except
You can do cat !(demo)
to output all files except a file called demo
. You
can separate patterns with a |
too such as cat !(demo|extras)
to skip files
named demo
or extras
.
To disable this behavior you can run shopt -u extglob
to bring your shell
back to normal.
Other Examples
Here’s the docs straight from man bash
:
?(pattern-list) Matches zero or one occurrence of the given patterns
*(pattern-list) Matches zero or more occurrences of the given patterns
+(pattern-list) Matches one or more occurrences of the given patterns
@(pattern-list) Matches one of the given patterns
!(pattern-list) Matches anything except one of the given patterns
If you’re wondering why it’s not enabled by default, there’s a good write up of that on StackOverflow.
If you did want to enable it by default you can put it into your shell’s rc file. You can also put it near the top of a shell script to have it apply to your whole script. You can optionally choose to turn it on and off on demand as needed.
# Edge Case with Line Parsing
Before going into the solution, it might help to understand a specific use case.
I had a bunch of shell scripts in a directory and I wanted to run shellcheck
on all of them. Normally you can do shellcheck *
to handle that but in my
case I had a few Python scripts without a .py
extension and the version of
shellcheck
I use tried to run against those Python scripts.
What I really wanted to do is “run ShellCheck against all files except these 3
Python scripts” so I used shellcheck !(a|b|c)
where a|b|c
would really be
the Python script names. Problem solved!
As for the edge case, I was running shellcheck
in a script so I put shopt -s extglob && shellcheck !(a|b|c)
but I noticed it didn’t work. It was like the
option wasn’t enabled.
It turns out that shopt -s extglob
must exist on its own line before any
lines that use it. All in all I just put it onto 2 separate lines and then it
worked fine.
If you have a hard requirement where it must exist on 1 line you can launch
Bash with extglob
enabled for just that 1 line command with bash -O extglob -c "shellcheck !(a|b|c)"
.
The video below demos using extglob
.
# Demo Video
Timestamps
- 0:00 – Default patterns
- 1:35 – All files except
- 1:53 – Enabling extglob
- 2:51 – Showing how it works
- 3:56 – Checking shopt options
- 5:00 – A specific use case
- 6:25 – Edge case in a script
- 8:42 – Toggling it in a script
When did you last use this pattern? Let me know below.