Updated on March 7, 2023 in #linux

Splitting Strings by a Delimiter with IFS or Bash's String Replace

There's a couple of ways to split strings into an array with a 1 character or multi-character delimiter.

Let’s start with wanting to split on 1 character. You could have a script where a user inputs a comma separated list of items. That could be defined as --fruits apple,orange,banana and now you want to loop over each fruit.

There’s a couple of ways to handle this.

# Using IFS

One of the most common use cases for the Internal Field Separator (IFS) in shell scripting is wanting to split a string by 1 or more individual characters and have an array that you can index into or loop over for each part.

That could look like this:

#!/usr/bin/env bash

set -o errexit
set -o pipefail
set -o nounset

# Let's pretend this came from a --fruits command line flag.
fruits="apple,orange,banana"

IFS="," read -r -a fruits_array <<< "${fruits}"
for fruit in "${fruits_array[@]}"; do
  echo "Fruit: ${fruit}"
done

# Output:
Fruit: apple
Fruit: orange
Fruit: banana

• <<< "${fruits} is our input that we pass in as stdin to the read command
• read -r -a fruits_array reads that input into a new fruits_array variable
• IFS="," splits our input by a specific character (comma)
• for fruit in ... loops over our new array and lets us operate on each item in the list

If you don’t set IFS it will use the default value which splits on white space such as a space, tab or newline character. That could be handy if we had a space separated list of fruits like apple orange banana, then we wouldn’t need to set IFS.

In my opinion I’d roll with this solution if you’re splitting on 1 character. It’s pretty robust, portable and decently readable for Bash.

You can pass multiple characters in such as IFS="orange" but that’s not going to produce a 2 element array with apple, and ,banana. Instead it’s going to individually split on each character that you supplied. That won’t work for our use case.

# Using Bash’s String Replace

If you want to split on multiple characters you might want to use Bash 4.2+’s string replace which works thanks to the parameter expansion feature:

fruits_array=(${fruits//orange/ })

for fruit in "${fruits_array[@]}"; do
  echo "Fruit: ${fruit}"
done

# Output:
Fruit: apple,
Fruit: ,banana

• fruits_array=(${fruits//orange/ }) replaces “orange” with " " for all occurrences

This gives us what we want and it also piggy backs on our prior knowledge about IFS because in this case the default value of using white space is being used.

If your input has spaces this won’t work but you can customize IFS if you need to such as IFS="!" fruits_array=(${fruits//orange/!}). In this case we’re picking ! to be the replacement text and delimiter for IFS. In your case you’d want to pick a character that you know won’t exist in your input string.

I think that works pretty well for most use cases.

Technically you can use this string replace method for single characters too but in that case I’d reach for IFS since in my opinion it’s slightly more readable and portable, although at this point you can count on most folks having Bash 4.2+. 3.2 is installed by default on macOS but you can still brew install bash to get a modern version.

# Throwing in the Towel?

If you have extremely strict use cases where you want proper splitting on any characters (single or multi), trimming, line ending support and all the bells and whistles then I suggest reading this StackOverflow answer.

It’s an epic post that demonstrates how difficult this problem is to handle in Bash.

It’s a good example of when to consider maybe using Python or another scripting language but if you have a hard requirement on using Bash it can be done based on that answer.

The video below demonstrates going over the IFS and string replace methods.

# Demo Video

Timestamps

• 0:55 – The use case of splitting a string by 1 character
• 1:44 – Using read and IFS to split a string into an array
• 2:51 – If you don’t define IFS it’ll use white space characters by default
• 4:27 – Can you name a fruit with 2 words?
• 4:47 – Splitting on multiple characters with IFS could be unexpected
• 5:48 – Using Bash’s parameter expansion and replacing strings
• 7:14 – This method works with 1 character too
• 8:00 – We’re still using IFS with the Bash string replace method
• 9:21 – Both methods work well most of the time but have footguns
• 9:44 – Splitting in Bash is hard

References

• https://www.shellcheck.net/wiki/SC2162

What’s your preferred method to split a string in Bash? Let me know below.