Run Parallel Background Commands in Linux and Wait Until They Complete
Sometimes it's useful to run parallel jobs and wait until they all finish before moving on, we can easily do this on the command line.
Quick Jump: Demo Video
This could be handy when you have a few tasks that can run in isolation. In my case I wanted to run a few sql dump commands in parallel and then do something once all of them finished.
In this video we’ll go how to combine using
& and the
Create the script:
touch work && chmod +x work
Copy / paste this into the above file:
#!/usr/bin/env sh # & instructs this command to run in the background and immediately begin # executing the next line in the script without waiting for it to finish. printf "Doing something...\n" sleep 3 & printf "Doing something else...\n\n" sleep 3 & # List all of the jobs and their PIDs. jobs -l # Wait until all of the jobs finish before continuing. wait printf "\n...and we're done!\n\n"
- Run and time the script
Doing something... Doing something else...  + 10254 Running  - 10253 Running ...and we're done! ./work 0.00s user 0.00s system 0% cpu 3.003 total
Notice that it finishes in about 3 seconds instead of 6 because it ran both sleep commands in parallel instead of sequentially waiting for each one. It also only prints out the last line once both jobs are done.
- 0:24 – Running the demo script
- 1:04 – Running commands in the background with
- 1:36 – The
waitcommand will let you wait until jobs finish
- 2:45 – The
jobscommand lets us get the PID and status of our jobs
- 3:35 – You can also wait for specific PIDs to finish running
- 4:30 – Running the jobs command with different flags
- 5:09 – Getting the status of each job after they’re done running
- 5:50 – Running things in parallel can save a lot of time
What are some use cases where you’ve used this pattern? Let me know below.