Monitor the Output of a Program for Changes Using the Watch Command
This sure beats running a command and hitting the up arrow dozens of times per minute while you wait for a change.
Quick Jump: Monitoring DNS Changes | A Few Other Examples | Demo Video
Prefer video? Here’s a demo video going over what’s mentioned below.
You can use watch for a number of things and we’ll go over a couple of examples, but first here’s the use case that provoked me to write this blog post.
Monitoring DNS Changes
The other day I was waiting for a DNS change to propagate so I was running dig TXT example.com waiting to see when my new TXT entry was live. However, just because my ISP’s DNS server says something doesn’t mean the rest of the world agrees.
So I found myself running dig TXT example.com, dig TXT example.com @8.8.8.8 and dig TXT example.com @1.1.1.1. That lets you check DNS records using popular DNS servers such as Google (8.8.8.8) and Cloudflare (1.1.1.1). If all 3 versions report the same info you can be reasonably confident that a decent amount of traffic is seeing your new DNS change.
I wasn’t going crazy up arrow spamming all 3 commands in 3 different terminals but I did run each of those a few times in a minute but then I remembered “oh yeah, there’s a command for this, duh!” – and here we are.
I landed on this:
watch --differences "dig TXT example.com \
; dig TXT example.com @8.8.8.8 \
; dig TXT example.com @1.1.1.1"
watch is available by default on Linux, if you’re using macOS you can brew install watch.
That will run all 3 dig commands in parallel and watch will show its output to you. You don’t need to launch each command in the background with & btw.
If nothing changes then the output won’t change. If the output changes then --differences (or -d) will highlight the changes in a different color. By default it’ll run every 2 seconds.
Eventually I saw my new TXT entry pop up in all 3 outputs and that was it.
A Few Other Examples
A classic example is watch date or watch --interval 1 date which shows updates every second. You can make the changes easier to see with watch --interval 1 --differences date or watch -n 1 -d date for short. This example isn’t super useful but it’s a good basic example to see how watch works.
Monitoring how many copies of a process are running:
This could come in handy once in a while. For example, you can run ps aux | grep vim to see how many copies of Vim are running.
In my case I see this. It’s a bunch of different projects inside of tmux sessions:
nick 2854 3.5 0.2 63376 23888 pts/3 S+ Jul14 742:04 vim .
nick 2894 1.1 0.2 62936 23960 pts/5 S+ Jul14 247:44 vim .
nick 2935 1.0 0.4 73052 37052 pts/8 S+ Jul14 216:07 vim .
nick 2974 0.8 0.4 72172 36908 pts/11 S+ Jul14 187:53 vim .
nick 3031 1.4 0.4 74368 39360 pts/14 S+ Jul14 312:10 vim .
nick 3067 0.8 0.4 72292 36828 pts/18 S+ Jul14 169:18 vim .
nick 3105 0.9 0.4 73860 39116 pts/21 S+ Jul14 191:12 vim .
nick 3141 1.9 0.2 63092 24324 pts/24 S+ Jul14 400:51 vim .
nick 3177 0.0 0.7 92080 57840 pts/27 S+ Jul14 16:10 vim .
nick 3219 0.5 0.9 158332 76536 pts/31 S+ Jul14 117:01 vim .
nick 15096 0.0 0.4 72536 38520 pts/2 S+ Jul27 0:00 vim .
nick 31009 0.0 0.0 4024 1928 pts/36 S+ 09:58 0:00 grep --color=auto vim
Now we can watch it with watch "ps aux | grep vim". We have to use quotes here because our watch command has a pipe (we’d need to do the same thing if we used &&).
Near the bottom you may have noticed lines that look like this:
nick 31095 0.0 0.0 3904 2968 pts/36 S+ 10:02 0:00 watch ps aux | grep vim
nick 31104 0.0 0.0 3904 992 pts/36 S+ 10:02 0:00 watch ps aux | grep vim
nick 31105 0.0 0.0 2888 948 pts/36 S+ 10:02 0:00 sh -c ps aux | grep vim
nick 31107 0.0 0.0 4024 2124 pts/36 S+ 10:02 0:00 grep vim
Maybe you don’t want to see these as part of the list, that’s ok, we can refine our command to be watch "ps aux | grep vim | grep -vE '(watch|grep)'" and now we’ll get back a list of Vim processes only. The 2nd grep command removes lines that contain watch or grep, it’s not a perfect solution but it works for now.
If you wanted to count the number of Vim processes you can run watch "ps aux | grep vim | grep -vEc '(watch|grep)'" instead. In this case I added -c to the last grep command to count the results:
Every 2.0s: ps aux | grep vim | grep -vEc '(watch|grep)' kitt: Sat Jul 29 10:05:50 2023
11
Exiting watch if a command fails
Let’s say you’re going away for a bit and want to leave a command running for minutes or hours and you don’t want it to be re-run if it fails. You can add --errexit|-e and then watch will stop watching if the command exits with a non-0 exit code.
You can test this one out by running a command that always fails such as watch -e false or watch -e "exit 1".
The watch process will remain running until you hit any key. That’s handy so that you can read the error message when you get back. What’s really nice is the title bar will show the time when the command failed since it’ll stop updating when the failure occurs.
You can hide that title bar too with --no-title|-t if for some reason you don’t want to see it.
Lastly if you have the beep package installed and your terminal supports it you can also add the --beep|-b flag to make watch emit a beep sound if a command fails.
Exiting watch if a command changes
Adding the --chgexit|-g flag will make watch quit out on any type of change. You can test that one with watch -g date which will exit after 2 seconds or so.
If you wanted to know how long a command ran before it changed you can run time watch -g -n 5 date. In this case I set a custom interval to let it run 5 seconds in which case we’ll see about 5 seconds in time’s total.
There’s a few other flags you can play with too, check out man watch for more details. The video below covers what was mentioned in this post.
Demo Video
Timestamps
- 0:22 – A quick demo of using the watch command
- 0:59 – Checking for DNS updates with dig
- 2:59 – Running multiple dig commands manually or scripting it
- 3:32 – Monitoring DNS changes with dig and watch
- 4:49 – Highlighting changes and modifying the interval of watch
- 5:56 – Monitoring how many copies of a process are running
- 8:38 – Monitoring a program to see if it exits with a failure
- 10:32 – Beep beep
- 11:47 – Removing the title
- 12:04 – Stop watching if the output changes
- 12:54 – Combining time with watch to see when a program’s output changed
What was the last thing you used the watch command for? Let us know below.