Learn Docker With My Newest Course

Dive into Docker takes you from "What is Docker?" to confidently applying Docker to your own projects. It's packed with best practices and examples. Start Learning Docker →

Updated on August 1, 2023 in #linux

Monitor the Output of a Program for Changes Using the Watch Command

monitor-the-output-of-a-program-for-changes-using-the-watch-command.jpg

This sure beats running a command and hitting the up arrow dozens of times per minute while you wait for a change.

Quick Jump:

Prefer video? Here’s a demo video going over what’s mentioned below.

You can use watch for a number of things and we’ll go over a couple of examples, but first here’s the use case that provoked me to write this blog post.

# Monitoring DNS Changes

The other day I was waiting for a DNS change to propagate so I was running dig TXT example.com waiting to see when my new TXT entry was live. However, just because my ISP’s DNS server says something doesn’t mean the rest of the world agrees.

So I found myself running dig TXT example.com, dig TXT example.com @8.8.8.8 and dig TXT example.com @1.1.1.1. That lets you check DNS records using popular DNS servers such as Google (8.8.8.8) and Cloudflare (1.1.1.1). If all 3 versions report the same info you can be reasonably confident that a decent amount of traffic is seeing your new DNS change.

I wasn’t going crazy up arrow spamming all 3 commands in 3 different terminals but I did run each of those a few times in a minute but then I remembered “oh yeah, there’s a command for this, duh!” – and here we are.

I landed on this:

watch --differences "dig TXT example.com \
  ; dig TXT example.com @8.8.8.8 \
  ; dig TXT example.com @1.1.1.1"

watch is available by default on Linux, if you’re using macOS you can brew install watch.

That will run all 3 dig commands in parallel and watch will show its output to you. You don’t need to launch each command in the background with & btw.

If nothing changes then the output won’t change. If the output changes then --differences (or -d) will highlight the changes in a different color. By default it’ll run every 2 seconds.

Eventually I saw my new TXT entry pop up in all 3 outputs and that was it.

# A Few Other Examples

A classic example is watch date or watch --interval 1 date which shows updates every second. You can make the changes easier to see with watch --interval 1 --differences date or watch -n 1 -d date for short. This example isn’t super useful but it’s a good basic example to see how watch works.

Monitoring how many copies of a process are running:

This could come in handy once in a while. For example, you can run ps aux | grep vim to see how many copies of Vim are running.

In my case I see this. It’s a bunch of different projects inside of tmux sessions:

nick      2854  3.5  0.2  63376 23888 pts/3    S+   Jul14 742:04 vim .
nick      2894  1.1  0.2  62936 23960 pts/5    S+   Jul14 247:44 vim .
nick      2935  1.0  0.4  73052 37052 pts/8    S+   Jul14 216:07 vim .
nick      2974  0.8  0.4  72172 36908 pts/11   S+   Jul14 187:53 vim .
nick      3031  1.4  0.4  74368 39360 pts/14   S+   Jul14 312:10 vim .
nick      3067  0.8  0.4  72292 36828 pts/18   S+   Jul14 169:18 vim .
nick      3105  0.9  0.4  73860 39116 pts/21   S+   Jul14 191:12 vim .
nick      3141  1.9  0.2  63092 24324 pts/24   S+   Jul14 400:51 vim .
nick      3177  0.0  0.7  92080 57840 pts/27   S+   Jul14  16:10 vim .
nick      3219  0.5  0.9 158332 76536 pts/31   S+   Jul14 117:01 vim .
nick     15096  0.0  0.4  72536 38520 pts/2    S+   Jul27   0:00 vim .
nick     31009  0.0  0.0   4024  1928 pts/36   S+   09:58   0:00 grep --color=auto vim

Now we can watch it with watch "ps aux | grep vim". We have to use quotes here because our watch command has a pipe (we’d need to do the same thing if we used &&).

Near the bottom you may have noticed lines that look like this:

nick     31095  0.0  0.0   3904  2968 pts/36   S+   10:02   0:00 watch ps aux | grep vim
nick     31104  0.0  0.0   3904   992 pts/36   S+   10:02   0:00 watch ps aux | grep vim
nick     31105  0.0  0.0   2888   948 pts/36   S+   10:02   0:00 sh -c ps aux | grep vim
nick     31107  0.0  0.0   4024  2124 pts/36   S+   10:02   0:00 grep vim

Maybe you don’t want to see these as part of the list, that’s ok, we can refine our command to be watch "ps aux | grep vim | grep -vE '(watch|grep)'" and now we’ll get back a list of Vim processes only. The 2nd grep command removes lines that contain watch or grep, it’s not a perfect solution but it works for now.

If you wanted to count the number of Vim processes you can run watch "ps aux | grep vim | grep -vEc '(watch|grep)'" instead. In this case I added -c to the last grep command to count the results:

Every 2.0s: ps aux | grep vim | grep -vEc '(watch|grep)'              kitt: Sat Jul 29 10:05:50 2023

11
Exiting watch if a command fails

Let’s say you’re going away for a bit and want to leave a command running for minutes or hours and you don’t want it to be re-run if it fails. You can add --errexit|-e and then watch will stop watching if the command exits with a non-0 exit code.

You can test this one out by running a command that always fails such as watch -e false or watch -e "exit 1".

The watch process will remain running until you hit any key. That’s handy so that you can read the error message when you get back. What’s really nice is the title bar will show the time when the command failed since it’ll stop updating when the failure occurs.

You can hide that title bar too with --no-title|-t if for some reason you don’t want to see it.

Lastly if you have the beep package installed and your terminal supports it you can also add the --beep|-b flag to make watch emit a beep sound if a command fails.

Exiting watch if a command changes

Adding the --chgexit|-g flag will make watch quit out on any type of change. You can test that one with watch -g date which will exit after 2 seconds or so.

If you wanted to know how long a command ran before it changed you can run time watch -g -n 5 date. In this case I set a custom interval to let it run 5 seconds in which case we’ll see about 5 seconds in time’s total.

There’s a few other flags you can play with too, check out man watch for more details. The video below covers what was mentioned in this post.

# Demo Video

Timestamps

  • 0:22 – A quick demo of using the watch command
  • 0:59 – Checking for DNS updates with dig
  • 2:59 – Running multiple dig commands manually or scripting it
  • 3:32 – Monitoring DNS changes with dig and watch
  • 4:49 – Highlighting changes and modifying the interval of watch
  • 5:56 – Monitoring how many copies of a process are running
  • 8:38 – Monitoring a program to see if it exits with a failure
  • 10:32 – Beep beep
  • 11:47 – Removing the title
  • 12:04 – Stop watching if the output changes
  • 12:54 – Combining time with watch to see when a program’s output changed

What was the last thing you used the watch command for? Let us know below.

Never Miss a Tip, Trick or Tutorial

Like you, I'm super protective of my inbox, so don't worry about getting spammed. You can expect a few emails per year (at most), and you can 1-click unsubscribe at any time. See what else you'll get too.

Comments