Learn Docker With My Newest Course

Dive into Docker takes you from "What is Docker?" to confidently applying Docker to your own projects. It's packed with best practices and examples. Start Learning Docker →

Updated on October 15, 2024 in #linux

Benefits of Running Your CI Tests on a Daily Schedule

benefits-of-running-your-ci-tests-on-a-daily-schedule.jpg

It's normal to run tests when pushing commits but you run the risk of having things break over time if you're not constantly pushing.

Quick Jump:

We all know the drill (hopefully!):

  • Push your code
  • CI kicks in and runs your test suite
  • It may or may not pass

Whether this happens in a PR or a direct push to your main branch is no different. In the end your code gets tested in an automated way which is nice.

But what happens if you’re not pushing changes on a very regular basis? Maybe you go a week or 2. It’s quite possible something can break during these periods of inactivity.

This happened to me recently after the official Python 3.12 Docker image stopped pip installing setuptools as an extra step after Python 3.12 removed it from the standard library. The Docker image kept it around a bit longer out of convenience.

Since the Python 3.12 tag is mutable (AKA it can change) and I didn’t use image digests in my Dockerfile, CI passed one day but failed after the package got removed and I didn’t know until someone opened an issue in one of my projects. Having the project’s test suite run daily would have let me know about this sooner since the CI tests would have started failing.

# Running Tests on a Cron Schedule

With GitHub Actions this is really easy, you can do:

on:
  schedule:
    - cron: "45 10 * * *"

That would run your action at 10:45 UTC every day, although it’s worth pointing out that GitHub can’t guarantee it will run exactly then. If lots of folks have scheduled tasks at the same time it might get delayed so they suggest avoiding popular times like 12:00, etc..

All of my example Docker starter web apps have this configured now. For example here’s the commit that added it to the Flask app.

Both GitLab and Bitbucket have ways to do this as well.

Overall I’m happy I set this up. I know every day these projects will get tested and if any of them fail I’ll receive an email notification from GitHub.

# Demo Video

Timestamps

  • 0:24 – What happens if you’re not pushing code regularly?
  • 1:38 – Running CI on a schedule
  • 2:42 – GitHub might delay your job depending on its load

Are you going to start scheduling your CI tests? Let me know below.

Never Miss a Tip, Trick or Tutorial

Like you, I'm super protective of my inbox, so don't worry about getting spammed. You can expect a few emails per year (at most), and you can 1-click unsubscribe at any time. See what else you'll get too.

Comments