Automate Suspending Kubernetes Cron Jobs with find and sed
Treat yourself by automating mundane tasks like setting a bunch of YAML properties from true to false and vice versa.
This example is specific to Kubernetes cron jobs but it can be applied to anything!
With Kubernetes you can set suspend: true or suspend: false on a cron job.
This is handy because a suspended cron job won’t get triggered, this is often
more user friendly than temporarily deleting the entire cron job and adding it
back later.
This is especially true if you subscribe to the GitOps philosophy and use tools like Argo CD to monitor the state of a git repo and sync it to be the true state of your cluster.
In other words, when you want to make changes to your cluster you commit and
push config file changes, not run kubectl commands directly against your
cluster.
You may choose to temporarily suspend cron jobs if you’re putting your site into maintenance mode such as when performing major database upgrades. You might block access to your public site at the firewall level but internally your Kubernetes cron jobs would still be firing unless you suspend them.
Let’s say you have 3 different apps and each of them have anywhere between 2
and 15 cron jobs. Let’s call it 20 cron jobs total. It’s mildly annoying to
open your code editor and do a project wide find / replace to set suspend: true or suspend: false so you can commit these changes afterwards.
Wouldn’t it be nice if you had a run script shortcut to where you can run ./run cronjobs:disable and ./run cronjobs:enable?
The run script functions could look like this:
function cronjobs:disable {
find "kubernetes/apps/overlays/"*"/cron-job.yaml" -type f \
-exec perl -pi -e "s/^ suspend: false$/ suspend: true/g" {} \;
}
function cronjobs:enable {
find "kubernetes/apps/overlays/"*"/cron-job.yaml" -type f \
-exec perl -pi -e "s/^ suspend: true$/ suspend: false/g" {} \;
}
In the above case, kubernetes/apps/overlays/"*"/cron-job.yaml would be a path
to wherever you keep your Kubernetes configs. I like using
Kustomize but that’s not
important here. The * in this case would expand to all of your apps, such as
app a, b and c. This way all of your app’s cron-job.yaml files get
modified.
We’re using perl instead of sed because I’ve found it to be more portable
on both Linux and macOS when it comes to performing in-place edits. The version
of sed on macOS does not work the same as the GNU version of sed but perl
does.
But if you want to use sed instead you can. The same logic applies to both.
We’re looking for very specific lines in the file and then globally replacing
them.
Here’s what the Kubernetes cron job config might look like. In this case it’s a
cron job that runs every 5 minutes and when it runs it curls a specific API
endpoint. Check out the suspend property on line 7, that’s what gets string
replaced:
---
apiVersion: "batch/v1"
kind: "CronJob"
metadata:
name: "app-do-something"
spec:
suspend: false
schedule: "*/5 * * * *"
concurrencyPolicy: "Forbid"
jobTemplate:
spec:
backoffLimit: 3
template:
spec:
restartPolicy: "OnFailure"
containers:
- name: "app"
imagePullPolicy: "Always"
image: "curlimages/curl:8.5.0"
command:
- "ash"
- "-c"
- |-
curl -s \
-H "Authorization:Bearer ${APP_API_BEARER_TOKEN}" \
-X POST \
http://a-app/api/v1/do_something
The video below shows disabling and enabling cron jobs on a few Kubernetes configs.
# Demo Video
Timestamps
- 0:18 – Looking at a Kubernetes cron job config
- 1:05 – This could be useful for temporary maintenance
- 2:07 – Demoing the disable and enable run script functions
- 2:52 – Going over the disable and enable functions
- 3:21 – Using Kustomize and Argo CD to stick with the GitOps philosophy
- 3:50 – Back to the disable and enable functions
- 5:55 – Was it worth it? Yes, now the intent is clear that we do this
Do you think you’ll add this to your project? Let us know below!