Using Python to Help Generate Personalized Email Templates
It didn't take long to bust out Python to help me manage upcoming podcast episodes and send interview questions out.
A few weeks ago I announced https://runninginproduction.com which is a new podcast / interview site and I have to say I’m really excited to see that folks are interested in this because I’ve gotten 18 podcast guest and 12 email interview requests in about 2 weeks.
In a previous article I talked about how I am using Google Forms to help deal with requests and that’s working out great, but the next steps after receiving a form submission are to read it, reply to the person with a list of questions customized for their project and if it’s a podcast then schedule it.
After I received a few responses I started to think about automating the process for helping me generate personalized questions for each guest.
Eventually I busted out Python to make a custom CLI tool but that’s not where I started. It took doing it manually a few times and an iteration before I got there.
But in the end it translates to being able to run:mkinterview -n "Jim Smith" -t "Flask / Python" -p DigitalOcean -m interview
And this will dump out a list of questions specific for that person and tech choices so that I can email them back quickly. It’s already saving me a lot of time.
One thing I’ve learned so far is, even running at a tiny bit of scale is super beneficial for figuring out ideas on how to automate things based on a real problem.
# Automating the Creation of Personalized Questions
It all starts with the work flow being done manually, so let’s go over that first.
Figuring Out the Work Flow Manually
The main problem is, I have a list of 12 questions that are pretty high level like “What’s your background and what site are you running in production?”. That’s mainly for the email based interviews. The podcasts are unscripted but I still send these questions over so people can get an idea of what to expect and prepare for the show if they want.
But then each of those high level questions have 4-10 bullet points with more specific sub-questions just to help flesh out an answer for that question.
That’s easy enough. I started with a single markdown document and had all that
written out, let’s call that email-template.md. This took a few hours to do
once.
And now let’s say someone submits the form and mentions they are using Flask. I don’t want to send out questions to them with things like “What motivated you to use your web framework of choice?”.
I want to personalize it for them with “What motivated you to use Flask / Python?”.
So for the first couple of replies I went through all of these questions and manually changed things. This involved editing their name, swapping in their framework in a couple of spots, adding specific questions about their hosting provider and a few other things.
Before I knew it, I was spending 5-7 minutes just carefully replacing things from the master email template. That’s not counting the time to read their reply in detail and checking out their site (that’s extra time, and will always be important to do manually).
But making ~15 changes in a long email takes a bit of time, especially if you want to double and triple check everything before you send it because if have typos or put in the wrong thing you look like you don’t pay attention to detail and maybe they won’t even follow up.
Even at this small scale of having ~30 submissions, that’s a non-trivial amount of time. Even with a pretty optimized manual work flow that’s ~150 minutes worth of tedious and error prone find and replacing.
That’s only half of the problem too. I also want to send emails with different text depending on if the person wants to be a podcast guest or get interviewed by email.
For example for podcasts I mention that the questions listed below are only to help prepare for the show and they don’t need to fill them out. I also add a bit about asking what timezone they are in so we can move forward with scheduling the call.
But for email interviews, I explain how the questions should be answered in Markdown format and also add a couple of different bullet points to certain questions. Then I finish it off with letting them know to reply back on whatever schedule works best for them, etc..
A First Shot at Automating the Process
After I did 3 of them I was like you know what, maybe I should create a
template-flask-email.md file along with a template-flask-podcast.md file.
And then as new technologies come across I’ll add more files, such as
template-rails-email.md and so on. Great idea I thought!
I actually took some time out and I did that for a bunch of them. I had templates for Flask, Phoenix, Rails and Django since that’s what was submit so far.
But then I started noticing that people were submitting variants on their cloud hosting provider too. Some people were using DigitalOcean, others were using Heroku, AWS or GCP, etc..
And the hosting providers being different drastically changes some of the bullet points for some of the questions.
So now it became an exponential problem. I can’t just have 2 templates for each framework. Now I would need 2 for each framework and then 2 more for each hosting provider.
That means with just the options listed above for Flask I would need:
template-flask-digitalocean-email.mdtemplate-flask-digitalocean-podcast.mdtemplate-flask-heroku-email.mdtemplate-flask-heroku-podcast.mdtemplate-flask-aws-email.mdtemplate-flask-aws-podcast.mdtemplate-flask-gcp-email.mdtemplate-flask-gcp-podcast.md
And then I would need to do the same for Phoenix, Rails, Django and also as new hosting providers came about like Netlify then I would need to keep adding them afterwards.
The amount of duplication here would be horrendous. Each one of those files would have a lot of questions. Imagine having to go back and edit one of those questions or add a bullet point. It’s a total maintenance nightmare.
It was at that point where I had to think of a new strategy.
Round 2: I Am a Developer So I Developed Something
It didn’t take too long to realize this is really a templating problem. I have a mostly static set of text where I want to situationally replace a few strings with other strings. That’s it.
I’m a big fan of creating single file / zero dependency Bash and Python CLI tools so the first thing I thought of was “have I solved a similar problem already?” and the answer was yes.
A few months ago I wrote a Python CLI tool called verdiff, I made a video on
how it works and how it was built if you want to check that out. The TL;DR
is it helps you see diffs of various web framework versions.
It’s a Python script
and the general pattern of that script is that I generate a custom Dockerfile
based on what web framework is passed in from the command line.
That’s pretty much exactly what I want to do here, except instead of a Dockerfile being generated it’s a list of questions. So I hacked away on that for about 2 hours and I ended up with something that’s working well so far.
The interview script isn’t open source but that’s only because it’s super
specific to my podcast. The verdiff source code covers all of the interesting
bits of code. I used the exact same patterns and strategies for the email
template as I did in the verdiff script.
I got a chance to use the new mkinterview script a bunch of times but the
funny thing is, as soon as the very first interview request came in after I
finished it, I was like “yay, time to enjoy the fruits of my labor!”.
However, when I read his tech stack he mentioned that he’s using his laptop on a home network to host his side project. That’s really cool but it was also an edge case I didn’t account for in the script.
Luckily the script is coded in such a way where if the hosting provider can’t be defined specifically like DigitalOcean, AWS, etc. then it falls back to generic bullet points that could apply to anywhere.
For example in the mkinterview -n "Jim Smith" -t "Flask / Python" -p DigitalOcean -m interview command from before, the -p DigitalOcean is
optional.
# Takeaways That Can Be Applied Elsewhere
I think one takeaway here is once you’re a programmer for a long enough time you’ll always be thinking of ways to optimize work flows and be thinking of ways to automate as much as you can when it makes sense.
I don’t know about you but for me doing stuff like this is really fun. Automating work flows and making things easier for myself is one of my favorite parts of being a developer.
I’m happy I spent a few hours writing that script because now instead of dreading waking up to seeing dozens of form submissions I’m very much looking forward to that and I hope one day it happens. It’s like a whole different perspective on life just from a single Python script.
Another takeaway is that shipping something and getting real data or feedback is the most important thing you can do. I never even thought about making this script until I had a real problem to solve and it took multiple iterations at different scales to get there.
What’s one of your stories where you automated something? Let me know below.