Talking with 52 Devs about Building / Deploying Web Apps & Tech Stacks
This panned out over the last 11 months of running a podcast where I chatted with folks about tech stacks, lessons learned and best tips.
It’s been almost a year since I started the Running in Production podcast and at the time of writing this post there’s been 52 published episodes where I chatted with 52 folks ranging from solo devs deploying their first project to organizations who serve billions of requests a month and have dozens of developers on their team, such as Mux.
I figured now would be a great time to reflect on a year’s worth of episodes and extract out some of the tech patterns and best tips that I came across.
Overall there were 161 distinct technologies used. That ranges from various languages / web frameworks, databases, hosting providers, SAAS tools and so on.
# Most Popular Web Framework
Let’s start with this one because it might be what you’re wondering first. Truthfully this can’t be determined based on the 50+ episodes due to how guests were found.
To find guests I reached out to folks individually and posted on a few message boards.
Almost half the episodes are related to Django because when I posted on Reddit over 30 people responded. I’m still posting Django episodes almost 6 months after recording some of them, it was crazy haha.
But that doesn’t mean Django is the most popular web framework ever created. It just means folks using Django were super open to being on the show. Also, I haven’t reached out to a number of other communities too but I will in due time.
It’s a tricky problem because I still have 18 Django episodes that are recorded but aren’t published yet. Once that back log is published I’ll find guests from other frameworks.
One thing I will mention is, the language or framework you pick isn’t that important. You can develop successful applications in any modern web framework. Just pick something that you like and run with it.
Also, if you’re curious why someone picked a specific web framework from the show, each episode has a time stamp reference in the show notes for “Motivation for choosing XYZ”. That’s when they talk about why they picked a specific language and framework.
Innovation Tokens
One thing related to picking a web framework that was mentioned a few times was being aware on why you’re building something in the first place. Is it to build a product or is it to learn a new language or tech stack?
As I’m sure you know, one of the best ways to learn something new is to really dive into it and build something real with it. So it’s super easy to talk yourself into using the latest shiny tools to implement your next idea.
That’s great if your goal is to learn something new, but if you’re trying to build a product then you might end up getting hung up and severely delayed while you learn whatever new tech you’re trying to learn. It might prevent you from ever shipping something too.
But there is a middle ground…
One great thing that was mentioned a few times was the idea of innovation tokens. For example, if your goal is to build a product you can still experiment with new technologies but you should limit yourself to only let’s say 2 or 3 tokens, AKA. new technologies.
That means if you wanted to build an elaborate SAAS app from scratch while solving problems you never solved before, maybe you shouldn’t try learning Elixir, OTP, Phoenix, Live View, Elasticsearch, Kubernetes and 5 other things all together.
Instead, pick some middle ground where maybe you decide to build the core of the app in whatever web framework you already know but bring in Elasticsearch and something else.
This way there’s still enough familiar ground to keep moving forward and you get to experiment with something new and useful for whatever you’re building.
There were a lot of examples of this in the episodes where someone decided to use a back-end framework they were already familiar with and experimented with something new on the front-end.
Of course all of this varies on what problems you’re solving. Sometimes a new back-end language is worth spending your innovation tokens on as long as you’re aware of what it takes to learn a new language / eco-system.
# Docker / Kubernetes / ECS / Serverless
23 out of the 52 episodes had someone using Docker in one way or another. That could either be in development, CI, production or all of the above.
I was kind of surprised at how many folks used Docker in production but not development. There were way more cases of that than using Docker in development but not in production.
Keep in mind, the goal of the podcast isn’t to grill each guest and question them on why they did or didn’t use a specific tech, but the overall census seems to be that folks not using Docker felt like it was either too complicated or didn’t bring enough to the table to make it worth using.
Still, as someone who does happily use Docker I was glad to see so many people using it. Especially when it ranged from solo devs to bigger teams.
Lots of folks ended up using Docker Compose on 1 server. By the way, if you want to learn about Docker and Docker Compose, I do have a video course that goes over both in detail.
As for Kubernetes. It wasn’t that popular for solo devs or devs with small teams. Only 6 people mentioned using it. Half of which were on huge deployments.
That was kind of interesting because if all you did was read HackerNews you would expect everyone would be using it and it would be glorious.
Now, I’m not here to poop on Kubernetes because I do think it’s an interesting technology but I will say that during most episodes where we talked about Kubernetes, it was very much the case that Kubernetes is not a 1 click solution where you write some YAML and suddenly you’re operating at web scale without any issues.
In most cases it took a lot of effort to get to the point where they were using it, and it also involved solving many problems outside the scope of Kubernetes itself to get there, such as developing their applications to be suited towards being scaled, such as saving uploads to S3 or S3-like object stores, and just about everything in the 12factor app manifesto.
Then it was onto learning and applying Kubernetes itself. In most cases folks were using a managed Kubernetes solution from whatever cloud provider they decided to go all-in with.
Only 1 person was using Docker Swarm by the way.
ECS, which is Amazon’s container orchestration service ended up having 6 people using it too. Most folks who went all-in with AWS ended up using ECS over EKS. In some cases it was because ECS was around longer and in other cases they much preferred ECS over EKS.
As for Serverless, 5 folks were using it in some way or another. 4 of them had a few functions here and there for specialized tasks. 1 person used Serverless for a majority of their back-end and that was for an Electron based password manager.
# Hosting Providers
It was pretty much all over the place ranging from using PaaS solutions such as Heroku and Render to spinning up a single server by hand on DigitalOcean.
I don’t have exact numbers but it felt like nearly half the episodes involved deploying to 1 server and having a few seconds of down time during each deploy.
I think that’s a reasonable approach for a lot of cases. I mean, even GoRails which is literally a million dollar business is being hosted on a single $20 / month DigitalOcean server (database included) and the server barely breaks a sweat.
As for the other half, it was a mixture of using PaaS solutions and some type of container orchestration tool.
For self managed servers, it wasn’t only DigitalOcean that people used. There was a decent mix of folks using EC2 and a few people using Linode as well. To put things into perspective 14 people used DigitalOcean and 2 used Linode.
I’m not saying one or the other is better but I’ve been using DigitalOcean for my own stuff for years. They are not a sponsor of the show (I don’t have any sponsors), but I do think they hit a sweet spot of value, performance, ease of use and have a solid track record for up-time and security best practices.
I don’t use affiliate links much on this site but if you ever do decide to use DigitalOcean, using this affiliate link will give you $100 in hosting credits and if you end up spending $25, I’ll get $25 in credits. I pay for hosting the podcast out of my own pocket with no corporate backing, so every bit helps!
# Continous Integration
Roughly a third of the episodes involved using CI or CI / CD. It was a healthy mix of Jenkins, GitHub Actions, CircleCI and all of the popular CI services you’re familiar with.
For the specifics you can always filter any podcast episode by tag by going to the main episode listing page. It has a break down and count of every technology.
One thing that I enjoyed hearing was folks not really caring about perfection, or the perfect set up. A bunch of folks still SSH into a server and run a few deploy related commands directly after running a test suite locally on their dev box.
These weren’t unprofitable solo devs working on their first project too. It was running legit SAAS apps with customers and since that method hasn’t failed for them yet they continue to do it.
I think the takeaway is if you’re a solo dev you can get away with murder in some regards. From what I remember, every project that had more than 1 developer on it had CI set up and the bigger projects all had automated CI / CD with thorough code reviews.
Truthfully I do think using a CI service is worth it even for a solo developer and the CD part is a bit more debatable but I do think it’s worth striving for, but at the same time if it means spending 3 weeks coming up with an iron clad pipeline or developing features for an app that customers are paying for I can’t really blame anyone for choosing to ship features.
# SPA or Sprinkles of JavaScript
A little less than half of the episodes had apps that were API driven using React, Vue, or Angular on the front-end.
As for the apps that were SPAs there was a healthy mix of them being developed like that because it is / was considered trendy, they had specific heavy client side needs, had a big dev team or they offer a public API and wanted to build their own app using the same API (sort of like Stripe’s dashboard and Stripe’s API).
A majority of apps were using server rendered templates with sprinkles of JavaScript. Those sprinkles came in using a combination of mostly vanilla JS and jQuery.
# Databases and Popular Libraries
I was really surprised at how many folks are using SQLite in production. It wasn’t just for smaller sites either.
Jon Calhoun’s Go course platform uses SQLite and he has 15,000+ active users. He’s moving to PostgreSQL in the next version of his platform but it’s not because SQLite was too slow or it let him down. It was mainly because he’s going to be using Docker then so why not.
8+ SQLite episodes can be found here: https://runninginproduction.com/tags/sqlite
But with that said, 37 people are using PostgreSQL. It was by far the most popular database. Then there were 5 people using Elasticsearch, 3 using MySQL, 2 for MongoDB and 1 for Neo4j. In most cases Elasticsearch was used in combination with PostgreSQL too.
Redis got a ton of usage too with 22 folks using it in one way or another. It really is a Swiss Army Knife for caching, acting as a queue back-end and more.
As for popular libraries, it seems like pretty much everyone who built a RESTful API with Django chose to use Django REST Framework. Also, SQLAlchemy was used in nearly every Flask project that used a SQL database.
Various linters were also popular. There’s too many to list here since it depends on what language was being used but there’s a solid amount of folks using linters in general.
# Best Tips and Lessons Learned
One question I try to ask near the end of every episode is:
After building, deploying and running a web app in production, what are your best tips and lessons learned?
As much of a cliche as it is, one of the most popular answers was to keep things as simple as they can be and avoid over engineering things.
I sort of wanted to condense everything into a few paragraphs but honestly, a lot of the advice is amazingly useful and pretty different so instead I grabbed 1 liners from the show notes of every episode that had a best tip, which was all but 1.
In case you’re curious, every episode has a “Best tips?” question in the show notes with a time stamp straight to that part. You can always head to https://runninginproduction.com/podcast/ to see a list of episodes if you wanted to get more context around each tip.
- #001 Love / learn Docker and use static typing when you can
- #002 Build a ton of apps and automate as much as possible
- #003 Keep your code and infrastructure simple early on and ship your app
- #004 Embrace the golden age of the solo web developer being viable
- #005 Just get moving to the best of your ability
- #006 Serverless couples your app logic to your provider, it’s not great
- #007 Focus on the data relationships before writing your code
- #008 Jump into the code, there’s a lot to take in but it’s manageable
- #009 Create a code contract to help keep your code sane for years to come
- #010 Look at what you have, what you want to do and how to get there
- #011 Stop over-thinking things because just starting helps a lot
- #012 Things are more changeable than you think, keep it simple initially
- #013 Be careful about over engineering and keep your eyes on the prize
- #014 Profile and load test your app before you launch your app publicly
- #015 If you’re not comfortable with a technology, don’t self host it
- #016 With undocumented APIs, write tests and pry into the details
- #017 Make a solid proof of concept in your new tech before switching to it fully
- #018 Don’t be afraid to break your code up into multiple repos
- #019 Don’t start out with a fancy infrastructure, just get your app up and running
- #020 Learning to self manage your time and break down problems
- #021 Leverage the Elixir community if you get stuck
- #022 Have automated tests for your code base
- #023 Focus on doing something you’re passionate about, it really helps
- #024 Get your workflows right and then automate them with scripts
- #025 Learn by doing and don’t worry about making mistakes
- #026 Use tools you can be productive with & be mindful of your dependencies
- #027 Deployment is tough but it’s OK to make mistakes as you go
- #028 Don’t build something in new tech just to use new tech
- #029 Be mindful of the SAAS tools you use and how they interact with your app
- #030 Start slow and expect failures, also don’t chase perfection
- #031 Video is hard and it keeps getting more and more complicated
- #032 Start simple and grow it from there, writing any code is important
- #033 Release as often as you can and invest in your release process
- #034 Pick the tool you’re the most productive with and ship something
- #036 Use the tools that you know unless you have a compelling reason not to
- #037 Keep learning and just get something up and running
- #038 Start with a base application and server architecture to help estimates
- #039 Avoid over engineering things and don’t chase perfection
- #040 Try to learn best practices when you can from podcasts, books, courses, etc.
- #041 Try to validate your idea before coding it to completion
- #042 Start simple and really stick to it
- #043 Try to get your app in front of your customers as soon as possible
- #044 Test and log everything and try to keep up with the updates
- #045 Keep learning but pick things you’re familiar with
- #046 Soft skills and focusing on making contributing code easy
- #046 Keep your tests easy to follow so you can see how something works
- #047 Get your product out ASAP and be mindful of what happens if you leave
- #048 Put in the research before implementing something
- #049 Use tools that a lot of people have thought long and hard about
- #050 Don’t worry about not knowing everything from the get go
- #051 Learn one or more web frameworks
- #052 Don’t be afraid to build something from scratch
Behind the Scenes to Assemble This List of Data
Instead of manually going through each episode and formatting the best tips in the above list I used grep, sort, cut, sed and Vim to grab the tips from each episode.
If you’re curious, this video breaks down how it was done:
# Self Reflection
Despite developing websites and web apps since the Geocities era I still learn at least 1 new thing in every episode.
I really like when guests come on with like 1 year of experience and end up creating a great service that people are using and paying for. It’s a refreshing reminder that all you have to do is pick a tech stack and write code.
As you get more and more experience, it’s so easy to talk yourself out of things because you think “yeah but if I ever need to do…” or “well, that seems pretty good but…” or “mmmm, maybe but what about xyz shortcoming?”.
For me, I guess it’s because when I’m working on my own projects I’m a perfectionist to an unhealthy level and I find it difficult to start things until I’m 100% convinced I’ve picked the “best” solution and it’s very hard to convince me of that.
So imagine how that pans out if you find 1 shortcoming in a language or web framework, suddenly you’re on this endless quest to find the perfect solution when in reality no perfection solution exists.
You just need to pick something you mostly like, make the best with what you have knowing full well it’s not perfect today and it’s not going to be perfect a year from now. Just build something to see if people even want what you’re building and re-evaluate later if needed.
So that’s kind of what I got out of all of this. There’s lots of folks out there building cool stuff with varying degrees of experience using many different tech stacks.
I wonder how different things will be next year after I record another 52 episodes. I’d like to continue doing this until I can’t find anyone else who wants to come on. If you want to become a guest on the show, you can fill out this short form to get the ball rolling.
Also, if you enjoy the show and want to see more episodes, since there’s no sponsors or donations the best way to support the show would be to check out my courses on building and deploying web apps. Recommending them to a friend is also very much appreciated!
Have you listened to any episodes? Which one did you like best? Let me know below.