I used to think capacity planning was about setting up CloudWatch alarms and hoping they’d fire before things broke. Spoiler: that’s not capacity planning—that’s just reactive firefighting with extra steps.
Real capacity planning means knowing you’ll need more database capacity three weeks from now, not three minutes after your site starts timing out. It means being able to confidently tell your team, “We’re good until mid-January, then we need to scale,” instead of scrambling during a production incident.
After years of being on the wrong side of this problem, I finally built a system that works. Here’s what I learned.
The Problem With How Most Teams Do Capacity Planning
Most engineering teams fall into one of two camps:
Camp 1: The Reactive Firefighters
They wait until something breaks, then throw resources at it. CPU hits 90%? Spin up more instances. Database slow? Upgrade the tier. This works until you’re burning budget on over-provisioned resources or, worse, you’re down because you scaled too late.
Camp 2: The Over-Engineers
They build elaborate forecasting models with machine learning, multiple data sources, and dashboards that take longer to understand than the problems they’re solving. These systems are impressive but rarely used because they’re too complex for day-to-day decisions.
I’ve been in both camps. Neither works long-term.
What Actually Makes Capacity Planning Work
After multiple iterations, I’ve learned that effective capacity planning needs three things:
- Simple, Actionable Metrics
You don’t need fifty metrics. You need the right five. For most systems, that’s:
- Resource utilization trends (CPU, memory, disk)
- Request rate growth patterns
- Database connection pool usage
- Storage growth rate
- Network throughput trends
The key is tracking these consistently and understanding what “normal” looks like for your system.
- Historical Context That Matters
Raw metrics are useless without context. You need to know:
- What happened during your last traffic spike
- How your system behaved during that product launch
- What “seasonal” patterns exist in your usage
I keep a simple log of significant events alongside our metrics—product launches, marketing campaigns, major features. When I look at a utilization spike from three months ago, I can see it coincided with a feature release, not a problem.
- Forward-Looking Projections Based on Reality
This is where most systems fail. They either use naive linear projections (useless for growing systems) or overly complex models (unusable for actual decisions).
What works: trend analysis with growth factors that you adjust based on what’s actually happening. If you’re growing 15% month-over-month, project that forward. If a major feature is launching next month, factor in expected load based on similar past events.
Building a System That Your Team Will Actually Use
Here’s the architecture that’s worked for me:
Data Collection Layer
Pull metrics from your existing monitoring (CloudWatch, Datadog, Prometheus, whatever you’re already using). Don’t build a new metrics pipeline. Use what you have.
I built a simple script that runs daily and pulls:
- Average and peak utilization metrics for the past 30 days
- Week-over-week growth rates
- Month-over-month trends
Analysis Layer
This is where the magic happens, but it’s simpler than you think. For each critical resource:
- Calculate the current growth rate (7-day, 30-day, 90-day averages)
- Project forward using the most conservative (slowest) growth rate
- Calculate time-to-threshold (when will you hit 80% capacity?)
- Flag anything that’ll hit threshold in the next 60 days
I use Python with pandas for this. The entire analysis script is under 200 lines.
Alert and Reporting Layer
Here’s the crucial part: make it impossible to ignore.
- Weekly email summary to the engineering team showing resources projected to need scaling
- Slack alerts when time-to-threshold drops below 30 days
- Monthly capacity review doc that auto-generates and gets dropped into our team drive
The key is automation. If you have to remember to check a dashboard, you won’t.
The Guardrails That Keep It Honest
Early versions of my system gave terrible predictions because I made these mistakes:
Mistake 1: Trusting short-term trends
A three-day spike doesn’t mean you’re growing 300% per week. I added a rule: never project based on less than 14 days of data, and always compare against 30-day and 90-day trends.
Mistake 2: Ignoring known events
If there’s a major feature launch next month, your linear projection is worthless. I maintain a simple calendar of planned events that might impact capacity and manually adjust projections around them.
Mistake 3: Setting thresholds too high
Waiting until you’re at 90% CPU to scale means you’re already feeling pain. I alert at 80% capacity with 60 days runway. That gives us time to plan, provision, and test without panic.
What This Actually Looks Like in Practice
Here’s a real example from last quarter:
Our RDS database was sitting at 52% CPU utilization. Traditional monitoring said we were fine. But my capacity system showed:
- 30-day growth rate: 3.2% per week
- 90-day growth rate: 2.8% per week
- Projected to hit 80% CPU: 47 days
Without this, we would’ve kept ignoring it until we hit 85% and had to do an emergency scaling operation during business hours. Instead, we scheduled a maintenance window, upgraded the instance class, and validated performance; all without impacting users.
That’s what predictive capacity planning actually delivers: boring, planned infrastructure changes instead of exciting 2 AM emergencies.
The ROI You Can’t Ignore
Since implementing this system, we’ve:
- Eliminated emergency scaling incidents (which used to happen every 6-8 weeks)
- Reduced over-provisioning by right-sizing based on actual projections
- Cut time spent in war rooms discussing “do we need to scale?” from hours to minutes
More importantly, our team sleeps better. When you know what’s coming, you can plan for it. When you’re constantly reacting, you burn out.
Starting Your Own Capacity Planning System
If you’re building this from scratch, start small:
Week 1: Pick your critical resources
What are the three things that, if they ran out of capacity, would take your system down? Database? Application servers? Cache layer? Start there.
Week 2: Set up automated data collection
Write a script that pulls utilization metrics for those resources daily. Store them somewhere: S3, a database, even a CSV in git. Just make sure it’s consistent.
Week 3: Build your first projection
Take the last 30 days of data, calculate growth rate, and project forward. It doesn’t need to be fancy. A spreadsheet works.
Week 4: Add alerting
When time-to-threshold drops below 60 days, send an email. That’s it. You now have a working capacity planning system.
Everything else—dashboards, ML models, sophisticated forecasting—can come later if you need it. But this core system will save you from most capacity-related incidents.
The Mindset Shift That Matters Most
The biggest change isn’t technical, it’s cultural. Capacity planning only works when your team believes that preventing problems is more valuable than heroically solving them.
Early in my career, I got praise for staying up all night scaling systems during an outage. Now I get praised for the outages that never happened because we scaled two weeks early.
That shift from reactive hero culture to proactive engineering culture is what makes capacity planning sustainable. The system I’ve described is just the tool that makes that culture possible.
What’s Next
Capacity planning isn’t a project you finish; it’s a practice you refine. As your system grows and changes, your capacity planning needs to evolve with it.
The system I’ve built continues to get better as I learn what matters and what doesn’t. Some quarters I add new metrics. Other times I simplify and remove things that weren’t adding value.
The goal isn’t perfection. It’s having enough visibility and lead time to make good decisions instead of panicked ones.
If you’re tired of reactive firefighting and want to build something better, start with the basics I’ve outlined here.
