DevOps for a Small Engineering Team

September 13, 2024 (2y ago) · 2 views

Most DevOps content online explains the same loop: CI, CD, IaC, monitoring, collaboration. That is fine as a glossary. It is not useful when you are a small team trying to keep a long-running product stable while also shipping new work.

This is what DevOps looked like for us — not as a job title, but as a set of decisions about where automation earns its keep.


The context

I took over a software company that had been running for 15 years. The product worked. Customers paid. But the operational layer had grown unevenly:

  • some deployments were manual
  • some environments were documented only in people's heads
  • some failures were recoverable in minutes, others required heroics
  • infrastructure knowledge was concentrated in one or two people

That is a common pattern in mature small companies. DevOps was not about adopting every tool in the AWS diagram. It was about reducing the number of things that only one person knew how to fix.


What we prioritized first

When the team is small, you cannot implement every DevOps practice at once. We ranked work by blast radius:

PracticeWhy it came firstWhat we skipped early
Repeatable deploysManual deploys were the biggest source of fear and downtimePerfect multi-region architecture
Secrets outside the repoCredential leaks are harder to undo than slow deploysFull secrets rotation automation on day one
Observability on the serving pathIf users cannot reach the app, nothing else mattersComprehensive tracing everywhere
Infrastructure as Code for productionConsole changes drift and break recoveryIaC for every throwaway experiment
Backup restore drillsBackups that never get restored are fictionFancy deployment orchestration before basics worked

The pattern is simple: automate the painful, frequent, and dangerous work first. Leave the impressive architecture for when the basics are boring.


CI/CD that matched our size

We did not need a platform team. We needed a pipeline that made production deploys boring.

What actually helped:

  1. One release path. Production deploys happen from a known branch through CI, not from someone's laptop.
  2. Build once, promote the artifact. Same Docker image or build output moves through environments.
  3. Migration step before traffic moves. Database changes are part of the deploy, not an afterthought.
  4. Rollback that is practiced. A rollback path that nobody has tested is not a rollback path.

What we deliberately did not do:

  • deploy every commit to production automatically on day one
  • maintain five nearly identical staging environments
  • optimize pipeline speed before pipeline reliability

For a small team, continuous delivery is often enough. Continuous deployment can wait until your tests and observability are strong enough to catch regressions without a human gate.


Infrastructure as Code where it mattered

IaC is easy to overdo. We used Terraform for the parts that are expensive to recreate by hand:

  • VPC and networking
  • production databases and caches
  • load balancers and ingress
  • IAM roles for CI
  • disaster recovery wiring

We did not Terraform every internal script, one-off cron, or local dev convenience. The goal was not "everything in Terraform." The goal was "production can be rebuilt without guessing."

That distinction matters. Small teams lose weeks turning IaC into a religion instead of a recovery tool.

The payoff showed up clearly when we rebuilt production on AWS with explicit disaster recovery. The DR work is documented in two posts:

Those projects were only feasible because production infrastructure was already treated as code, not as a pile of console decisions accumulated over years.


Observability without a platform team

We did not build a full internal observability platform. We did make sure the basics existed:

SignalMinimum barWhy
LogsSearchable request and error logs with correlation IDsYou need to answer "what broke?" quickly
MetricsLatency, error rate, queue depth, CPU/memory on critical servicesTrends matter more than dashboards with 200 charts
AlertsPages for user-visible failure, not for every metric twitchAlert fatigue kills response quality
Deploy visibilityKnow which version is live in which environmentDebugging production without this is guesswork

The best observability investment for us was not a new tool category. It was agreeing on one canonical log line per request with identifiers that survive across services. That made cross-service debugging possible without a dedicated SRE team.


Communication is part of the system

DevOps is often framed as tools. In a small company, the operational bottleneck is just as often communication:

  • who is allowed to deploy
  • what counts as an incident
  • when to roll back versus push forward
  • where runbooks live
  • how on-call rotation works when the team is tiny

We kept runbooks short and tied them to real failure modes we had seen: deploy rollback, database restore, queue backlog, provider outage. Long theoretical runbooks are shelfware.


What I would tell a small team starting now

  1. Do not copy enterprise DevOps maturity on day one. Start with repeatable deploys, secrets hygiene, and logs you can search.
  2. Treat restore drills as part of delivery. If you cannot restore, you do not have backups.
  3. Use IaC for production recovery paths, not for vanity completeness.
  4. Automate the steps people get wrong at 2 AM. That is where DevOps pays rent.
  5. Write down tradeoffs. Future you needs to know why you skipped something, not just that you skipped it.

DevOps for a small team is not a toolchain checklist. It is the discipline of making production boring enough that the team can keep shipping without betting the company on one person's memory.