How we ran 20+ microservices with a team of 4

I once worked on a team which ran not 5, not 10, but yes, 20+ microservices (and yes, we did consider logical boundaries — each service had a well-defined purpose). The size of the team varied between 3–5 members at any given point in time. I have had a few recent conversations where developers have been afraid to embrace the microservice architecture paradigm because of the perceived additional maintenance and startup cost which may come with microservices. Below are some practices my team followed which helped us run many services at scale successfully.

Templated code repository structure

Over time, as my team developed a command over the language and a preference for specific libraries and patterns, we created a templated code repository for microservices. With the help of a command-line tool, and answering a few questions about the capabilities of the service — voilà! — a new repo for a microservice was easily created. The questions asked were along the lines of the following: is this an HTTP service, does it need a datastore, does it need to be a producer/consumer on an event bus, etc. This allowed developers to focus on the business and architectural problems, and not spend time on solved tasks like creating a base service.

Code patterns

With time, through conversations via pull requests, the team also developed agreed-upon best practices on how to use channels, goroutines, etc. For example, the first time we wanted to limit the number of concurrent database queries made by a given service, we had detailed conversations in the PR about the best approach. But when a second service needed to implement the same guardrails, we adopted the already established pattern. We didn’t have to resolve the problem, and it improved our developer velocity. With time, we had established a number of patterns and when writing a new service, the time was spent only solving new problems.

Areas of expertise

We couldn’t all be experts for all 20 services, so we instead focused on developing our areas of expertises. This was helpful because in case someone outside the team had a question, we had a go to person who knew the answer right away. Though some knew some services better than others, but because we used common code structure and patterns, it never took anyone much time to dig into another repository and understand what was going on.

Operational patterns

Just like our services, our dashboards were also stored in code and templated. This made it very easy to generate a stat dashboard and alerts for a new service through code generation.

Updating dependencies

When you need to update libraries, due to security vulnerabilities or to leverage new functionality, updating 20+ services can seem a daunting task. When we first started, we were updating our dependencies for each service by hand.With time, we were able to leverage the fact that our services had fairly uniform layout, which allowed us to use scripts to not only update the code, but also make PRs.

Make deployments easy

When you have to manage a large number of services, automating deployment and testing is of paramount importance. Once a change was merged, the code was tested, deployed, and graduated automatically through our environments, reducing the time and effort required by developers. In case a post deployment check failed, the switch to the new services was never made, always ensuring we had a working version of the code in any given environment.

I hope some of this was useful and would love to hear what your team does to manage and scale developing microservices!