During the last decade I was involved with several software endeavors that made the mistake of creating an initial architecture based on microservices, or adopting them too early. On all occasions, the approach backfired, causing deep frustration and resulting in a significant overhead in development costs. This is because microservices are a solution to a problem - how to enable multiple engineers or engineering teams to make continuous changes to the software and the production runtime at scale, without stepping on each others’ toes.
However, microservices incur a significant overhead; They create rigid context boundaries that are harder to change, as opposed to a system that runs in a single process. Since at an early stage, we don’t know what are the right context boundaries, and need to frequently change our software due to changing requirements in our attempts to find a product-market fit, a microservice architecture at this stage will solidify our design too soon and reduce the flexibility of our system. In order to keep our codebase malleable and flexible, it is preferable to start new endeavors in a single, monolithic project, that can gradually evolve into a monorepo consisting of multiple microservices - if and when the need arises.
When engineers hear the word “monolith”, they often envision a big ball of mud that is impossible to change due to coupling, duplication and other violations of engineering best practices. But in fact, it is completely possible to create a modular monolith where at any point in time, it’s easy to extract a module into a separate process - yielding a microservice.
A system composed of microservices experiences another set of problems, in the domain of distributed systems. When two logical modules communicate with each other within the same process, one can safely assume that the communication is reliable. As soon as a call crosses a network boundary, we must take into account possible failures such as packet loss, temporary unavailability of the callee (for instance, during deployment) and latency issues - including timeouts. In addition, sending data over the network requires it to be serializable, which often results in hidden bugs, when some data is serialized in an unexpected manner - for instance, dates.
Another disadvantage of a microservice-based architecture is the complexity of testing. To test a system that runs in a single process, all you need to do is start the process. Starting a Kubernetes-based cluster of microservices, each with its own database, incurs more complexity and latency and as a result, software systems that started their life as microservices often have system tests for each microservice but lacking E2E test coverage - meaning that there are no high-level tests that assert that the system actually works. If we, instead, started with a monolith, the simpler system tests could evolve over time into complex E2E tests, when microservices are split off the monolith.
How can we ensure that our monolith evolves in a modular fashion and retain our ability to extract any module to a microservice when needed? One might be tempted to come up with a complete high-level design and define the context boundaries in advance, hoping that when we want to extract microservices from the monolith, the task would be relatively painless. However, my experience has been that this is a futile attempt. In a project I have been involved in back in 2018, the CTO spent a significant amount of time creating a well-defined set of high-level services defined in Protocol Buffers IDL, and used code generation to convert them into Go interfaces. As a result, changes to the context boundaries became a complicated effort, and it required all communication between modules inside the monolith to be serializable into Protobuf messages, thus limiting our ability to facilitate emergent design and incurring accidental complexity. Eventually, the sole service that actually needed to be run as a separate process was dealing with a problem the original design didn’t even consider, rendering the whole effort moot.
A better approach would be to gradually evolve the monolith, using quick iterations and frequent refactors, to facilitate an emergent design. Initially the code would be very messy, but as we iterate over the product in our attempt to find a product-market fit, we’re bound to throw a lot of code away anyway. As soon as a product concept solidifies enough, clear context boundaries emerge and the interfaces can solidify as well. A few months of frequent refactors and a sufficient focus on tidiness should yield a monolith with clear context boundaries and separation of concerns, that can be broken down into microservices, that actually represents the current problems rather than what we initially envisioned when we kicked off our journey.
