Engineering is a Process Not an End State
Product people aren't the only ones that should think this way
According to any number of pedestrian startup books out there, a marker of a good product team is the ability to build a product iteratively, learning lessons along the way that might even shape what the final end-state product looks and feels like. Build an MVP, experiment with pricing plans, do some A/B testing, accept that not all of the products you launch will work. Most importantly, prioritize speed above almost everything else, and learn from failures faster than your competitors.
This style of thinking has saturated the tech industry and has proven to be incredibly powerful. Engineers have adopted workflows like sprints, built continuous integration/deployment systems so that code can go into deployment the same day it’s written, and invested heavily in application monitoring with the understanding that more frequent code changes will lead to more bugs. These are all great developments, but in my day-to-day experience as an engineer, I don’t feel like they always go far enough.
Even though engineers have accepted that the products they’re building are subject to change and adaption, they still generally treat the systems they work on as “end-states”. They judge code by how well it functions currently instead of how well it functioned when it was built. They strive to build perfect systems for very experimental features, and then get secretly upset when their work doesn’t pay off because the feature wasn’t as useful to customers as expected.
I don’t want to spend too much time speculating about why we engineers tend to view engineering as an “end-state” task, but I think it’s for roughly two reasons: our training and our desire to model our systems like those at big tech companies. Academic computer science programs prioritize teaching students about how write efficient algorithms and learn what sorts of tools exist to do specialized tasks. I remember one computer science class that made us write tests for each function we wrote and pretty much just graded that section pass/fail on whether we wrote some form of success and failure test for each.
Systems at large, at-scale companies are also bad models of how code should be written at smaller companies still focused on product experimentation. Large companies will often write blog posts about how they added a layer of complexity to a system in order to improve one important or expensive operation. Much of the tone of these sorts of articles makes it seem like this complexity is something that could apply everywhere (of course these blog posts are mostly marketing and not education). Since engineers with experience in these types of environments make their way to startups quite frequently, and since all engineers are naturally interested in solving hard problems, we often get carried away when designing our own code.
Engineers trying to build a finalized, complex system from scratch is like a painter painting one perfect square inch at a time.
So, if building complicated, optimized systems from scratch isn’t the killer engineering skill to learn for adding value at a startup, what is?
In short, acting like a painter would. Paint the light colors first, those that could be overwritten with darker ones to cover up a mistake. Start with outlines, and only move onto details once those outlines are figured out.
Top skills of process-driven engineers:
They know where and when to performance-optimize the code that they write, and expend only the necessary energy doing this (since the rest of the energy goes into building faster).
They know generally what the product could evolve into and build designs that will be compatible with the likely product evolution, without optimizing too much for any one “final product”.
They write tests when their code is about to affect a production system or helps speed up their development, but no sooner. When they do write tests, they focus on writing integration tests (that test how the entire system should work) instead of isolated tests.
They prioritize any part of their system that might affect someone else’s work. This makes sure that any disagreements that might (and will!) arise won’t cause someone to have to undo a bunch of previous work to fit into another person’s framework.
In essence, process-driven engineers embrace uncertainty and accept that reasonable mistakes are going to be made.
On the surface, this may sound like a mindset or something that any manager could start to enforce within their team, but in reality it’s a long, hard process to grow these skills. In many cases it’s much harder to build a system that could work well-enough for either of two possible product use-cases than to build a system that works perfectly for one. It takes a really deep knowledge of a codebase to know which lines of code need to be battle-tested and which would only cause problems that would be cheap to fix later.
I don’t have a silver bullet to help guide engineers on making these tradeoffs well — it’s so context specific. Here are some general thoughts for engineering leaders, though:
Shift time from talking about what the final system will look like to what order it should be built in.
Shift time from talking about what the product will look like two steps from now to instead talking about what it might look like three steps from now.
Decide what types of bugs are tolerable since they can be fixed cheaply, and try to steer people towards avoiding expensive optimizations or testing around these cases.
Give credit for simplicity and for fast development.
It’s time that these difficult iterative engineering skills become highly recognized and prized the way that iterative product development is.
No man ever steps in the same codebase twice, for it's not the same codebase and he's not the same man.
[Did Heraclitus code??]