Tests as Institutional Memory

testing integration-testing engineering-culture leadership
RSS Feed

Integration tests aren't just good practice -- they're how teams retain fragile knowledge. People quit, teams are restructured, RIFs happen. Confluence documents lag behind new features. Jira tickets pile up and go stale. Eventually the only person who understood why something works the way it does is gone.

Tests can't be broken without blocking development. That forcing function is the point. A passing test suite is a living specification -- not what someone wrote in a doc two years ago, but what the system actually does today. Add a comment explaining why a test exists and you've captured an architectural decision that survives headcount changes.

We had an opensearch cluster that was undocumented and critical to the business. It housed most of the company's content and powered search. When new feature requests came in, we had no idea what we'd break. The problem? Jira tickets were vague or straight up missing, the PRs descriptions were all AI summaries of work rather than reasons behind changes, and you can forget about a confluence document.

You can read the full story in Reverse Engineering Legacy Systems.

I went in fully blind, not knowing the service or its behaviors at all. I wanted to clean up the code so that people would be able to read it and learn it easier. I definitely violated the "Chesterton's Fence" argument, but I was never going to get answers since no one I asked could answer questions about the service. My tests allowed me to learn the system and simplify it confidently. However, that's not the most significant payoff.

Two weeks later, I received an email that the CI pipeline failed. It was my integration tests, catching a regression that would have slipped through unit testing, and it certainly would have gone unnoticed for months due to it affecting rankings, not validity. Since I wrote it, I have noticed it fail at least two additional times, not to mention the countless times it has aided me on local development. This should not be a one-off exceptional example, this should be the default.

Why Testing Gets Skipped

If you're reading this blog, you're likely not a junior engineer. We all know the countless reasons tests get skipped. Leads, Managers, Product, other departments, and often even the engineers themselves, all want velocity. They want to finish their tasks and move on. They want to be told good job as quick and as often as possible. There's pressure from all directions to just move as fast as possible -- especially inside of a feature factory. Writing tests and documentation feel like they don't provide value to the business. And to an extent, they're right... At least in the moment. No one is rewarded for writing tests. They feel slow and unrewarding.

A Reframing of Tests and How to Introduce Them

Tests are an upfront cost to velocity, but they can be a godsend when observed over time. Reduced incidents, fewer QA gotchas (or worse, misses), fast feedback, and documentation after the author eventually moves on. So long as the tests aren't coupled, flaky, or fragile they should provide value over years.

They improve getting new engineers up to speed by acting as documentation. They reduce risk to working on the unfamiliar code. Adding a new test or playing with test data can help understand the actual functionality and provides a playground. They allow adding new features with confidence that the existing work flows continue to work or break in the expected way.

Here's what you can do at your org to make a significant change:

  1. Add code coverage requirements to your pipeline. This should be a floor, not a ceiling. Even a 10% coverage requirement will pay dividends.
  2. Ask for tests in PRs, even if it seems annoying. Don't be overly prescriptive, but by making sure there's one or two valuable tests, we can start our journey.
  3. When touching unfamiliar code, read the tests or add some if they're missing.
  4. When we add or review tests, make sure there are comments around WHY we're testing, not just what or how. This is our documentation. This is important.

Tests become the lifeblood of the company. You don't need to be the one adding all of them, but you should make sure the knowledge doesn't leave with someone forever.