How to enforce code quality without sacrificing CI speed

Better metrics than just test coverage

Imagine building a massive LEGO castle with thousands of tiny pieces. You and your friends each add a few bricks at a time. But without rules, someone might use a brick that is the wrong color or shape, and soon the whole thing looks messy and might even collapse. That is what happens in software when teams grow and code changes fast without quality checks. Teams need a way to automatically enforce rules so every new piece of code fits perfectly, without slowing everyone down.

Many teams use unit tests and code coverage as quality gates. The idea is simple: if new code does not have enough test coverage, it should not be merged. But measuring only coverage can be misleading. Think of it like a school report card that only counts how many assignments you turned in, not how well you did on them. A student could submit all assignments but still fail the class if the answers are wrong. Similarly, code can have high test coverage without being correct or maintainable. For example, a developer might write tests that only cover the happy path and ignore edge cases, or they might write overly complex tests that are hard to maintain. This is why many teams end up with slow CI pipelines full of false positives they have to babysit.

• Use tools that integrate with your CI system
• Start with warnings instead of failures to educate teams
• Focus on metrics that matter for your context
• Use phased approaches with stricter rules in production branches

Putting it all together: An example pipeline

Instead of just checking if code is covered by tests, teams should check how well the code is designed. One way is to use mutation testing, which changes small parts of the code to see if tests catch the errors. If tests pass even after introducing a bug, then those tests are not good enough. Another way is to use linters, which are tools that scan code for common mistakes or complex patterns that are hard to maintain. For example, a linter can flag code that is too complex, like a function that has too many decisions points, making it hard to test and maintain. Teams can set rules like ‘no function should have a complexity above 20’, and the CI pipeline will block any change that does not meet that standard. This is like having a spell checker and grammar checker for code. It does not just check if tests exist, but if the code is well-structured and easy to maintain.

For teams starting out, the easiest way is to use a tool that runs on every pull request. For example, a Python project can use `ruff` to check for style issues and common bugs. The maintainers can set it up so that if `ruff` finds any errors, the pipeline fails and the author must fix them. But if the team is not used to such tools, they can start by having `ruff` only warn on violations, not fail the build. After a few weeks, when everyone is used to it, they can switch to failing the build. Similarly, for test coverage, instead of requiring 80% coverage on every change, require that new code does not reduce the overall coverage. This avoids punishing teams for the past and focuses on future changes. Another approach is to use mutation testing only on the main branch, so it runs less often but still ensures quality. The key is to start simple and automate what you can.