How We Measure Feature Flags’ Success 

We often get asked how to measure feature flags’ success, and for us, the answer is qualitative rather than quantitative. It also boils down to a single factor: release confidence for developers. There are metrics that feature flags affect in terms of team productivity but in our opinion, the single most important change is the feeling of not being impeded in your daily workflow as often. 

When we speak to development teams that use feature flags, almost everyone says the same thing. You can and will track metrics, but developer confidence—and the knock-on effect on release velocity—is our main measure of success. 

Feature flags and daily workflows

Quite often when you’re writing code, you know you want to finish a feature but can’t because there’s a dependency on another team or another platform that’s not ready. That blocks you. Not only that, it blocks the build, the code and the release. 

With feature flags, you don’t have to worry about that block. Wrap the feature in a flag, push your code and then when whatever you’re waiting on is ready, you can go into Flagsmith and enable it. 

If you can push to production with the safety net of a feature flag, your release velocity is going to go up and your confidence is going to go up with it. You’re also going to reduce the number of instances where you get into messy branch situations. You’ll notice fewer monster pull requests with a heap of changes and human error from bad merging. 

When production is updated more frequently, it generally means you don’t have too many long-lived feature branches and those nightmares happen less. This means releasing code more regularly and more confidently.

Releasing rapidly for more stability (and less scary PRs)

You would think that releasing more rapidly would decrease stability but usually, the inverse is true. If releases are less of a big deal, you make fewer mistakes because it’s more manageable to release code. 

We’ve found that to be the case, and would rather (on the front end at least) release iteratively and know that a feature will be a work in progress than leave it in a long-lived feature branch for weeks or months. 

We would rather put code that we know won’t work or has an issue behind a feature flag than deal with a 5000-line pull request that’s riddled with collisions. Those PRs are terrifying. When you’re running production code with a high-traffic API, it’s much more stable to use a feature flag than to add the PR to your main branch. 

What are some metrics to look out for? 

There will be quantitative changes to look out for as well. We speak to customers who are deploying versions of their applications more often and pushing to production more frequently, and that’s going to positively affect metrics. A few examples of metrics to watch—and the way that feature flags commonly affect them: 

• Reduced open pull request times
• Lower incident rates
• Fewer merge conflicts 
• Higher frequency of rollouts
• More successful rollbacks
• Increased release velocity and team velocity

Conclusion

If you analyse quantitative things like incident rates after using feature flags, you’ll likely see release metrics improve. Arguably, though, the most important benefit is developer confidence. If a feature is wrapped in a feature flag, you can push it to production, test it there, and roll it back with confidence and safety—all without having to wait for other teams and features to be ready.