The Metric System
Sean Moore
You only improve what you measure. But what to measure, and what to improve?
Metrics are the name of the game nowadays. Interaction tracking, A/B tests, user feedback, all of it to better shape the things we use and love. We’ve gotten quite good at the collection - most of it is automatic, watching our every action within a system and silently recording the decisions. It’s a giant snare of complexity. Yes, we’re quite good at raking in information. What we’re still struggling with is making sense of it all.
So let’s talk about Lift.
Lift is a nifty little social/ productivity hybrid app that I recently have taken a liking to. It lets me keep track of all the habits that I’d like to build, giving me some reinforcement as I build up a count over a week, and track my long-term progress as well. There’s also a social aspect, allowing comments and “props” to your progress, and the progress of others who are working towards the same goals.
In a recent app update, the developers shared some of their data that influenced one of the changes they made to the interface:
[P]eople check in to habits at a rate that’s six times greater than they give props and sixty times greater than they leave comments.
Knowing that data is extremely informative in directing product revisions. So the lovely people at Lift took this insight and came to a perfectly valid conclusion: habit check-ins are what most users want to do with the app. And then they used that conclusion as a basis for changes to the app to make the check-in process be as frictionless and enjoyable process.
There’s a fork in the road in all of this though, because between the data and the design decision, an important assumption was made: that little conclusion that that the fine folks at Lift came to. But it’s not the only possible one. I, for example, check-ins happen more often than other activity because “props” and comments are difficult actions for users to perform, or users don’t understand why they would want to give them in the first place.
And even if the conclusion was correct, and it is very much so in the realm of possibility that it could be, there is still the question of whether making what users already actively do – nearly to the point of exclusion of all other actions in the app – is in line with what the overall goals of the app are aiming for. This app is designed to put users on track with habits through positive reinforcement, and if the designers and developers thought that “props” and comments were an important enough part of that positive feedback to design, implement and test in the first place, does it really do justice to the vision of the app to further bury these interactions beneath the “frictionless” check-ins?
Obsession for the truth drives us into this collection craze; we want to know what works, and what doesn’t so we can make the best things. And so we collect, and we analyze, and we conclude. But having numbers to back up reasoning doesn’t necessarily make your logic any better or worse; it just gives more weight to whatever decision you do eventually settle on and that inertia may make it harder to change course at a later date.
By all means, let data inform your decisions. But don’t forget that design isn’t about making the most well-supported decisions. It’s about making the right ones for your users, your company, and your vision.
Don’t let the facts that are steer you away from the future you hope to be.