I recently had an interesting discussion with a friend about how to “measure the value of a product management team”, and it really got me thinking - what are the metrics that we can stand in front of and unambiguously say represent “success” for a product management team? It seems obvious; good product teams are successful if their products are successful. But some people might wonder if the products would have been equally successful had the “product management” just been done by the Sales team and the CEO. Are they wrong? How can we tell?
As usual, I asked Twitter and got a variety of responses back, ranging from “Why the hell would you even measure this?” to very specific metrics that are probably more measures of development team efficiency. Personally, I don’t have an answer but I am super curious and will be doing more research! I’d be interested to hear your takes too.
A message from our sponsor
Yes, yes, I’m sponsoring myself again but, hey, my kids need to eat. As you probably know, I've moved into freelance coaching and consulting for B2B product companies. I want to help your company, your team and your... well, you, get better at product management. If you want to chat with me about what I can do for you, why not head over to book a call with me? We can discuss your needs and how I can help.
New Podcast Episode: Embracing Change to Innovate in Product Management
I recently had the pleasure of a sit-down with Greg Coticchia. Greg’s the CEO at Sopheon, an innovation management platform, and he’s passionate about helping companies throughout their innovation lifecycle. He’s also been in the product management game since the 80s, so we reflected on what has changed, what has stayed the same and some of the barriers to innovation in established companies.
Check out the episode on your favourite podcast app, or get it right here.
Innovation is about more than just shiny new tech
It's easy to get excited about new tech, but we should get equally excited about repositioning existing products or building new business models to serve novel segments. Everything should always be focused on the users!
Why You Should NEVER Divide by Effort When Prioritising
I was recently chatting to a product leader about the perennial topic of product prioritisation. Prioritisation is, of course, one of the key parts of a product manager’s job. It’s also one of the hardest, so it’s no surprise that there have been various attempts to provide frameworks to make it easier to manage, more explainable and more data-driven. I even made my own, the NEVER framework, which I believe is the de facto standard in sales-led B2B organisations (please don’t use this framework).
There are numerous popular frameworks out there. Some, like MoSCoW and Kano, are more qualitative. Others attempt to be more quantitative, including frameworks like “Value vs Effort”, Weighted Shortest Job First (WSJF), RICE and ICE (the latter championed by my former podcast guest Itamar Gilad - check the episode out here!). There’s also other stuff like opportunity scoring and “buy a feature”. Each of these frameworks or approaches brings something different to the table.
Personally, I’m not an advocate for any particular framework and don’t really advise using them. I do, however, understand the desire to try to bring order to chaos. But, when I think of numeric frameworks, I do have a few concerns.
Numeric frameworks are often still based on opinion, just wearing a quantitative false beard
It seems to make sense that we can score things on a scale, turn the handle and have some numbers come out. It feels neat and somewhat scientific. We have numbers! But, ask yourself, where do those numbers come from? In almost all cases they are, at best, guestimates. Ask 5 people about the “impact” of an initiative and I’m sure you’ll get 5 different results depending on their own pet projects. “Effort” is always difficult to estimate upfront. Confidence can vary wildly depending on who you ask, although I do like Itamar’s “confidence meter”. The numbers you’re putting in are probably all guesses (or, worse still, averages of various people’s opinions).
Scoring frameworks drive an expectation that we just start working on things in that order
One of the worst prioritisation exercises you can undertake is to try to score an entire backlog full of ideas, sort it all on the score and just start work building things in that precise order. Now, the best product teams understand that any prioritisation framework is really just the starting point to provoke the right questions and discussions. They also appreciate that priorities can (and should) change throughout the lifecycle of a product, rather than sticking to some yellowing old document they produced months ago. On the other hand, having a big old list of features scored in priority order drives the subconscious (and often conscious) expectation that things will be worked on in exactly that order and, worse still, that they’re all going to get done eventually. It’s just a matter of time, after all!
Dividing by effort can lead to delaying the things that can really make a difference
Any framework that divides by effort is, by definition, deprioritising things that are more effort. On the face of it, this makes sense. If something has a Value of 6 and an Effort of 3, it’s going to be easier to get done than something with a Value of 10 and an effort of 10. Now, if you only have a handful of initiatives then maybe you’ll get to that Value 10 thing soon enough. But, product management doesn’t work like that; there are always more ideas than we can build. Dividing by effort might mean we never even get started on the Value 10 initiative, which is the most valuable initiative of all!
So, what to do instead?
Simply put… prioritise by value, and ignore effort. If the most valuable thing we can do also takes longer, it’s probably time to get started!
But, wait! Doesn’t that mean we’re not going to see any value for a really long time? Sure, if you love waterfalls. We can do better than that, though. Rather than prioritising big, really important initiatives lower than small, less important initiatives, let’s just break up the big, really important initiatives into lots of smaller initiatives.
User story mapping is a wonderful approach here. There’s a really good book on it that you should read right now! It’s a great way to break up big, complicated initiatives into smaller tranches of work, focused with laser precision on your users. But, even if you don’t use this approach, it’s important that you find ways to deliver incremental value in ways that represent progress and, ultimately, value for your users.
After all, I’m pretty sure that no one ever built a great product by just building all the easy stuff!
Ok, so the NEVER thing might be a bit strong. Maybe there are times that you legitimately need to hit everything in a big bang, and there’s some reason that you can’t break it down. I do always encourage people to work small though. Small is relative, depending on your own context, but I strongly believe that you can always try to work smaller.
Writing this newsletter is fun, and I love to give stuff away for free. But, if you want to buy me a coffee, you can always buy me a coffee. In any case, if you enjoyed this issue, please share it with your friends!
Thanks for reading One Knight in Product newsletter! Subscribe for free to receive new posts and support my work.
Quantitative false beard sounds fun as long as I can couple it with a matching mustache.
I haven't really heard ditching effort framed up like this. In all honesty, I've often used effort as a way to politely kill requests from high ranking folks in my different orgs who have a tendency to throw out partially baked ideas and try to push them to the top of the stack. But I could get behind what you're saying.