Hoarding: How to Prioritize Features to Clean Up the Clutter

It’s time to name and shame a behavior that costs our economy billions each year, wastes most big businesses’ precious resources, and makes our jobs in software development just that little bit more miserable and conflict ridden. I call it hoarding.

Last year, I was driving along with my two daughters in the back of our car. Ash, the eldest, then nine, started telling six-year-old Alice about how bees pollinate flowers when they make honey and how that caused food to grow and how without bees the world would run out of food (and honey) and we’d probably all die.

Alice nodded all the way through Ash’s explanation, made a thoughtful face at the end, then said, “I’d probably die without potatoes.”

Alice loves potatoes—especially roasted ones.

Once, a couple of years ago, we ran out of potatoes at the end of a meal and wee Alice—who is, honestly, the cutest thing you’ve ever seen—was grumpy for the rest of the day.

Nowadays, Alice attempts to hoard spuds at the start of the meal, just in case. As soon as my wife puts the dish of spuds out on the table, Alice grabs as many as she can, loading her plate high up. Why? She's afraid that if she doesn't, there'll be none left later on and she will miss out. It's the tragedy of the commons in miniature, and it wouldn't happen if Alice didn't perceive a scarcity of spuds. If we cooked 200 spuds for our family of four, then there'd be no need for her to hoard. But we don’t, so she hoards, which encourages the rest of the family to hoard, lest we miss out; we often end up throwing out a few of the hoarded potatoes.

Hoarding, generally speaking, happens whenever there is competition for a valued and scarce resource. You see hoarding before and after natural disasters when people dash to the supermarkets and empty the shelves, selfishly—but understandably—grabbing as much food as they can for fear that they’ll starve. Big manufacturers, on the other hand, competitively hoard components and raw materials they speculate will be scarce in the future.

Likewise, hoarding is an incredibly common—but usually unnamed and invisible—phenomenon in corporate software development. Our customer's hoard scarce IT time by loading as many features into their requirements document as they possibly can, even though many of the requirements are of questionable value. Their reasoning is understandable. They may think something like this: “If we don’t ask for feature X now, and we find out later on that we need it, then we’re out of luck.” The customer knows they’ve waited two years for their project to get to the top of the software development queue. They know that once their project finishes, they'll not receive any more IT resources to work on feature X for a long, long time—if ever. So, just like my Alice, customers hoard their metaphoric spuds.

This resulted in requirements documents being loaded with low value containing a lot of “we might want it one day in the future, but we might not” features. The product is much bigger to build than it needs to be, it’s more challenging and considerably more time consuming to test, and the projects take considerably longer to deliver. Financially speaking, it costs more to deliver and the project’s financial benefits, which are only earned by shipping, are delayed, as are the start and end of the next big project.

If you’ve been doing agile for a while, you are no doubt aware of the cost of hoarding and you’ve removed much of it by the simple act of getting your customer to prioritize features and stories then releasing them as soon as you can. By working that way, it usually becomes obvious when you’ve started working on the low-value features, at which point it would be better to move your development team on to the next big project and start working on its valuable features.

But what can you do if you’re not doing agile yet, or if you’re struggling to get your customers to prioritize? Start by collecting data. Go back over the software produced in a sample of past projects and guesstimate what percentage of the features built back then were actually needed. If you can, ask for help and validation from users and customers. You won’t come up with a precise number, and some of the little-used features are—just like the self-destruct button on a space ship—necessary, but hopefully never used. Nonetheless, you’ll be able to come up with a range.

Let’s say the range of needed features is between one quarter and one third of the features built. Now, let’s assume that you could finish each future project 25 percent sooner. It might be more than that, in reality, but it might be less. You’ll never know for sure, so pick a nice round sounding number then start to play around with it. For instance, if you can finish each future project 25 percent sooner just by prioritizing, then that’s like getting 33 percent more staff for free. FOR FREE! And, pretty soon, you’ll start finishing projects before they otherwise would have been started. (Fire up your spreadsheet software and simulate an ongoing series of projects that takes four months in column A and compare them to a series of projects that takes three months).

The next step is the most difficult: If you don’t know what to do with those numbers, then find someone who does (Your boss maybe? Or your bosses’ boss) and ask them for help. Most of us don’t have the skills to have the hard conversations like this.

The first thing your new partner is going to ask is this: I understand the problem. What do we do about it?

There are at least two significant steps. The first is to create a new role called a “flowmaster,” which you can think of as a high-level product owner. The flowmaster’s job is to dish out the potatoes profitably. He manages the list of projects. He decides when each project stops and what new project should be started to take its place. This is a senior role and an unpopular one because the flowmaster has to make some hard commercial decisions.

The second step is to either start working iteratively and incrementally, as we do in agile, or, at the very least, to limit the size of projects. If a project typically takes a year, you’ll want the flowmaster to split it into three or four much shorter projects. That forces prioritization. To put it in terms my Alice might understand: Not only are we going to ask her to not hoard potatoes, but we’re going to give her a smaller plate to make it easier not to hoard. If you still have an appetite for more spuds near the end of the meal then you can always ask the flowmaster (or mum or dad) for more.

One final word of warning: Be careful how you use the word “hoarding.” It’s an emotive term. It sounds like you’re blaming your customers when, in reality, they’re not to blame. They’re just doing the best they can given the way we have traditionally built software. Your two suggestions—authorizing someone to prioritize projects and putting a limit on the size of projects—change the system. Within months, you’ll discover roughly one third more capacity, and you’ll push more projects through your development system than you ever expected too. And you know that scarcity that drove the hoarding behavior? Much of it will disappear as your project backlog shrinks.