SAFe & Cost of Delay: a suggested improvement

I have previously shared my view on the way SAFe teaches Cost of Delay. It’s possible that the feedback came in too large a batch, so maybe I can break it down and suggest some incremental improvements. I’ll start with the part I struggle with the most and see if we can make it just a little bit better…

The SAFe “Cost of Delay” formula today

Here is SAFe’s formula for Cost of Delay, as it is today on the website:

Cost of Delay = User-Business Value + Time Criticality + Risk Reduction and/or Opportunity Enablement

One of my main challenges with this definition is that it adds ‘Time Criticality” to the other two parameters. To see why this doesn’t make sense, consider an option for which the “User-Business Value” = 0 and “Risk Reduction and/or Opportunity Enablement” = 0 but for which “Time Criticality” = 21.

In this case, SAFe is suggesting that

Cost of Delay = 0 + 0 + 21 = 21

This result clearly shows that there is a problem with the formula. If an option has zero value, then it shouldn’t matter how time critical it is, the Cost of Delay should be zero.

It has since been explained to me that SAFe practitioners do indeed recognise this as a problem. To cater for this, there is some sort of “strategy filter” which weeds out these sort of nonsense results. In effect, they put a lower bound of 1 on each of the components.

The need for this filter highlights that the SAFe formula poorly reflects the underlying principles of Cost of Delay. The filter doesn’t fix the problem though, it merely hides the most obviously flawed results.

What follows is a proposal for making a small, simple and hopefully as painless an improvement as I can think of. Assuming we can only use the same three qualitative inputs, how might we improve this?

Value

The current SAFe formula has two terms which are both about Value (“User-Business Value”, and “Risk Reduction/Opportunity Enablement”. Depending on the sub-type, these map perfectly well to the four buckets of value that I normally use, which (when they are all denominated in the same scale or currency) we can combine into total Value like this:

Value = Increase Revenue + Protect Revenue + Reduce Cost + Avoid Cost

In the same way, we can continue to add UBV and RROE together, like so:

Value = User or Business Value + Risk Reduction and/or Opportunity Enablement

Quick sanity check: either one of these can be zero, but if both are zero, then value (and subsequently, Cost of Delay) should also be zero. So far, so good – and no change yet, so this should be easy to swallow.

Combining with Urgency

The main issue lies in how we treat the parameter that SAFe calls “Time Criticality”. I usually call this “Urgency” and ask people to consider which of the four most common Urgency Profiles that a value proposition can have. This helps them do a better job of estimating what the impact of time is on the outcome you’re interested in. The result is Cost of Delay, for which the units are $/week or £/month.

As we’ve already said though, we cannot simply add “Time Criticality” to the other two value terms. Urgency is orthogonal to Value. I’ve also tried to make the case previously that urgency alone is not enough, you need both. Urgency is the part that turns a feature for which the value is estimated to be worth, say, $2m into a feature for which the Cost of Delay is $10,000/week. The same feature with a different urgency could have a Cost of Delay of $20,000/week, or as low as $0/week.  For those allergic to numbers, I already proposed a simplified qualitative Urgency. In this simplified model I proposed that Urgency is, in effect, multiplied by Value to get Cost of Delay.

Cost of Delay = Value x Urgency

Applying the same underlying principles to the SAFe formula give us:

Cost of Delay = (User or Business Value + Risk Reduction and/or Opportunity Enablement) x (Time Criticality)

If you have read this post about Qualitative Cost of Delay you should be able to see how these two align. In this case, you’re adding together two terms (U-BV and RR/OE) as Value on the vertical axis, with “Time Criticality” in place of Urgency on the horizontal axis.

Cost of Delay 9-box Qualitative per week

This approach also means that practitioners who aren’t yet ready to try quantifying Cost of Delay can use a simple matrix for visualising Value x Urgency as above. Feel free to use SAFe terminology, or whatever words resonate and make sense in your context. It’s the meaning that matters.

Making this simple change would immediately put SAFe in a better position than the more common Value over Effort approach to prioritisation, which fails to account for urgency.

Going further…

The aim was to improve the approach using the same three qualitative parameters. If those constraints were relaxed, we could go further. Of course, there will always be a continuum in the way people approach and start using Cost of Delay. (I am an Engineer, not a “purist”, and I am far more interested in the successful application of sound theory than religious adherence to an impractical theory.)

At least the underlying principles are now better reflected in the building blocks.

Comments 12

  1. Thx Josh, really helpful.

    More importantly, like you said, if we can get organisations to starting thinking about CoD in order to sequence jobs for maximum benefits but initially finding some building blocks to start with, then that can only be goodness.

    Its got to be better than the HiPPO and the LVD!

    Hope life under is treating you well.

  2. Time criticality has no place in the formula at all: When you are assessing the business value lost by delaying a feature you are explicitly taking time criticality into account – business value has to relate to a loss over a specific_period_of_time – i.e. “how much will be lost if I miss a specific release date?”

    From a practical perspective, duration of implementation also has no bearing if you’re simply deciding what to develop to go into a release on a FIXED DATE in future – it’s the development capacity in the period leading up to that release date that matters, since the length of time it takes to implement has no impact on the value lost – it’s either in the next release or it isn’t.

    So given a (fixed) future release date, a better way of generating an index value per feature to decide what should go into the backlog TODAY should be:

    net_loss_through_missing_that_release / effort_estimate

    (since as mentioned earlier, “duration” has no bearing when fixed dates are involved – something will either make it into that release or not – it’s a binary thing, not a sliding date, so we’re looking to prioritise the items with the highest value-to-effort ratio so we can fit more in…)

    1. Post
      Author

      Thanks for reading and taking the time to comment.

      Whilst I agree with much of the sentiment of what you say, what I observe in most organisations is that business value (especially of the relative, qualitative sort) tends to be treated as independent from when it is delivered. For example, a score of 8 points worth of “business value” would score the same whether it is delivered this month or next month. What you’re describing is an all-rolled-in Cost of Delay figure – in which case that would be fine, but it’s not how most organisations are treating it. (YMMV)

      I also see “Duration” as being useful as a separate parameter, especially in a world where Continuous Delivery is not only possible but a stated objective for even the biggest and slowest organisations. For as long as the things we choose to work on block the pipeline for different lengths of time it is going to make sense to divide by duration in order to maximise ROI for any length of time you wish to optimise over. You can find more discussion about why CD3 gives the optimum result here.

  3. Minor quibble, as I like what you are saying here and will do some experimentation with it on real work… but..My understanding is that the minimum value is 1, not 0

    1. Post
      Author

      I think you might need to reread the post. Here is the part where I specifically speak to the point you’ve raised:

      “It has since been explained to me that SAFe practitioners do indeed recognise this as a problem. To cater for this, there is some sort of “strategy filter” which weeds out these sort of nonsense results. In effect, they put a lower bound of 1 on each of the components.

      The need for this filter highlights that the SAFe formula poorly reflects the underlying principles of Cost of Delay. The filter doesn’t fix the problem though, it merely hides the most obviously flawed results.

      Perhaps there some way I can make this more clear. Open to suggestions…

  4. “This result clearly shows that there is a problem with the formula. If an option has zero value, then it shouldn’t matter how time critical it is, the Cost of Delay should be zero.”

    …or – when an option has zero value, we shouldn’t bother with any COD or WSJF calculation at all – problem solved?

    1. Post
      Author

      Yes, if an option has either zero value (or zero urgency) then we shouldn’t bother with any CoD or WSJF calculation. (This should be obvious.)

      However, this doesn’t solve the problem with the formula that SAFe has invented, it merely hides it!

      The point of plugging in zero values is to expose that the formula is nonsensical. Until SAFe changes their formula then the underlying problem remains.

  5. Cost of Delay is a powerful concept. Reading books like Lean Enterprise and Lean UX makes one wonder more how it would fit into an experimental approach to product work. Most feature ideas have negative actual value. (They either have zero or negative impact on key outcome metrics, and yet still cost money to implement.) So, instead of funding solutions, gathering requirements and building roadmaps, we capture assumptions and create a runway of hypotheses to test.

    How does this square with the application of CoD? If most feature ideas are wrong, there would seem to be a risk in just throwing them in a backlog and focusing on their better prioritization. Would you say one should first capture assumptions, prioritize them, write and test hypotheses, maybe do some user story mapping and THEN start building a backlog and looking at something like CoD?

    Thanks for your time.

    1. Post
      Author

      Hi Charles,

      Yes, Cost of Delay is a powerful concept :) We owe Don Reinertsen a huge debt of gratitude for his work. I’m merely applying what I’ve learned from him on the topic, and learning more about it along the way.

      You ask how does it fit into an experimental approach? Well, it really depends on what you mean by this, but I have seen it used in lots of situations where there is high levels of uncertainty about what is valuable or not, and where the probability of developing something of non-negative value is very high. In fact, I’d say that’s the vast majority of Product Development (whether we recognise this reality, or not).

      Completely on board with your “capture assumptions and create a runway of hypotheses to test”. I talk about this briefly in this post on Product Roadmaps.

      The question is, how are you capturing assumptions? And, how are you ordering your hypotheses to test? For the latter, I suggest ordering hypotheses by CD3. For the former, I generally find that to surface assumptions it really helps to switch to System Two and try to quantify Cost of Delay. For very early ideas, most of the value is likely to be in the form of information. Either way, Cost of Delay does a much better job of surfacing hidden assumptions than if you’re just doing relative scoring or some other surface-level method.

      I’ve made this point before, here:

      “The other advantage we have seen is that making an attempt to put numbers on the Cost of Delay forces us to surface assumptions. In doing so, we are confronted with the reality that our intuition about value and urgency is very poor, until we wake up System Two thinking. By surfacing our assumptions we get a chance to disinfect them with some sunlight, by making them visible and more open to questioning.”

      To be clear, I’m not saying that Cost of Delay is the be all and end all. It’s not. (There are no silver bullets, I’m afraid.) Does Cost of Delay help tilt the playing field in your favour? In my experience, yes, it’s better than the alternatives I’ve seen or heard so far.

      Hope that helps,
      /Joshua

  6. This is the perfect web site for anybody who hopes to find out about this topic.
    You know so much its almost tough to argue with you (not that I personally would want to… HaHa). You certainly put a new spin on a topic which has been discussed for
    ages. Excellent stuff, just excellent!

  7. Seems that even if one wanted to rate the business value at zero (instead of one), I can’t think of a situation where the time criticality or urgency would be high. If a feature has no business value, how can it be argued to be urgent? Your 0+0+21=21 example would lead me to question the competence of the person trying to determine the CoD for that feature.

    1. Post
      Author

      It’s not just my example. The creators of the formula apparently find it necessary to filter out such situations, so clearly it happens. Avoiding these situations, either through process or by self-filtering is quite beside the point though: which is, that the formula itself doesn’t reflect the underlying principles. These are just the really obviously flawed results. Because the operators are wrong, the SAFe formula will produce other spurious results that just aren’t so obvious, (since all the inputs are qualitative and relative).

      So, instead of questioning the competence of a person trying to determine CoD using the SAFe formula, perhaps you might question the competence of whoever came up with the formula in the first place? It’s pretty simple to fix, and I’ve provided the first baby-step suggestion for doing so here. It costs you nothing to consider this alternative alongside the official SAFe formula, so you’ve got absolutely nothing to lose.

      Let me know how you get on…

Leave a Reply to Darren Wilmshurst Cancel reply

Your email address will not be published. Required fields are marked *