Buying an off-the-shelf solution? Don’t forget the caravan!

Nearly every company I have worked with in the last 15 years has at some point embarked on a major system replacement, or spent a serious amount of time and effort investigating doing so. I wrote a bit about this here, referring to Joel Spolsky’s advice to never do this as well as Dan Milstein’s excellent reply: if you absolutely must do this, do it like this.

There is a variation of this theme though, where the approach to a system replacement is to migrate to a commercial off-the-shelf or COTS solution. The drivers for the change are usually the same as for the rewrite — the desire to reduce the cost of maintenance as well as making it faster and cheaper to provide new functionality, (some of which may be already built in to the off-the-shelf system).

The logic is clear. It’s like replacing your Reliant Robin with the bright and shiny new yellow Lamborghini that the dealer is offering for a really good price. The old Reliant Robin system is typically referred to with the slightly pejorative name “legacy”. The new Lamborghini often goes by various, more hopeful, forward-looking names.

The promise of the Lamborghini is a marketing feature list: the latest and greatest technology with a modern decoupled, service oriented design. It is streamlined and built for speed and high performance. It can of course do multi-channel “out of the box”, offering a simple solution to the very real need to have a mobile offering. Unsurprisingly, the business can hardly wait to sign the lease agreement, get it registered and get their hands on the keys. Enthusiasm and expectations are sky high. In the meantime, everything but the absolutely critical maintenance to the old Reliant Robin is put on hold immediately (usually before the decision to switch is made). Many of the key people who know the quirks and idiosyncrasies of the Reliant see the writing on the wall and start polishing their CVs and looking for the door.

Eventually, after a protracted business case process to confirm the decision made usually by people who don’t fully understand the implications the project gets the green light. The keys to the new Lamborghini are handed over and it’s ready to speed off down the road and impress everyone. This is where you strike your first problem: traffic!

It turns out that much of the slowness of the Reliant actually had little to do with the technology or even the rust and muck in the joints. It was primarily the upstream processes (prioritisation, funding, resource allocation, analysis and approval batching) and downstream processes (making it work with all the other systems and keeping it running and secure) that was slowing things down. These are kind of like the traffic lights, stop signs and intersections, tight bends, as well as “traffic-calming” devices like judder bars and sleeping policeman that force you to slow down.

These environmental aspects are not inherent to the technology, they are part of the wider organisational culture — especially it’s approach to the uncertainties and risk that are an inherent part of the complexity of developing software solutions.

Most organisations more than a few years old have grown a whole system designed specifically to control and funnel the demand for change. The annual budgeting process. The business case and approval process. The change decision boards and the out-of-band testing and quality control, (sometimes outsourced and offshored). It was the system for allocating resources and attempts to fix schedule, cost and scope in large batches — well in advance of all the discoveries about what was really needed and before we really knew how we were going to actually build it.

To add to the Reliant Robin vs Lamborghini analogy, there’s one vital piece missing: as much as you want to cut away the complexity and simplify the technology, unfortunately much of what makes the technology complex and a bit messy is actually a reflection of what your customers want and the environment that you operate in. In effect, the Reliant Robin not only has to navigate the traffic lights, intersections and other cars and trucks on the road, it also is hitched to something that is actually quite valuable to you: a caravan stuffed full of your customers. You know, that slightly inconvenient thing, the reason why the organisation exists at all?

The Lamborghini might be faster on the race track, but when it needs to also pull a caravan with product complexity, regulatory complexity, pricing complexity and other considerations of the market you’re in — this seriously slows it down. Typically, (just like the Lamborghini), your off-the-shelf system doesn’t have a tow-bar to hook the caravan up, or at least not one that can handle the weight of it.

This is something that both the vendor and decision-makers tend to massively underestimate. The vendor simply doesn’t understand the specific environment you operate in, or your customers and the “jobs” you take care of for them. Instead, they share lots of beautiful screenshots of the new Lamborghini racing at full speed on the open road with nary a leaf in the way, and nothing dragging it down. These pictures and the dream they sell are seductive for a business that has to live with the reality of a Reliant Robin stuck in a sea of red lights and heavy traffic and towing a heavy caravan.

Of course, there are some situations where it does make sense to migrate from the old to the new. Even off-the-shelf solutions can make perfect sense for those areas where you foresee no opportunity to differentiate from your competitors, and you have no concerns that a new entrant might also get the benefit of using the same system. If you’re considering this though, make sure you go into it with an awareness of what the real reasons for slowness are – your organisational culture and complexity, and don’t forget the caravan!