OSS that are painful and full of denial

It’s quite common, especially in enterprise technology, for something to propose a new way to solve an existing problem. It can’t be used to solve the problem in the old way, so ‘it doesn’t work’, and proposes a new way, and so ‘no-one will want that’. This is how generational shifts work – first you try to force the new tool to fit the old workflow, and then the new tool creates a new workflow. Both parts are painful and full of denial, but the new model is ultimately much better than the old.”
Ben Evans
from the same post as yesterday’s blog on PAOSS.

The other part that Ben glosses over, but which is highly relevant to OSS is staying with the old tools and old workflows because the change chasm is so wide – this third part is even more painful and full of denial.

The monolith OSS of our recent past (and current state in many cases) is no longer a feasible option, These monoliths tended to be built around highly structured, hyper-connected relational databases, which was/is a strength but also a weakness.

There are nuances of course, but a lot of inventory relationships are unchanging – A conductor / fibre goes in a sheath, in a cable in a duct, in a trench. A port is on a card which is in a device, that’s in a rack that’s in a room, in a building.

In a relational database, these are all built upon joins. And in many OSS, there are thousands of hours of development that have built layers and layers of joins onto these base capabilities to give more advanced capabilities. But the computational complexity of all these joins leads to response times that can run into many minutes, not viable for operators that require near-real-time responsiveness – think network fault rectification where SLAs are tight.

For inventory in particular, the generational shift mentioned by Ben appears to be the graph database.
“[The] difference between graph databases and relational databases is that the connections between nodes directly link in such a way that relating data becomes a simple matter of following connections. You avoid the join index lookup performance problem by specifying connections at insert time, so that the data graph can be walked rather than calculated at query time.
This property, only found in native graph databases, is known as index-free adjacency, and it allows queries to traverse millions of nodes per second, offering response times that are several orders of magnitude faster than with relational databases for connected queries (e.g., friend-of-friend/shortest path)
Johan Svensson here

A transition to a graph DB represents new ways with new tools, and obsolescence of a lot of past efforts. But the next generational shift that obsoletes graph DBs surely isn’t far away either.

The big bets on the monoliths no longer seem viable. It seems to me that the highly modular, interfaced, small-grid OSS is the way of the future.

Leave a Reply

Your email address will not be published. Required fields are marked *