“Always speak politely to an enraged dragon.”
Steven Brust.
Regular readers will have noticed my penchant for referring to the Pareto Principle (aka the 80/20 rule). I’m a big believer in focusing on streamlining the high-use or highly important items of OSS implementations. OSS are simply too big and too complex to get bogged down in the minutiae.
But for those with a slant for the details, I’m clearly missing a very, very important point for CSPs with a high volume of transactions. This blog from 37signals about slaying “Dragons on the far side of the histogram” will be music to the ears of detailed-minded readers.
So if 37signals gets hit with 50 million requests a day, it’s quite likely that OSS at large CSPs get orders of magnitude larger again.
To quote their blog, “But when you process about fifty million requests a day, there’s still an awful lot of requests hidden on the far side of the 99th. And there, young ones, is where the dragons lie.
A while back we started shining the light into that cave. And even while I expected there to be dragons, I was still shocked at just how large and plentiful they were at our scale. Just 0.4% of requests took 1-2 seconds to resolve, but that’s still a shockingly 200,000 requests when you’re doing those fifty million requests.
Yet it gets worse. Just 0.0025% of requests took 10-30 seconds, but that’s still a whooping 1,250 requests. While some of those come from API requests that users do not see directly, a fair slice is indeed from real, impatient human beings. That’s just embarrassing! And a far, far away land from that pretty picture painted by the 60ms mean. Ugh.
Finally, there was the true elite: The 0.0001%, for a total of 50 instances. Those guys sat and waited between 30 and 60 seconds on their merry request to complete. Triple ugh.”
To translate that into unhappy customers, that’s a lot per day on 50 million transactions.
How does that map to your OSS performance tuning? How far past the 80th percentile, or even the 99th percentile, do you shine a light?
Yes, I admit it. You details-guys are an absolutely vital component of delivering high levels of satisfaction to a CSP’s customer base. 😀
Have you ever had an OSS or BSS project that’s impacted by poor data quality? The problem of data integrity gets raised on almost every
“One of my favorite things about designing software is designing the defaults. The defaults define the experience for everyone out of the box. And, therefore,
Have you ever been tasked with mapping out the processes for the telco you work for? Either as a way of documenting current state, or
In the last week alone, I’ve had two potent reminders that the world of OSS is simply (pun intended) too complex. This manifests in many
Here at Passionate About OSS, we are exactly that – Passionate About OSS (Operational Support Systems). We’re also passionate about BSS, NMS, or any other names you call the tools that help to operationalise your network.
