Why KPIs, QoS and QoE aren’t good enough. What else must we do

Earlier this week we talked about the data management challenge being solved, for the most part. However, we still have large gaps in the decision support challenge, mostly because we don’t have tools to help us understand the complex “systems / environments” in which decisions need to be made.

I thought the graph below might help to further articulate this thought.

Source: Rohde and Schwarz

Our OSS and data management solutions are fantastic at collecting and presenting KPIs (Key Performance Indicators), at the bottom of the pyramid. KPIs tend to be nodal in nature. That is, each network device publishes information that it sees locally. Large networks collect and present terabytes worth of telemetry and log data every day. The firehose of KPIs generally doesn’t really help much. There’s too many metrics / KPIs for a human to possibly consume or evaluate, let alone make decisions with. Unfortunately, many OSS stop at the KPI level.

More advanced OSS know that we must help filter the KPIs. This is where quality of the network or Quality of Service (QoS) comes into play. QoS numbers tend to relate to end-to-end indicators such as jitter, packet loss, throughput, etc in packet-switched networks. This is a quantitative measure of the quality of a given service.

Unfortunately, QoS isn’t a direct representation of the customer’s actual experience. As customers, we might experience issues with voice, video, connectivity, etc when using applications whose packets are carried over the network. Quality of Experience (QoE) metrics are used to capture the customer’s perception of how satisfactory the service has been, regardless of how good the QoS metrics look. End-user experience metrics such as voice mean opinion score (MOS) have been developed to bridge the gap between QoS and QoE. QoE metrics are often measured more subjectively or qualitatively than QoS. 

Perfect. As network operators, we now know whether our customers are satisfied with their holistic experience using the network, applications, consuming information, etc. Some more advanced OSS manage to present metrics at the second (QoS) or third (QoE) layer of this pyramid.

Unfortunately, we still have a problem. If we see deterioration in KPI, QoS or QoE numbers, simply presenting this information in graphical format doesn’t actually tell us what we should do to fix the problem or to perform network / capital optimisation activities. The ideal situation is for these fixes / optimisations to be performed automatically – such as self organising / optimising networks (SON).

Most OSS still stop short at point 1 on the chart from the earlier article, providing a multitude of graphs or tables, irrespective of where on the pyramid they are.

What we actually want is for specific information about what to do next – decision support – to change the e-tilt on an antenna to more optimally target where mobile phone users are, to add an extra 1 Gbps of capacity to a bearer link so that customer download speeds improve, to replace a copper pillar with a fibre node so that customers don’t experience so many outages, to perform a node-split, etc.

Taking the outputs of our OSS from the pyramid of metrics at point 1 to decision support at point 2 on this chart, will make our OSS far more valuable. It will use the power of computers to show us how to continuously optimise the network. Unfortunately, to do so, we have to get far better at complex system modelling / analysis (or what the cool kids call “systems of engagement” according to Tony N – thanks for the heads-up on the latest nomenclature Tony! )