“The goal is to transform data into information, and information into insight”
CSPs hold vast repositories of data in their OSS / BSS. The aim of the data collection process is to allow the OctopOSS to provide valuable insights for the CSP.
The data collection effort is one of the most important factors of an OSS implementation. The software can be perfect, the processes can be perfect and the people can be perfect, but the solution will be useless if the data integrity is poor.
Moreover, the OSS tools only function when loaded with suitable data, so the data load phase is a lynchpin of the entire project. Hint: From a career perspective, if you want to become a cross-domain OSS guru, get yourself on the data migration team and you’ll quickly learn about all of the touch-points of an OSS, including technical, procedural, organisational, etc. He who holds the data holds the power.
This is also the stage where collaboration between CSP and vendor/integrator is essential, as shown in the following pieces of the jigsaw:
- Understanding the environment, network, hardware and software (including underlying data structures) – expert understanding primarily resides within the vendor’s team to specify and build the OSS environment. However, the CSP will be called upon to establish accounts, access rights, etc to be able to reach and manage the OSS hardware over the CSP’s network. Both parties will need to collaborate to identify the required environments (eg Production, Development, Test, Training, Data migration, etc) and redundancy mechanisms (eg active-active, hot standby, disaster recovery, archiving, backup / restore, etc)
- Understanding the insights to be presented to CSP operators – the CSP’s experts will have the best understanding of the information that is most important for operating the network. The vendor’s experts will have the best understanding of how to present information within their applications and how to create the data model that supports the data presentation
- Data naming conventions – The CSP will already have naming conventions. This may need revision and/or supplementation to work within the new OSS environment. The Vendor will be able to provide insights into whether the existing naming convention is supportable and can provide suggestions for refining to best suit their applications
- Understanding the data sources – The vendor will know what information is required to support the proposed data model. The CSP will know where to source the required information. Data sources could include exports from other systems (BSS, NMS, EMS, NE, etc), interfaces from other systems (eg APIs), other files maintained by CSPs (eg spreadsheets), manual entry from printed documents and possibly others. It is also important to know which data sources will be master (eg does OSS push data to EMS or does EMS push data to OSS)
- Perform data mapping – The vendor will normally analyse the various data sources and identify where key data should reside within their applications. However, once the data mapping exercise has been conducted, the CSP should closely review to confirm the data will be presented to meet their needs (ie the first step in quality assurance).
- Processing and populating the information – The vendor will normally lead this process based on the data sources provided by the CSP. The data will often require cleansing, parsing, supplementation to fully populate the required OSS data fields.
- Refine the data to ensure that it is providing valuable insights to the CSP operators
As the points above suggest, the CSP is unable to do this without help from the vendor and the vendor is unable to do this without help from the CSP. Neither party can just throw the hand grenade over the fence and expect the other party to complete this task alone. The data modelling and load exercise is likely to be very iterative and will probably be planned in a number of discrete phases. Both parties will need to collaborate to refine the presented data during this evolution.
Further to point 1 in the list above, the specification of hardware and software will be designed around projected data volumes, transaction rates and overall application performance. The initial specification should include projected future needs and data growth as well as scalability paths and/or limitations.
Sequence of Data Load
It is common that careful consideration must be given to the sequence of data loaded (eg reference data loaded before the main data load) due to data constraints. The method of data population could be auto-discovery (the OSS tools collect information from the data sources and populate into the OSS), script loading (pre-defined or custom scripts will populate the information into the OSS) or manual loading.
If you have access to an auto-discovery capability, it is recommended to develop this interface and load data using this approach rather than script loading as it is the long-term solution for data reconciliation.
Data mapping may also include identification of the sequence of data load between the application environments discussed in point 1 above (eg from DATA-MIG to DEV/TEST to PROD).
Iterative test loads are important because data loads are key entry criteria to the various test phases (eg Unit Testing, System Integration Testing, User Acceptance Testing, etc). The data must be suitable by each of these test stages to mimic the future Production environment to minimise regression testing.
Since the CSP will be the ultimate user of the data, it is incumbent on them to validate and approve all data before it is loaded into the Production environment.
Mediation Devices (MD) or Mediation Device Drivers (MDDs) are the custom software tools that act as intermediaries between interfaces. For example, if an Element Management System (EMS) has a CORBA interface and needs to pass information to your OSS via it’s standard SNMP interface, an MDD can conduct protocol conversion, communications control, data mapping and data forwarding.
It is common for vendors to indicate that their mediation platforms are “out of the box,” meaning that they can start collecting information from the network immediately. This is often not completely true. At best, they need configuration to support naming conventions, data mapping, address ranges, etc. At worst, they need significant customisation to enhance functionality. For example, some legacy voice switches have enough commands to fill more than 10,000 pages of documentation. However, each CSP will only use a small subset of the available commands depending on how they’ve configured their network. Each CSP will generally use a different sub-set of commands. As such, a vendor will tend to only configure the commands on an as-required basis rather than programming all possible commands.
The vendor’s claims that they have already interfaced to a particular device may be completely correct, but they still might need to conduct significant additional programming to make their mediation platform work for a new customer.