“It’s more important to have the right people involved than it is to follow the process exactly right.“
Just as vendors have learnt the importance of context-sensitive help, there is also the need for context-sensitive training and context-sensitive testing (and context-sensitive everything else for that matter).
I once worked on a project where a team of about 20 testers flew into the country, stayed 3-4 weeks and submitted the OctopOSS to a barrage of tests. The results came back with a resoundingly positive outcome with only a handful of minor bugs for the developers to resolve.
The only problem was that I’m not sure exactly what they tested because in the week after the testers flew home, our in-country data modelling team (a team of 2 – one colleague and I) identified a dozen bugs that caused complete system meltdown (of a US$30M OSS by the way) and loads of other problems that needed urgent rectification.
The morals of that story were:
- The developers should not define the test cases and data sets to hand over to the testers. They should be written by subject matter experts in operations, people who intimately understand the network and services that need to be managed. To quote Brian Kernighan, “If you are as clever as you can be when you write it, how will you ever debug it?”
- The developers and testers should have basic understanding of the network technologies that they’re testing via the OSS framework
- The testers (if not the developers) should gain familiarity with the customer-specific context and test with the data that the customer will use
- Other disciplines, such as trainers, business analysts, etc should also be imbued with a context-sensitive understanding of the customer’s environment and data sets