Customer Experience – Product Evaluation

Fear cannot be banished, but it can be calm and without panic; it can be mitigated by reason and evaluation
Vannevar Bush

In the Christmas Day blog, I indicated that I was going to begin a series on how OSS vendors / integrators can start with the customer experience and work back to improve their own products, services and systems. Today’s blog is the first in this series.

The customer experience during the product evaluation stage basically revolves around fear. There are generally many people / departments involved in the evaluation and the decisions these evaluators make will have a bearing on their organisation’s success (or otherwise) for years to come. OSS products tend to be so sticky that the vendor remains an important partner for 5+ years and the evaluators know that a poor decision will have long-term ramifications.

The fears are perfectly understandable too because the evaluators will never have perfect information upon which to base their decision. There’s just too much complexity. As such, there will always be an element of risk in the customers’s minds and they will justifiably seek to build an evaluation process that mitigates as many fears as possible.


From experience helping customers with their vendor selection, I believe the following are the key fear types that need resolution by the evaluators:

  1. Have we been able to classify and articulate our needs well enough for the vendor to understand and respond accordingly
  2. How do we determine which vendor’s total solution best meets those needs, knowing that:
    1. It is impossible for us to verify 100% of the vendor’s solution, so we must propose a way to evaluate what is most essential (eg the Pareto Principle) and mitigate against the rest as best we can
    2. It is unlikely that any vendor’s solution will be perfectly aligned (ie they’re all compromised against our needs in some way)
    3. Many of our needs are intangible, thus making them difficult/impossible to measure
    4. It is difficult for us to validate a vendor’s claims in some cases
    5. As indicated above, there are likely to be many people involved in the vendor / product selection, each with their own unique interpretation of needs and fears
  3. Financial fears:
    1. Will the solutions fit within our budgetary constraints and if not, what compromises will we need to make
    2. Will the solution provide us with tangible returns on our investment rather than being an ongoing money-pit
  4. What are the risks relating to buying (or not buying), including risks that are beyond the conclusion of the sales cycle

(The numbering of the items below match with the fear types above)

  1. The customer requires substantial feedback from the vendor to describe their understanding of the customer’s needs
  2. This is a tough one due to the complexities involved, but the customer will seek any forms of justification they can get, particularly proof that is in their own voice (ie demonstrations using their terminologies, naming conventions, network models, services, business processes, organisational structures, key locations, etc). Other methods of proof would ideally include testimonials, references, benchmarks, etc that are beyond the vendor’s prejudice
  3. The customer wants clear and simple pricing models, with simple terms and conditions so that they’re not left with doubts about hidden costs or gotchyas. The customer also wants to be able to collaborate closely with the vendor to build up a cost or value justification. OSS have many intangibles, so this can often be difficult for both vendor and customer, but the effort should allay many customer fears as well as building their trust that the vendor is seeking win-win outcomes
  4. The vendor will hopefully have a clear understanding of the types of risks that have arisen during their past project implementation experience. Rather than trying to hide these risks, vendors that present key risks and work with the customer to prepare mitigation strategies will generally also engender the customer’s trust and confidence

Note: There are earlier stages in the life-cycle of an OSS than product evaluation (eg strategic analysis, requirement analysis, road-map planning, etc), but the product evaluation stage is the first where the vendors are essentially involved. The vendors may be involved earlier, but often the customer conducts their own pre-planning.

Read the Passionate About OSS Blog for more or Subscribe to the Passionate About OSS Blog by Email

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.