“Just because you’re afraid of doing something doesn’t give you a permission slip to not do it.”
There’s a lot of fear in OSS. So many things can go wrong (the OctopOSS theory), so much incompetence is created, so many nearly insurmountable integration challenges await and their complexity means that there is no perfect plan going into a project.
They do require a leap of faith, a confidence in your team to work your way through all the challenges and a commitment from senior stakeholders to help drive change through (with compassion for those whose working life is about to be impacted of course). They also need an eye for simplification, like the Mechanical Turk model.
OSS are often delivered into large organisational structures, structures that are functionally siloed. For large OSS, even the OSS team can have multiple functional silos.
Where there are functional silos, there are activities within OSS that need to be delivered across silos. That’s where things can get a bit dysfunctional. Jurisdictions, ownership of responsibilities, agreements on approach, misalignments of performance indicators, downstream impacts and dare I say it, turf wars, can make it more difficult to deliver an OSS organisationally than technically… and OSS can be incredibly difficult to deliver from a technical perspective.
Organisational change management is often completely overlooked, or only brought to bear far too late in the delivery process. Often there are so much dysfunction between silos, even where each silo has the best of intentions, that a bigger change management accord needs to be invoked.
A burning platform – communicating the need for urgent, radical changes brought about by dire circumstances
A moon shot – focussing the attention of the entire organisation on incredibly ambitious challenge
Dictatorship of decision making rather than democracy
Alternatively, insert any other method to help ensure all members of the team, across all silos, have a clear understanding of the greater objectives the team is trying to meet. I’d love to hear of examples that you’ve invoked to get great team results on complex OSS projects (or any other project type for that matter).
“When in doubt, connect.
That’s what fast-growing, important organizations do.
Making stuff is great.
Making connections is even better.”
Seth Godinin his post here.
Simple words. Simple concept. Interesting message…. with a traffic-light of OSS layers.
Layer 1 – A connection red light
The more connections an OSS has, the more enriched the data tends to become. However, by contrast the more interconnected an OSS gets, the more inflexible and difficult it gets to maintain. The chess-board analogy and the handshake analogy attest to the challenges associated to a highly connected OSS. In this case, when in doubt, don’t connect.
Layer 2 – A connection orange light
Five of the top seven companies (by market cap) in the world are tech companies (1. Apple, 2. Alphabet (Google), 3. Microsoft, 6. Amazon, 7. Facebook). They have become valuable through the power of connection. China Telecom represents one of the original connectors, the telecommunications carriers, and just makes it into the top 10 whilst Verizon and AT&T are both worth over $250B too. Our OSS allow these connections to be made – but they’re making a connection “for” the customer rather than making a connection “with” the customer. Until brand-OSS starts making a connection with the customers, we’ll never be fast growing like the tech titans. When in doubt, connect at a deeper level.
Layer 3 – A connection green light Tripods or Linchpins are the most valuable of OSS resources because of their ability to connect (ideas, people, products, workflows, contracts, etc). They are the pinnacle of OSS implementers because their breadth of connections allows them to interconnect the most complex of OSS jigsaws. If you want to become an OSS tripod, then Seth’s lead is a great one to follow – When in doubt, connect.
When planning out a full-stack business / network / services management solution, I tend to follow the top-down, bottom-up design process.
Let’s take the TMN pyramid as a starting point:
Image courtesy of www.researchgate.net
Bottom-up: When designing the assurance stream (eg alarms, performance, etc), I start at the bottom (Network Elements), understanding what devices exist in the network and what events / notifications / monitors they will issue. From there, I seek to understand what tool/s will manage them (Element Management / Network Management), then keep climbing the stack to understand how to present key information that impacts services and the business.
Top-down: When designing the fulfilment stream (eg designs, provisioning, moves/adds/changes, configuration, etc), I generally start at the top* (Business Management), what services are being offered (Service Management) and figure out how those workflows propagate down into the network and associated tools (such as ticketing, workforce management, service order provisioning, etc).
This helps to build a conceptual architecture (ie a layered architecture with a set of building blocks, where the functional building blocks start out blank). From the conceptual architecture, we can then identify the tools / people / processes that will deliver on the functions within each blank box.
This approach ensures we have the big picture in mind before getting bogged down into the minutiae of integrations, data flows, configurations of tools, etc.
To get momentum quickly, I tend to start with the bottom-up side as data flows (eg SNMP traps) are more standardised and the tools tend to need less configuration to get some (but not all) raw alarms / traps into management tools, perhaps just sand-pit versions. For the next step, the top-down part, I tend to create just one simple service scenario, and perhaps even design the front-end to use The Mechanical Turk model described yesterday, and follow the flow manually down through the stack and into element management or network layers. Then grow both streams from there!
* Note that I’m assuming services are already flowing through the network under management and/or the team has already figured out the services they’re offering. In a completely green-fields situation, the capabilities of the network might determine the product set that can be offered to customers (bottom-up), but generally it will be top-down.
OSS can be complicated beasts with many tentacles. It can make starting a new project daunting. When I start, I like to use a WBS to show all the tentacles (people, processes, technology, contracts) on a single page, then look for the lead domino (or dominoes).
A lead domino is the first piece, the one that starts the momentum and makes a load of other pieces easier or irrelevant.
Each project is different of course, but I tend to look to get base-level software operational as the lead domino. It doesn’t have to do much. It can be on the cloud, in a sandpit, on a laptop. It can show minimal functionality and not even any full use cases. It can represent only a simplistic persona, not the many personas an OSS normally needs to represent.
But what it does show is something tangible. Something that can be interacted with and built upon. Most importantly, it gives people a context and immediately takes away a lot of the “what-if?” questions that can derail the planning stage. It provides the basis to answer other questions. It provides a level of understanding that allows level-up questions to be asked.
Or it might be hearts and minds exercise to get the organisation deeply engaged in the project and help overcome the complexities and challenges that will ensue. Or it could just be getting key infrastructure in place like servers, databases or DCNs (Data Control Networks) that will allow the pieces of the puzzle to communicate with each other.
On an uplift project, it might be something more strategic like a straw-man of the architecture, an end-to-end transaction flow, a revised UI or a data model.
For OSS implementations, just like dominoes, it’s choosing the right place/s to kick-start the momentum.
During the last week, this blog-roll has talked about the benefits, but also the challenges facing implementation techniques like Agile in the world of OSS. There’s no doubt that they’re good at breaking down challenges into smaller pieces for implementation. Unfortunately there’s also the risk of doing for the sake of doing – stuffing more stuff into the backlog – without necessarily thinking of the long-term implications.
Warren Buffett’s “two-list” prioritisation strategy could be an interesting adjunct to Agile, microservices and the like for OSS. Rather than putting all 25 of the listed goals into backlog and prioritising the top 5, the Buffett technique sees only the top 5 entering the backlog and the remaining 20 put into an avoid-at-all-cost bucket… at least until the top 5 are complete.
I know I’m as guilty as Mike Flint in not just tackling the top 5, but keeping the next 20 bubbling away. Is your OSS implementation queue taking the same approach as Buffett or me?
What does the integration map of your OSS suite look like? Does it have lots of meatballs with a spaghetti of interconnections? Is it possibly even too entangled that even creating an integration map would be a nightmare?
Similarly, how many customisations have you made to your out-of-the-box tools?
In recent posts we’ve discussed the frenzy of doing without necessarily considering the life-time costs of all those integrations and customisations. There’s no doubt that most of those tweaks will add capability to the solution, but the long-term costs aren’t always factored in.
We also talk about ruthless subtraction projects. There are many reasons why it’s easier to talk about reduction projects than actually achieve them. Mostly it’s because the integrations and customisations have entwined the solutions so tightly that it’s nearly impossible to wind them back.
But what if, like many start-ups, you had an exit strategy in mind when introducing a new OSS tool into your suite? There is an inevitability of obsolescence in OSS, either through technology change, moving business requirements, breakdowns in supplier / partnership relationships, etc. However, most tools stay around for longer than their useful shelf life because of their stickiness. So why not keep rigid control over the level of stickiness via your exit strategy?
My interpretation of an exit strategy is to ensure by-play with other systems happens at data level rather than integrations and customisations to the OSS tools. It also includes ruthless minimisation of snowball dependencies* within that data. Just because integrations can be done, doesn’t mean they should be done.
* Snowball dependencies are when one builds on another, builds on another, which is a common OSS occurrence.
“I’ve been involved in telecom operations for decades, and I’ve learned that there is nothing in networking as inertial as OSS/BSS. A large minority of my experts (but still a minority) think that we should scrap the whole OSS/BSS model and simply integrate operations tasks with the service models of SDN and NFV orchestration. That’s possible too, and we’ll have to wait to see if there’s a sign that this more radical approach—which would really be event-driven—will end up the default path.”
Another thought-provoking article from Tom above. It’s worth clicking on the link for an extended read.
I agree with his assertion about OSS / BSS being inertial and designed against principles of the distant past.
I believe the glacial speed of change primarily comes down to one thing – variants.
Networks support lots of variants. Product offerings / bundles introduce more. Processes more still. Entwined best-of-breed integrations introduce yet more still. Etc. Now multiply these numbers and you get the number of variants that OSS need to handle.
That’s the number of variants that must be designed for, developed for, integrated / configured for, tested against, data migrated for, supported and handed over into ops… not to mention the defects that arise from overlooking variants or new variants being introduced with new networks, services, etc.
So whether it’s the old approach or a new event-driven approach, if we can collectively slash the number of variants, we can go a long way to reducing the entanglement.
My fear at the moment is that virtualised networks are going to add a vast number of new variants before anyone thinks of stripping any of the old ones out.
“But what if it’s ALL tech-debt? Everything we build needs to be supported for its natural life. The more we accumulate, the more that needs supporting. Support represents all the debt we leave behind, like going on a credit-card fuelled spending spree. And like any other debt, the more you collect, the more it compounds (due to the handshake analogy).” More details here.
If we consider every investment of effort or cash in our OSS as a debt of support that the organisation has to carry forward, does it change our expectation of what we need from our OSS in return?
Taking on debt to buy assets (eg property, shares, businesses) is considered good debt. Taking on debt to buy stuff that produces no financial return is considered bad debt.
By that definition, is the expenditure of time and cash on our OSS good debt or bad debt?
Are all of our efforts producing:
A tangible financial benefit
An intangible benefit or
No perceivable benefit at all
I suspect that in most environments it’s probably all of the above but heavily weighted towards B.
Clearly we need mechanisms to drive more of our tech debt into A initiatives. That means more “revenue generation” thinking though.
“Information overload is happening at all career levels. Companies restructure, and current staff absorb additional responsibilities, requiring new skills to be learned. Technology changes, and new systems, tools and processes need to be mastered. Markets change, and new strategies or client prospects or industry sectors need to be researched. To be successful in today’s times you need to master not only the skills relevant to your own job, but also the skills of learning , adapting to change quickly, and sustainably dealing with change without burning out.”
Sounds like a day in the life of OSS doesn’t it?
I’m a huge fan of AFL. The game, like OSS, is getting increasingly sophisticated, faster and strategic. Despite this, there’s almost not a week that goes by without one of the top-Line coaches stating that it’s really a simple game. The truth of the matter is that it is not a simple game but the best coaches have a way of simplifying the objectives so that their players can function well within the chaos.
They train and strategise for the complexity but simplify the game-day messaging to players down to only 2-3 key indicators and themes [at least that’s what the coaches say in interviews, but I wouldn’t know for sure having not been a professional footballer].
On any given day on most of the OSS projects I’ve worked on, there is more chaos than clarity. A chaos of technologies, meetings, deadlines, deliverables, processes, designs, checklists, etc.
How often do we stop to run a pre-mortem before death by complexity has occurred?
“Most customers buy the basic and learn to love the features, but the whole customer experience is based on trying to sell the features.”
This statement appears oh-so-true in the OSS sales pitches that I’ve observed.
In many cases the customer really only needs the basic, but when every vendor is selling the features, customers also tend to get caught up in the features. “The basic” effectively represents Pareto’s 20% of functionality that is required by 80% (or more) of customers. However, since every vendor has the basic they don’t feel differentiated enough to sell that 20%. They sell the remaining 80%, “the features,” that give the perspective of uniqueness compared with all the other vendors.
When I help customers through the vendor selection process, I take a different perspective though. I work with the customer to understand what is “the basic” of their business model and what the OSS will channel most impact through. It’s not the hundreds of sexy features, the ones which will get used on rare occasions, but “the basic” that get used hundreds (or thousands, or millions) of times every day. I then work with the customers to figure out a way of benchmarking which vendor solution delivers the greatest efficiency on their basic.
A couple of side benefits come from this strategy too:
Once delivery momentum is established and the customer’s operators are using the basic, there are still hundreds of features to play with and enhance over time. Hundreds of more little wins that enhance the customer experience, building the love
“When you evaluate the market, you’re looking to understand where people are today. You empathize. Then you move into storytelling. You tell the story of how you can make their lives better. That’s the “after” state.
All marketing ever does is articulate that shift. You have to have the best product, and you’ve got to be the best at explaining what you do… at articulating your value, so people say, ‘Holy crap, I get it and I want it.’”
How does “the market” currently feel about our OSS?
See the comments in this post for a perspective from Roger, “It takes too long to get changes made. It costs $xxx,xxx for “you guys” to get out of bed to make a change.” Roger makes these comments from the perspective of an OSS customer and I think they probably reflect the thoughts of many OSS customers globally.
The question for us as OSS developers / integrators is how to make lives better for all the Rogers out there. The key is in the word change. Let’s look at this from the context of Theseus’s paradox, which is a thought experiment that raises the question of whether an object that has had all of its components replaced remains fundamentally the same object.
Vendor products have definitely become more modular since I first started working on OSS. However, they probably haven’t become atomic enough to be readily replaced and enhanced whilst in-flight like the Ship of Theseus. Taking on the perspective of a vendor, I probably wouldn’t want my products to be easily replaceable either (I’d want the moat that Warren Buffett talks about)… [although I’d like to think that I’d want to create irreplaceable products because of what they deliver to customers, including total lifecycle benefits, not because of technical / contractual lock-in].
Customers are taking Theseus matters into their own hands through agile / CI-CD methodologies and the use of micro-services. They have merit, but they’re not ideal because it means the customer needs more custom development resourcing on their payroll than they’d prefer. I’m sure this pendulum will shift back again towards more off-the-shelf solutions in coming years.
That’s why I believe the next great OSS will come from open source roots, where modules will evolve based on customer needs and anyone can (continually) make evolutionary, even revolutionary, improvements whilst on their OSS voyage (ie the full life-cycle).
“In the world of OSS, the blind men are like the 6 (or more) different departments – operations, business, IT, sales, marketing, executive, etc. Each interacts with the OSS elephant and each has a different experience with it.”
Yesterday’s post on Six blind men and an OSS elephant.
Each of us, and each of the stakeholder groups we represent, has a different perception of our OSS. We have different interactions with it and different biases that influence the perceptions. Similarly, we each have different knowledge about it.
But who knows the solution best? Operations, business, IT, project team, system integrators? The flippant answer would be none do because of the different perspectives / biases. Another flippant answer is that collectively they all do, as they all contribute to a shared vision.
The more likely answer is the person / people / group that makes the effort to understand the others. That’s why thrashing is so important in the early stages of an OSS project. It’s also why I find it interesting in the organisations that intentionally compartmentalise OSS knowledge and insert organisational barriers to prevent wisdom sharing beyond demarcation points. Sure, it constrains the effort to areas of specialisation, but you end up with teams who think the elephant is just the tail, the tusks, the ears, etc.
The concept of digital transformation can be massive… or it can stem from something much smaller.
In digital transformation, there’s a mantra of simplify then transform. That makes perfect sense, in alignment with, “The chess-board analogy,” but often overlooked.
But there’s another simplification perspective to take. Let’s take the 6 blind men and the elephant parable into consideration. Each of the six blind men gets a different perception of what they’re interacting with.
In the world of OSS, the blind men are like the 6 (or more) different departments – operations, business, IT, sales, marketing, executive, etc. Each interacts with the OSS elephant and each has a different experience with it.
The MVP (Minimum Viable Product) or sand-pit environment is one great strategy to building momentum in large organisations. Identify something small that is at the heart of the solution – a place where all parties interact and get their chance to touch the new elephant. Then build something at that central point that presents something that’s encouraging / enticing to all who interact with it.
For a start, the cloud represents an environment that isn’t “owned” by any of the business units and isn’t bound up in all the security or other checks that slow delivery down. Secondly, with an MVP ensure you’re touching simulated components inside the cloud rather than exposed components inside the business (think “the implications of a handshake“). Thirdly, make it a sand-pit – segregated from production (and potentially from each of the six blind men) then encourage interactive play.
Have you ever worked in a big telco (or any large organisation for that matter)? Have you ever noticed the disconnection in knowledge / information? On a recent project, I was brought in as an external consultant to find potential improvements to one business unit’s specific key performance indicator (KPI).
As a consultant, my role is a connector – a connector of people, ideas, projects, products, technologies, approaches, etc. On the abovementioned project, I posed questions within the business unit, but also reached out to adjacent units. There were no less than 6 business units that were investigating a particular leading-edge technology that could’ve improved the KPI. Unfortunately, they all only had budget for investigations, with one also having the budget to conduct a very small Proof-of-Concept (PoC). Collectively, they would’ve had the budget to do a fairly significant implementation for all to leverage but separately, none were within 1-2 years of implementation .
I later found out that there was an additional, completely unrelated business unit that had already taken this technology into production with a small trial. It was for a completely different use case but had stood up a product that could’ve been leveraged by the rest.
That got me thinking – the challenge for us in technology is finding a way to remove this waste through “perfect” communication, without bogging us down with information gathering and grooming. Can a conjunct of existing technologies be used, as follows?
We already have:
Knowledge management systems (KMS) to gather and groom knowledge
Chat-bots to collect verbal information and respond with recommendations (more on OSS chat bots tomorrow BTW)
Artificial Intelligence (or even just basic analytics tools) to scan data (from KMS, chat-bots, OSS, etc) and provide decision support prompts
We have all the technologies required to significantly augment our current poor communication methods. The question I have for you is do Communications Support Systems (CSS) become an adjunct to OSS or just another IT system that happens to consume OSS data as well as any other sources?
Note: I’ve bracketed “Ambient Listening” in the chat-bot because the six different groups above were all working on research into the cutting-edge technology and talking about it, but there was little in the way of official documentation stored – mostly just emails, communicator conversations, verbal discussions, etc. I do acknowledge the invasion of privacy concerns with this thought experiment though!! 🙂
I’ve been seeing a lot of references to continual development methodologies lately (ie references to agile, DevOps, CI/CD, etc). Quotes like focus on projects rather than the mission statement, focus on products rather than projects, etc. The core principle is in chunking workload down into modular pieces that can be implemented to give flexibility and agility to meet evolving customer demands. These approaches are claimed to improve customer-centricity as well as improving our ability to deliver.
This is great thinking, but we aren’t quite there yet. Do you notice that these measures have a tendency to be introspective (ie the service provider’s business goals, usually product delivery-related goals)? A customer’s experience isn’t directly related to mission statements, projects, epics or even products. It relates to their end-to-end experience with the service provider’s brand, which means all the components that make up their service. It’s the whole pyramid of pain (as opposed to the service provider’s product definition) and all the disparate business units that contribute to the customer experience.
Continual development is about the slicing and dicing into manageable chunks. The next step is the ability to slice those chunks smaller still, into chunks per service instance, and stitch all of these together to give a service stream. Not just orchestration, but omni-experience orchestration that traverses the entire pyramid experienced by every individual customer.
This is probably too atomic to be managable for a service provider and their current OSS. However, improvements in AI / ML have the potential to have a huge part to play in the next methodology evolution, taking feeds from dev data sources as well as ops data sources. It has the ability to guide customer-driven development rather than product-driven development.
Not just marketing to the segment of one, but development driven by the experiences of one.
Over the years, I’ve dealt with (and worked with) many vendors, as I’m sure you have too.
Some will bend over backwards to help their customers (or potential customers) out, finding workarounds to their own internal rules to help make something good happen.
Others will persuade their customers into signing a contract before bending their customers over backwards, finding internal rules to their own workarounds to make sure nothing good happens.
Which group do you think customers prefer dealing with? The answer’s obviously the first.
But having been the impartial advisor in vendor selection processes, the group the customer prefers dealing with doesn’t always equate with who they sign a contract with. That answer seems less obvious – to me at least (acknowledging though that there are many factors that go into an OSS purchasing decision).
“It is easier to act yourself into a new way of thinking, than it is to think yourself into a new way of acting.”
The agile, lean documentation delivery models of today have the huge advantage of gaining momentum on OSS projects. No longer do we have the big-bang cutovers, but we start with barely viable solutions and progressively build on them. It starts out with the Minimum Viable Product (MVP) delivery model.
To Millard Fuller’s point, generating momentum by actions (ie product releases) brings about new ways of thinking and associated operational change – far faster than trying to generate momentum through talking / thinking / documenting OSS transformations.
The caveat on this model though is that somebody on the team has to have the clear understanding of where the product is going. Otherwise there can be the tendency for the agile backlog to drag the team down rabbit-holes.
“The truth is that whale-sized companies would love to do business with smaller suppliers. The relationship is more immediate in every way. Small suppliers can deliver goods more quickly than larger ones. It’s much easier to get the CEO of a small supplier on the phone for discussion of the transactions. Small suppliers are more flexible. So why are so many small companies unsuccessful in entering this larger arena? Because they do not fully understand the fears of whale-sized companies, fears that have arisen because of all the factors affecting business during the last few decades. It may be true that you have prepared yourself and your company to respond to the immediate demand for an excellent product. We call this demand the “whale’s pain.” But fear trumps pain every time. If you don’t know how to understand and allay the whale’s fear, you will not be able to make the sale.”
Barbara Weaver Smith and Tom Searcy, from their book, “Whale Hunting: How to Land Big Sales and Transform Your Company”
When it comes to OSS customers, most are whales relative to their vendors (ie service providers tend to have larger market cap than the OSS vendors that they partner with).
Barbara and Tom make a fascinating point here – “If you don’t know how to understand and allay the whale’s fear, you will not be able to make the [OSS] sale.” And this is not just the literal sale. Te statement rings true not just for vendor sales teams, but for internal project initiators and implementation teams who are making changes mid-flight.
I’ve seen scenarios where one vendor has been clearly better able to overcome a customer’s pain points, but haven’t been able to overcome their fears, whether that’s been the perception of size, the perception of the ability to deliver or other fears.
OSS projects tend to be complex and costly. There tend to be a lot of unknowns, and with that comes a lot of fear.
When helping customers with their product / vendor selections (or OSS consultancies in general), I try to overcome customer fears with a methodology that steps through the evaluation of pain points, the demonstration of capabilities and the opportunity for customer and vendor to build trust in each other through the interaction process. But a vendor selection can’t overcome all of the unknowns prior to entering into OSS implementation contracts (partnerships), so there are often still residual fears.
Do you agree with Barbara and Tom’s premise? Does fear trump pain every time? Do you have any techniques that you’ve found to be successful in allaying customer fears (and/or pain points) on OSS projects?
To leave you with one last thought – The confused mind says no. How do you remove the confusion from what seem like complex projects in the eyes of your customers / stakeholders / sponsors?
Many telcos around the world have a sometimes subtle, sometimes not, turf-war going on between networks / operations and IT groups. Virtualisation of the network potentially amplifies this because it increases the scope of possible overlap.
As described in yesterday’s “Noah’s Ark of OSS success,” one of the ways of improving the success of an OSS is to make it relevant to a larger set of users. Theoretically, this includes representatives from networks, ops and IT, which isn’t always easy to achieve.
Is there anything you can do with your OSS to engineer a meet-in-the-middle collaboration space? Are there adjacencies where people from different business units need to collaborate but don’t always succeed in doing so? Are there areas of potential overlap where demarcation and validation can be defined?
Notwithstanding the technical and user interface considerations, there’s also the perspective of who actually owns the meet-in-the-middle tool. For example, if it’s an IT-owned tool, there are risks that networks / ops might not want to contribute to its success. Ditto if it’s owned by networks or ops.
As an integrator I’ve seen many examples of technically relevant solutions not succeeding because of people-related issues. Organisational change management is an often-underestimated tool that isn’t factored into OSS projects until too late, particularly within large, complex organisation structures.