Can LLMs help us to reimagine what the OSS of the future looks like?

I love blogs. One of the amazing things about them is that it allows you to hear the wisdom of exceptional individuals who you’d otherwise never have access to or ever have the chance to meet. They’re the best mentors you never had. One example of that was in last week’s post from David Heinemeier Hansson about triggering people to run A/B tests in their own minds. A pretty novel concept!

Today, I’ll share some wisdom from Benedict Evans, a person who is arguably most famous from his time as a partner at Andreessen Horowitz, but whose writings are now followed by 175,000+ people (including me). Anyway, back to a couple of quotes from a recent article of Ben’s about Building AI Products:

“a useful way to think about generative AI models is that they are extremely good at telling you what a good answer to a question like that would probably look like. There are some use-cases where ‘looks like a good answer’ is exactly what you want, and there are some where ‘roughly right’ is ‘precisely wrong’.”

This is important in the context of their use in OSS/BSS and telco use cases. In most scenarios, OSS need deterministic answers (ie a given input will always produce the same output), versus the probabilistic answers provided by LLMs (ie a given input can give multiple different outputs based on probability / uncertainty).  Moreover,

“There are two ways to try to solve this. One is to treat it as a science problem – this is early, and the models will get better.

…the other path is to treat this as a product problem. How do we build useful mass-market products around models that we should presume will be getting things ‘wrong’?

A stock reaction of AI people to examples like mine is to say “you’re holding it wrong” – I asked 1: the wrong kind of question and 2: I asked it in the wrong way. I should have done a bunch of prompt engineering! But the message of the last 50 years of consumer computing is that you do not move adoption forward by making the users learn command lines – you have to move towards the users.”

The move from Yahoo to Google to GenAI to find answers to your questions is a perfect example of moving towards the users and reducing friction to more effectively deliver what they need.

Ben then describes

“…there are also at least two other approaches.

The first is to contain the product to a narrow domain, so that you can create a custom UI around the input and output that communicates what it can and can’t do

…But the other approach is that the user never sees the prompt or the output, or knows that this is generative AI at all, and both the input and the output are abstracted away as functions inside some other thing.”

This is all interesting, but here’s the part where blog technology gives access to the profound insights of people like Ben who I’d otherwise never have the chance to absorb:

“Looking at this on another axis: with any new technology, we begin by trying to make it fit the problems we already have, while the incumbents try to make it a feature (hence Google and Microsoft spraying LLMs all over their products in the last year). Then startups use it to unbundle the incumbents (to unbundle search, Oracle or Email), but meanwhile, other startups try to work out what we could build that would be truly native to the new technology. That comes in stages. First, Flickr had an iPhone app, but then Instagram used the smartphone camera and local computing to add filters, and further on again, Snap and TikTok used the touch screen, video and location to make something truly native to the device. So, what native experiences do we build with this, that aren’t the chatbot itself, or where the ‘error rate’ doesn’t matter, but abstracts this new capability in some way?

This of course is proposing a paradox, that I’ve talked about before: here we have a general-purpose technology, and yet the way to deploy is to unbundle it into single-purpose tools and experiences. But this might just be misplacing the right level of abstraction. Electric motors are a general-purpose technology, but you don’t buy a box of electric motors from Home Depot – you buy a drill, a washing machine and a blender. The technology is instantiated into use cases. PCs and smartphones are general-purpose tools that replaced single-purpose tools – typewriters, calculators, voice recorders and music players – but each of those functions is achieved through a piece of single-purpose software: most people don’t use Excel as a word processor. One reason that some people are so excited about LLMs is the they might not follow that pattern: they might move up through all of those levels of abstraction to the top. That would leave no room for ‘thin GPT wrappers’. Yet I don’t think they can really do that yet, and so everything I’ve just written is really just wondering what you can build to change the world even if that never happens.”

This is the part where we can really start to reimagine what the OSS of the future looks like. No doubt you’ve seen many OSS vendors starting to place thin GPT wrappers around their existing solutions. We’re starting to see evidence of “Flickr having an iPhone app.” What native experiences do we build with this that aren’t the chatbot itself?

Do we start with use cases where the error rate doesn’t matter or where responses can be iterated upon to answer the questions one might have? An example of this might be the GPT board member. You can imagine the board meeting of the past where one of the board members posed a question that nobody had the answer for. A colleague was then tasked with sending their minions to collect the information required to answer the question… at the next board meeting… which then posed further questions and further scurrying minions. The appropriately trained GPT board member can answer those questions on the spot and then iterate on them. The question then becomes whether the human board members have suitable awareness to interpret the “correctness” of the answers. Do we create single-purpose GUIs to solve these single use-cases?

Or do we start with a general purpose capability that can perform all use cases, effectively bypassing all of the logic built into all of the OSS/BSS tools and allowing the GPT to connect a novel user interface to reason directly with the ocean of data gathered from devices and EMS/NMS/etc?

The UI / UX is effectively the “coal-face” for any OSS. It has a significant influence over purchasing decisions. It has a significant influence over staff satisfaction and efficiency. Yet, I find it interesting that it’s often overlooked in our industry.

As per the link already cited above, I believe that UI/UX was a significant factor in ChatGPT leap-frogging the many other AI LLM leaders to gain massive market-share and attention around the world. It’s also one of the reasons Apple is currently a trillion dollar brand. We’re yet to have that UI/UX upsurge in the OSS industry but I’m sure it will happen. Perhaps in combination with an LLM.

As always, I’d love to hear your thoughts about making an OSS that is truly native to, and leverages the possibilities of, LLM (Large Language Model) platforms and the data that underpins them.

Have you already seen an astonishing OSS UI that isn’t just a thin GPT wrapper around an existing OSS? [FWIW, here are a couple that have impressed me recently – article 1, article 2]

PS. If you would like PAOSS to perform a UI/UX audit and review, please get in touch with us.