The Vocabulary Problem — what 'context' actually means

VivekWhen did you realize the problem wasn't the model?

AlanIt depends what kind of model you mean. My view was shaped by years of research on databases and query. Go back to the 2010s and the hot topic was big data — commodity hardware piling up data lakes. But the data lakes weren't helping much. There's a kind of modeling that lives under the header of ontologies, taxonomies, and controlled vocabularies. Librarians have used it for decades, but enterprises stayed ignorant of it, because they were committed to structured data, relational databases, and SQL. Those are fine — but that data-modeling approach doesn't scale to big-data environments. That's where ontologies are used to create FAIR data: whatever the type of data, you can bring it together into one big graph.

VivekWhy is migrating between AI systems so much harder than migrating from Salesforce to Microsoft Dynamics? In the AI context it's not a JSON file you can export and import.

AlanBecause we've had this fragmentation for so long, and there hasn't been a good-faith effort by enough companies to address it. There's a wonderful book, Software Wasteland by Dave McComb, about application-centric versus data-centric. The application-centric approach is the fragmented one: a lot of logic trapped in application code, recreated by developers over and over, with instance data duplicated everywhere — data warehouses, lakehouses, data lakes. The graph approach exists to address that fragmentation. It's not an easy fix, and it requires commitment.

Most large incumbents are committed to older architecture, because they grew up in the application-centric era, before AI really took hold. Their claims are about two things: real innovations they're justifiably proud of, and the old stuff they still want to keep selling. They've got one foot in the past and one foot in the present. Companies can't afford to commit to whatever their favorite software vendors want them to do anymore. They have to take an independent route to transformation — and it's not just technological. These departments have to be transformed too.

The goal isn't to eliminate language models, large or small — it's to use them together with a knowledge-graph approach. You enrich the data upstream and capture quality data upstream, so that what you feed the AI is trustworthy when it comes out the other end. Current AI is a black box; it needs quality input to produce trustworthy output. The only way to get that is to take ownership of the work stream.

What knowledge is, is conceptual abstractions. You want agents to have access to those abstraction layers, so they understand not just the concept at hand but the whole field it belongs to and its different meanings in each context. Take the ContextOps podcast — there are all these domains, and different layers of abstraction you're building. If you take an ontological approach — start with controlled vocabularies and taxonomies, then build the ontology — the machine gets layers of abstraction it can navigate, so it interprets correctly what the data means in a particular context.

VivekIs this just how markets form — concepts get named, overextended, and then it takes time to settle into a real definition? Or is something about this specific cycle different, which makes the vocabulary drift more harmful than usual?

AlanSomething's different now. Go back 20 to 25 years to Nick Carr's IT Doesn't Matter. His argument was that IT is all commoditized — companies don't need to worry about the technology, just commit to your favored vendor and they'll steer you right. That might have been okay for a while. So companies subscribed to suites, then to software as a service — large enterprises now have hundreds of thousands of SaaS subscriptions, all perpetuating the older architecture. There's a ton of noise from vendors struggling to hold market share, all making AI claims. You have to be a contrarian to sift through it and realize there isn't much trusted signal out there.

VivekYou've talked about foxes and hedgehogs. When there's this much noise, how should vendors actually work with the buying committee on the other side?

AlanAt PwC — hundreds of thousands of people globally — the formal org chart wasn't how you got things done. If you needed someone's help, you worked the informal network. I used internal social media to find people curious about the same things, and we piloted knowledge-graph projects that way. The caveat: a few pockets of innovation don't get you a firm-wide result. Most companies experimenting with this adopt it in small pockets. The hedgehogs are good at keeping the lights on, keeping the established business running — they're important. But the foxes are the ones who have to lead the effort to adopt AI the right way.

VivekWe had a similar experience at CogniSwitch. The POC worked, and the person who owned data management was aligned in principle. But they needed someone to sign off on the knowledge-curation part — a true guardian of the source of knowledge. The business buyer said: not us. We don't have the time, we don't even have a knowledge function — we barely have a data function. Standing one up for something they didn't yet fully understand felt like a bet they weren't ready to take, so they went with a different deployment.

Take a product manager told to score reps on mortgage-collection calls — short calls, ten or fifteen minutes. The temptation is: we have a million-token context window now, so just drop the call in and ask ten questions. Why does that simple prompt-based approach on unstructured data fall short?

AlanLarge language models are useful for certain things — it's about using the tools the right way, and I've been on that learning journey myself. I found I was wasting a lot of time with the language model, and my approach has changed significantly over the months. Companies face the same education: decide how to use the tools at hand in the best way, and make sure the tooling available now doesn't get in the way of the longer-term strategy. Not all of these tools are useful for all use cases — you have to be judicious, and people shouldn't waste time using a tool for a purpose it was never intended for.

Leadership really needs to be immune to the hype. The board ought to insist on it. Whoever gets appointed should be savvy about not being subject to hype, and not putting the organization through a spending cycle that's just wasteful. Companies aren't going to survive and thrive unless leadership is savvy about this.

VivekOne thing you'd change about how enterprises do this — and it actually happens tomorrow. What is it?

AlanYou have to stop buying into the notion that the stuff you're purchasing will solve your problems. You need a much larger commitment to being hands-on and building, rather than just buying the solution. If you have enthusiastic people building in a small way, and you keep the company on that track, it works — but not immediately. It can't just be one person's idea; everyone has to be committed. It's a bigger management challenge than companies have faced in a long time — maybe since before World War II, when they had to retool for wartime manufacturing. People don't really realize the nature of the challenge.