Own your meaning, or rent it

VivekHere's a setup. A specialty pharmacy runs a small contact center — providers on long calls with cancer patients, helping them understand and adhere to their medication. Not five- or ten-minute urgent-care calls; these run forty-five minutes to an hour, and there are thousands of them. The pharmacy is heavily incentivized to understand provider performance and close the care gap. The CTO says: Gemini has a two-million-token context window, I'll just pipe the transcripts in with a prompt and get my provider performance. Where should that break — and at what scale does it stop being a transcription problem and become a knowledge or semantic problem?

ElliottIf we dump every call into a context window and start asking questions, these models — when they hit contradictions — usually won't make a claim about the things that conflict, unless you prompt them specifically to find inconsistencies. If you don't, it slides over them, and as the context grows it pays less attention to those specific things. For the first ten or twenty calls it might work fine. But as you go on, certain things won't get flagged. And when you end up in court because the wrong person got the wrong thing — one line item changed and it wasn't caught — you're going to have to say "the computer made a mistake." That's not what they want to hear.

A semantic system imposes stricter, deductive guardrails. It's not just descriptive — here's all the information — it's prescriptive: here's the form the information must take, here's what's allowed and what isn't. A system like that forces, before the call ends, that this documentation has to be gathered: this date, this confirmation, this clip from the call. You need a precise paper trail. These LLMs aren't deductive — they're making a series of highly, highly educated guesses I rely on every day. But when you're dealing with people's lives, the answer needs to rest on something stronger than "the million-token window should have held."

VivekStep back — what do you mean by "deductive"? It does seem to deduce things for me, at least when I read its output.

ElliottThink about an inference: moving from one set of facts to another — if these are true, then this follows. The inference can be deductive — if this, then necessarily that — or inductive — if this, then to a very high probability that. That difference, "necessarily" versus "probably," even when probably is very high, is the line between a deductive and an inductive system. LLM inferences are inductive: they predict the next token from a preponderance of evidence. A deductive system has specific rules — if these values, then that value.

Say the internet decides, as a meme, that Volkswagen Beetles are no longer cars. We keep training models on the internet, so eventually the model "learns" a Beetle isn't a car — and if you ask it, off the basis of all that training, it says: of course a Beetle isn't a car. A deductive system says: this object is a Volkswagen; Volkswagens are cars; therefore it's a car — every single time, necessarily. There's no contingent state of the world that changes that.

We don't actually want the LLM itself to be deductive — we want it squishy, so it can handle human language, which is never perfectly precise. But the process that pulls information into its context — that's the part you want tight control over. You want that to be a deductive process, modeled after your language and how you understand your terms. The LLM is the interface, the thing talking to the human. If you tell it exactly what to say right before it says it, the chance of it messing that up is very low.

JoshuaWe keep throwing around "semantic layer," and everyone has a different definition. How would you define it for a CTO?

ElliottA semantic layer is your enterprise's governed layer of meaning. It tells your systems, your people, and now your agents what your business concepts mean, how they relate, and how to read data through those concepts — even when that data lives in many different systems you don't own. It is not another copy of your data. Most enterprise data lives in applications whose schemas were designed for that application's use case, not for your company. The semantic layer lets you connect that data and bend it toward what you mean by those concepts.

How ready are people? It spans the gamut. The financial institutions we work with are more ready than anyone I've seen — they almost always have a knowledge team, because they have to. Random startup tech? They're just trying to tread water; they don't have the time, so a semantic layer gets treated as a luxury. But it doesn't have to be — you don't build the whole thing at once. You make a clean strike, put a stake in the ground, and build out from there.

Ask yourself: who's going to own the meaning? If you adopt another SaaS and everything routes through it, and you don't own the model — it's not portable, you can't get it out — then you're renting your meaning, renting your infrastructure. Later, once they've drowned out everyone around them, they raise the rent, and you're beholden to them. We've seen this before; we saw it in the cloud. None of this is anti-vendor — I love vendors. But the thing you should own is the meaning, so you can connect into everything else and move when a better vendor comes along.

When I go into an organization, I want to find the most pedantic person — I love those people. They have an encyclopedic understanding of every piece of meaning, even when it's written down nowhere. If you can't find them, start asserting things — give people something to respond to. The ones who respond strongest usually have these maps hidden away, sometimes without even knowing it. People have feelings about what everything means. Language is a form of life, and they live in it every single day.

These "context operating system" ideas — I have them in applications I use personally, but they're not all in production yet; we're still sorting out the ins and outs. If someone tells you they've completely solved context, they might be selling you a bridge.

We're at the beginning. AI is a steam engine for a new generation. Anyone who thinks agents are the end of this — right now I'm trying to build things where I don't have to make LLM calls. I want LLM calls, they matter, but I want speed and lower token costs. These bigger, better systems aren't going to scale with the current agent loops; we'll have to find better and better ways to do this.