Knowledge Is a Conversation

·9 min read
#mentat

Somewhere in your company's Notion wiki is a page titled "Deployment Pipeline (Last Updated: August 2024)" that three people have bookmarked and nobody trusts. It describes what to do when everything goes right. It says nothing about what to do when the service is slow at 3am, because the person who wrote it didn't think that counted as a step.


"Write It Down" Is the Wrong Verb

Everyone says "write it down." That's the command that gets issued when the new VP of Product proposes something that was tried 18 months ago and killed after a painful customer conversation. The command assumes the problem is compliance, but it's not.

Michael Gerber's E-Myth Revisited diagnosed part of this decades ago - the technician who starts a business can't naturally systematize what they know. Atul Gawande's Checklist Manifesto made the case that checklists save lives in surgery and aviation. The Phoenix Project turned the knowledge bottleneck into a novel - Brent, the one engineer who knows everything, becomes the constraint that grinds the whole organization to a halt. All three are right, and all three solutions work for explicit procedures. They don't touch the tacit half - the judgment calls that experts can't put into a checklist because they don't know they're making them.

The Curse of Knowledge means experts can't simulate what it's like not to know. They write the intentional parts and skip the judgment calls that are automatic to them. A senior engineer documents the deployment pipeline in Confluence and leaves out "if this service is slow, check that first" because it doesn't feel like a step. It feels like breathing, and writing captures what's explicit but cannot capture what's tacit. And 42% of institutional knowledge is tacit - exists in experience, judgment, pattern recognition, not in any procedure.

And even the docs that DO get written? They go stale in 3-6 months, with engineering SOPs obsolete within a quarter. Onboarding guides that took 20-40 hours to produce become irrelevant within 6 months. For a 50-person company, shadow work costs real money: people answering questions that are already "documented" somewhere in a wiki but unfindable or untrusted. And this cost scales faster than headcount. At 10 people, everyone knows who knows what. At 50, the knowledge graph fragments - people start asking "who would know about this?" At 100, the gap between what the organization collectively knows and what any individual can access becomes the primary bottleneck, and it gets more expensive to close with every new hire.

So if writing doesn't work, what does? Structured conversation - not any conversation, but structured elicitation that pulls knowledge out through questions experts didn't know they needed to answer.


Why Experts Write Badly (It's Not What You Think)

Experts don't write bad docs because they're bad writers. They write bad docs because the task itself is broken.

Writing is a monological act. The expert manages three simultaneous cognitive processes: retrieving the information, organizing it structurally, and handling the mechanics of writing - grammar, syntax, tool navigation. That's "extraneous cognitive load" that has nothing to do with the expertise itself. When the medium eats up working memory, the content suffers.

The output: muddy, cloudy, convoluted docs that assume too much context. Or worse, docs that are technically accurate and completely useless because they describe the "what" without the "why." The new VP of Product reads the doc on the API design and gets the architecture. She doesn't get why approach B was rejected after the August customer conversation, which is the only thing that would prevent her from proposing it again.

There's a neurological reason for this gap. Research on memory decay shows that contextual intent - the "why did I choose A over B?" - decays more than 90% within 72 hours. The expert remembers the decision but not the reasoning chain. By the time someone asks them to document it, the most valuable part is already gone.

The healthcare data shows what happens when you MANDATE documentation under pressure. An audit of 621 physician progress notes found 92.73% were copy-pasted, and 58% of those copies contained identifiable errors. Even in a legally regulated environment with life-or-death stakes, written documentation degrades under cognitive load. This isn't a software problem - healthcare, aerospace, the military, law firms. Every field that runs on expert judgment hits the same wall when it tries to write that judgment down.


Conversation as Elicitation Tool

The knowledge engineering field has known this for 40 years. I think the Critical Decision Method (CDM) is the closest thing we have to a gold standard: you don't ask "how do you do this?" You ask "tell me about the last time this went wrong." Incident-based elicitation - walk me through that specific case, tell me what you were noticing at this point, what someone less experienced would have missed, and what changed your approach.

This works because conversation is dialogical. The listener acts as an external regulator - their confusion signals force the expert to backtrack and fill gaps they didn't know they'd left. The expert dedicates working memory to the actual expertise instead of wasting it on Confluence formatting and guessing what the reader needs.

NASA's Ares I-X program used structured conversation sessions with storytelling prompts to capture knowledge across the program, documented in their multi-volume knowledge capture report (May 2010). The US Army has studied structured elicitation for decades: their After Action Review (AAR), introduced in the 1970s, has been validated across a meta-analysis of 61 studies covering 915 teams and 3,499 individuals, with an effect size of d=0.79 - larger than most training methods. The key finding: highly structured AARs are significantly more effective than unstructured ones. Structure matters.

Software engineering's equivalent - the post-mortem - shows how much the written medium loses. Research on code incident analysis found that manual reviews only captured 54-63% of the relevant commit context. Nearly half of what actually happened never made it into the write-up.

The tool works because it matches how expertise is stored - in pattern libraries built from lived cases, not in step-by-step procedures.


Who Does the Asking (And Why It Matters)

This could be a skilled analyst following a playbook. Critical Decision Method exists as a human protocol. Knowledge engineers have been doing this for decades. But at scale, you need consistency and availability that humans can't provide. A senior engineer leaving in two weeks can't wait for the analyst to be free. The sales VP onboarding in Singapore can't sync schedules with someone in New York.

An AI interviewer with the right protocol can, arguably, conduct the same elicitation at 2am on a Sunday or across 15 experts in parallel. It doesn't get tired, doesn't need scheduling, doesn't bring ego or politics into the conversation. It's a facilitator, not a recorder. It reads what's already documented and uses gaps to generate questions. "I see you documented the happy path for deployment. What happens when the service is slow?" It makes elicitation feel like helping a colleague, not filling out a form.

The async dimension matters more than the AI dimension. Traditional knowledge capture requires two or more people to align their schedules - sometimes flying team members in from other locations - and block out one to three hour time slots. That's extremely expensive for the organization, and by the time you get the session scheduled, the knowledge is already expiring. Ebbinghaus's forgetting curve shows 50% of information is lost within 20 minutes and 90% within a week - the "why" behind a decision is the first thing to go. It's like buying produce that's already going soft because you had to wait two weeks for the grocery store to open. Async elicitation means the conversation happens in the gaps: ten minutes after a production incident while the context is fresh, a follow-up question that arrives Tuesday morning about something the expert mentioned Friday. Knowledge capture stops being a project with a deadline and becomes a continuous background process that fits into how people actually work. No documentation sprints, no "knowledge transfer week" before someone leaves.

And the system comes back. During a conversation, an expert says "we're planning to overhaul the CI/CD pipeline next quarter." That's a trigger point - a future reference the system extracts and schedules for follow-up. On the other side, the AI agent watches for changes: doc updates, incident reports, code commits. Three months later, the deployment docs get updated, two incidents touch the pipeline, and the scheduler fires because the convergence is there: the future the expert described has started arriving, and reality doesn't match the plan. That pressure point triggers re-engagement: "Hey, you talked about upcoming changes to the pipeline, and since then the docs changed and we had a couple of incidents. Can you walk me through what actually happened and what didn't make it?" No human interviewer would remember that context or notice the drift. The system does, because it's reconciling what experts said against what actually changed.

From expert-as-documentarian (high friction, low fidelity) to expert-as-conversationalist (low friction, high fidelity). The expert talks, the system structures.


Expert Glorification (Not Extraction)

The companies that get this wrong treat experts as resources to be mined. Extract the knowledge, replace the person. Experts can smell that from across the building, and they will never participate.

The companies that get this right treat experts as heroes willing to share. Frame the conversation as freeing them from being the bottleneck. Being Brent is exhausting - every question routes through you, every decision waits for your calendar. "What you know took 10 years to learn. Help the next person get there in 10 months" isn't about replacing Brent. It's about giving Brent his weekends back. Their judgment becomes available to 50 people instead of the 3 who can get on their calendar. A good test: could your company survive losing your Brent within six months? If the answer is no, the knowledge is trapped in a person, not available to the organization. That's not a compliment to the expert - it's a failure of the system around them.

Motivation research is clear: monetary rewards don't drive knowledge sharing. Enjoyment in helping others drives knowledge sharing, as does knowledge self-efficacy and the sense that "what I know matters and I can make it accessible." Design for that, not for compliance. An Elo rating per knowledge domain makes this concrete. Every contribution - answering questions, walking through incidents, clarifying decisions - builds visible, domain-specific reputation. The deployment pipeline expert sees that reflected differently from the payments expert. Knowledge sharing stops being a chore and starts being a game where the score tracks real authority.

There's a cognitive science reason this works. Research on the protégé effect shows that explaining concepts to others reinforces learning and identifies gaps. The conversation isn't just extracting knowledge, it's refining it. The expert who talks through their decision-making process is doing structured reflection, converting raw experience into explicit understanding. And sometimes the act of explaining itself generates new understanding - the expert connects two ideas they hadn't linked before, or spots a gap in their own mental model they'd been working around unconsciously. Engineers call this rubber duck debugging. Physicists call it the Feynman technique. You start explaining the problem and solve it mid-sentence. That's why it feels valuable instead of draining.


Multi-Expert Reconciliation (Disagreement Is Data)

When you ask three experts the same question, you get three different answers. That's not a bug, but the signal.

Engineer A says "if deployment is slow, check the cache first - stale entries are the usual culprit." Engineer B says "skip the cache, start with the logs - you'll waste an hour in the cache and the logs would have told you in two minutes." Both are right, depending on context neither of them made explicit. Forcing them to converge into one "official answer" destroys the nuance. The right move is to preserve the disagreement with full reasoning. Document that A's mental model starts from known patterns, B's prioritizes faster diagnosis. Let the person querying this knowledge see both and choose based on their context.

The Delphi method works: gather views independently, show anonymized summaries, iterate without forcing consensus. 70-80% agreement is consensus. The remaining 20-30% is often where the most valuable knowledge lives - the edge cases, the judgment calls, the "it depends" that separates real expertise from procedural knowledge.

Disagreement can indicate epistemic health, not failure - preserve that, because the alternative is groupthink.


The Output: Living, Not Static

You don't get a Confluence page titled "Deployment Best Practices (Last Updated: March 2024)." You get a system that can answer "What do I do if the deployment is slow?" and pull from Engineer A's cache-first approach, Engineer B's log-first approach, and the incident from last quarter where both approaches failed and someone discovered a third path. The knowledge reconciles across sources and surfaces the right context at query time, which basically means the knowledge stops moving when people stop talking.

Knowledge-as-artifact (write once, slowly decay) versus knowledge-as-conversation (evolve continuously, synthesize on demand).


The Real Choice

The companies reaching 50+ people mark right now have a choice: keep treating knowledge capture as a writing task, or shift the frame and treat it as a conversation task that can be structured, coordinated, and scaled in ways that writing never could.

Next: how do you know the AI interviewer is capturing what the expert actually said, not what the model thinks they meant? Multi-expert reconciliation as a fact-checking mechanism.