The Core Thesis
When experienced practitioners use AI systems to do real work, they externalize information about how they think. Every prompt, correction, refinement, and rejection produces data about how problems are framed, how quality is evaluated, and how judgment is applied. At scale, this constitutes a continuous record of organizational expertise that has historically been impossible to capture.
The most significant output of AI deployment may not be the work the model produces. It may be the cognitive data workers generate while using it.
This process does not require deliberate planning. It emerges structurally from the interaction between AI systems and experienced practitioners. But whether organizations recognize it or not, AI-assisted workflows convert previously tacit expertise into durable, analyzable artifacts. The long-term implications extend well beyond productivity gains; they concern who owns expertise, how it transfers, and how much of professional work can ultimately be reproduced through systems rather than individuals.
The Tacit Knowledge Problem
Organizations have spent decades attempting to capture expertise, with limited success.
The difficulty is that much of what experts know is tacit. It exists as intuition, pattern recognition, and accumulated judgment rather than explicit procedure. Experts often cannot fully articulate how they arrive at conclusions because the reasoning process has become internalized through years of practice.
Traditional knowledge management approaches consistently encounter this limitation. Documentation captures procedures but misses judgment. Interviews capture what experts can articulate, not necessarily what they actually do. Job descriptions describe responsibilities rather than decision-making processes. Direct observation is expensive, difficult to scale, and often alters behavior.
AI-assisted work changes this dynamic.
When an experienced practitioner uses an AI system to solve a real problem, they frequently externalize information that would otherwise remain invisible: objectives, constraints, assumptions, failure conditions, quality standards, edge cases, and contextual considerations. They reveal not only what they are trying to accomplish but how they think about accomplishing it.
A prompt is not a complete representation of expertise. It is often incomplete, inconsistent, and noisy. Senior experts may omit critical context because it feels obvious to them; novices may produce verbose prompts that reveal little understanding. Many interactions involve routine tasks with limited informational value.
Yet at scale, across thousands of workers and millions of interactions, meaningful patterns emerge. The aggregate becomes a partial map of how practitioners actually approach their work.
Not All Expertise Extracts Equally
Before building the automation argument, one qualification matters: tacit knowledge is not uniform, and not all of it is equally visible through AI interactions.
Some forms of professional knowledge translate naturally into digital artifacts. Technical analysis, document production, software development, data processing, research synthesis, and structured decision-making leave extensive traces that can be observed, recorded, and modeled.
Other forms are more resistant. Negotiation, political influence, stakeholder management, trust-building, and institutional memory depend on relationships and context that are difficult to reduce to prompts or workflow logs.
Cognitive telemetry does not make every aspect of a profession legible. It makes certain categories of expertise significantly more observable than they were before. The distinction matters because automation risk is unlikely to be distributed evenly. Roles whose value derives primarily from structured cognitive tasks are more exposed than roles whose value derives from social positioning, accountability, or relationship capital.
From Prompts to Judgment
The highest-value signal in this process is not the prompt itself; it is the sequence of judgments that follows.
When a practitioner rejects an output, modifies instructions, rewrites a section, or accepts one version over another, they reveal information about quality. They distinguish between outputs that are merely plausible and outputs that are genuinely useful in context.
This distinction matters because factual correctness is often the easiest part of professional work. What organizations have historically struggled to capture is judgment: knowing which solution fits the situation, which trade-offs matter, and which outcomes satisfy real-world requirements.
Traditional workflows rarely surface this information in structured form. AI-assisted workflows generate it continuously.
Whether this feedback is ultimately used for model training, workflow optimization, or internal knowledge systems is a secondary question. The more fundamental shift is that expert judgment is being converted into data. The organization now possesses artifacts that describe not only what work was done, but how quality was evaluated.
The Standardization Feedback Loop
A less obvious dynamic emerges once AI systems become integrated into everyday work: roles begin adapting to the tools.
Workers restructure tasks to fit what the systems handle well. Processes become more standardized. Inputs become more structured. Deliverables become easier to evaluate. Workflows increasingly conform to patterns that produce reliable outputs.
This creates a feedback loop. Standardized work is easier for AI systems to assist with. Increased assistance encourages further standardization. Further standardization makes the work easier to model and, eventually, easier to automate.
A concrete example: a legal team that previously relied on senior attorneys to draft contract clauses from experience begins using AI to generate first drafts. Attorneys shift toward reviewing and correcting outputs rather than originating them. Over time, the drafting task becomes increasingly structured; inputs are templated, outputs are evaluated against explicit criteria, and the judgment required narrows to a smaller set of decisions. The role has not been automated, but it has become more legible.
The system is not only learning the role. The role is reshaping itself around the system.
This feedback loop may be the most significant long-term effect of AI adoption; not because it eliminates roles directly, but because it gradually converts less-structured work into more-structured work, closing the gap between what humans do and what computational systems can model.
A Case Study: Software Engineering
Software engineering is the domain where cognitive telemetry is most technically legible. The tooling is widely deployed, the data flows are concrete, and the boundary between what is captured and what remains out of reach can be drawn with reasonable precision.
The Telemetry Stack
Modern AI coding tools operate across several instrumentation layers simultaneously. Understanding what each layer captures is necessary before assessing what the aggregate reveals.
Completion telemetry is the most visible layer. When a tool like GitHub Copilot generates a suggestion and the engineer accepts, modifies, or rejects it, that interaction is logged. The raw event includes the suggestion content, the surrounding context window sent to the model, the engineer's action, and in some implementations the latency between suggestion display and decision. Copilot's telemetry documentation confirms collection of acceptance rates, suggestion content, and editor context. What this captures is a continuous stream of binary and partial-acceptance judgments against generated code, anchored to the exact context in which the judgment was made.
Edit sequence telemetry goes deeper. Tools integrated at the language server level can observe not just accept/reject events but the full edit history between suggestion and commit: what was deleted, what was rewritten, in what order, and how many iterations preceded a stable state. This is structurally more valuable than acceptance rate alone. An engineer who accepts a suggestion and immediately rewrites half of it has revealed more about their quality standards than one who simply rejected it. The delta between suggested code and committed code is a high-resolution signal; it encodes judgment the engineer never articulated explicitly.
Chat and inline prompt telemetry captures the natural language layer. When an engineer uses a chat interface to ask for a refactor, explain an error, generate a test, or work through a design decision, the prompt text, the model response, and any follow-up corrections are logged. GitHub Copilot Chat, Cursor, and similar tools retain conversation history for the session and in some configurations beyond it. These prompts are qualitatively different from completion telemetry; they contain explicit reasoning, stated constraints, and problem framing that completion interactions only imply.
Diagnostic and context telemetry captures the surrounding state: open files, active errors, recent git diff, imported packages, and the language server's understanding of the codebase graph at the time of the interaction. This context is sent to the model to improve completion quality, but it also means that every interaction is logged with a snapshot of what the engineer was working on, what was broken, and what the codebase structure looked like. The interaction is not recorded in isolation; it is recorded with its full operational context.
The degree of capture varies significantly by tooling configuration and organizational policy. Enterprise deployments may restrict telemetry transmission; privacy-conscious teams may disable certain logging features; tooling fragmentation means different engineers on the same team may be instrumented differently. What is consistent across mainstream systems is the direction: observability is increasing, not decreasing, and the default configurations of widely adopted tools retain more than most engineers assume.
What Seniority Looks Like in Prompt Data
The most consequential aspect of this telemetry is that it stratifies meaningfully by experience level, and the most valuable layers correspond to the most expensive engineers.
A junior engineer's interactions cluster around syntax errors, library API lookups, boilerplate generation, and error message interpretation. These prompts are high volume and low cognitive density. "Fix this TypeScript error." "Write a React component that does X." "What does this error mean." The interactions reveal familiarity with tooling but limited problem decomposition.
A mid-level engineer's interactions shift toward pattern-level reasoning. Prompts begin to specify constraints: performance requirements, existing architectural patterns to conform to, edge cases to handle. Completion rejections become more selective; they accept syntactically correct code less readily and push back on implementations that violate conventions the model cannot infer from context. The edit sequences after acceptance become more targeted; specific lines rewritten, not wholesale replacement.
A senior engineer's interactions reveal a different cognitive layer entirely. Prompts encode system-level context that requires judgment to assemble: "We use event sourcing here, don't introduce direct state mutation; this service is downstream of X and needs to handle out-of-order delivery; previous attempts to solve this used Y approach and failed because of Z." The prompt is not a task specification; it is a compressed transfer of architectural knowledge and failure history. Completion rejections at this level frequently involve correct code being rejected for reasons the model cannot observe: it will cause a performance regression under load patterns not visible in the codebase, it introduces a dependency that conflicts with a future migration plan, it solves the immediate problem but creates a maintenance burden in a specific module that the engineer knows is already fragile.
A staff or principal engineer's prompts, when they use chat interfaces for design work, encode something more valuable still: the reasoning process behind architectural decisions. "We need to decide between these two approaches; the first has better write throughput but makes the read path more complex; our current bottleneck is reads not writes; here are the failure modes I'm worried about." This is not a prompt for code generation. It is the engineer externalizing the exact decision framework they would otherwise apply silently and commit to in a design document that captures the conclusion but not the reasoning.
Code Review as Implicit RLHF
Beyond direct AI tool interactions, a parallel extraction mechanism operates through AI-assisted code review.
When an engineer reviews a pull request using an AI tool that summarizes changes, flags potential issues, or suggests improvements, their accept/reject/modify decisions on those suggestions constitute preference feedback over code quality judgments. This produces preference signals analogous to RLHF feedback data, generated passively through normal review workflow rather than through an instrumented training process.
The signal here is particularly high-fidelity because code review judgment is among the hardest forms of engineering expertise to capture. It involves simultaneously evaluating correctness, performance characteristics, maintainability, conformance to team conventions, security implications, and fit with the system's existing architecture. Senior engineers apply all of these criteria in parallel, often without being able to articulate the weighting. When they override an AI suggestion during review, the override is a labeled example of that composite judgment applied to a specific case.
At scale across a codebase with hundreds of engineers performing thousands of reviews, this data maps what good code looks like in that specific organizational and technical context; not in the abstract, but against the actual constraints, conventions, and quality standards of a real system.
What Remains Out of Reach
The extraction mechanism has real limits, and being precise about them matters.
Pre-code architectural reasoning is largely invisible. The decision to use an event-driven architecture rather than a request-response model, to separate a service boundary at a particular seam, or to defer a capability rather than build it now; these decisions are made in conversations, documents, and internal deliberation before a line of code is written. They produce no prompt, no completion event, no edit sequence. The telemetry stack captures implementation expertise far more completely than it captures design expertise.
Institutional and political context does not surface in code interactions. An engineer who avoids a particular technical approach because it conflicts with another team's roadmap, because it failed in a previous system two years ago, or because it will require coordination with a team that moves slowly; this judgment is applied silently. The code that results looks identical to code written without that context. The telemetry captures the output but not the reasoning.
Failure mode intuition is partially captured but mostly lost. A senior engineer who rejects a correct implementation because they recognize it will fail under a specific load pattern is generating a useful signal; but only if they articulate the reason in a prompt or comment. If the rejection happens as a silent edit, the telemetry records the modification without the rationale.
Cross-session reasoning is structurally absent. AI tools operate within session context windows. The accumulated understanding an engineer carries across months of working on a system; the mental model of which components are fragile, which abstractions are leaking, where technical debt is concentrated, what the system's actual failure modes are under production load; none of this is transferred through the telemetry layer. Each session begins without it.
The Net Assessment
What the telemetry stack captures, in aggregate, is a detailed record of implementation-level engineering judgment: how experienced engineers decompose tasks, what quality thresholds they apply to generated code, how they specify constraints and failure conditions, and what corrections they make to outputs that are technically correct but contextually wrong.
This is not a complete picture of software engineering expertise. It is heavily weighted toward the execution layer and largely blind to the design and architectural layers where senior judgment is most concentrated and most difficult to replicate.
Three distinctions are worth holding separately here. Observability is what the telemetry stack captures: interaction events, edit sequences, preference signals. Modelability is whether a model can learn useful generalizations from that data: observable interactions are necessary but not sufficient, since noisy or low-density signals may not produce learnable patterns. Automatability is whether the learned capability is sufficient to replace a workflow without human oversight: a model can be highly accurate on implementation tasks while remaining entirely dependent on human judgment for the upstream decisions that determine what to implement. The telemetry stack advances observability broadly, modelability for implementation work specifically, and automatability only for the narrowest and most structured execution tasks.
The practical implication is that the compression problem is more tractable for implementation work than for system design. AI systems trained on this telemetry will become better at generating code that fits existing patterns, at avoiding the classes of errors that experienced engineers consistently reject, and at producing outputs that require less correction from mid-level practitioners. They will not, from this data alone, become better at the upstream reasoning that determines what should be built and how the system should be structured.
The boundary between implementation and design is not fixed. As AI tools extend further into design workflows; architecture diagram generation, API design assistance, system design review; the telemetry layer will expand upward. The out-of-reach category is shrinking, not stable.
Capability Accumulation
A common assumption is that automating knowledge work requires solving general intelligence. In most cases, it does not.
Organizations do not need systems capable of matching human performance across all domains. They need systems capable of producing acceptable outputs for economically relevant tasks. The challenge is not replicating human cognition in its entirety; it is accumulating sufficient coverage of a role's operational requirements.
As organizations accumulate prompts, evaluations, workflows, corrections, and task histories, they build a dataset of how roles function in practice. Automation then becomes partly a compression problem: the objective is not to recreate the full richness of human expertise, but to capture enough of a role's input-output relationships to produce acceptable performance across the tasks that matter.
For narrowly defined domains with structured inputs and measurable outputs, this threshold is reachable without breakthroughs in general AI capability. The data generated through normal AI-assisted work moves organizations and AI providers closer to that threshold for the specific roles generating the data.
The Organizational Incentive
The incentives surrounding this process do not require coordinated intent.
AI providers have an obvious interest in understanding how professionals perform work. Better understanding produces better products and greater capability. Enterprises adopting AI tools are typically motivated by productivity gains, cost reduction, or competitive pressure. Most are not deploying AI primarily as knowledge extraction mechanisms.
Yet structural incentives often matter more than explicit intentions.
Organizations have always sought to reduce dependence on individual expertise. Expertise concentrated in specific people creates institutional risk. Expertise that can be documented, standardized, or embedded into systems creates leverage. AI-assisted workflows naturally move expertise in that direction, even when productivity is the stated objective.
The knowledge extraction is often not the goal. It is the byproduct of pursuing a goal that organizations already had.
What This Implies
The implications are frequently framed as a question of replacement. That framing misses the more immediate effect.
For most occupations, the near-term consequence is compression rather than elimination. If AI systems allow fewer workers to produce the same output volume, organizations may require smaller teams, slower hiring, or reduced headcount growth. The economic effect can be substantial without full automation occurring.
Compensation structures may shift. Entry-level roles may narrow as the tasks that once served as apprenticeship become AI-assisted. Organizations may place increasing value on individuals who can effectively direct, supervise, and integrate AI systems, rather than those who primarily execute the underlying cognitive tasks.
None of this requires artificial general intelligence. None of it requires organizations to consciously pursue workforce reduction. It requires only that AI systems become useful enough to reduce the marginal cost of producing certain forms of cognitive output; and that organizations, responding rationally to reduced costs, adjust how they staff and structure work accordingly.
The broader shift is this: AI-assisted work makes expertise observable with more detail than previously possible. Once observable, expertise becomes measurable. Once measurable, it becomes transferable. Once transferable, it becomes increasingly possible to reproduce through systems rather than individuals.
The future of knowledge work will be shaped less by breakthroughs in machine intelligence than by the growing capacity of organizations to capture, store, and operationalize human expertise itself. Most of that capture is already underway, embedded in the ordinary interactions of people using tools they adopted for entirely different reasons.