THE VERY GOOD GUYS / R05 / RESEARCH ANALYSIS
Can someone do the job from these notes?
A guide can read well but leave out steps. The real test is whether someone else can use it to do the job.
Read the full paper ↓- What the evidence shows
- A work guide used nine recorded sessions and existing files. It kept a list of choices that still needed approval.
- The limit
- The guide exists. We have not measured whether staff use it or whether someone new can do the job from its steps.
- The practical decision
- Ask someone new to try the steps. Keep the source and any open decisions with the guide.
FULL WORKING PAPER
Decision-Preserving Synthesis: From Recorded Knowledge to Executable Procedures
Retrospective evidence and proposed evaluation. Historical checks, observed results and future tests are distinguished in the paper.
Abstract
A recorded interview can support a well-written procedure without establishing that the procedure is correct, authorized, or usable by another operator. This paper examines an operating manual compiled from nine recorded sessions and prior documents, including seven scoring rules and six rulings still awaiting approval. [A1] Supporting recordings show instruction, learner difficulty applying the method without workplace inputs, and a domain review that required procedure corrections before rollout. [A2] [A3] We propose decision-preserving synthesis: every actionable instruction retains its evidence, scope, authority, unresolved conflicts, and execution dependencies. A formal readiness rule and a proposed handoff feasibility pilot distinguish source support from operational adoption. The contribution is a practical representation and evaluation design, not a claim that transcription captures all tacit knowledge or that the proposed method has improved workplace outcomes.
Keywords
- knowledge capture
- procedure synthesis
- provenance
- retrieval augmented generation
- handoff evaluation
Contributions
- A proposed decision-preserving instruction record linking source support, authority, scope, conflict, and execution dependencies.
- An evidence-bounded analysis of manual compilation, unresolved rulings, teaching, and domain correction.
- A paired handoff evaluation separating document fidelity, independent execution, safe abstention, and operator effort.
Introduction
Operational knowledge is distributed across conversations, examples, records, habits, and disputed decisions. Turning recordings into a manual can make that material easier to find, but fluent synthesis can also erase the uncertainty that matters most. A tentative scoring rule may become an imperative. Two descriptions of a process may be merged into a sequence that nobody actually follows. The resulting document can look complete precisely because the unresolved decision has disappeared.
The motivating manual was compiled on July 16, 2026, from nine recorded working sessions and an existing document corpus. It preserved seven scoring rules alongside six rulings awaiting approval. Those counts describe the source inventory and pending decisions, not employee adoption or operational improvement. We ask how a synthesis process can preserve that distinction at the level of individual instructions, so another operator can identify both what to do and what still requires a decision. [A1]
Method
The retrospective method inspects the manual's compilation statement, provisional definitions, and open-decision register, then uses timestamped audit findings from relevant recordings. It separates authored synthesis from domain knowledge supplied by other participants. The laboratory's role in interviewing, organizing, or generating text does not imply authorship of an operator's technical procedure. The private provenance map preserves source locations; this paper uses generic references and omits identifying details. [A1] [A2] [A3]
Evidence is classified by what it can establish. A transcript can show that someone proposed a workflow or corrected a draft. A compiled manual establishes the existence of a synthesis artifact. An execution log can establish that a particular operator completed a particular task under particular conditions. These evidence types are related but not interchangeable. We do not infer a deployed knowledge system from an instructional demonstration or infer adoption from the presence of a document.
Evidence and Results
The manual's nine-session inventory and its seven rules with six unresolved rulings show an attempt to retain incomplete decisions. They do not establish whether the rules were later approved, whether the underlying source inventory was complete, or whether staff used the resulting manual. No measured completion rate, comprehension score, or before-and-after error rate is available from that artifact. The defensible result is a documented synthesis with visible uncertainty. [A1]
The teaching record adds a practical limit. A learner reported difficulty applying the method without suitable workplace inputs; the instructor explained using a recorded interview as source material, while final handoff generation was deferred to homework. A separate field review challenged the assumption that compliance was the remaining problem: the domain operator said the written procedure first required correction and review. These are observed teaching and review interactions, not proof that a corrected workflow was subsequently adopted. [A2] [A3]
Proposed Framework: Decision-Preserving Synthesis
Define decision-preserving synthesis as producing actionable text while retaining the distinctions that determine whether the action is justified. Each instruction is a record containing a trigger, action, responsible role, prerequisites, source spans, scope, authority status, conflict status, and version. The record separates an observed practice from an approved policy and separates both from a proposed improvement. An unresolved contradiction remains a decision item rather than being smoothed into a plausible compromise.
Let D be the dependency set for an instruction. Readiness requires source support for its factual premises, authorization for its normative choices, current applicability to its operating context, and resolution of critical conflicts in D. A supported quotation alone cannot satisfy all four conditions. For example, an interview may accurately describe historical stock handling while a later approved procedure changes when inventory is decremented. The synthesis must preserve that temporal relationship and cannot choose solely by retrieval similarity.
The representation also records safe branches. When an input is missing or approval is unresolved, the handoff states what may proceed, what must stop, and which role can decide. This is more useful than attaching a vague confidence percentage to the document. Criticality is defined before evaluation: an unsupported step that changes operational state carries a different consequence from an uncertain explanatory sentence. Both should remain traceable, but they need not block the same actions.
The modest contribution is a readiness rule joining evidence support, decision authority, and task dependencies in one instruction-level record. It adapts established knowledge and provenance ideas to the observed gap between recorded conversation and executable handoff. The prediction is fewer unsupported critical actions and more successful independent attempts on ambiguous tasks, with additional preparation and review time. Report justified holds separately from avoidable blockage; neither counts as task completion.
I = (trigger, action, role, prerequisites, source spans, scope, authority, conflicts, version)
The instruction record preserves both the content and the conditions under which it can be acted upon.
ready(I) = supported(I) AND authorized(I) AND current(I) AND resolved(critical dependencies of I)
Readiness is a conjunction of distinct judgments. Retrieval relevance, citation presence, and model confidence do not independently establish it.
Evaluation Design
The planned handoff study is a feasibility pilot using six fictional operating tasks: stock withdrawal, approval routing, shift handoff, duplicate correction, missing-input recovery, and a stale procedure conflicting with a newer artifact. Each receives a consented role-play recording, transcript, source packet, simulated interface, and prewritten truth sheet. Inputs are not yet frozen and no participants have been assigned. The packet deliberately includes exceptions and disagreement so the method is tested where preserving decisions might matter.
Both conditions receive the same packet, model version, generation settings, output-length limit, and domain-review allowance. The baseline produces a chronological summary and basic handoff. The proposed condition produces instruction records and then readable task guidance. One output per task per condition is retained, with raw drafts and reviewer edits. Distinct receivers perform the two conditions for each task; assignment is counterbalanced across tasks and no receiver sees both versions of the same task.
The primary outcome is correct first-attempt completion without clarification, counted over six assigned attempts per condition. Report justified holds for genuinely missing prerequisites separately from avoidable blockage caused by a deficient handoff. Both remain in the assigned-work denominator and neither counts as completion. Freeze that distinction in the truth sheet before testing. Secondary measures include unsupported critical instructions, exception preservation, retrievable claim support, wrong actions, clarification requests, and recovery time. Record preparation, review, correction, execution, and rescue minutes separately. A fixed answer bank supports a separately scored recovery phase.
Before execution, agree the scoring rubric, critical actions, eligibility rules, time limits, and operator-cost ceiling. The proposed pilot advancement rule requires no unsupported critical instructions, preservation of all critical exceptions, and better completion than the baseline within the agreed human-time budget. It decides whether to invest in a larger study, not whether a general method advantage has been established. With one generation and receiver attempt per task and condition, counterbalancing cannot eliminate receiver differences or estimate generation variability. Report paired descriptive outcomes; a tie shows no completion benefit on this set. The pilot compares synthesis formats given shared material and does not independently test interview effectiveness.
Limitations
An instruction record cannot recover facts that nobody recorded. Experts may omit habitual checks, disagree for legitimate contextual reasons, or remember a process inaccurately. Speaker attribution and transcription can also fail. Correctly citing such a statement establishes where it came from, not its truth. Domain review and a walk-through remain necessary, and reviewers must be able to reject the synthesized procedure rather than merely edit its wording.
The manual already preserved unresolved rulings without the proposed instruction schema, which is evidence against treating the schema as a prerequisite for useful synthesis. Moreover, the learner's difficulty concerned unavailable workplace inputs, not a demonstrated defect in summary format. Better source preparation or a direct expert walkthrough could outperform added structure. The proposed comparison must allow that outcome and cannot attribute every execution problem to a missing provenance field. [A1] [A2]
The proposed representation can become burdensome for routine tasks, and a detailed record may overwhelm the receiver if displayed without prioritization. It may also fail where authority is disputed or no owner will resolve a critical gap. Six synthetic tasks cannot establish workplace adoption, durable learning, or general transfer. Longitudinal use and permission to study real operations would require a separate protocol. The present paper reports no new execution results.
Conclusion
A useful procedure preserves the decisions that make action legitimate. The audited material supports real synthesis, teaching, and corrective review, while leaving adoption unproved. Decision-preserving synthesis makes source support, authority, conflict, and execution dependencies inspectable at the instruction level.
Begin with a small set of consequential instructions, resolve their source conflicts with the appropriate domain role, and test them with an operator who did not write the handoff. Retain ordinary prose for presentation while keeping decision records available for review. Advance the structure only if it improves correct action within the agreed human-time budget. That is the necessary next evidence beyond the existence of a polished manual.
References
Numbered entries link to public literature. Entries beginning with A describe retained private materials; identifying records and archive locations are not published.
- [1]
A Dynamic Theory of Organizational Knowledge Creation (opens in a new tab)
Organization Science 5(1), 14-37
Original paper, hosted by an academic institution; used for knowledge conversion distinctions.
https://www.lamsade.dauphine.fr/~giard/Organization_Science_Nonaka94.pdf
- [2]
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (opens in a new tab)
NeurIPS 2020
Primary author manuscript; retrieval supplies evidence but does not establish organizational authority.
https://arxiv.org/abs/2005.11401
- [3]
Enabling Large Language Models to Generate Text with Citations (opens in a new tab)
EMNLP 2023, 6465-6488
Used for separate evaluation of correctness and citation quality.
https://aclanthology.org/2023.emnlp-main.398/
- [4]
Why and Where: A Characterization of Data Provenance (opens in a new tab)
ICDT 2001, 316-330
Primary author-hosted paper; adapted as an analogy, not a new provenance theorem.
https://homepages.inf.ed.ac.uk/opb/papers/ICDT2001.pdf
- [A1]
Operating-manual compilation and unresolved-rule register
Source manual dated July 16
Nine recorded sessions, seven scoring rules, and six unresolved rulings. Adoption is not established.
Private source · Description only - [A2]
Recorded handoff instruction and learner feedback
Timestamped transcript audit
Teaching and proposed homework are distinguished from completed independent execution.
Private source · Description only - [A3]
Domain review requiring procedure correction before rollout
Timestamped transcript audit
The technical correction is attributed to the domain operator. Later adoption is not established.
Private source · Description only