The research-to-output cycle in science is longer than in most professions: months or years from research question to published paper, with multiple experimental iterations, peer review revisions, and the ongoing need to maintain coherence across a complex evidence base. A workflow that works for a two-week project fails at 18-month multi-experiment papers.
The workflow described here addresses the specific failure modes that emerge in scientific research at scale: research questions that drift from the literature they're grounded in; experimental designs that aren't explicitly connected to the questions they're answering; analysis decisions made for convenience rather than documented rationale; and research outputs that produce a published paper but don't route the accumulated knowledge back into the scientific knowledge infrastructure for future work.
The Four Research-to-Output Failure Modes in Science
Failure 1 — Research question drift. The project begins with a clear question grounded in the literature, but as experiments accumulate and results come in, the framing shifts without being explicitly updated. The paper that results addresses a question that's related to but different from the original — and the relationship between the accumulated evidence and the stated claims is unclear.
Failure 2 — Experimental design not grounded in the literature. Experimental parameters are chosen based on what's available or convenient, not on what the literature suggests is appropriate for the specific question being asked. Results are difficult to interpret against prior work because the design choices weren't explicitly connected to the prior literature.
Failure 3 — Analysis decisions made post hoc without documentation. Data analysis proceeds through a sequence of decisions — what to include, what to exclude, how to normalize, what statistical test to apply — without documenting the rationale for each decision at the time it's made. When reviewers or collaborators ask why specific choices were made, the rationale is uncertain.
Failure 4 — Knowledge not routed after publication. The research that produced a paper — experimental protocols developed, methodological innovations made, analytical approaches validated — goes into the paper and not into the lab's knowledge base. The next project in the same area starts with less accumulated knowledge than it should.
Stage 1: Research Question Grounding
Every research project begins with an explicit statement of the research question and its grounding in the literature.
RESEARCH QUESTION GROUNDING — [Date]
PROJECT: [Project name or code]
RESEARCH QUESTION: [The specific question this research addresses — one sentence]
IMPORTANCE:
Why does this question matter? [What scientific or practical significance does answering it have?]
What gap in the literature does it address? [What does the field not know that this will establish?]
LITERATURE GROUNDING:
What does the field know about this question? [Summary of the relevant evidence]
Key papers establishing the prior work: [Reference manager keys or citations]
What does the field not know? [The specific gap this research addresses]
What preliminary data supports the feasibility? [Evidence that the approach is viable]
HYPOTHESES:
Primary hypothesis: [The specific claim the research will test]
Alternative hypotheses: [What else the results might show]
Predictions: [What specific experimental results would support or refute each hypothesis]
SIGNIFICANCE STATEMENT:
If the hypothesis is confirmed, what would that establish?
If the hypothesis is refuted, what would that establish?
What would be the field-level significance of either outcome?
This document is the anchor that the rest of the project returns to. When experiments accumulate and results come in, the research question grounding document is the reference for: are we still asking the same question we started with? If the question has evolved, update the document explicitly and document why.
Stage 2: Experimental Design Documentation
Before any experiments begin, the design is documented with explicit connections to the research question and the literature.
EXPERIMENTAL DESIGN DOCUMENTATION — [Date]
PROJECT: [Project name]
Research question reference: [Pointer to the research question grounding document]
EXPERIMENTAL APPROACH:
What will we measure? [The dependent variables]
What will we manipulate? [The independent variables]
What will we hold constant? [Control conditions]
Why these choices? [Explicit reference to literature that supports these design choices]
SAMPLE AND EXPERIMENTAL UNIT:
What is the unit of analysis? [Cell / Animal / Patient / Sample / Other]
Sample size rationale: [Power analysis or field-standard justification for n]
Selection criteria: [Inclusion and exclusion criteria — before data collection]
PROTOCOL:
Which protocol(s)? [Protocol name + version]
Any modifications to standard protocol? [What and why]
PLANNED ANALYSIS:
Primary analysis: [What statistical test or analytical approach will address the primary hypothesis]
Secondary analyses: [Additional analyses planned before data collection]
Exclusion criteria: [Pre-specified rules for what data will be excluded from analysis]
WHAT WOULD CONSTITUTE A POSITIVE RESULT?
[Explicit pre-specification: what result would support the primary hypothesis?]
WHAT WOULD CONSTITUTE A NEGATIVE RESULT?
[Explicit pre-specification: what result would refute the primary hypothesis?]
WHAT WOULD CONSTITUTE AN INCONCLUSIVE RESULT?
[What result would not clearly support or refute — and how would we respond?]
The pre-specification discipline: specifying the primary analysis before collecting data protects against post hoc analysis selection — choosing the analysis that produces the most favorable results rather than the analysis most appropriate to the question. In fields where pre-registration is standard (clinical research, many areas of psychology), this is formalized; in fields where it's not yet standard, the lab's own documentation creates the same protection.
Stage 3: Data Collection and Documentation
Data collection produces both the experimental results and the experimental record — what was done, when, by whom, under what conditions, and what was observed.
EXPERIMENT RECORD — [Experiment ID: Date-ProjectCode-SequenceNumber]
PROJECT: [Project name]
Research question: [Pointer to research question document]
Protocol used: [Protocol name + version — exact version critical]
Date(s) conducted: [Date range]
Conducted by: [Scientist name(s)]
EXPERIMENTAL CONDITIONS:
[Specific experimental conditions as run — any deviations from planned protocol?]
[Key instrument settings and parameters]
[Environmental conditions if relevant]
RAW DATA LOCATION: [Exact path to raw data files]
OBSERVATIONS DURING COLLECTION:
[Anything unusual observed during data collection]
[Any equipment issues or anomalies]
[Any samples or conditions that appeared compromised at the time of collection]
PRELIMINARY OBSERVATIONS:
[Initial impression of results — before formal analysis]
[Any obvious quality issues]
QUALITY CONTROL RESULTS:
[QC checks performed and their outcomes]
[Any samples excluded at QC stage — with reason]
POST-COLLECTION ROUTING:
[ ] Raw data backed up in [location]
[ ] Experiment record linked to project file
[ ] Any immediate observations relevant to experimental design communicated to [collaborators]
The deviation documentation discipline: any deviation from the planned protocol — a different reagent lot, a different instrument, a procedural adaptation — is documented at time of collection. Post hoc discovery of deviations (finding them in lab notes months later) creates uncertainty about whether the deviation affected the results. Real-time documentation maintains the clear experimental record.
Stage 4: Data Analysis with Decision Documentation
Data analysis is not a mechanical process of applying pre-specified tests — it requires judgment at multiple stages. Documenting the judgments and their rationale at the time they're made is what makes the analysis reproducible and defensible.
ANALYSIS DECISION LOG — [Date]
PROJECT: [Project name]
Experiment(s): [Which experiments' data this analysis covers]
Analysis code location: [Repository + commit ID or file path]
DATA PREPARATION:
Raw data characteristics: [What the raw data looks like — key quality metrics]
Data cleaning decisions:
Decision 1: [What was done + why + alternatives considered]
Decision 2: [...]
Normalization: [What normalization was applied + why this approach]
Exclusions applied: [What was excluded + pre-specified or post hoc + reason]
Were exclusions pre-specified? [Yes → reference to design doc / No → note that they were post hoc]
ANALYSIS CHOICES:
Primary analysis method: [What was used + why]
Was this the pre-specified approach? [Yes / No — if No: why was the pre-specified approach not used?]
Parameter choices: [Key analytical parameters and their justification]
Software and versions: [Analysis software and package versions]
RESULTS SUMMARY:
Primary result: [The key quantitative result with uncertainty estimate]
Secondary results: [Additional findings]
Unexpected findings: [Anything that wasn't predicted by the hypothesis]
INTERPRETATION:
What does the primary result say about the hypothesis?
[Supported / Refuted / Inconclusive — with specific reference to the pre-specification]
What are the most important caveats?
WHAT WOULD CHANGE THIS INTERPRETATION:
[What additional data or analyses would change the interpretation?]
[What are the most significant alternative explanations?]
CODE COMMIT:
[ ] Analysis code committed to repository with this analysis version
[ ] Code README updated to reflect this analysis
[ ] Output files (figures, tables) linked to this code version
The post-hoc analysis transparency principle: post-hoc analyses — analyses not pre-specified before data collection — are legitimate and valuable. They are documented as post-hoc and interpreted with appropriate caution: as hypothesis-generating rather than hypothesis-testing. The failure mode is treating post-hoc analyses as if they were pre-specified, which inflates confidence in the results.
Stage 5: Manuscript Preparation
Manuscript preparation is the synthesis stage: the accumulated experimental evidence and analysis is structured into a coherent scientific argument.
The outline-before-writing discipline:
Before any manuscript prose is written, the manuscript outline is drafted from the accumulated evidence:
MANUSCRIPT OUTLINE — [Date]
TITLE (working): [What does this paper establish?]
Journal target: [Where will this be submitted?]
ABSTRACT (one sentence per section):
Background: [The gap in the field this paper addresses]
Approach: [How the question was addressed]
Key finding: [What was established]
Significance: [Why it matters]
INTRODUCTION (logic):
[Paragraph 1: The broad context]
[Paragraph 2: The specific gap]
[Paragraph 3: The approach and its rationale]
[Paragraph 4 (optional): Organization of the paper]
KEY FIGURES (planned):
Figure 1: [What it shows + what it establishes]
Figure 2: [...]
Figure 3: [...]
RESULTS LOGIC:
[The logical sequence in which results will be presented]
[Does each figure follow logically from the prior?]
[Is the argument coherent as a whole?]
DISCUSSION LOGIC:
[What does the primary finding mean?]
[How does it connect to the prior literature?]
[What are the most important caveats?]
[What are the implications for future work?]
CONTRIBUTION STATEMENT:
[One sentence: what does this paper add to the field?]
The outline is the architecture of the argument before the prose is written. Writing from a well-constructed outline produces manuscripts faster than writing exploratorily and restructuring prose.
Stage 6: Post-Publication Knowledge Routing
After a paper is published, the research that produced it is routed back into the knowledge base.
POST-PUBLICATION KNOWLEDGE ROUTING — [Date]
PAPER: [Citation]
Publication date: [Date]
KNOWLEDGE BASE UPDATES:
Research synthesis entries to update:
[ ] [Research question 1]: Paper resolves / contributes to / complicates this question
[ ] [Research question 2]: [...]
Protocol updates:
[ ] [Protocol 1]: Paper validates / modifies this protocol
[ ] [Protocol 2]: [...]
Analytical methodology updates:
[ ] [Analysis type]: Paper validates / modifies this analytical approach
LITERATURE CONNECTIONS:
[ ] Papers that this paper will be cited in connection with — add cross-references
[ ] Follow-up questions this paper opens — add to research synthesis as open questions
GRANT AND PROPOSAL UPDATES:
[ ] Preliminary data sections in pending proposals updated to cite this paper
[ ] Future directions established by this paper documented for next proposal cycle
NEW RESEARCH QUESTIONS:
[ ] Questions opened by this paper's findings → new research synthesis entries created
A Worked Example: Single-Experiment Paper
Context: a cell biology lab is studying how a specific kinase affects cell migration. The PI designs an experiment to test whether pharmacological inhibition of the kinase reduces migration velocity.
Research question grounding (Week 1):
The kinase has been shown in the literature to phosphorylate a key cytoskeletal regulator; published data suggests the phosphorylation promotes cytoskeletal dynamics; the hypothesis is that inhibiting the kinase will reduce the dynamics that drive migration. Three key papers are identified as the basis. The prediction is specified: inhibition will reduce migration velocity by more than 20% (field-standard significance threshold) compared to vehicle control.
Experimental design (Week 2):
Sample size calculated from published variance in migration assays in the same cell type — 12 wells per condition based on published coefficient of variation. Inhibitor concentration chosen based on published IC50 for the target kinase in the cell type to be used. Controls: vehicle, positive control (a known migration inhibitor). Primary analysis pre-specified: unpaired t-test comparing mean migration velocity between inhibitor and vehicle conditions.
Data collection (Weeks 3-4):
Three independent experiments conducted. Experiment records created for each. One experiment excluded based on a pre-specified QC criterion (high background in the vehicle condition indicating contamination).
Analysis (Week 5):
Analysis decision log records: one experiment excluded (QC, pre-specified); normalization to cell count performed (pre-specified); primary analysis performed as pre-specified; result: 34% reduction in migration velocity, p=0.003. One post-hoc analysis performed (migration persistence, not pre-specified) — documented as exploratory/hypothesis-generating.
Manuscript preparation (Weeks 6-9):
Outline drafted from the accumulated evidence. Four key figures planned. Writing proceeds from outline.
Post-publication routing:
Research synthesis entry updated: kinase inhibition confirmed to reduce migration velocity in this cell type (specific paper). Protocol knowledge base updated: migration assay protocol validated for this application. New research question created: does kinase inhibition affect other aspects of migration behavior (persistence, directionality)?
Key Takeaways
-
The scientist research-to-output workflow has six stages. Research question grounding (literature + hypothesis + predictions), experimental design documentation (pre-specified design + analysis plan), data collection with deviation documentation, analysis with decision documentation (pre-specified vs. post-hoc explicitly distinguished), manuscript preparation from structured outline, and post-publication knowledge routing.
-
Research question grounding is the anchor for the entire project. The research question, its literature basis, and the predictions are documented before any experiments begin. When the project drifts, this document is the reference.
-
Pre-specification protects against post-hoc analysis inflation. Primary analyses pre-specified before data collection are hypothesis-testing. Post-hoc analyses are hypothesis-generating. Documenting which is which, at the time of analysis, maintains the appropriate confidence level for each.
-
Analysis decision documentation is the reproducibility foundation. Every analysis judgment — what to exclude, how to normalize, what statistical test — is documented with its rationale at the time it's made. This documentation is what makes the analysis reproducible and defensible.
-
Post-publication knowledge routing converts each paper's research into compounding infrastructure. Research synthesis updated, protocols validated, analytical methodology documented, new questions opened. Without this routing, each paper is an output from one project rather than a contribution to the lab's growing knowledge base.
Conclusion
The research-to-output workflow for scientists creates the infrastructure that allows scientific research to be rigorous, reproducible, and cumulative. The research question grounding anchors the project to the literature. The experimental design documentation grounds the design choices in the prior literature and pre-specifies the analysis. The data collection and analysis documentation maintains the clear experimental record that reproducibility requires. And the post-publication routing step converts each paper's accumulated research into permanent scientific knowledge infrastructure. The discipline required at each stage is modest; the compound return across multiple projects and years is substantial.
To go deeper, check out Web Clipping for Research Papers.