Data science work generates notes at multiple levels: the analysis session where methodology decisions are made, the literature read that informs those decisions, the experiments run and their results, and the domain knowledge accumulated about the data environment. Without structure, these notes become a hard drive full of notebooks, PDFs, and documents that are technically present but practically inaccessible.
The four templates below structure the note-taking contexts that most directly accumulate into retrievable data science intelligence: analysis session notes, research literature notes, experiment design and results, and domain knowledge capture. Each template includes the fields that make notes retrievable — not just organized.
Template 1: Analysis Session Notes
Analysis sessions — exploratory data analysis, model iteration, hypothesis investigation — generate methodology decisions and findings that need to be captured in the moment. The notebook is good for code; it's not designed to capture the reasoning behind the code.
The gap between notebooks and analysis notes:
A Jupyter notebook contains code, outputs, and whatever markdown comments the data scientist added. It doesn't naturally contain:
- Why this approach was chosen over alternatives
- What the data revealed that changed the analytical direction
- What was tried that didn't work and why it was abandoned
- What the finding means for the business question
Analysis session notes capture this reasoning layer separately from the code.
ANALYSIS SESSION NOTES
Date: [Date of session]
Project: [What project / business question]
Session goal: [What I was trying to accomplish in this session]
WHAT I DID:
Approach taken: [What I tried in this session]
Key code or notebook reference: [Where the code lives]
DECISIONS MADE:
Decision: [A methodology choice made in this session]
Options I considered: [What else I thought about]
Why I chose this: [The reasoning]
What would change this: [What would make me revisit]
Decision: [...]
WHAT I FOUND:
Finding: [A result or observation from the session]
What it means: [Interpretation — in plain terms]
Confidence: [How confident I am in this finding]
Finding: [...]
WHAT SURPRISED ME:
[A result that was different from what I expected]
What I expected: [Prior expectation]
What actually happened: [Observed result]
How I updated my thinking: [What this changes about my approach or understanding]
WHAT DIDN'T WORK:
[What I tried that failed + why it failed]
[What I won't try again in this context + why]
OUTSTANDING QUESTIONS:
[ ] [Question to investigate in next session]
[ ] [...]
KB UPDATE:
[ ] Experiment log updated
[ ] Methodology KB updated (if applicable)
[ ] Domain knowledge KB updated (if applicable)
The "what didn't work" section is the most frequently skipped and most valuable field. The approach that failed — the feature that degraded performance, the model that didn't generalize, the preprocessing that introduced leakage — represents information about the data environment that should be captured to prevent repeating the same work.
The "what surprised me" field is the most signal-rich field. Surprises are where intuitions are updated. The finding that contradicted expectations is the learning that compounds; the finding that confirmed expectations confirms but doesn't expand the model.
Template 2: Research Literature Notes
Research literature notes convert reading into retrievable methodology intelligence. The paper read and not captured disappears; the paper read with structured notes becomes a searchable resource organized by the problems it addresses.
RESEARCH LITERATURE NOTES
Date read: [Date]
Source: [Paper title / Blog post / Documentation]
Authors: [If paper]
Publication: [Journal / Conference / Blog / Documentation site]
Publication date: [When originally published — may differ from when you read it]
URL + Archive URL: [Link + archive.org backup for web sources]
TYPE OF CONTENT:
[ ] Academic paper (peer reviewed)
[ ] Practitioner blog (company blog or individual)
[ ] Official documentation
[ ] Tutorial or course material
[ ] Survey or review paper
WHY I READ THIS:
What problem or question prompted this reading: [The specific question that sent me to this source]
MAIN CONTRIBUTION / KEY INSIGHT:
What this paper/post is about: [2-3 sentence summary]
The key insight for applied data science: [The most practically useful takeaway]
METHOD DETAILS (for methodology papers):
What the method does: [Plain language description]
When to use it: [Problem types + data characteristics where this applies]
When NOT to use it: [Where this method is inappropriate]
Key assumptions: [What the method assumes about the data]
Computational complexity: [If relevant — does this scale to your data?]
IMPLEMENTATION NOTES:
Available implementations: [Which libraries implement this]
Code reference: [If I've implemented or found a good implementation]
Pitfalls to avoid: [Common implementation errors]
QUESTIONS THIS RAISES:
[ ] [Open question that this paper prompted]
[ ] [...]
HOW THIS APPLIES TO MY CURRENT WORK:
Relevant to current project?: [ ] Yes: [How] [ ] No [ ] Potentially: [When]
CREDIBILITY ASSESSMENT:
Publication quality: [High / Medium — consider source type]
Currency: [Is this recent enough to be current best practice?]
Claims I'd verify: [Anything in this source I'd check against another source]
KB UPDATE:
[ ] Research literature KB updated
Problem type tagged: [Which problem type KB entry this was added to]
Method tagged: [Which method family KB entry this was added to]
The "publication date" vs. "date read" distinction matters for methodology literature. A blog post published in 2019 about a technique that was superseded in 2022 looks current if only the read date is captured. The publication date makes the currency of the methodology guidance visible when it's used.
The "when NOT to use it" field is the most practically useful and most commonly absent field in methodology notes. The paper that describes a method typically emphasizes when it works well; the data scientist needs to know when it doesn't work so they don't misapply it.
Template 3: Experiment Design and Results Notes
Experiment design and results notes create the organized record of what was tried and what happened — the experiment history that prevents re-doing work that's already been done.
EXPERIMENT DESIGN AND RESULTS
EXPERIMENT DESIGN (written before running):
Date designed: [Date]
Project: [What project]
Experiment type: [ ] Model comparison [ ] Feature impact [ ] Hyperparameter search
[ ] A/B test [ ] Data split comparison [ ] Other
Hypothesis:
What I expect to find: [Clear statement of the prediction]
Why I expect this: [Reasoning behind the hypothesis]
Design:
What I'm comparing: [Treatment and control configurations]
Evaluation metric: [How success will be measured — chosen before seeing results]
Data used: [What data, what version/snapshot]
Random seeds: [If applicable — for reproducibility]
Success criteria:
What result would confirm the hypothesis: [Specific threshold]
What result would reject it: [What would falsify the hypothesis]
EXPERIMENT RESULTS (filled in after running):
Date run: [Date]
Results:
[Configuration 1]: [Metric value]
[Configuration 2]: [Metric value]
Difference: [Magnitude + interpretation]
Statistical significance (if applicable):
Test: [What statistical test used]
Result: [p-value or confidence interval]
Interpretation: [What this means — is the difference meaningful?]
Was hypothesis confirmed?: [ ] Yes [ ] No [ ] Partially
WHAT THE RESULTS MEAN:
Interpretation: [What this finding implies — plain language]
Unexpected: [Any result that didn't match expectations]
WHAT I'D TRY NEXT:
Informed by these results: [What follows from what was found]
REPRODUCIBILITY NOTE:
Can this experiment be reproduced?: [ ] Yes [ ] Partially [ ] No
What's needed to reproduce: [Random seeds + data version + environment note]
The "design before running" discipline is the scientific discipline that prevents unconscious result-shopping — adjusting the hypothesis or evaluation metric after seeing results to match what was found. The hypothesis written before the experiment is the ground truth against which the results are evaluated.
Template 4: Domain Knowledge Notes
Domain knowledge notes capture the organizational and data context that accumulates through direct experience with the data environment — the information that isn't in any documentation but lives in the experience of having worked with the data.
DOMAIN KNOWLEDGE NOTES
Date of discovery: [When this was learned]
Domain: [Data source / business area / system]
Knowledge type: [ ] Data quirk [ ] Business logic [ ] Definition [ ] Process [ ] Gotcha
WHAT I LEARNED:
Description: [Clear statement of the knowledge]
How I discovered it: [What led to discovering this]
WHY IT MATTERS:
What happens when you don't know this: [The error or wrong result that would occur]
Analyses it affects: [What types of analyses this knowledge is relevant to]
HOW TO HANDLE IT:
The right approach: [What to do given this knowledge]
The workaround (if needed): [Code or process to handle this correctly]
VERIFICATION:
Is this still accurate?: [ ] Confirmed [ ] Uncertain [ ] Known to have changed
Date last verified: [When this was confirmed to still be accurate]
WHO ELSE KNOWS THIS:
Team members who should know this: [Who this is relevant for]
SOURCE:
How this knowledge was confirmed: [Other team members who confirmed + documentation if any]
KB UPDATE:
[ ] Domain/data KB updated
The "what happens when you don't know this" field is the field that converts passive domain knowledge capture into active institutional value. Framing each piece of domain knowledge in terms of the analysis error it prevents gives future analysts the context to understand why the knowledge matters, not just what it is.
The "date last verified" field prevents domain knowledge from becoming confident obsolescence. Business processes change, data schemas evolve, and the data environment shifts. Domain knowledge needs to be verified periodically — and the date makes it obvious when it hasn't been.
Key Takeaways
-
Analysis session notes capture the reasoning layer that notebooks can't. Notebooks contain code and outputs; analysis session notes contain why this approach was chosen, what was tried that didn't work, and what surprised. The reasoning layer is what makes future work faster.
-
Research literature notes should include publication date, not just read date. The blog post from 2019 may describe a technique superseded in 2022. Publication date makes the currency of methodology guidance visible.
-
"When NOT to use it" is the most important field in methodology notes. Papers describe when a method works; the data scientist needs to know when not to apply it to avoid misapplication.
-
Experiment design notes are written before running. The pre-analysis plan — hypothesis and evaluation metric written before seeing results — is the scientific discipline that prevents result-shopping.
-
Domain knowledge notes include "date last verified." Business processes evolve; data definitions change. Domain knowledge needs periodic verification, and the date makes it visible when it hasn't been updated.
Conclusion
Four note-taking templates structure the most important data science note-taking contexts: analysis session notes that capture the reasoning layer behind code decisions and the value-rich "what surprised me" and "what didn't work" fields; research literature notes with publication date (not just read date), "when NOT to use it" methodology bounds, and KB update steps; experiment design notes with the pre-run hypothesis and evaluation metric that create the scientific discipline; and domain knowledge notes with discovery date, "what happens when you don't know this" framing, and date-last-verified currency tracking. The KB update step on each template — the explicit instruction to update the relevant KB domain after each note — is what converts individual notes from isolated records into the compound knowledge system that makes data science productivity improve over time.
Related reading: Web Clipping vs. Bookmarking.