How I read a paper when I want the idea to last

I used to treat a good paper note as a compressed paper: every section represented, every term defined, every result filed away. The notes were complete—and almost impossible to return to.

Now I read in three passes. Each pass has a different job. Together, they leave me with something shorter than a summary but more useful than a highlight reel.

Pass one: map the claim

Before I get absorbed in implementation details, I try to write the paper’s main claim as one sentence:

Compared with what, this method changes which outcome, under which conditions?

That sentence exposes ambiguity quickly. “Improves performance” is not yet a claim. Improves which metric, on what distribution, relative to which baseline, and at what cost?

On this pass I read the abstract, introduction, figures, and conclusion. I write down:

the problem the authors chose;
the comparison that makes the contribution meaningful;
the result the paper most needs to be true; and
the assumptions that define where the claim applies.

The goal is orientation, not judgment. If I cannot describe the argument without borrowing the paper’s wording, I probably do not understand it yet.

Pass two: pressure-test the evidence

The second pass is slower. I follow the evidence chain from the central claim back through the experiments and methods.

Three questions do most of the work:

Which result carries the argument? Not every table matters equally.
What alternative explanation is still possible? A gain may come from scale, data, tuning effort, or a favorable evaluation setup.
What would make the result disappear? Distribution shift, a stronger baseline, a different metric, or a change in the unit of analysis can all matter.

This is not about trying to “defeat” a paper. It is about finding the boundary of what the evidence supports. A precise boundary makes a contribution more useful, not less.

Pass three: look for transfer

On the final pass, I stop asking only what the paper says and start asking what it gives me.

I look for three kinds of transfer:

Method transfer: Could the experimental design answer a question in another domain?
Concept transfer: Does the paper give a name to a pattern I have seen elsewhere?
Failure transfer: Is the limitation a special case of a broader problem?

This is where a reading becomes part of a research direction. A technique may not transfer, but a measurement warning might. A benchmark result may be narrow, but the way its controls were designed may generalize.

The note I keep

My final note fits on one screen before any detailed appendix:

Claim — the strongest version the evidence supports.
Mechanism — why the authors think the result happens.
Evidence — the experiment or argument doing the most work.
Friction — one unresolved assumption, limitation, or competing explanation.
Connection — where the idea touches something I already care about.
Next question — the smallest follow-up I can state clearly.

I still keep technical details when I may need to reproduce something. But they sit below the thinking, not in place of it.

What I am trying to optimize

I am not optimizing for the number of papers finished. I want to recognize recurring structures: the same measurement mistake in a new domain, the same hidden assumption under a different name, or the same useful idea surviving a change of setting.

The real test of a note is simple: months later, can it help me ask a better question?