How to read research papers for fun and profit

One skill that I’ve learned after a year in grad school is how to effectively read research papers. Previously I had found them impenetrable, but now I find them a great source of information about cutting-edge science while it is being done and before it’s made its way into textbooks. Now I read about 4-5 of them every week.

My research area is natural language processing and machine learning, but I read papers in lots of fields, not just in AI and computer science. Papers are my go-to source for a myriad of scientific inquiries, for example: does drinking alcohol cause cancer? Are women more talkative than men? Was winter in Toronto abnormally cold this year? Etc.

我在研究生院学习一年后学到的一项技能是如何有效地阅读研究论文。 以前我发现它们难以穿透，但现在我发现它们是关于尖端科学的重要信息来源，而它正在完成并且在它进入教科书之前。 现在我每周阅读大约4-5个。

我的研究领域是自然语言处理和机器学习，但我在许多领域阅读论文，而不仅仅是人工智能和计算机科学。 论文是我进行无数科学调查的首选来源，例如：喝酒会导致癌症吗？ 女性比男性更健谈吗？ 今年多伦多的冬天异常寒冷吗？ 等等。

Why read scientific papers?

If you try to Google questions like these, you typically end up on Wikipedia or some random article on the internet. Research papers are an underutilized resource that have several advantages over other common sources of information on the internet.

Advantages over articles on the internet: no matter what topic, you will undoubtedly find articles on it on the internet. Some of these articles are excellent, but others are opinionated nonsense. Without being an expert yourself, it can be difficult to decide what information to trust. Peer-reviewed research papers are held to a much higher minimum quality standard, and for every claim they make, they have to clearly state their evidence, assumptions, how they arrived at the conclusion, and their degree of confidence in their result. You can examine the paper for yourself and decide if the assumptions are reasonable and the conclusions follow logically, rather than trust someone else’s word for it. With some digging deeper and some critical thinking, you can avoid a lot of misinformation on the internet.

Advantages over Wikipedia: Wikipedia is a pretty reliable source of truth; in fact, it often cites scientific papers as its sources. However, Wikipedia is written to be concise, so that oftentimes, a 30-page research paper is summarized to 1-2 sentences. If you only read Wikipedia, you will miss a lot of the nuances contained in the original paper, and only develop a cursory understanding compared to going directly to the source.

为何阅读科学论文？
如果你尝试谷歌这样的问题，你通常最终会在维基百科或互联网上的一些随机文章。研究论文是一种未充分利用的资源，与互联网上其他常见的信息来源相比具有几个优势。

互联网上的文章的优势：无论什么主题，你无疑会在互联网上找到文章。其中一些文章非常好，但其他文章都是自以为是的废话。如果不是自己的专家，可能很难决定要信任哪些信息。经过同行评审的研究论文的最低质量标准要高得多，对于他们提出的每一项要求，他们都必须清楚地陈述他们的证据，假设，他们如何得出结论，以及他们对结果的信任程度。您可以自己检查论文，并确定假设是否合理，结论是否符合逻辑，而不是相信别人的话。通过深入挖掘和一些批判性思维，您可以避免在互联网上发生大量错误信息。

维基百科的优势：维基百科是一个非常可靠的事实来源;事实上，它经常引用科学论文作为其来源。然而，维基百科的写作简洁，因此通常将30页的研究论文概括为1-2个句子。如果您只阅读维基百科，那么您将错过原始论文中包含的许多细微差别，并且与直接访问源代码相比，只能进行粗略的理解。

Finding the right paper to read

If your professor or colleague has assigned you a specific paper to read, then you can skip this section.

A big part of the challenge of reading papers is deciding which ones to read. There are a lot of papers out there, and only a few will be relevant to you. Therefore, deciding what to read is a nontrivial skill in itself.

Research papers are the most useful when you have a specific problem or question in mind. When I first started out reading papers, I approached this the wrong way. One day, I’d suddenly decide “hmm, complexity theory is pretty interesting, let’s go on arXiv and look at some recent complexity theory papers“. Then, I’d open a few, attempt to read them, get confused, and conclude I’m not smart enough to read complexity theory papers. Why is this a bad idea? A research paper exists to answer a very specific question, so it makes no sense to pick up a random paper without the background context. What is the problem? What approaches have been tried in the past, and how have they failed? Without understanding background information like this, it’s impossible to appreciate the contribution of a specific paper.

找到合适的论文阅读
如果您的教授或同事为您指定了特定的论文，那么您可以跳过本节。

阅读论文的挑战很大一部分是决定阅读哪些论文。那里有很多论文，只有少数与你有关。因此，决定阅读什么本身就是一项非常重要的技能。

当您遇到特定问题或疑问时，研究论文最有用。当我第一次开始阅读论文时，我走错了方向。有一天，我突然决定“嗯，复杂性理论非常有趣，让我们继续研究arXiv并查看最近的一些复杂性理论论文”。然后，我打开一些，尝试阅读它们，感到困惑，并得出结论我不够聪明，不能阅读复杂性理论论文。为什么这是一个坏主意？一篇研究论文的存在是为了回答一个非常具体的问题，因此在没有背景背景的情况下选择随机论文是没有意义的。问题是什么？过去曾尝试过哪些方法，它们如何失败？如果不理解这样的背景信息，就无法理解特定论文的贡献。

Above: Use the forward citation and related article buttons on Google Scholar to explore relevant papers.

It’s helpful to think of each research paper as a node in a massive, interconnected graph. Rather than each paper existing as a standalone item, a paper is deeply connected to the research that came before and after it.

Google Scholar is your best friend for exploring this graph. Begin by entering a few keywords and picking a few promising hits from the first 2-3 pages. Good, this is your starting point. Here are some heuristics for traversing the paper graph:

将每个研究论文视为大规模互连图中的节点是有帮助的。 一篇论文不是将每篇论文作为独立项目存在，而是与之前和之后的研究密切相关。

Google Scholar是您探索此图表的最佳朋友。 首先输入一些关键词，然后从前2-3页中选择一些有希望的点击。 好，这是你的出发点。 以下是遍历纸质图表的一些启发式方法：

• To go forward in time, look at works that cited this paper. A paper being cited usually means one of two things: (1) the future paper uses some technique or result developed in the current paper for some other purpose, or (2) the future paper improves on the techniques in the current paper. Citations of the second type are more useful.
• To go backward in time, look at the paper’s introduction and related work. This puts the paper in context of previous work. Occasionally, you find a survey paper that doesn’t contribute anything novel of its own, but summarizes a bunch of previous related work; these are really helpful when you’re beginning your research in a topic.
• Citation count is a good indicator of a paper’s importance and merit. If the paper has under 10 citations, take its claims with a grain of salt (even more so if it’s an arXiv preprint and not a peer-reviewed paper). Over 100 citations means the paper has made a significant contribution; over 1000 citations indicates a landmark paper in the field and is probably worth reading. Citation count is not a perfect metric, especially for very recent work, but it’s a useful heuristic that’s applicable across disciplines.

The first pass: High level overview

Great, you’ve decided on a paper to read. Now how to read it effectively?

Reading a paper is not like reading a novel. When you read a novel, you start at the beginning and read linearly until you reach the end. However, reading a paper is most efficient by hopping around the sections as appropriate, rather than read linearly from beginning to end.

The goal of your first reading of a paper is to first get a high level overview of the paper, before diving into the details. As you go through the paper, here are some good questions that you should be asking yourself:

第一遍：高级概述
太棒了，你决定读一篇论文。 现在如何有效地阅读它？

读报纸不像读小说。 当你读一本小说时，你从头开始并线性阅读直到你到达终点。 但是，通过适当地跳过这些部分来阅读论文是最有效的，而不是从头到尾线性地阅读。

首次阅读论文的目的是首先获得论文的高级概述，然后再深入了解详细信息。 在阅读本文时，您应该问自己以下一些好问题：

• What is the problem being solved?
• What approaches have been tried before, and what are their limitations?
• What is this paper’s novel contribution?
• What experiments were done, using what dataset? How successful were the results?
• Can the method in this paper be applied to my problem?
• If not, what assumptions are needed for this method to work?

Above: Treat each paper as a node in a massive graph of research, rather than a standalone item in a vacuum.

When I read a paper, I usually proceed in the following order:

当我阅读论文时，我通常按以下顺序进行：

1. Abstract: a long paragraph that summarizes the entire paper. Read this to decide if the rest of the paper is worth reading or not.
2. Introduction, diagrams, tables, and conclusion. Often, reading the diagrams and captions gives you a good idea of what’s going on with minimal effort.
3. If the field is unfamiliar to you, then note down any interesting references in the introduction and related works sections to explore later. If the field is familiar, then just skim these sections.
4. Read the main body of the paper: model, experiment, and discussion, without getting too bogged down in the details. If a section is confusing, skip it for now and come back to it on a second reading.

That’s it — you’ve finished reading a paper! Now you can either go back and read it again, focusing on the details you skimmed over the first pass, or move on to a different paper that you’ve added to your backlog.

When reading a paper, you should not expect to understand every aspect of the paper by the time you’re done. You can always refer back to the paper at a later time, as needed. Generally, you don’t need to understand all the details, unless you’re trying to replicate or extend the paper.

就是这样 - 你读完了一篇论文！ 现在，您可以返回并再次阅读，重点关注您在第一遍中浏览的详细信息，或者转到已添加到待办事项中的其他纸张。

阅读论文时，您不应期望在完成论文时理解论文的各个方面。 您可以根据需要稍后再参考该论文。 通常，您不需要了解所有细节，除非您尝试复制或扩展论文。

Help, I’m stuck!

Sometimes, despite your best efforts, you find that a paper is impenetrable. It’s not necessarily your fault — some papers are hastily written hours before a conference deadline. What do you do now?

Look for a video or blog post explaining the paper. If you’re lucky, someone may have recorded a lecture where the author presents the paper at a conference. Maybe somebody wrote a blog post summarizing the paper (Colah’s blog has great summaries of machine learning research). These are often better at explaining things than the actual paper.

If there’s a lot of background terminology that don’t make sense, it may be better to consult other sources like textbooks and course lectures rather than papers. This is especially true if the research is not new (>10 years old). Research papers are not always the best at explaining a concept clearly: by their nature, they document research as it’s being done. Sometimes, the paper paints an incomplete picture of something that’s better understood later. Textbook writers can look back on research after it’s already done, and thereby benefit from hindsight knowledge that didn’t exist when the paper was written.

Basic statistics is useful in many experimental fields — concepts like linear / logistic regression, p-values, hypothesis testing, and common statistical distribution. Any paper that deals with experimental data will use at least some statistics, so it’s worthwhile to be comfortable with basic stats.

救命，我被困住了！
有时候，尽管你付出了最大的努力，但你发现一篇论文是难以理解的。这不一定是你的错 - 有些文件是在会议截止日期前几个小时写的。你现在做什么？

寻找解释该论文的视频或博客文章。如果你很幸运，有人可能会录制一个作者在会议上提交论文的讲座。也许有人撰写了一篇总结论文的博客文章（Colah的博客对机器学习研究有很好的总结）。这些通常比实际论文更能解释事物。

如果有很多背景术语没有意义，那么最好先咨询教科书和课程讲座等其他来源，而不是论文。如果研究不是新的（> 10年），则尤其如此。研究论文并不总是最能明确地解释一个概念：就其本质而言，它们将研究记录在案。有时候，这篇论文描绘了一幅不完整的画面，这些画面后来会被更好地理解。教科书作者可以在已经完成后回顾研究，从而受益于在撰写论文时不存在的事后知识。

基本统计在许多实验领域都很有用 - 线性/逻辑回归，p值，假设检验和常见统计分布等概念。任何处理实验数据的论文都会使用至少一些统计数据，所以对基本统计数据感到满意是值得的。

That’s it for my advice. The densely packed two-column pages of text may appear daunting to the uninitiated reader, but they can be conquered with a bit of practice. Whether it’s for work or for fun, you definitely don’t need a PhD to read papers.

这是我的建议。 密集的两栏文本对于不熟悉的读者来说可能显得令人生畏，但它们可以通过一些练习来征服。 无论是为了工作还是为了好玩，你绝对不需要博士学位来阅读论文。