That dreaded literature review looms large. Your inbox overflows with journal articles. But, some sparkle with statistical significance, others hide conflicts of interest behind fancy phrasing. And, I remember staring at my table of contents, wondering if my source selection would impress or implode during peer review. So do many authors, who are just getting started. They simply don't know which sources to trust. Therefore, we need a simple system to help us check whether any source we find on Google Scholar or our academic search engine of choice is actually research that we can trust. Let me present to you seven criteria that help you check whether or not a source is worth citing:
1. Published papers prove their worth
Authority isn't just a fancy word for "seems legit." Pull up the author's profile. Yes, legit researchers have a website (the uglier, the better, just kidding).
- Do they teach at respected institutions?
- Have they published other work in this field?
- Are they cited by other scholars?
Don't be the student, who cites a "Dr. Smith" who turns out to be a wellness blogger with a mail-order degree in crystal healing. Hard Nein. A better approach if you're completely new to this: Google Scholar the author. If they're legitimate, you'll find a trail of academic breadcrumbs—papers, citations, and institutional connections. Follow the publications.
2. Know whether journals justify trust
Not all journals deserve your trust. Look for peer review processes, impact factors, and institutional backing. Watch for red flags: promises of rapid publication, fees without clear services, or websites that look like they were designed during the dial-up era. There are too many shady players in this game. A good starting point is Beall's List of Potential Predatory Journals and Publishers. Fun fact: I've published in Frontiers journals and they are on the list. But that publisher is so large that some parts of their brand are shady and others aren't (long discussion). Either way, tread with caution when a name pops up on the list.
Another pro move is to check if the journal appears in your field's major indexing services. If it's missing from Web of Science or Scopus, treat it like that leftover sushi from last week—with extreme caution.
3. Study design divides wheat from chaff
Strong academic work backs claims with data, not dramatic declarations. Count the citations. Check the methodology. If a paper makes sweeping statements without support, it belongs in the same category as your uncle's Facebook posts about conspiracy theories. No, the earth is not flat, Theodore.
Here's a smart strategy: Flip to the methods section first. No methods? No dice. Empirically speaking that is. Now, there are fields that don't stress empirical studies and still provide valid additions to theoretical literature. Especially in the humanities and other fields, there is plenty of high-impact work that never flirted with empiricism. Then, again rockstars like Michel Foucault conducted empirically grounded historical research, too. His methodology combined theoretical analysis with detailed examinations of concrete practices and local phenomena. What a cool person.
4. From low-quality to literature review
Follow the paper trail. That's a sure way to find work that has bubbled to the surface over the years:
- Do other respected scholars cite this work?
- More importantly, how do they cite it?
Sometimes a widely-cited paper is famous for being spectacularly wrong (like those studies on dubious health benefits of chocolate, many of which are not as effective as we'd like them to be).
My tactical tip here: Use Google Scholar's "cited by" feature to see who's referencing the paper and what they're saying about it.
5. Browse better, research smarter
In some fields, a five-year-old paper is ancient history. In others, classical works from decades ago still shape current thinking. Know your field's freshness dating system and a paper's half-life. My archaeology colleagues would joke that anything from this century counts as "breaking news."
For this, it really takes getting to know your field a bit. Check your field's major journals. How old are their typical references? That's your freshness benchmark. Work from there. You might also find that there is a major need for updated citations from a related field (like when Bayesian statistics made their rounds in human-computer interaction).
6. Limitations are just honesty
Every paper has a perspective, but good academic work acknowledges its limitations. Watch for language that sounds more like a sales pitch than scholarly analysis. This should usually not get through peer review and if it did, then that's a surefire sign that the peer-review quality of the publication venue is lacking. If it promises to revolutionize everything you know about underwater basket weaving, grab your skepticism hat.
In the discussion (at the end of it or in a separate section after it), look for a limitations section. Its absence speaks volumes about the quality of the venue.
7. The writing makes the scholar
Professional academic writing isn't just about content. The ability to write good manuscript is a critical skill for researchers. Much excellent research can get rejected because of poorly written manuscripts. Our recent research emphasizes that researchers should be the 'primary drivers' of their manuscript writing (and not AI).
But that doesn't mean they should invite spelling mistakes everywhere. Spelling mistakes happen, but a paper riddled with errors suggests careless work. If the authors couldn't be bothered to proofread, did they really triple-check their data? And why was there no editorial process that helped put the paper into top shape?
Bottom line: Three typos? Maybe. Three typos per paragraph? Problem.
What to do about it?
Start with these three steps today:
- Create a source evaluation checklist in your reference manager (I've included one below for paid subscribers)
- Set up Google Scholar alerts for key authors in your field
- Bookmark the major indexing services for quick verifications
Good sources are much more than correct citations. Good sources are credible, current, and connected to the broader academic conversation.
Until next week, my friends.
