Skip to content

afeezadebayo/research-learning-skills / source-evaluation

source-evaluation

Verified

Evaluate sources before relying on them: lateral reading and the SIFT method, deeper checks of authority, method, evidence, currency, purpose and transparency, red flags for statistics, surveys, studies, reports, news, websites, social media and AI outputs, and confidence labels with reasons. Use whenever someone needs to check whether a source is credible, evaluate a statistic, study or report, verify AI-cited sources, judge a website or decide how much to trust information before using it.

Source Evaluation

Not all sources deserve the same trust. A statistic in a slick report may come from a tiny self-selected survey; a news story may misread a study; a website that looks official may be a lobby group; an AI tool may cite a page that does not say what it claims. Research by the Stanford History Education Group found that professional fact-checkers judged online sources more accurately than other experienced readers because they read laterally: they left the page early to see what other sources said about it. [Research, Wineburg and McGrew, Lateral Reading, 2017] Mike Caulfield's SIFT method turns that into four quick moves: Stop, Investigate the source, Find better coverage, Trace claims to the original. [Reference, Caulfield, SIFT]

This skill evaluates sources before relying on them: quick SIFT checks, deeper assessment of authority, method, evidence, date, purpose and bias, special checks for statistics, studies, reports and AI-generated content, and a clear confidence label for each source.

Operating principles

  1. Read laterally. Check what independent sources say about the source before trusting it.
  2. Trace to the original. Claims about studies and statistics must be checked at their origin.
  3. Check method, not just reputation. Even reputable sources publish weak analyses.
  4. Consider purpose and incentives. Who funded it, and what do they want readers to believe?
  5. Date matters. Old data may no longer be true.
  6. Label confidence. Every source used gets high, medium or low confidence with a reason.

SIFT in practice

MoveWhat to doTime
StopNotice your reaction; ask whether you know this sourceSeconds
Investigate the sourceSearch the publisher or author in a new tab; look for independent descriptions1 to 2 minutes
Find better coverageLook for reputable coverage of the same claim2 to 5 minutes
Trace claimsFollow quotes, statistics and studies to the original5 to 15 minutes

Deeper assessment

CriterionQuestions
AuthorityWho wrote or published it? What expertise and track record?
MethodHow was the data collected? Sample size, selection, definitions, time period?
EvidenceAre claims supported by data or citations? Can they be checked?
CurrencyWhen was it published and when was the data collected? Has anything changed since?
PurposeInform, sell, persuade, lobby or entertain? Who funded it?
ConsistencyDoes it agree with other reliable sources? If not, why?
TransparencyAre methods, data and conflicts of interest disclosed?

Checks for specific source types

SourceRed flagsChecks
StatisticsNo method, round numbers, no dateFind the original; check definition and sample
SurveysSmall or self-selected sample, leading questionsSample size, who was asked, exact wording
Academic studiesNot peer-reviewed, small samples, one study onlyJournal quality, replication, systematic reviews
Industry reportsVendor-sponsored conclusionsSponsor, method appendix, independent confirmation
NewsSingle anonymous source, headline beyond the storyRead the full piece; find the primary source
WebsitesUnclear ownership, no contact detailsLook up the organisation laterally
AI outputsConfident claims, citations that do not matchOpen every cited source; verify key facts
Social mediaScreenshots without links, viral claimsFind the original post and context

When to use, and when not to

Use it whenever research supports a decision, report, article or public claim, especially for statistics, health, finance, legal or political topics and AI-generated summaries.

Don't spend deep evaluation time on trivial, low-stakes facts; a quick SIFT is enough.

Inputs

  • The source or claim.
  • What it will be used for and how important it is.
  • Other sources already found.

Workflow

Step 1. Decide how much checking is needed

High-stakes claims (money, health, law, public statements) get deeper checks.

Step 2. Run SIFT

The four moves above, in a few minutes.

Step 3. Assess method and evidence

For data and studies, read the methodology: sample, definitions, period, analysis.

Step 4. Look for conflicts of interest

Funding, authorship, sponsorship, affiliations.

Step 5. Compare with other sources

Agreement among independent, reliable sources raises confidence; conflicts need explanation.

Step 6. Label confidence

High, medium or low with a one-line reason.

Step 7. Record

In the research log: source, assessment, confidence, how it will be used.

Confidence labels

LabelTypical profile
HighPrimary or highly reputable source, transparent method, recent, confirmed by others
MediumReputable but secondary, some method gaps, or older data
LowUnclear method, interested party, unconfirmed, or old

Reading a study quickly

  1. What question did it ask, and who was studied (sample size, population, place)?
  2. What did it measure, and how?
  3. Was it an experiment, an observational study or a survey?
  4. How big is the effect, not just whether it is significant?
  5. What limitations do the authors state?
  6. Has it been replicated or included in a systematic review? A single observational study rarely proves cause and effect; systematic reviews and meta-analyses usually give a more reliable picture.

Evidence hierarchy (rough guide)

StrongerWeaker
Systematic reviews and meta-analysesSingle studies
Randomised experimentsObservational studies
Large, representative samplesSmall or self-selected samples
Transparent methods and dataUndisclosed methods
Independent fundingInterested sponsors
The hierarchy guides judgement but does not replace it: a well-designed observational study can be more useful than a poor experiment.

Using AI to help evaluate

An AI assistant can summarise a source's methodology, look up the publisher and list potential conflicts of interest, but the final judgement needs a person to open the original and read the key sections, especially for high-stakes claims.

Edge cases and judgement calls

SituationWhat to do
Only one source existsUse with a low or medium label; say it is unconfirmed
Expert disagreementReport main positions and the evidence for each
Source reputable but study weakJudge the study, not the brand
Sponsored researchUse cautiously; look for independent confirmation
Retracted or corrected articlesCheck for retractions and corrections before citing
PreprintsLabel as not yet peer-reviewed

Handoffs to related skills

  • Finding sources: desk-research
  • Checking specific claims: fact-checking
  • AI research outputs: ai-assisted-research
  • Academic studies: literature-review
  • Synthesis: research-synthesis

Worked example

A team planned to quote "70% of buyers research online before contacting sales" in a pitch. A quick SIFT showed the figure appeared on dozens of marketing blogs; tracing it led to a vendor report several years old with no published method. Better coverage found more recent independent surveys with disclosed samples showing a related but different finding. The team replaced the claim with the newer source, stated the sample and year, and labelled confidence medium.

Common mistakes

  • Judging a page by how professional it looks.
  • Citing a statistic from an article instead of the original.
  • Ignoring dates and methods.
  • Trusting AI citations without opening them.
  • Treating one study as settled evidence.

Output format

1. Source and claim
2. SIFT findings
3. Method and evidence assessment
4. Conflicts of interest
5. Comparison with other sources
6. Confidence label and reason

Self-check before you answer

  • Was the source checked laterally?
  • Were key claims traced to their original?
  • Were method, date and purpose considered?
  • Does each source have a confidence label with a reason?

Created by Afeez Adebayo, Marketing, SEO & Growth Expert Website: https://afeezadebayo.com LinkedIn: https://www.linkedin.com/in/afeez-adebayo/ X: https://x.com/za_icon Source: https://afeezadebayo.com/skills-md/productivity/research-learning/source-evaluation/

Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). You may copy, share and adapt this skill, including for commercial use, as long as you credit Afeez Adebayo and link to the source URL above. Keep this notice in any copy or adapted version.

slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 193kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 193kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 193kUpdated Claude CodeCodexVerifiedInstall