Evaluating

A book open in front of a keyboard.You have articles and books in hand. Now it’s time to evaluate their quality.

Let’s say you’re new to campus and you’ve decided to join an organization so you can meet people. How do you evaluate which organization fits you best? You gather information about several organizations:

  • Club websites list social events and service opportunities
  • Your advisor identifies activities that would look good on your resume
  • Club officers invite you to sit in on a meeting and ask questions

You evaluate your choices in light of your own interests and goals. You find the organization that fits you best.

Doing research is similar. You’ll find plenty of sources of information, but some will fit your assignment better than others.

Finding information is not the end of research. You want information that supports the point you’re trying to make.  Some sources can be outdated, biased, or just plain wrong, and using that information makes it a lot more difficult for you to present a convincing argument.

Taking the time to critically evaluate information as you find it will help you to avoid wrong turns in the research process.

How Can You Know If Information Is Appropriate for Your Research?

Consider the source. You can apply the CAARP test:

  • Currency
  • Authority
  • Accuracy
  • Relevance
  • Purpose

Clue 1: Currency

Currency is important because information can quickly become obsolete. Supporting your thesis statement with facts that have been superseded by new research or recent events weakens your argument. Of course, not all assignments require the most current information; older materials can provide an historical or comprehensive understanding of your topic.

How Do You Know If the Timeliness of Your Information Is Appropriate?

  • When was the information published or last updated?
  • Have newer articles been published on your topic?
  • Are links or references to other sources up to date?
  • Is your topic in an area that changes rapidly, like technology or popular culture?

Clue 2: Authority

A gavel lying on a table with documents underneath itAuthority is important in judging the credibility of the author’s assertions. In a trial regarding DNA evidence, a jury gives far more authority to what a genetics specialist has to say compared to someone off the street.

How Do You Know If an Author Is an Authority on Your Topic?

  • What are the author’s credentials?
  • Is the author affiliated with an educational institution or prominent organization?
  • Can you find information about the author from reference books or the Internet?
  • Do other books or articles cite the author?

Clue 3: Accuracy

How Do You Know If Your Source Is Accurate?

  • Are there statements you know to be false?
  • Are there errors in spelling, punctuation, or grammar?
  • Was the information reviewed by editors or subject experts before it was published?
  • What citations or references support the author’s claims?
  • What do other people have to say about the topic?

Clue 4: Relevance

Relevance is important because you are expected to support your ideas with pertinent information. A source detailing Einstein’s marriage and family life would not be germane to his theories in physics.

How Do You Know If Your Source Is Relevant?

  • Does the information answer your research question?
  • Does the information meet the stated requirements of the assignment?
  • Is the information too technical or too simplified for you to use?
  • Does the source add something new to your knowledge of your topic?

Clue 5: Purpose

Purpose is important because books, articles, and Web pages exist to educate, entertain, or sell a product or point of view. Some sources may be frivolous or commercial in nature, providing inadequate, false, or biased information. Other sources are more ambiguous concerning their partiality. Varied points of view can be valid, as long as they are based upon good reasoning and careful use of evidence.

How Do You Determine the Purpose of Your Source?

  • Why did the author or publisher make this information available?
  • Is there an obvious bias or prejudice?
  • Are alternative points of view presented?
  • Does the author omit important facts or data that might disprove a claim?
  • Does the author use strong or emotional language?

Apply the CAARP Test

Questions

  1. Your research question is “What are the social benefits and liabilities for tribes developing casinos on Indian reservations?” Select the source that would likely be more current for your topic.
    1. A book published in 1998 about gaming on American Indian lands
    2. Pechanga.net, Indian gaming news on the internet
  2. Your research question is “What are the social benefits and liabilities for tribes developing casinos on Indian reservations?” Which database would be more likely to point you to articles relevant to your topic?
    1. Sociological abstracts
    2. Biological abstracts
  3. Your research question is “What are the social benefits and liabilities for tribes developing casinos on Indian reservations?” Which source would likely be more authoritative on your topic?
    1. A peer-reviewed article published in UNLV Gaming Research & Review
    2. An article published in Sports Illustrated
  4. Your research question is “What are the social benefits and liabilities for tribes developing casinos on Indian reservations?” Which source is more likely to be accurate?
    1. Employment statistics from a newsletter published by a grassroots organization opposed to gambling
    2. Employment statistics from the U.S. Department of Labor website
  5. Your research question is “What are the social benefits and liabilities for tribes developing casinos on Indian reservations?” Which source would more likely suit the purpose of your report?
    1. Pechanga.net, Indian gaming news on the Internet
    2. A peer-reviewed article published in UNLV Gaming Research & Review

Answers

  1. Internet resources like Pechanga.net are generally more current than books; a website should indicate the date it was last updated. A book published in 1998 will not be as current as most websites, although it may provide good background information.
  2. Sociological Abstracts indexes journals that focus on social consequences of human behavior, which is more relevant to your topic. Biological Abstracts indexes journals in the life sciences, which would not likely cover social aspects of your topic.
  3. An article in a peer-reviewed journal is more likely to have been written by an expert and will have been evaluated by other experts before it is published. Writers for Sports Illustrated are not likely to have expertise in social issues surrounding gambling.
  4. The U.S. government is considered a reliable source for statistics. You can identify a government web site by the “.gov” at the end of the URL. A grass roots organization opposed to gambling may use legitimate statistics but exclude those that don’t support a specific agenda.
  5. The title UNLV Gaming Research & Review is a clue that the journal is likely to publish studies related to issues surrounding gaming. Pechanga.net would probably give you a handle on what’s going on in the world of Indian gaming, but it would not provide scholarly research.

An article we found in UNLV Gaming Research & Review meets all our criteria:

A screenshot of an article abstract “A Special Look at Indian Gaming.” Its currency (2004), accuracy (published by UNLV Gaming Research and Review Journal), purpose (This article probes into the Native American gaming), relevancy (social implications), and authority (author is a professor at the University of Houston Hotel College) are all indicated.

The criteria for evaluating sources are just guidelines to help you think critically about the information you find. Depending upon how you are using the information, some criteria may be more useful than others:

  • For finding just a few facts, accuracy is more important than relevance.
  • Reading a book that’s out of date can still add to your understanding of a subject, especially if the author is often cited as an expert or if your topic is in a field that doesn’t change quickly, like the humanities.
  • A biased source can contribute relevant facts and data. The National Rifle Association compiles plenty of data to promote its agenda. It’s your job to support or refute the arguments of impartial authors.

In each of these situations, relying on more than one source for information will help you decide what is or isn’t accurate and whether something is relevant to your assignment. Think about how the new information fits in with what you already know and how you want to present your argument. Making notes about and organizing information helps you keep track of what might or might not be useful to your project.

Let’s Wrap Up

Think about information resources as “evidence.”
Viewing information as a tool to prove a point or support an argument is a useful starting point for evaluation.

Don’t assume that one format of information is better than others.
All kinds of information should be evaluated carefully, including books, articles and web sites.

Evaluation is an art, not a science.
There is no “one size fits all” set of guidelines for this important activity.

Test Yourself!

  1. Which of the following questions should you ask yourself when considering the relevance of a potential source?
    1. Was the source published in the past five years?
    2. Does the source add something new to your knowledge of the topic?
    3. Is the source free of bias?
    4. Is the author an established expert in the field?
  2. In which type of research would currency be most important as you consider which sources to use?
    1. Health Sciences/Medicine
    2. Literature
    3. History
    4. Philosophy
  3. Citing corroborating sources reinforces the ___________ of an article.
    1. accuracy
    2. currency
    3. relevance
    4. purpose
  4. In gathering data for your project about freedom of information conflicts in China, which resource will provide the most accurate and least biased information?
    1. Chinese newspapers
    2. American newspapers
    3. The article “Status of Media in China” from the Encyclopedia of International Media and Communication
    4. An interview with your uncle who immigrated to the US from Tibet.
  5. All of the following statements are true except one. Which statement is FALSE?
    1. All research assignments require the most current information possible.
    2. Internet resources are generally more current than books.
    3. A web site should indicate the date it was last updated.
    4. Reading a book that’s out of date can still add to your understanding of a subject.
  6. A grass roots organization opposed to gambling would be a good source for accurate statistics to support your research on the social benefits and liabilities for tribes developing casinos on Indian reservations.
    1. true
    2. false

Answers

  1. 1.2: A source is relevant when it adds something new to your knowledge of the topic. Relevance is important because you are expected to support your ideas with pertinent information.
  2. 2.1: When doing research in a field that changes quickly, like medicine or health, currency is especially important.
  3. 3.1: Citing corroborating sources is one way of reinforcing the accuracy of an article.
  4. 4.3
  5. 5.1
  6. false: A grass roots organization opposed to gambling would not be a good source for accurate statistics to support your research on the social benefits and liabilities for tribes developing casinos on Indian reservations. A grass roots organization opposed to gambling may use legitimate statistics but exclude those that don’t support a specific agenda.