Evaluating Research Resources

Evaluating Research Resources

As you gather sources, you will need to examine them with a critical eye. Smart researchers continually ask themselves two questions: “Is this source relevant to my purpose?” and “Is this source reliable?” The first question will help you avoid wasting valuable time reading sources that stray too far from your specific topic and research questions. The second question will help you find accurate, trustworthy sources.

Determining Whether a Source Is Relevant

At this point in your research process, you may have identified dozens of potential sources. It is easy for writers to get so caught up in checking out books and printing out articles that they forget to ask themselves how they will use these resources in their research. Now is a good time to get a little ruthless. Reading and taking notes takes time and energy, so you will want to focus on the most relevant sources.

To weed through your stack of books and articles, skim their contents. Read quickly with your research questions and subtopics in mind. Table 11.3 “Tips for Skimming Books and Articles” explains how to skim to get a quick sense of what topics are covered. If a book or article is not especially relevant, put it aside. You can always come back to it later if you need to.

Table 11.3 Tips for Skimming Books and Articles

Tips for Skimming Books Tips for Skimming Articles
  1. Read the dust jacket and table of contents for a broad overview of the topics covered.
  2. Use the index to locate more specific topics and see how thoroughly they are covered.
  3. Flip through the book and look for subtitles or key terms that correspond to your research.
  1. Skim the introduction and conclusion for summary material.
  2. Skim through subheadings and text features such as sidebars.
  3. Look for keywords related to your topic.
  4. Journal articles often begin with an abstract or summary of the contents. Read it to determine the article’s relevance to your research.

Determining Whether a Source Is Reliable

All information sources are not created equal. Sources can vary greatly in terms of how carefully they are researched, written, edited, and reviewed for accuracy. Common sense will help you identify obviously questionable sources, such as tabloids that feature tales of alien abductions, or personal websites with glaring typos. Sometimes, however, a source’s reliability—or lack of it—is not so obvious. For more information about source reliability, see Chapter 12 “Writing a Research Paper”.

To evaluate your research sources, you will use critical thinking skills consciously and deliberately. You will consider criteria such as the type of source, its intended purpose and audience, the author’s (or authors’) qualifications, the publication’s reputation, any indications of bias or hidden agendas, how current the source is, and the overall quality of the writing, thinking, and design.

Evaluating Types of Sources

The different types of sources you will consult are written for distinct purposes and with different audiences in mind. This accounts for other differences, such as the following:

  • How thoroughly the writers cover a given topic
  • How carefully the writers research and document facts
  • How editors review the work
  • What biases or agendas affect the content

A journal article written for an academic audience for the purpose of expanding scholarship in a given field will take an approach quite different from a magazine feature written to inform a general audience. Textbooks, hard news articles, and websites approach a subject from different angles as well. To some extent, the type of source provides clues about its overall depth and reliability.

High-Quality Sources

These sources provide the most in-depth information. They are researched and written by subject matter experts and are carefully reviewed.

  • Scholarly books and articles in scholarly journals
  • Trade books and magazines geared toward an educated general audience, such as Smithsonian Magazine or Nature
  • Government documents, such as books, reports, and web pages
  • Documents posted online by reputable organizations, such as universities and research institutes
  • Textbooks and reference books, which are usually reliable but may not cover a topic in great depth

Varied-Quality Sources

These sources are often useful. However, they do not cover subjects in as much depth as high-quality sources, and they are not always rigorously researched and reviewed. Some, such as popular magazine articles or company brochures, may be written to market a product or a cause. Use them with caution.

  • News stories and feature articles (print or online) from reputable newspapers, magazines, or organizations, such as Newsweek or the Public Broadcasting Service
  • Popular magazine articles, which may or may not be carefully researched and fact checked
  • Documents published by businesses and nonprofit organizations

Questionable Sources

These sources should be avoided. They are often written primarily to attract a large readership or present the author’s opinions and are not subject to careful review.

  • Loosely regulated or unregulated media content, such as Internet discussion boards, blogs, free online encyclopedias, talk radio shows, television news shows with obvious political biases, personal websites, and chat rooms

Tip

Free online encyclopedias and wikis may seem like a great source of information. They usually appear among the first few results of a web search. They cover thousands of topics, and many articles use an informal, straightforward writing style. Unfortunately, these sites have no control system for researching, writing, and reviewing articles. Instead, they rely on a community of users to police themselves. At best, these sites can be a starting point for finding other, more trustworthy sources. Never use them as final sources.

Evaluating Credibility and Reputability

Even when you are using a type of source that is generally reliable, you will still need to evaluate the author’s credibility and the publication itself on an individual basis. To examine the author’s credibility—that is, how much you can believe of what the author has to say—examine his or her credentials. What career experience or academic study shows that the author has the expertise to write about this topic?

Keep in mind that expertise in one field is no guarantee of expertise in another, unrelated area. For instance, an author may have an advanced degree in physiology, but this credential is not a valid qualification for writing about psychology. Check credentials carefully.

Just as important as the author’s credibility is the publication’s overall reputability. Reputability refers to a source’s standing and reputation as a respectable, reliable source of information. An established and well-known newspaper, such as the New York Times or the Wall Street Journal, is more reputable than a college newspaper put out by comparatively inexperienced students. A website that is maintained by a well-known, respected organization and regularly updated is more reputable than one created by an unknown author or group.

If you are using articles from scholarly journals, you can check databases that keep count of how many times each article has been cited in other articles. This can be a rough indication of the article’s quality or, at the very least, of its influence and reputation among other scholars.

Checking for Biases and Hidden Agendas

Whenever you consult a source, always think carefully about the author’s or authors’ purpose in presenting the information. Few sources present facts completely objectively. In some cases, the source’s content and tone are significantly influenced by biases or hidden agendas.

Bias refers to favoritism or prejudice toward a particular person or group. For instance, an author may be biased against a certain political party and present information in a way that subtly—or not so subtly—makes that organization look bad. Bias can lead an author to present facts selectively, edit quotations to misrepresent someone’s words, and distort information.

Hidden agendas are goals that are not immediately obvious but influence how an author presents the facts. For instance, an article about the role of beef in a healthy diet would be questionable if it were written by a representative of the beef industry—or by the president of an animal-rights organization. In both cases, the author would likely have a hidden agenda.

As Jorge conducted his research, he read several research studies in which scientists found significant benefits to following a low-carbohydrate diet. He also noticed that many studies were sponsored by a foundation associated with the author of a popular series of low-carbohydrate diet books. Jorge read these studies with a critical eye, knowing that a hidden agenda might be shaping the researchers’ conclusions.

Using Current Sources

Be sure to seek out sources that are current, or up to date. Depending on the topic, sources may become outdated relatively soon after publication, or they may remain useful for years. For instance, online social networking sites have evolved rapidly over the past few years. An article published in 2002 about this topic will not provide current information. On the other hand, a research paper on elementary education practices might refer to studies published decades ago by influential child psychologists.

When using websites for research, check to see when the site was last updated. Many sites publish this information on the homepage, and some, such as news sites, are updated daily or weekly. Many nonfunctioning links are a sign that a website is not regularly updated. Do not be afraid to ask your professor for suggestions if you find that many of your most relevant sources are not especially reliable—or that the most reliable sources are not relevant.

Evaluating Overall Quality by Asking Questions

When you evaluate a source, you will consider the criteria previously discussed as well as your overall impressions of its quality. Read carefully, and notice how well the author presents and supports his or her statements. Stay actively engaged—do not simply accept an author’s words as truth. Ask questions to determine each source’s value. Checklist 11.1 lists ten questions to ask yourself as a critical reader.

Checklist 11.1

Source Evaluation

  • Is the type of source appropriate for my purpose? Is it a high-quality source or one that needs to be looked at more critically?
  • Can I establish that the author is credible and the publication is reputable?
  • Does the author support ideas with specific facts and details that are carefully documented? Is the source of the author’s information clear? (When you use secondary sources, look for sources that are not too removed from primary research.)
  • Does the source include any factual errors or instances of faulty logic?
  • Does the author leave out any information that I would expect to see in a discussion of this topic?
  • Do the author’s conclusions logically follow from the evidence that is presented? Can I see how the author got from one point to another?
  • Is the writing clear and organized, and is it free from errors, clichés, and empty buzzwords? Is the tone objective, balanced, and reasonable? (Be on the lookout for extreme, emotionally charged language.)
  • Are there any obvious biases or agendas? Based on what I know about the author, are there likely to be any hidden agendas?
  • Are graphics informative, useful, and easy to understand? Are websites organized, easy to navigate, and free of clutter like flashing ads and unnecessary sound effects?
  • Is the source contradicted by information found in other sources? (If so, it is possible that your sources are presenting similar information but taking different perspectives, which requires you to think carefully about which sources you find more convincing and why. Be suspicious, however, of any source that presents facts that you cannot confirm elsewhere.)

Writing at Work

The critical thinking skills you use to evaluate research sources as a student are equally valuable when you conduct research on the job. If you follow certain periodicals or websites, you have probably identified publications that consistently provide reliable information. Reading blogs and online discussion groups is a great way to identify new trends and hot topics in a particular field, but these sources should not be used for substantial research.