Assessing Reliability

There is health information all around us.  Children, in particular, are vulnerable to inaccurate or misleading health information.  This is critical, as the information we utilize impacts the decisions and health behaviors that influence our level of health and wellness.  As adults, we need to know how to assess health information for reliability so that we can guide children in healthy ways.  We can teach young children about the various sources of health information they will encounter.

What does Reliable mean?  Reliable is an adjective that means capable of being relied on or dependable. It also means, yielding the same or comparable results in different clinical experiments.

Potential hazards of not using reliable, evidence-based resources include financial loss, health risk, injury, disability or death. 

Potential Issues with Reliability:

Ask yourself the following questions to help determine reliability.

1. Does the headline/article/source contain enough information?
2. Is the information or author biased?
3. Who is the author?
4. Is the information subject to interpretation?
5. Has the information been distorted?
6. Has the information been replicated? Can you verify the info in more than one source?
7. Does the information contain proper references?
8. How current is the information?

There is a lot of unreliable health information in many places.  Let’s talk about people as sources of health information.  Do you get your health information from friends and family members?  Although it is most convenient to obtain health advice from people who are close to us, most friends and relatives are not the best sources of health information…………..unless they utilize evidence based research information in a skeptical way!

Assessing Publications and Posts – Which of the following would be most or least reliable and why?

• Book
• Newspaper
• Magazine
• Health-Related Blog
• Research Journal (Scholarly Journal)
• Social Media Post

Let’s talk about books. Anyone can write a book. If someone writes a book about their life experiences related to health, it may be very interesting, however, it is only one person’s experience.  We can’t gain reliable health information by looking only at the life or perspective of one individual person.  If it’s a book based on one person’s experience or their opinions, then it’s not reliable. Reliable health information involves looking at many, many individuals and then using the research process to control for factors that may have caused certain results by chance.  Some people do write books that are based on solid research and provide good references, but this would be the minority of books.

Newspaper and magazine articles related to health will generally have significant issues with reliability.  They are usually written by someone who is taking the information from another source.  This makes it subject to interpretation.  If you’ve ever played the telephone game, you know that messages can get altered when they go through other sources.  If the writer is taking the article directly from an original research article, then the interpretation problem probably isn’t as big as it would be if they had taken it from another author who had already interpreted the study.  News media, in particular, is known for wanting to grab attention and therefore will put out one-line attention grabbing statements that may be based on only one study. Also, critical information is often omitted, and frequently the original source is not provided for follow up.  This also happens frequently with radio and TV news bits.  Some people think they need to change a health behavior based on a one-sentence news flash.  In reality, it may be information that defies the results of hundreds of other studies.  Individuals listening to the news flash don’t realize this, and don’t realize that for health information to be reliable, you want to see it replicated, or repeated, in a number of studies.

Sometimes magazine or newspaper articles are written or sponsored by a for-profit entity and will be biased toward use of a product.  It is very important to look at who the author is or who the article is funded by. We should always be on the lookout for bias in advertising with every source of health information.  Many times, the information used as a basis for sales is not reliable.

There are many social media posts about health that are not reliable information. Some are based on research, but individuals rarely check before sharing and having the information go viral.  There are also numerous blogs on the internet.  Do they provide reliable information?  Generally, no.  Blogs usually involve opinion without research backing.  Could there be a reliable researcher who is providing research based information through a blog?  Yes, but that would be unlikely.

Research in a peer-reviewed journal is going to be one of the best sources of reliable health information that you can find.  When original research is published in a research journal, it is first hand and a primary source.  It is the raw data provided by the researchers. Therefore, the issue of interpretation, where another author may have interpreted something in a different way, is reduced.  Review articles in peer-reviewed journals are a secondary source, however, the researcher(s) compiling the review are experts in the field and their work is critically scrutinized by other experts.  (Peer-reviewed means that it has been critically reviewed by other professionals in the field. Peer-reviewed journal articles are not published unless they comply with rigorous standards, from the review of existing literature to the analysis of data and listing of possible limitations.)  Also, any funders or conflicts of interest must be listed so the reader knows of potential bias.

Be sure to note that just because a publication has the word “journal” in it does not mean that it is a research journal.  For example, you may have heard of “The Wall Street Journal” or “Ladies Home Journal”. These are not research journals and do not contain actual research studies.  “The New England Journal of Medicine”, however, is an example of a research journal that contains actual research studies.  If you are questioning whether or not your source is a research journal, look for the characteristics of a research journal study article within.

How do I know if an article is a study from a Scholarly Journal/Research Journal?

You will usually find most all of the following elements in such an article, as well as a lengthy list of references:

• Abstract
• Review of Literature
• Methods
• Data Analysis
• Results/Discussion
• Limitations
• Reference List (Usually Long!)

If you have not spent time reading research journal articles, at first, the research articles may be difficult to read through.  Key point – Do not expect to understand everything within the article, especially when it comes to the data analysis. It is very helpful to start by reading the abstract, as it is a simplified summary of the research. As you become more familiar with these types of articles, you will feel more comfortable.

What about the INTERNET? 

There is a lot of incorrect health information on the internet. The internet is full of information.  We even have to watch out for “fake news”………..”news” that is posted with the intention of deception in order to sway the reader’s viewpoint and/or wallet.

Headlines, whether they are on the internet or from another source, can be very misleading.  It’s all about attention-grabbing and money making!

True or False? _____ If the first five items that come up as a result of a Google search say the same thing, it must be true.

False-  Unfortunately, people may think this is true, but search engines are using a system of algorithms that sort and rank webpages, some of which contain inaccurate and/or misleading information. These websites can appear to be reliable, but unfortunately those who are trying to deceive are very proficient at making false information appear true. Unfortunately, some individuals have carried out harmful behaviors as a result of believing false internet information. Luckily, we have tools we can use to evaluate information we find on the internet.

As previously discussed, there is a lot of health information on the web and the majority of what people see is not reliable.  Much of the information is interpretation and/or opinion.  Typically, .edu and .gov sites are going to be more reliable than .coms. The Center for Disease Control (CDC) site can be very useful for reliable data regarding a number of US health topics. www.cdc.gov  Check the World Health Organization (WHO) site for global information www.who.int/en. The .org sites can go either way.  Just because an organization has non-profit status, does not mean that their sources are sound.  There are some exceptions, of course, but we need to be cautious. The .edu and .gov organizations are much more reputable overall.  The institutions behind them tend to have familiarity with reliable research, the peer review process, have less bias, will not be involved with making a profit, and work with larger sample sizes.

Look for peer-reviewed journal research.  DO NOT just Google or do a regular search, as it is unlikely you will be able to access original research journal articles.  Colleges have subscriptions to a number of research journals, which can be accessed by students.  Peer-reviewed studies from scholarly journals can be accessed via the internet.

Why is Research so Important?

If we make our decisions based on only our individual experience or even a small group of people’s experience, that would be extremely problematic.  The experience of one person or a small group cannot be generalized or applied to other people, as the probability is low that the same outcome would be the result.  If research is reliable and done well, it provides us with a much more educated view than we would otherwise have.

For example, if we are making a decision regarding dietary intake, taking a medication or whether or not to start smoking, it is very helpful for us to look at reliable research.  We may have a close friend or relative who has had great success with a diet with no apparent negative effects.  If we rely on individual experience, that may influence us to participate in an unhealthy behavior.  If we look at some research, however, we will likely see a very different picture regarding the probability of having health issues.  With the power of studying large numbers of people, we have much better information. If we see that out of thousands of people studied a large or “significant” number had healthy outcomes, that would be good evidence to consider.

How Does Research Work?

Research is done in various ways. In many cases, large groups of individuals are compared. The more individuals that are involved in the study, the stronger the outcomes, as that decreases the probability that the results were coincidental.  Researchers must do their best to control for “confounding factors”.  This means that if the impact of physical exercise on ADHD were being studied and the children in the group with the physical exercise was showing a decrease in ADHD symptoms, researchers would need to make sure that other factors, such as diet or medications were not influencing that difference. Repeating the same study on additional individuals helps validate the results as well.