Is It Fact or Fiction? How to Size Up a Health Care Study

Key Takeaways

• How a research study is designed and who takes part in it are very important.
• Peer review is not a fail-safe, and it may not always detect bias or even fraud.
• Researchers have a motive to find newsworthy results—and media outlets to hype them.

Wine is good for your heart—unless, that is, it hastens your demise.

CrossFit is the best way to get healthier—although, yoga might be better.

Eggs are a great source of protein—or maybe they’re dangerous cholesterol bombs.

If you’re like many Americans today, you encounter lots of news about what scientists (and others) think you should and shouldn’t do with regard to your body. And undoubtedly, improvements in medicine and research over the years have led to amazing findings and breakthroughs that can potentially help us make better decisions about our health.

But the fact is, medical research often contradicts itself—or at least creates confusion when the conclusion of one study (“Could your morning coffee be good for your eyes?”) seems to clash with another’s results (“Does drinking coffee damage your eyesight?”).

This type of “he-said-she said” health care news makes it tempting to throw up one’s hands and ignore the science altogether. But there’s a way to better understand health data so that it can help you live your best life: Learn how to vet a health care study so you can decide whether to pay attention or chuck it.

With that in mind, here’s advice on how to assess the never-ending stream of health care headlines—and, more important, the research behind them.

More important than ever

The need for due diligence on health care research and its claims is hardly new, of course. But in some ways, it’s become more vital than ever for consumers to vet the studies that can drive both the media headlines we read and the recommendations we get from our health care providers. These are some big reasons:

1. The aging population. By 2030, all the living baby boomers—more than 70 million people—will be at least 65 years old. That means a huge percentage of the population is at (or soon will be at) an age where they need to know whether the health information they’re getting is reliable.
2. Poor health. The prevalence of many chronic diseases has risen over time. Today, according to the CDC, six in 10 U.S. adults have a chronic disease—while 4 in 10 have two or more chronic diseases.
3. Social media information—and misinformation. Due to social media, health care studies (and articles/videos highlighting them) can be seen by more people than ever. While that helps the reliable studies gain more traction, it also means unreliable research and claims can spread rapidly and become widely accepted before there’s a chance to stop such misinformation.?

A tough road to navigate

In June 2022, the American Medical Association and the U.S. Food and Drug Administration launched a medical education program focused on dietary supplements designed to help physicians better understand what their patients are hearing about these over-the-counter products.

If even physicians need help separating fact from fiction, how are the rest of us supposed to have any clue?

Enter Dr. Stephen Soumerai, professor of population medicine at Harvard Medical School and Harvard Pilgrim Health Care Institute. He’s asked that question for years and is the lead author on a CDC research study that, well, studies research studies. His goal: Help consumers understand the strengths and weaknesses of different types of health care reports, and enable them to detect the kinds of study designs most likely to be accurate and trustworthy.

One of Soumerai’s biggest concerns: More evidence is showing that even the fact that a study is peer-reviewed and published in a medical journal doesn’t necessarily make it valid. Too many studies, he says, don’t show the cause-and-effect relationships they purport to. It starts with faulty research design, then snowballs as inaccurate conclusions are reported by media outlets that don’t understand the science.

What to ask and what to look for

So what are some important things to think about when deciding for yourself whether a given health study is reliable, well researched and truthful?

Here are some questions that Dr. Soumerai and others suggest we consider:

1. What is this study actually saying, and how was it developed?

The point of most health studies is to show how a food or drug (or an exercise or a new technology) can have a positive effect on a person’s health and wellness. But that equation is not always as obvious as one might assume. As Soumerai points out, the basic fact of whether certain interventions effect change that’s “different from what would have happened anyway (i.e., what happened to a control group)” is not always readily apparent and definable.

?Best advice: Look past the headlines and be sure you understand what a given health study is actually claiming. Example: A headline might imply that full-fat dairy is now considered to be good for you—but when you read the study, the actual claim is that it’s simply not as bad for you as margarine.

Beyond that, it’s important to look at a study’s methodology to get an idea of how the research was conducted. When research is poorly designed, researchers might apply “strenuous statistical machinations” to “adjust for” disparities between the study and control groups—which can lead to skewed results, says Soumerai. Those results usually don’t provide useful information for the casual consumer.

Always try to determine how a given health care study was conducted. By far, the best type of research design for proving cause and effect is a randomized control trial (RCT), which groups participants into separate cohorts that receive different treatments or diet regimens. Look for RCTs wherever possible.

Important: Because RCTs cost a lot of money and take a long time, many published studies are based instead on correlational research—which (by definition) cannot prove causation and are difficult to control for a wide range of variables other than what’s being studied. This is not to say correlational studies are worthless—just that they should be viewed with greater skepticism.

2. Who is providing the funding?

This might seem obvious, but it’s not always as apparent as you might think. Money talks, of course—and sometimes it may be subtly whispering through the research study you’re reading.

Action step: Look at the fine print. Is a given research study performed by impartial clinical researchers at a respected academic medical center? Was it funded by a nonpartisan think tank or public health agency? Or is it just a thinly veiled advertisement for a new “superfood” or prescription drug? Even a peer-reviewed, double-blind study from Stanford or Penn Medicine with a wide-ranging cohort of trial participants might not be 100 percent perfect. But it’s a whole lot more trustworthy than one sponsored by Wally’s Wacky Weight Loss Pills that polled a small group of paid participants.

Most reputable medical journals will include funding information. That said, media outlets often don’t note such details. And if that information isn’t readily apparent, you can look for other clues. Example: As Dr. Robert H. Shmerling, a physician and researcher with Harvard Medical School, has explained, “studies with positive findings are more likely to get published than those with negative (and often disappointing) results.”

That’s not to say that a health product company would falsify findings outright to make their product look good, of course. But—to Soumerai’s earlier point—it might affect how they approach the design of the study. If research is sponsored by a pharma firm, for instance, that company has “little incentive to perform high-quality, expensive studies if the results might hurt chances for FDA approval or future sales,” notes Shmerling. Instead, the studies could be crafted in a way that maximizes the chance for success. For instance, “rather than comparing a new drug against a competitor’s drug, it might be compared to a placebo or an old drug at a dose too low to be effective.”

3. Who else is benefiting?

Don’t let skepticism stop with the vitamin industry lobbyist or exercise equipment manufacturer that’s funding a study. As Shmerling notes, there are plenty of folks in the media and even academia who have a vested interest in seeing research published—however dubious it might be. For example, it’s not uncommon for newspaper or magazine editors to seek out studies with sensational results that confirm their readers’ views or hopes. Such stories get more web traffic (or sell more papers) and can boost ad sales. What’s more, even well-meaning, uncynical editorial leaders might not have the health care or science expertise to truly judge a given study with a critical eye.

When it comes to academics, “promotion, better jobs, higher pay, tenure and prestige all ride on the ability to publish,” notes Shmerling. “So, even if the research is preliminary, applies to few people, or is of uncertain impact, casting the research in the most favorable and dramatic light is standard operating procedure.”?

Of course, you likely won’t know if study authors or media shot-callers have hidden agendas. That’s why it’s so important to read multiple studies from multiple sources that don’t simply repeat the same findings from the same sources.

4. Who is being studied?

It’s also smart to learn about the people participating in a study. Much attention has been paid in recent years to non-representative samples of trial participants, representing only a small subsection—rather than a wide cross-section—of the human population.

Example: Having a cohort that’s all or mostly white suburbanites will skew a study’s findings. It’s important that research subjects of different races, genders and socioeconomic status be included in studies to help ensure a representative sampling is being studied.?

For most health research, it’s hugely important that a study group be as diverse as possible. And the bigger the sample size, the better. That’s why the National Institutes of Health’s landmark All of Us Research Program, for example, is going out of its way to build a broadly diverse cohort of 1 million or more Americans—of all shapes and sizes, from all walks of life—in pursuit of perhaps the most diverse health database in history.

A limited study of folks who look just like you, on the other hand, could be hampered by small sample size—or compromised by confirmation bias.

Conclusion

This information won’t guarantee that you’ll always zero in on only the most reputable medical studies and weed out the unreliable ones. But it can give you a foundation from which to assess the barrage of health care data and findings that come at us virtually every day. It can also potentially help you have better, more informed conversations with your medical providers—particularly when they make a suggestion or recommendation based on a study they’ve read.?

Ultimately, your health care decisions can have a huge impact on the quality of your life. Make sure you’re taking steps to understand the real facts behind those decisions.?

VFO Inner Circle Special Report?

By John J. Bowen Jr.

? Copyright 2023 by AES Nation, LLC. All rights reserved.

No part of this publication may be reproduced or retransmitted in any form or by any means, including but not limited to electronic, mechanical, photocopying, recording or any information storage retrieval system, without the prior written permission of the publisher. Unauthorized copying may subject violators to criminal penalties as well as liabilities for substantial monetary damages up to $100,000 per infringement, costs and attorneys’ fees.?

This publication should not be utilized as a substitute for professional advice in specific situations. If legal, medical, accounting, financial, consulting, coaching or other professional advice is required, the services of the appropriate professional should be sought. Neither the author nor the publisher may be held liable in any way for any interpretation or use of the information in this publication.

The author will make recommendations for solutions for you to explore that are not his own. Any recommendation is always based on the author’s research and experience. The information contained herein is accurate to the best of the publisher’s and author’s knowledge; however, the publisher and author can accept no responsibility for the accuracy or completeness of such information or for loss or damage caused by any use thereof.

Unless otherwise noted, the source for all data cited regarding financial advisors in this report is CEG Worldwide, LLC. The source for all data cited regarding business owners and other professionals is AES Nation, LLC.?

?

Securities offered through LPL Financial. Member FINRA / SIPC. Investment advisory services offered through NewEdge Advisors, LLC, a registered investment adviser. NewEdge Advisors, LLC and Congruent Wealth, LLC are separate entities from LPL Financial.