The Statistical Illusions in Cancer Research

A 4-Part Reality Check on Keto and Cancer

Part 2/4: "Are These Studies Designed to Trick You? How to Spot Statistical Nonsense"

Numbers don’t lie—but the way they’re presented can be misleading. In cancer research, and especially in dietary studies, statistics are often used in ways that make results seem more impressive than they actually are. A small benefit can be exaggerated with clever framing, and inconvenient results can be hidden behind complex wording.

This is particularly true in ketogenic diet (KD) research, where studies often report metabolic changes as if they are clinical improvements. It’s one thing to show that blood sugar is lower, but does that actually mean patients live longer? Many studies use percentages, relative risk reductions, and selective reporting to suggest strong benefits, when in reality, the impact might be minor—or nonexistent.

This is where it becomes crucial to ask: Are these studies actually proving something meaningful, or just dressing up weak results with impressive-sounding numbers?

Expanding on the Key Questions and Their Relevance to the Study

Was the study large enough, and was it designed to be reliable? (https://www.mdpi.com/2072-6643/14/18/3851 - Ketogenic Diet in the Treatment of Gliomas and Glioblastomas)

Small studies are inherently weak. The smaller the sample size, the greater the chance that results are due to random variation rather than a real effect. A study with just 6 or 12 or 20 patients, for example, tells us almost nothing—it’s too easy for a few extreme cases to skew the outcome.

How It Relates to the Study: The "Ketogenic Diet in Gliomas" study relies on small sample sizes, sometimes fewer than 20 patients, making it difficult to draw any real conclusions. Without a large enough group, it’s impossible to determine whether the effects seen are due to the diet or just natural differences among individuals.

What to Look for in Studies:

• How many people were included, and was the study randomized?
• Was there a control group, or were results compared to general expectations?
• Have similar results been replicated in other, larger studies?

Are the results clinically relevant, or just statistically significant?

Statistical significance means that an effect is unlikely to be due to random chance—but that doesn’t mean it’s important in the real world. A treatment might increase survival by only a few days, yet still be labeled as "statistically significant." But if a cancer patient only gains a few extra days weeks or months, with therapy versus no therapy and if therapy involves endless hospitalization, or deprivation of pleasures and the loss of the proximity of loved ones - is it still meaningful?

How It Relates to the Study: The "Ketogenic Diet in Gliomas" study highlights changes in glucose metabolism but does not demonstrate that these changes lead to improved survival or quality of life. The study suggests potential benefits in tumor metabolism, but without proving that these metabolic shifts actually translate into longer survival.

What to Look for in Studies:

• Are the reported changes significant in terms of patient outcomes, or just in lab results?
• Do survival times or remission rates improve, or do tumors simply slow their growth temporarily?
• Are patient quality of life measures included, or is the study only focused on biochemical markers?

Did the study use relative percentages to make effects seem bigger than they are?

One of the most common statistical tricks is using relative risk reductions instead of absolute risk reductions. For example, if a study finds that the risk of cancer recurrence drops from 2% to 1%, they might report this as a "50% reduction"—which sounds far more impressive than saying "a 1% absolute reduction."

How It Relates to the Study: The "Ketogenic Diet in Gliomas" study, like many in the field, reports relative changes in metabolic markers but does not provide clear data on absolute improvements in survival. This makes it difficult to tell whether the observed effects are clinically meaningful or just statistically convenient.

What to Look for in Studies:

• Are improvements shown in actual survival time, or just metabolic markers?
• Do they report absolute numbers, or only percentages?
• Does the study exaggerate results with dramatic phrasing without providing clear data?

Clinical Generalizability: Can This Study Apply to Me?

A key concept in evaluating studies is clinical generalizability—the ability to apply the study’s findings to a broader population, including individuals with different health conditions, genetic backgrounds, and lifestyles.

People often ask, "How many humans were included?"—which is important, but the real question should be:

"Were the humans in this study similar to me?"

A study done on healthy young athletes does not necessarily apply to older cancer patients undergoing chemotherapy. Likewise, a study on patients with early-stage brain tumors does not automatically tell us whether the ketogenic diet will help patients with aggressive, late-stage glioblastomas.

Without clinical generalizability, even a well-conducted study with a large sample size may not be relevant to a specific individual’s situation.

Meta-Analysis: "Effects of Ketogenic Diets on Cancer Outcomes: Are the Benefits Overstated?"

This meta-analysis examines multiple ketogenic diet studies and finds that while KD may influence metabolic markers, there is no strong evidence that it consistently improves survival or quality of life in cancer patients. Many studies use small sample sizes, relative risk reductions, and short timeframes, making their findings difficult to apply to real-world cancer treatment.

Link: https://onlinelibrary.wiley.com/doi/full/10.1111/nbu.12693

Final Takeaway:

Many ketogenic diet studies appear impressive at first glance, but a closer look often reveals small sample sizes, selective reporting, and misleading statistics. While metabolic changes are interesting, they do not automatically translate into improved survival or meaningful clinical benefits.

Before accepting a study’s conclusions, ask the right questions:

• Was the study large enough, and was it designed to be reliable?
• Are the results clinically relevant, or just statistically significant?
• Did the study use relative percentages to make effects seem bigger than they are?

Without clear, well-designed studies showing long-term survival benefits, the ketogenic diet remains an experimental approach—not a proven cancer therapy.