Designed to Fail: Why Short-Term Diet Trials Can’t Solve Chronic Disease

Originally published in STAT News on April 22

Imagine a clinical trial involving sedentary, overweight adults. One group continues their inactive lifestyle, while the other begins an intensive regimen of daily running, calisthenics, and sports. After just a week or two, participants in the exercise group would likely feel sore, fatigued, and possibly even show a temporary drop in performance. No one would seriously conclude from this short-term snapshot that exercise is harmful. It’s clear we’d need a longer trial to understand the benefits of physical activity.

Yet this same flawed thinking underpins many of the short-term diet trials being used in today’s major nutrition research efforts—including the high-profile, federally funded Nutrition for Precision Health initiative. In a recent article in The BMJ (British Medical Journal), we highlighted how such short trials are structurally incapable of answering the complex question they’re meant to solve: What kind of diet prevents chronic disease?

A Bold but Flawed Approach

The Nutrition for Precision Health program aims to harness artificial intelligence to determine the ideal diet for each individual. To do so, thousands of volunteers will be assigned to follow three distinct dietary patterns:

1. A conventionally healthy diet rich in vegetables, fruits, and whole grains.
2. A highly processed diet loaded with sugar, refined grains, and meats.
3. A low-carbohydrate, high-fat diet that strictly limits sugars and grains.

To ensure compliance, researchers are pulling out all the stops—providing fully prepared meals and, in some cases, housing up to 1,000 participants in inpatient facilities where every bite can be monitored. Advanced technologies will track everything from gut microbes to metabolism, using tools with intimidating names like metagenomics, metabolomics, and the Automatic Ingestion Monitor.

Despite the sophistication and expense, there’s a catch: each dietary phase lasts only two weeks.

Two Weeks Is Not Enough

Two weeks is woefully inadequate to assess the long-term effects of diet on chronic diseases such as obesity, diabetes, and heart disease. The evidence from prior studies confirms this.

Take, for example, an influential two-week trial comparing unprocessed and ultra-processed diets. Participants on the ultra-processed diet initially consumed about 600 more calories per day, leading to short-term weight gain. But the difference shrank by about 25 calories per day, suggesting the body was adjusting. Extend the study by just a few more weeks, and the calorie intake between the groups might equalize. A replication of the study showed the effect started fading after just one week.

Day-to-day eating behavior is influenced by minor variables like utensil size, plate color, temperature, and whether you’re eating alone or with others. But we intuitively know that these are short-lived effects. Nobody expects to achieve lasting weight loss simply by switching spoon sizes.

So, does ultra-processed food cause obesity? Maybe. But we won’t get the answer from two-week trials.

The Carry-Over Conundrum

Adding to the problem is the cross-over design used in many of these trials, including Nutrition for Precision Health. In this format, every participant tries each of the three diets, one after another, over a few months. The goal is to control for individual variability and let researchers compare how the same person responds to different diets.

Sounds smart in theory—but it’s statistically dangerous in practice.

Let’s go back to our exercise example. If one group starts with strenuous activity, they’ll be tired and sore. Then, during their sedentary phase, they’ll rest and recover—and potentially begin reaping delayed benefits of that initial exercise. Conversely, participants who begin sedentary might become even less fit, making the later exercise phase seem more taxing.

The result is a carry-over effect—residual impacts from one treatment that bleed into the next. In diet trials, carry-over effects are especially insidious because physiological adaptation to diet takes time. For example, transitioning to a low-carbohydrate diet often causes “keto flu”—temporary fatigue and brain fog. When people used to high-carb diets start low-carb regimens, their initial symptoms can distort results.

In one major cross-over trial comparing low-fat and low-carb diets, the carry-over effect skewed energy intake by an astounding 2,000 calories per day.

When carry-over effects exist, the trial results become not just ambiguous—but misleading. A beneficial diet might appear ineffective (or worse), and a harmful one might look good.

What the Stats Say

This isn’t just theoretical. For more than 50 years, statisticians have issued consistent warnings about carry-over effects in cross-over trials:

• Varma (1974): Only the first diet period can be reliably analyzed if carry-over is suspected.
• Hill (1979): If carry-over is detected, comparisons should be limited to the initial phase.
• FDA (2001): If unequal carry-over can’t be ruled out, no valid within-subject comparison can be made.
• Jones & Kenward (2003): Carry-over is statistically indistinguishable from treatment-period interactions—rendering the design unusable without proper washout periods.

These problems are not exceptions—they are baked into the design. And yet, they continue to plague short-term dietary trials.

What We Need Instead

If we truly want to understand whether a diet can support metabolism, reduce inflammation, delay aging, or protect the brain, we need long-term studies—with enough time for the body to fully adapt. Each diet phase should last at least two months, with equally long wash-out periods in between to eliminate carry-over effects.

Ultimately, only long-term, parallel-group trials—where different people follow different diets over years—can provide the answers we need. We’d never approve a new drug for diabetes or heart disease based on just two weeks of data. Why would we do so for something as fundamental as diet?

Public Support is Crucial

This kind of research is expensive and not likely to be funded by industry. Drug companies may spend over $1 billion to develop a single medication. But there’s no similar financial incentive to fund rigorous diet trials—no company profits when people prevent disease through food.

That’s why government investment is essential. The Nutrition for Precision Health program has the right ambition—but with a different design, its funding could support several long-term trials comparing low-carb, ultra-processed, and whole-food diets. These trials would give us solid evidence for future dietary guidelines and clinical recommendations.

While we’ve spent decades debating diets, chronic disease rates have only gone up. If we’re serious about reversing that trend, we need to stop relying on short, flawed studies. It’s time to fund and execute the kind of research that can actually lead to solutions.

Stats 101: The Fatal Flaw of Carry-Over Effects

The scientific consensus is clear: carry-over effects invalidate cross-over trials when not properly addressed. Here’s what experts have said for decades:

• “Carryover effects are completely confounded with treatment-by-period interaction.” – Wang (1997)
• “If carryover exists, direct comparisons of treatments can be invalidated.” – Ambrosius (2007)
• “Such an estimate can only be obtained by analyzing the first treatment period or assuming no differential carry-over.” – Reed (2004)
• “Crossover study designs are unsuitable when outcomes have long carryover effects.” – Lichtenstein (2021)