Lots of treatments work very well in mice but fail to show benefits in human trials. They’re false positives, and they get lots of people excited over treatments that never end up working in humans.

(Why do they work so well in mice, I wonder? Is it because so much of our research uses them? I wonder, if we were willing to completely throw morals out the window, could we get those sorts of results in humans by experimenting on them directly? Not that I’m advocating this.)

I just realized something blindingly obvious: there are false negatives too. But how are these handled? Treatments that don’t work in mice never make it to human trials, even though they may work in humans. Without doing human trials on treatments that failed to work in mice, we can’t evaluate a false negative rate, but it could potentially be high. Certainly it’s nonzero in any case.

This is another example of snap judgments shooting down ideas, but this is far less clear-cut than most criticism because failing to analyze the treatment prior to human trials can endanger people’s health.

I think that what we need are better computer models.