How to Do Bad Biomarker Research
Biomarker Uncertainty Principle: A molecular signature derived from high-dimensional data can be either parsimonious or predictive, but not both. We have more data than ever, more good data than ever, a lower proportion of data that are good, a lack of strategic thinking about what data are needed to answer questions of interest, sub-optimal analysis of data, and an occasional tendency to do research that should not be done.
Before discussing how much of biomarker research has gone wrong from a methodologic viewpoint, it’s useful to recall statistical principles that are used to guide good research.
A paper in JAMA Psychiatry (McGrath et al. (2013) ) about a new neuroimaging biomarker in depression typifies how violations of some of the above principles lead to findings that are very unlikely to either be replicated or to improve clinical practice. It also illustrates how poor understanding of research methodology by the media leads to hype.
The Times article claimed that we now know whether to treat a specific depressed patient with drug therapy or behavioral therapy if a certain imaging study was done. This finding rests on assessment of interactions of imaging results with treatment in a randomized trial.
