The functional charlatans

This year, functional brain imaging celebrates its 25th anniversary. Basically, functional imaging or fMRI is a misleading term because it is mainly used for the depiction of changes of local blood supply in the brain activated by specific stimuli. In contrast to electroencephalography (EEG) and magnetoencephalography (MEG), it does not provide a direct measure of neural activity.¹

In 1990, Dr. Jack Belliveau and colleagues published the first observation of the stimulation of the human visual cortex by MRI.² They watched the first pass effect of a contrast agent after bolus injection to demonstrate changes in cortical perfusion upon activation with a photic stimulus. However, this approach required two contrast agent injections. This disadvantage was resolved by the demonstration of brain activation using the blood oxygenation level-dependent (BOLD) effect developed by Dr. Seiji Ogawa,³ also in 1990. Over the last few years, this very elegant technique has led to a fast proliferation of fMRI all over the world.

Dr. Peter Rinck, PhD, is a professor of diagnostic imaging and the president of the Council of the Round Table Foundation (TRTF) and European Magnetic Resonance Forum (EMRF).

A real explosion of medical, paramedical, neurosocial, and neuroeconomic publications ensued. The outcome of commercial applications went through the media: of the German automobile manufacturer trying to find out which car design attracts more men, of a major U.S. soft drinks company checking the most bewitching taste ... you name it. Functional MRI is even used as a lie detector: no science, just fiction.

Soon, research in this topic was in the hands of amateurs playing with MRI and functional MR, lacking the background in physics, chemistry, and medicine, as well as the scientific rigor necessary to work in a new field. They saw pictures with colorful enhancement of the brain and overnight became cognitive social neuroscientists, new-age phrenologists -- because color pictures can be interpreted by everybody. But nobody bothered to ask: What is normal and what is pathological -- or, what is an artifact?

Unfortunately, BOLD studies have a very low sensitivity and signal-to-noise ratio. The signal changes related to cerebral activation are close to the noise level and therefore numerous signal processing and, beyond this statistical techniques are used to overcome this handicap. Many blood flow alterations described in functional brain imaging rely on signal-intensity changes of less than 5%. More so, T2* to estimate blood oxygen saturation is only one singled-out factor; oxygen supply and saturation are dependent on several additional and independent parameters, among them lung and heart function, vessel size, and hematocrit.

Even the inventor, Ogawa, added some very detailed and critical remarks about the techniques in a review article 22 years after the first description of BOLD.⁴

As Dr. Gustav von Schulthess from Zurich pointed out in the early days of fMRI:

[A] caveat for fMRI: It is a very interesting technique, but signal changes are but a few percent. Hence, the method is technically demanding and "the threshold of nonsense production is low."⁵

Outstanding proof of his claim can be found reading one of the most famous publications in this field published in the last years: the fMRI story (at 1.5 tesla) of a dead Atlantic Salmon (Salmo salar).⁶ Here are some excerpts:

[From the Methods Section:] "The task administered to the salmon involved completing an open-ended mentalizing task. The salmon was shown a series of photographs depicting human individuals in social situations with a specified emotional valence, either socially inclusive or socially exclusive. The salmon was asked to determine which emotion the individual in the photo must have been experiencing. The photo stimuli were presented in a block design."

[From the beginning of the Results Section:] "A t -contrast was used to test for regions with significant BOLD signal change during the presentation of photos as compared to rest. The parameters for this comparison were t (131) > 3.15, p (uncorrected) < 0.001, 3-voxel extent threshold. The relatively low extent threshold value was chosen due to the small size of the salmon's brain relative to voxel size. Several active voxels were observed in a cluster located within the salmon's brain cavity. The size of this cluster was 81 mm³ with a cluster-level significance of p = 0.001."

This article presents in a really imaginative way the often overlooked main problem of fMRI. If the fMRI study of the little brain of a dead fish appears to give cognitive social scientists indications of brain functions and answers to some of their puzzles, how much confidence can we have in studies that follow the same or similar paradigms in far bigger live human brains? With their very catching experiment and a later paper, Bennett and collaborators stressed how pivotal it is in fMRI to apply statistics properly and scrupulously because random noise may yield spurious results in the acquired images.⁷

Some years ago I pointed out that the number of good medical -- among them radiological -- papers is less than 1% of all papers published. Much of the rest is without rhyme or reason, more chaff than grain. It seems even worse in fMRI. For that reason, good papers are laudable -- and who cares about bad papers, as long as they disappear in the sea of scientific trash? Still, the scientific reputation of fMRI research is poor, and the charlatans ruin the standing of the serious scientists in the field.

Sadly, a few papers are bad and ugly. During the last few years I followed the publications of a group of authors that now should be marked as dubious, worrying, and malign. These people did something similar with presumed pedophiles as the others did with the dead salmon. They tried to identify pedophiles by changes of BOLD effects after showing the study subjects pictures, in this case of nude adults and children. For these experiments, the authors used small heterogeneous groups of presumed pedophiles attracted to either boys or girls, and similarly sized comparison groups of healthy heterosexual and homosexual men. They found BOLD activation when applying (uncorrected) thresholds, and claim to be able to distinguish between pedophiles and nonpedophiles; their accuracy is 95%.⁸

In a study published two years later about stimulation of what appears the same study subjects, this time shown pictures of adult and children faces, whole sections are poorly written and incomprehensible, the materials and methods section is irreproducible, the statistics are unsound and not applicable, and all results are probably false positives.⁹ More so, there are always the caveats of very small comparison groups and the lack of knowledge of what is normal.

The problem is not the bad research performed; this is common; it's the ugly and deeply disturbing conclusion, where the authors turn dilettantism into a weapon:

"Functional brain response patterns to sexual stimuli contain sufficient information to identify pedophiles with high accuracy. The automatic classification of these patterns is a promising objective tool to clinically diagnose pedophilia."⁸

This conclusion is, politely phrased, highly problematic. Using fMRI as a biomarker (i.e., a detector) for pedophilia is unethical -- because the technique does not make it possible to identify pedophiles. The employment of fMRI to diagnose pedophilia may have unforeseen consequences. It is a misuse and abuse of medical imaging. None of these articles has the rock-solid foundation that would be necessary for the conclusions the authors draw at the end.

The impact of such papers might be hurtful and detrimental, even deadly for some members of our societies. Readers of the articles might draw conclusions and take actions that are not appropriate, taking for granted that "scientific" publications even in obscure journals can be taken as the last truth.

The authors (Ponseti et al. from Kiel, Germany; Siebner from Copenhagen; and Beier et al. from Berlin) and the ethical committees of the respective universities as well as the editors and reviewers of the journals the articles were published in are responsible for possible harm caused. The authors' claims are false and have to be actively countered as forcefully as possible. The papers should be retracted.

Note: Writing this column has taken a good year. References six, eight, and nine have been retrospectively peer-reviewed by four leading scientists in the field. All of them thought that paper six is one of the best papers ever written in fMRI; and all of them rejected papers eight and nine. Meanwhile, some critique of the mentioned papers has appeared on the Web. It concurs with what is said in this column.

More so, statements that many neuroimaging papers are of inferior quality are also being underlined in numerous blogs, for instance in reference 10, together with a discussion to retract all these papers, en masse, because they can be harmful: "Neuroscientists working with such controversial populations need to be especially careful in analyzing their data, and aware of how their work may be used in a broader social context."¹¹

Dr. Peter Rinck, PhD, is a professor of diagnostic imaging and the president of the Council of the Round Table Foundation (TRTF) and European Magnetic Resonance Forum (EMRF).

References

1. Rinck PA. Functional Imaging. In: Magnetic Resonance in Medicine. The Basic Textbook of the European Magnetic Resonance Forum. 8th edition; 2014.
2. Belliveau JW, Rosen BR, Kantor HL, et al. Functional cerebral imaging by susceptibility-contrast NMR. Magn Res Med. 1990;14(3):538-546.
3. Ogawa S, Lee TM, Kay AR, Tank DW. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc Natl Acad Sci USA. 1990;87(24):9868-9872.
4. Kim SG, Ogawa S. Biophysical and physiological origins of blood oxygenation level-dependent fMRI signals. J Cereb Blood Flow Metab. 2012;32(7):1188-1206.
5. Von Schulthess G. Clinical MR in the year 2010. Mag Res Med. 1999;8(3):133-145.
6. Bennett CM, Baird AA, Miller MB, Wolford GL. Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for proper multiple comparisons correction. J Serendipitous and Unexpected Results. 2010;1(1):1-5.
7. Bennett CM, Miller MB. fMRI reliability: influences of task and experimental design. Cogn Affect Behav Neurosci. 2013;13(4):690-702.
8. Ponseti J, Granert O, Jansen O, et al. Assessment of pedophilia using hemodynamic brainresponse to sexual stimuli. Arch Gen Psychiatry 2012;69(2):187-194.
9. Ponseti J, Granert O, van Eimeren T, et al. Human face processing is tuned to sexual age preferences. Biology Letters. 2014;10(5):20140200.
10. Bor D. The dilemma of weak neuroimaging papers. www.danielbor.com/dilemma-weak-neuroimaging.
11. The Neurocritic. Let's face it: Publishing weak data on face processing in pedophiles is pointless. http://neurocritic.blogspot.fr/2014_05_01_archive.html.

The comments and observations expressed herein do not necessarily reflect the opinions of AuntMinnieEurope.com, nor should they be construed as an endorsement or admonishment of any particular vendor, analyst, industry consultant, or consulting group.