Publication bias

One of the field of medicine’s most attractive qualities is that it is inhabited by people of skeptical minds, and to convince anyone of anything requires quite a substantial feat of proof. I suppose the underlying reason for this is that do no harm remains one of our central tenets, and when patients’ lives are at stake, we want to be absolutely sure we know what we’re doing.

In contrast, I’m continuously amazed at how low the threshold for acceptable evidence seems to be in the public sphere. The guilt by association smear is a particularly common and egregious example of this, as it insinuates without offering any evidence at all. (e.g. Nancy Pelosi sounds a lot like Michael Moore these days, talking about limiting the sale of handguns.) More dangerous is when evidence is used to draw conclusions that it doesn’t actually support. I remember watching a debate about gay and lesbian adoption rights on the Oprah Winfrey show a few years ago and getting irritated as one of the speakers kept citing evidence that children raised without fathers do less well than other children, in support of his claim that lesbian couples make inferior parents. Nobody on the show pointed out the ridiculousness of this logic, but in a medical conference he’d have been a laughing stock. Let me get this right – instead of studying the children of lesbian couples, you want to make assumptions about them using the children from broken homes as a proxy? I have a better idea, how about I prescribe you a medication for attention deficit disorder that has been proven safe in tortoises! Never mind that by his logic gay male parents would have made the best child-rearers, as they'd have provided the steady influence of not one but two fathers. I’d have found this man much less objectionable had he simply stated that he was opposed to lesbian parents because he was opposed to lesbianism.

One of the ways that we can stay honest, in medicine and in life, is by identifying bias in the information that we receive. The point isn’t necessarily to eliminate the bias, but simply to recognize it so that we can factor it in when we form opinions. Conservatives, for example, would like us to correct for what they see as a liberal media bias. Claiming that the news is biased, however, is a dangerous game to play; if the news seems biased, that is, if reality seems biased, it implies that the bias actually lies in the person perceiving it.

A kind of bias that I find particularly fascinating and that is commonly described in medicine is publication bias. Publication bias results from considering only information that has been published and neglecting other information. If you think about it, publication bias in one form or another influences everything that we learn and know, since we are unable to access information that hasn’t been published or broadcast in some way. By filtering out information that is not worth being heard, publication bias serves as a useful form of quality control, but it does mean that we must rely on others to supply us with information that’s relevant, important, and entertaining. On the micro level, in conversation between people, publication bias becomes speech bias. When talking with someone, you only hear the thoughts that they decide are worth or safe speaking. Since the dullest, dumbest, and riskiest thoughts are filtered out, speech bias makes people seem smarter than they actually are, and probably leads us to doubt our own capability and worth. Maybe those really arrogant people out there in the world would seem less intimidating if we could somehow eliminate this bias. There are definitely certain minds where I would love to implant a thought recorder and register the thinking as is. Of course, unfiltered speech can also be hazardous – I’ve been exposed to enough of it in the psych ward to know its capacity for harm. A wish to be able to read minds is exactly the kind of wish that the devil loves to grant. Yet, speech bias does act as an impediment towards peace and understanding between people in conflict. If we could all just plug our brains into a global neural network where ideas and information were exchanged freely, wouldn’t the world be a better, more peaceful place?

Wait a minute, that sounds an awful lot like the blogosphere.

posted by Wyatt @ 3:57 AM Comments: 2