Take it with a pinch of salt
Findings presented at scientific meetings may be preliminary and may have undergone only limited peer review. Any study should be regarded as tentative if the results are at odds with the evidence accumulated so far.
THERE IS an eight per cent risk reduction of ever having cardiovascular disease among women who had taken birth control pills in the past. And if women used oral contraceptives early on, they are probably going to be protected later in life. These were the findings of Rahi Victory and colleagues from Wayne State University in Detroit based on the data taken from the Women's Health Initiative (WHI) study.
The results overturned a long held scientific view on contraception and health. The results were presented in October at the annual meeting of the American Society for Reproductive Medicine (ASRM) and also published in an abstract form in Fertility & Sterility, the magazine of ASRM. The research paper was awarded the first prize by the Society for Reproductive Endocrinology and Infertility. It attracted wide media attention too.
Study under fire
The prize-winning study is now under fire. The Women's Health Initiative study, a long-term national health study undertaken in the U.S. to focus on strategies to prevent heart disease, breast and colorectal cancer and osteoporosis in postmenopausal women, in a statement released to the press had warned the public that the benefits claimed by the study are questionable.
"Analyses of these data must be regarded as exploratory since the Women's Health Initiative was not designed to answer these important questions. Furthermore, we have found that analyses of these data do not support inferences of either cardiovascular disease benefit or risk when additional account is taken of the complex relationships between the ages of participating women and both oral contraceptive use patterns and cardiovascular disease," the WHI press release said.
Misleading the public
Who now is responsible for misleading the public, particularly the women, about the health benefits of oral contraceptives the media or the scientific community? "The public places a lot of trust in the health-news media, in part because the reports begin with research findings from medical experts whom the public trusts even more," noted Edward W. Campion in an editorial in a recent issue of the New England Journal of Medicine.
In this case the findings were not only from medical experts but the raw data on which the researchers based their findings were from the WHI study. Add to this the most important factor the paper presented by the scientists won a prize. So was the media right in giving wide publicity? "The media and the public see publication in peer-reviewed journals as validation of the research," wrote Mr. Campion. "Diligent reviewers and careful editors can identify mistakes, provide balance, and restrain over interpretation."
Biases and errors
Incidentally, Dr. Victory presented his findings in the annual meeting of the American Society for Reproductive Medicine. Although the attendees were scientists and not lay public, Dr. Victory's results escaped the tough peer-reviewing process that precedes publication in a journal.
The WHI press release notes many such biases and errors that prove this point. For instance, it states that all analyses were done based on the women's ability to recall events. Their medical records were not checked; care was not taken to find out the timing of cardiovascular disease and oral contraceptive use; and it studied only those who were healthy. Those who died of cardiovascular problems were obviously left out; age of the women and the possibility of oral contraceptive use were also not factored in.
"If we do our job as editors well, a newspaper, magazine, or television report on a journal article is likely to be more reliable than it would have been had the article not gone through the peer-review and revision process," Robert Steinbrook wrote in an editorial in the New England Journal of Medicine. Papers presented at conferences unfortunately never go through the peer-review process before being presented.
"Analogous to the (electronic) preprint is the conference presentation or abstract, which airs research findings before it has been formally `written up' and peer reviewed," noted Tony Delamothe in an editorial in the British Medical Journal.
The root cause
A paper published in 2002 in the Journal of the American Medical Association finds out why media coverage of scientific meetings leads to problems. "... it is also easy to understand why media coverage of scientific meetings could be a problem. Scientific meetings are intended to provide a forum for researchers to present new work to colleagues; the work presented may be preliminary and may have undergone only limited peer review.
"Frequently, the presentations represent work in progress. Unfortunately, many projects fail to live up to their early promise; in some cases, fatal flaws emerge," the paper states.
It goes further to say, "Press coverage at this early stage may leave the public with the false impression that the data are in fact mature, the methods valid, and the findings widely accepted. As a consequence, patients may experience undue hope or anxiety or may seek unproved, useless, or even dangerous tests and treatments."
"Nearly every clinical research study should be seen as preliminary. No matter how important the conclusions, they should be considered tentative until a body of evidence accumulates pointing in the same direction," noted Marcia Angell and Jerome P. Kassirer in another editorial in the New England Journal of Medicine about findings published in journals. "Any one study should be regarded as tentative, the more so if the results are spectacular or at odds with the evidence accumulated so far."
Need for caution
The last point mentioned is all the more pertinent in this case as the body of evidences correlating oral contraceptive use and the increased risk of suffering from cardiovascular disease is staggering.
"Other research studies have shown that women who take oral contraceptives have a small increased risk for cardiovascular disease, like blood clots, heart attacks and stroke.
"Those studies provide better scientific data than the recent analyses, because they focused on specific questions about oral contraceptives and cardiovascular disease," the WHI press release pointed out. So what does all this mean?
You pick up the paper one day and read that cholesterol causes heart attacks and you pick it up the next day and read that is doesn't. It becomes easy for people to feel that scientists don't know what they're doing," Dr. Ronald B. Turner of the University of Virginia was quoted as saying in the `reviews' in the British Medical Journal. Perhaps it is time the media and public stopped taking the findings presented at conferences on their face value.
Send this article to Friends by