As a former research engineer who spent a career right here in West Windsor conducting research on a variety of physical phenomena — primarily in the field of fluid dynamics — I am often bothered by the misuse of the word “study” in the press. That is “study” the noun — and I don’t mean a room in your house where you sit and read. Because the local press in West Windsor and Plainsboro can be just as guilty of this as can the press anywhere, I feel it’s time to tell it like it really is.
More often than not the bothersome phrase goes something like this: “Numerous studies have shown that the effect is serious and should be given more attention by researchers in seeking a solution.” The problem with this is that there may actually have been — or very likely had been — only one study in the first place. All the other so-called “numerous studies” were simply reviews or citations of the original and only study.
Many writers and reporters make no distinction, primarily because they lack the special knowledge needed to recognize legitimate contributions to a specialized field. That’s unfortunate because the lay public can easily get the idea that a huge amount of research has shown that the problem is or isn’t serious, when, in fact, only one true “study” of the subject exists. I realize that for some writers this approach is looked upon as simple exaggeration designed to hold the readers’ attention. Strict factual accuracy is viewed as less important. “Numerous studies — wow! This must be important.”
All this is doubly unfortunate if there really have been “numerous” studies on a specific subject that show that the problem really is serious. This is where editors must be vigilant in deciding what to print. But if that is what they really want to do, they should make a real effort to check their facts with an “unbiased” source. Sometimes that’s not easy to do, and it should then be acceptable to mention that the opinion may, in fact, be biased or uncertain. I should also mention that this is a way for the writer to “get off the hook” if something goes wrong as a result of misinterpretation of what was written.
Once, about 30 years ago, I was asked by the local school board to look into the matter of how effective seat belts might be on school buses in preventing injury to riders in an accident. I and the others on my committee — including a professional nurse who was determined to have seat belts made mandatory on buses — found that at that time only one true study had ever been made of the subject, wherein real school buses had been tested with instrumented dummies in the seats with and without belts.
I think the study had been conducted by the U.S. Department of Transportation. The buses were put through real accident situations — not simulations — and the results recorded to determine what hazards and resulting damage to the dummies actually occurred. The results showed that seat belts were very effective in preventing such damage.
But then we found that bus manufacturers and other “anti-seat belt” commentary referred to “numerous studies” that showed seat belts in buses to be “ineffective” or “too expensive” or not worth the trouble for a variety of reasons. One reason given was that students would hit each other with the seat belts and buckles. So where were these “numerous studies”? Answer: they didn’t exist. Each time the one and only study was the subject of a news story or magazine article, including its biased commentary, that story or article itself became a “study.” Thus, after only one round of review of the subject there were now “several studies,” many of which contained biased, opinion-based commentary not based on the real test results.
Then, after those articles were written about, there was thus another tier of “studies” created. Yet there was still only one true study of the subject that provided the basis for what were now referred to as “numerous studies.”
Readers and editors should be wary when they see the phrase “numerous studies show…” It’s often an exaggeration or just plain untrue, no matter what the subject. Over the years many articles in the Science section of the New York Times have referred to “studies” of various subjects. Unless the same article explains exactly what the word “study” really refers to, including the scientific reference or original source, the reader should be skeptical. If the author or editor is unsure, the article shouldn’t be printed in the first place, period.
I am sure that real professional investigators are familiar with the “study” effect, since their research is subject to true scrutiny by their peers. Scientific journals in many fields will not publish the results of new research without subjecting the work to peer review. This is as it should be. Such journals are not intended for assessment by the lay public. In fact, highly technical and specialized language would not be understandable to such an audience.
Sometimes, of course, peer review may involve “peers” who are also rivals and somehow in competition with the author of the study, and, therefore, may be critical in a biased way. That is when the editor of the journal should explain the situation — if competent to do so. In such cases, there should be an editorial note to give the reader some guidance in interpreting the claims given.
Sometimes, of course, contrary opinions about results are legitimate, and absolute opinions by an author are legitimately questioned. This is when further research is definitely legitimate and necessary. But when you see the word “study” in a newspaper or magazine, beware.
Getting back to seat belts in buses, there was no doubt that pertinent research showed that using them would be highly desirable, despite what the “anti-seat-belt” faction said. At the time — in the 1980s — there was doubt among many drivers that belts were even necessary in automobiles, and some in the public fought hard to avoid or overturn new laws that required their use — not because they didn’t increase safety, but because some people thought they were “a nuisance” or “inconvenient.”
Some people claimed that belts could even be dangerous because they might prevent you from “escaping” from a damaged car after an accident. I even remember one time I was driving into the parking lot at West Windsor town hall and an officer in a patrol car was ticketing a driver who was about to park while not wearing his seat belt.
But little by little the public began to stop objecting to seat belts. Some states, including New Jersey, enacted legislation that made their use mandatory for drivers and front-seat passengers. And then, so was their use in New Jersey school buses. Some larger buses were also allowed to have “compartmentalized” seats, so that occupants would be held in the area of their seats in case of an accident, instead of being thrown through open areas of the bus interior
It took a few years, but the effort our committee and others helped start in the 1980s finally had a positive outcome — and we didn’t have to publish a “study” to accomplish it.