William Miller, MD
How to Judge Scientific Information
COVID-19 has had many effects on our lives including how we think about information we receive from various sources. We have been bombarded with information about it from the very beginning of the pandemic. There is so much out there, especially on the internet, and much of it contradictory, that it can be difficult to tell what to believe sometimes.
During the past 16 months that I have been writing the Miller Report, I have tried to provide information that is based on scientifically known facts and less on opinion. I have received much feedback from the readership and I appreciate it. One thing that I have been asked is why I do not provide alternative points of view on such topics as the safety of vaccines, for example.
The reason is that the information I provide is based on what is published in scientific journals that are highly respected in the healthcare field and upon information provided by the FDA and the CDC. So much of what is out there on the internet is not what one should consider reliable information from reliable sources. Here are some guidelines that I use which you might find helpful to judge what you are reading on the internet as being reliable or not.
First, when we Google a topic or question, we need to understand how the search engine arranges the answers. Literally, there will be hundreds of thousands of responses. Part of what arranges those responses as to which ones come up first is based on our own preferences. In other words, we will always get a biased search response based on what we have tended to look at in the past. So, if you often go to websites that are, say, anti- vaccination, you will be preferentially directed towards that type of site. This becomes a bias on the information you are receiving. This is especially true for social media websites such as Facebook. If you want to avoid such bias, you need to evaluate the websites and the information on them.
Websites of scientific and medical journals specifically try to avoid bias. Articles in these types of journals undergo something called “peer review”. This means that no article gets published without being judged by other experts in the field. They critically look at how a study was performed, whether the analysis of the data was done correctly and did the results of the study actually support the conclusions. Another important aspect of scientific articles is that they must reference previously reported studies. That way, the article is based upon the body of knowledge and less upon the opinion of individuals.
A few months ago, one of my readers sent me an article that appeared to be published in a scientific journal. The article made a number of claims that vaccines in general are dangerous and that the COVID vaccines are unsafe. The article was an editorial piece, meaning an opinion, and not a scientific review of the subject. I was curious and so I looked into it further. It turns out that the “journal” was in fact an on-line blog maintained by the author herself. She had given her blog a name that sounded very authoritative and actually included the word “journal” in the name. In the editorial piece, she referenced three other “articles”, all of which were her own. This is an example of a situation that a critical reader should consider with some skepticism. It is not peer reviewed, the “journal” may sound like a scientific publication because of its name, but it is not. Referencing one’s own authored “articles” also suggests that the article is more based on personal opinion of the author and less so on any body of scientific knowledge.
Here is another way to spot something that may not be reliable. Ask the question, is the headline “sensational”? Taking the vaccination safety question further, I received another response from a reader that was a link to a video entitled, “Say goodby to your loved ones if they get the COVID vaccine.” It was a video claiming that there are all sorts of unreported deaths from the COVID vaccines that are being kept secret by our government so that big pharma can profit. The headline alone should alert a skeptical reader seeking the truth to be wary such a link.
A common error made in many pseudo-science reports is claiming correlation as causation. To illustrate, I will give an obvious example. “The number of pirates on the ocean has been decreasing since the year 1800 while at the same time global temperatures are warming, therefore, the loss of pirates is causing global warming.” This is a very common type of error. When a person died shortly after getting a COVID vaccine, the immediate claim of many was that the vaccine caused the death. In fact, when hundreds of millions of people are getting vaccinated, one would expect that occasionally something else happens to some of those people shortly thereafter. If one of them won the PowerBall shortly after being vaccinated, I doubt that anyone would claim a causation. This type of error must be guarded against when it supports a preconceived bias. Scientific studies are designed to detect the difference between what is simply coincidence and what is an actual cause-and-effect. Using large numbers of subjects in a study helps limit such problems and if the sample size is large enough and the effect is common, then the claim of cause-and-effect might be supported.
Both the CDC and the FDA have teams of scientists and healthcare experts that spend their careers looking at data from numerous scientific sources. They make recommendations and decisions based on the scientific evidence and on what they believe is in the best interest of the public. In this time of political turmoil, it may be easy to become suspicious of what any government entity is telling us. However, at least with respect to the CDC and the FDA, the data upon which they are basing decisions is available to the public. There is transparency. The recommendations are supported by references to scientifically respected, peer reviewed, journals. So, if a member of the public has a question or concern about the basis for any decision the FDA and the CDC has made, that person is free to look it up and review the data themselves.
I trust what the CDC and the FDA put out because I have reviewed what they base their decisions on and I trust the sources they are using as reliable. In America, we believe that everyone has a right to their own opinion, but we should keep in mind that this does not mean everyone’s opinion is right. I will try to continue to report information based as much as possible on what is supported by sound scientific evidence.