Use and abuse of statistical analysis

A vegan recently tried to convince me of the health merits of his lifestyle over mine by quoting a study.  The main point was that when comparing a certain set of markers associated with good health between a population of vegans and the general population, the vegans scored better.  Obviously, this meant that being a vegan was better than eating meat.

I remain unconvinced.  At first glance the figures seem very convincing, in a way, but they do not stand up to scrutiny.  The biggest flaw is that the study is comparing a population with a high awareness of what they eat to a general population that range from meat eating health fanatics to sedentary fast food consumers.  It is not a focused study, and all it really says is that people who follow a consistent diet without most junk food do better than a sample that doesn’t.  For the study to be convincing it needs to start by comparing two like groups where one eats animal protein and one doesn’t.

Unfortunately, the use and abuse of statistics might as well be regarded as best practice. When it comes to influencing decisions or creating a story it is painfully effective, and unfortunately most people do not have the knowledge needed to call bullshit. What is frustrating is that often the data used is obviously poorly selected, gathered with little thought and misrepresented to produce the most bizarre insights.