Are You Cherry Picking Data? Stop Trying to Prove You’re Right! (evidence matters)


how do you research something on the internet do you start with an open mind or do you set out to try and prove that what you believe is the truth about a subject what you type into Google will give you a clue as well as what you do with the results once they appear if you are in fact cherry-picking evidence then there is a pretty good chance that your conclusions may be completely wrong let me explain hi I’m dr. Alex from ourpetshealth.com helping you and your pet to live a healthier happier life and welcome to another evidence matters video now if you’ve spent any time online you’ll have seen plenty of people cry do your research well research is all well and good but only if you’re doing it properly and this plea is normally made by someone with alternative views and I’ve bet in most cases their cereal cherry pickers so let’s start with the Wikipedia definition of cherry picking it goes cherry picking suppressing evidence or the fallacy of incomplete evidence is the act of pointing to individual cases or data that seems to confirm a particular position while ignoring a significant proportion or related cases or data that may contradict that position so let me explain what this means with an example using my old friend homeopathy let’s say you come from a family that strongly believes in homeopathy but you want to research if it really works or not chances are you will go into your research with the starting opinion that homeopathy does work even if this belief is a subconscious one the worst thing you could Google would be what is the evidence homeopathy works and leave it at that so looking at the first page of Google you’ll find that four of the results are proclaiming the wonder of homeopathic treatment and homeopathy in general and making bold claims about how effective it is one results it’s on the fence and the other four results on the first page claim that homeopathy is completely ineffective now if you’re a believer looking at these results in all likelihood you will just click on the four results that support your view you might follow any links they contain you’ll make copious notes proving what you already believed and claim with satisfaction knew your own independent research confirms homeopathy is the best thing since sliced bread what you will have done is cherry-pick the evidence that supports your view and completely disregarded any evidence that suggested otherwise now of course if all you did was read the evidence that proves homeopathy doesn’t work you’d be doing exactly the same thing you’d be cherry-picking the evidence that you wanted to believe now if you read all the results and then googled what is the evidence homeopathy doesn’t work you’d be doing a much better job because you’d be trying to find evidence both for and against to allow you to try and form an overall opinion you’d then read all about the systematic reviews of all the evidence and see the many institutions that have all drawn the same conclusion now in all likelihood you’d come to the same opinion so in reality you can find evidence for anything that you can imagine and this evidence may be nothing more than story’s opinion or anecdote and I’ll discuss it in a future video why this isn’t really evidence at all but if it supports your viewpoint and is the only thing you look at then it will just do nothing but confirm your beliefs so just for example take the belief that the earth is flat if your research simply meant that you read evidence on the page from the Flat Earth Society then what do you think would happen you do nothing more but reinforce your belief and that is not research remember as well that cherry-picking isn’t always done consciously so it isn’t always as obvious as these examples making a conscious decision though to avoid cherry-picking is very important whenever you’re doing any research but when you’re especially when you’re researching particular ideas or treatments and remember that evidence can be found and quoted to support almost every opinion so another category of cherry-picking that to be aware of and I’d like to discuss is that mm manufacturers or whichever drug or supplement that you might be researching so most products will often make bold claims like proven to result in a 59 percent faster cure rate if you actually look at where this data comes from though the actual conclusions may be very different equally they’re made just to pick data from the one study that showed a positive result and completely ignore the majority that failed to demonstrate any benefit the actual publishing of research studies is another area where there’s the potential for cherry-picking so running studies that are conduct in a way so as to be able to draw firm conclusions does take a lot of money and as a result drug companies will often be the ones funding studies into their drugs it is quite possible then that they will only publish the studies which show the best results leaving those with less enthusiastic outcomes unpublished and unseen now that’s not to say that the positive studies that have been carried out have been done incorrectly or that all pharmaceutical studies should be ignored absolutely not but the overall body of evidence may be subject to bias as a result of cherry-picking which studies to publish so a really good book that discusses this issue and more about the pharmaceutical industry in more detail is the very readable bad farmer and while you’re at it it’d be a good idea to pick up bad science as well by the same author which goes through a lot of the ideas that I’m discussing in my evidence matters series that are really key to evidence-based veterinary medicine but also human medicine and will stand you in good stead for researching topics in the future so cherry-picking is just one way to misinterpret evidence make sure you know all about the placebo effect in pets as well as reduction to the mean which I’ve discussed in previous videos and make sure you subscribe to make sure to be sure that you don’t miss out on future videos all about helping you make informed decisions so that you and your pet can live healthier happier lives and until next time i’m dr. alex from our pets health because they’re family

One Comment

Add a Comment

Your email address will not be published. Required fields are marked *