{"id":19209,"date":"2016-05-10T02:25:53","date_gmt":"2016-05-10T06:25:53","guid":{"rendered":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii.html"},"modified":"2017-11-29T08:05:52","modified_gmt":"2017-11-29T13:05:52","slug":"are-most-scientific-studies-false-and-biased-part-ii","status":"publish","type":"post","link":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/","title":{"rendered":"Are Most Scientific Studies False and Biased? Part II"},"content":{"rendered":"

In my homepage
\nblog<\/a>, I discussed the flaws and biases in supplement and drug trials. This is
\npart II of the discussion.<\/p>\n

\nA 2005 review tilted, “Why Most
\nPublished Research Findings Are False,” provides a good summary of factors that
\ninfluence conclusions of studies (bold emphasis mine):<\/p>\n

\n

“There is
\nincreasing concern that most current published research findings are false. The
\nprobability that a research claim is true may depend on study power and
\nbias, the number of other studies on the same question, and, importantly, the
\nratio of true to no relationships among the relationships probed in each
\nscientific field.<\/b> In this framework, a research finding is less likely
\nto be true when the studies conducted in a field are smaller; when effect sizes
\nare smaller; when there is a greater number and lesser preselection of tested
\nrelationships; where there is greater flexibility in designs, definitions,
\noutcomes, and analytical modes; when there is greater financial and other
\ninterest and prejudice; and when more teams are involved in a scientific field
\nin chase of statistical significance.<\/b> Simulations show that for most study
\ndesigns and settings, it is more likely for a research claim to be false than
\ntrue. Moreover, for many current scientific fields, claimed research findings
\nmay often be simply accurate measures of the prevailing bias. In this essay, I
\ndiscuss the implications of these problems for the conduct and interpretation
\nof research.” (http:\/\/journals.plos.org\/plosmedicine\/article?id=10.1371\/journal.pmed.0020124)<\/i><\/p>\n<\/blockquote>\n

<\/p>\n

So, Do We Ignore the Data?<\/b><\/p>\n

No, no, and no…there’s
\nways to use this information to make informed decisions about what the “evidence”
\nis actually saying. But, it takes a lot of detective work. Most of my research and looking into studies
\nhas come from my own review of studies reporting on biases and lessons learned
\nfrom my mentors and teachers. In no way am I methodology whiz, but I do have a
\nbasic grasp of why our model and interpretations need to be interpreted with caution.<\/p>\n

These are some of
\nthe considerations that I always review when applying studies to my clients:<\/p>\n

1. Read the
\nactual study and be wary of media spin. In one cross-sectional analysis of 130
\nstudies of health news reported on google and found the following:<\/p>\n

\n

In total, 78% of the news did not
\nprovide a full reference or electronic link to the scientific article. We found
\nat least one spin in 114 (88%) news items and 18 different types of spin in
\nnews. These spin were mainly related to misleading reporting (59%) such as not
\nreporting adverse events that were reported in the scientific article (25%),
\nmisleading interpretation (69%) such as claiming a causal effect despite
\nnon-randomized study design (49%) and overgeneralization\/misleading
\nextrapolation (41%) of the results such as extrapolating a beneficial effect
\nfrom an animal study to humans (21%). We also identified some new types of spin
\nsuch as highlighting a single patient experience for the success of a new
\ntreatment instead of focusing on the group results. (http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4608738\/)<\/i><\/p>\n<\/blockquote>\n

2. Consider
\nmethods used.<\/p>\n

Subjects: Who
\nwere the participants\/what population was studied (differences in gender\/ethnicity\/health
\nstatus)? What were their characteristics? What was the dropout rate? Who was
\nexcluded and why? What were the factors controlled for in the subjects? <\/p>\n

Type of study:
\nWas it observational and correlational study which look for relationships
\nverses cause-and-effect or was it a case-control randomized trial? Was there a
\ncontrol or was it a comparison trial? (Too much or too little control both have
\nweakness. For example, too much control prevents extrapolation of the
\nintervention to the real world and too little prevents interpretation that the
\nintervention caused the change.)<\/p>\n

Intervention: What
\nis the form of intervention? Was it the appropriate dosage? How long was the
\nstudy? How was it taken? What was the placebo effect?<\/p>\n

3. Search the
\nresults for inconsistencies:<\/p>\n

How are the
\nresults reported? For example, is it the use of an odds ratio<\/a>, is it relative
\nor absolute risk? What is the NNT? Is the
\np-value of significance<\/a> truly reflective of compatible data with the
\nstatistical model? <\/p>\n

Do the charts
\nand statistics match the author’s conclusions?<\/p>\n

\n
\n4. More can be found here for the geeks…<\/p>\n

\n

\u00b7Epidemiological
\nstudy interpretation (http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3077477\/)<\/p>\n

\u00b7Biases
\nto search for (http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/18582622) <\/p>\n

\u00b7Design, analysis and interpretation of
\nmethod-comparison studies (http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2944826\/)\n<\/p>\n

\u00b7Quasi-experimental study designs (http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC1380192\/)\n<\/p>\n<\/blockquote>\n

Where to Go
\nfrom Here?<\/b><\/p>\n

1. Physicians and practitioners need to be honest about interventions and
\nbe transparent about what they have experience with. Both parties should look
\nup the NNT and find a few studies to examine if the intervention is new.<\/p>\n

2. Consumers and patients need to be aware that some of the studies and
\nstandard of care physicians are using could be flawed. Don’t just accept
\ntreatment that isn’t helping without studying the data or asking your doctor
\nfor more information. Most importantly, look for if you’re getting results with
\nan intervention (nutrient, herb, oil, supplement, medication) and use that in
\nyour basis for the final decision.<\/p>\n

Get the references
\nhere on my homepage. <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

In my homepage blog, I discussed the flaws and biases in supplement and drug trials. This is part II of the discussion. A 2005 review tilted, "Why Most Published Research Findings Are False," provides a good summary of factors…<\/p>\n","protected":false},"author":109,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[98,362,848,926,1050,1156],"class_list":["post-19209","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-bias-and-studies","tag-flaws-in-evidence-based-medicine","tag-naturopathic-and-functional-medicine-and-albany-ny","tag-nutrition-and-evidence-based-research","tag-research-bias-and-supplements","tag-supplements-and-health"],"yoast_head":"\r\nAre Most Scientific Studies False and Biased? Part II - Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog<\/title>\r\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\r\n<link rel=\"canonical\" href=\"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/\" \/>\r\n<meta property=\"og:locale\" content=\"en_US\" \/>\r\n<meta property=\"og:type\" content=\"article\" \/>\r\n<meta property=\"og:title\" content=\"Are Most Scientific Studies False and Biased? Part II - Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog\" \/>\r\n<meta property=\"og:description\" content=\"In my homepage blog, I discussed the flaws and biases in supplement and drug trials. This is part II of the discussion. A 2005 review tilted, "Why Most Published Research Findings Are False," provides a good summary of factors...\" \/>\r\n<meta property=\"og:url\" content=\"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/\" \/>\r\n<meta property=\"og:site_name\" content=\"Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog\" \/>\r\n<meta property=\"article:published_time\" content=\"2016-05-10T06:25:53+00:00\" \/>\r\n<meta property=\"article:modified_time\" content=\"2017-11-29T13:05:52+00:00\" \/>\r\n<meta name=\"author\" content=\"Dr. Sarah\" \/>\r\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\r\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Dr. Sarah\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\r\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/\",\"url\":\"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/\",\"name\":\"Are Most Scientific Studies False and Biased? Part II - Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.saratoga.com\/living-well\/#website\"},\"datePublished\":\"2016-05-10T06:25:53+00:00\",\"dateModified\":\"2017-11-29T13:05:52+00:00\",\"author\":{\"@id\":\"https:\/\/www.saratoga.com\/living-well\/#\/schema\/person\/6872284eed515076e2993a69584827e3\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.saratoga.com\/living-well\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Are Most Scientific Studies False and Biased? Part II\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.saratoga.com\/living-well\/#website\",\"url\":\"https:\/\/www.saratoga.com\/living-well\/\",\"name\":\"Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.saratoga.com\/living-well\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.saratoga.com\/living-well\/#\/schema\/person\/6872284eed515076e2993a69584827e3\",\"name\":\"Dr. Sarah\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.saratoga.com\/living-well\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b37e59a320fe3e87499bec9e197eda9c?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b37e59a320fe3e87499bec9e197eda9c?s=96&d=mm&r=g\",\"caption\":\"Dr. Sarah\"},\"description\":\"Dr. LoBisco has been in holistic healthcare for over 10 years. She became interested in holistic medicine when she was able to heal two herniated discs through nutrition, yoga, supplementation, and chiropractic. She has mentored with holistic practices throughout New York, Vermont, and Connecticut. In addition to her Naturopathic and Functional Medical training, Dr. LoBisco has extensive training in a variety of healing modalities, including therapeutic essential oils, nutraceuticals, herbs, whole food supplements, nutritional medicine, and mind-body therapies. She is a graduate of the accredited, four year post-graduate program in Naturopathic Medicine at the University of Bridgeport in Connecticut. This program includes clinical rotations and a demanding scientific curriculum in integrating conventional and natural medicine. Dr. LoBisco holds her license from the state of Vermont. Dr. LoBisco has completed her postdoctoral training as a certified functional medicine practitioner. She is also certified in Applied Kinesiology and holds a BA in psychology from SUNY Geneseo. She has contributed as an item writer for the North American Board of Naturopathic Examiners (NABNE) and has several articles that have been published in the Naturopathic Doctor News and Review Digest (NDNR) and the Townsend Letter, both physician- based journals. Dr. LoBisco is also a hired speaker on integrative medical topics for medical professionals. Dr. LoBisco currently incorporates her training in holistic medical practices and conventional medicine through writing, researching, private practice, and through her independent contracting work for companies regarding supplements, nutraceuticals, essential oils, and medical foods. She has a small, private wellness consultation practice through telephone and Skype. Dr. LoBisco also enjoys continuing to educate and empower her readers through her blogs and social media. Her new book, BreakFree Medicine, is now available on Amazon and through Barnes & Noble.\",\"url\":\"https:\/\/www.saratoga.com\/living-well\/author\/slobisco\/\"}]}<\/script>\r\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Are Most Scientific Studies False and Biased? Part II - Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/","og_locale":"en_US","og_type":"article","og_title":"Are Most Scientific Studies False and Biased? Part II - Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog","og_description":"In my homepage blog, I discussed the flaws and biases in supplement and drug trials. This is part II of the discussion. A 2005 review tilted, "Why Most Published Research Findings Are False," provides a good summary of factors...","og_url":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/","og_site_name":"Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog","article_published_time":"2016-05-10T06:25:53+00:00","article_modified_time":"2017-11-29T13:05:52+00:00","author":"Dr. Sarah","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Dr. Sarah","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/","url":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/","name":"Are Most Scientific Studies False and Biased? Part II - Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog","isPartOf":{"@id":"https:\/\/www.saratoga.com\/living-well\/#website"},"datePublished":"2016-05-10T06:25:53+00:00","dateModified":"2017-11-29T13:05:52+00:00","author":{"@id":"https:\/\/www.saratoga.com\/living-well\/#\/schema\/person\/6872284eed515076e2993a69584827e3"},"breadcrumb":{"@id":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.saratoga.com\/living-well\/2016\/05\/are-most-scientific-studies-false-and-biased-part-ii\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.saratoga.com\/living-well\/"},{"@type":"ListItem","position":2,"name":"Are Most Scientific Studies False and Biased? Part II"}]},{"@type":"WebSite","@id":"https:\/\/www.saratoga.com\/living-well\/#website","url":"https:\/\/www.saratoga.com\/living-well\/","name":"Living Well In Saratoga Springs NY: Saratoga's Holistic Health Forum Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.saratoga.com\/living-well\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.saratoga.com\/living-well\/#\/schema\/person\/6872284eed515076e2993a69584827e3","name":"Dr. Sarah","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.saratoga.com\/living-well\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b37e59a320fe3e87499bec9e197eda9c?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b37e59a320fe3e87499bec9e197eda9c?s=96&d=mm&r=g","caption":"Dr. Sarah"},"description":"Dr. LoBisco has been in holistic healthcare for over 10 years. She became interested in holistic medicine when she was able to heal two herniated discs through nutrition, yoga, supplementation, and chiropractic. She has mentored with holistic practices throughout New York, Vermont, and Connecticut. In addition to her Naturopathic and Functional Medical training, Dr. LoBisco has extensive training in a variety of healing modalities, including therapeutic essential oils, nutraceuticals, herbs, whole food supplements, nutritional medicine, and mind-body therapies. She is a graduate of the accredited, four year post-graduate program in Naturopathic Medicine at the University of Bridgeport in Connecticut. This program includes clinical rotations and a demanding scientific curriculum in integrating conventional and natural medicine. Dr. LoBisco holds her license from the state of Vermont. Dr. LoBisco has completed her postdoctoral training as a certified functional medicine practitioner. She is also certified in Applied Kinesiology and holds a BA in psychology from SUNY Geneseo. She has contributed as an item writer for the North American Board of Naturopathic Examiners (NABNE) and has several articles that have been published in the Naturopathic Doctor News and Review Digest (NDNR) and the Townsend Letter, both physician- based journals. Dr. LoBisco is also a hired speaker on integrative medical topics for medical professionals. Dr. LoBisco currently incorporates her training in holistic medical practices and conventional medicine through writing, researching, private practice, and through her independent contracting work for companies regarding supplements, nutraceuticals, essential oils, and medical foods. She has a small, private wellness consultation practice through telephone and Skype. Dr. LoBisco also enjoys continuing to educate and empower her readers through her blogs and social media. Her new book, BreakFree Medicine, is now available on Amazon and through Barnes & Noble.","url":"https:\/\/www.saratoga.com\/living-well\/author\/slobisco\/"}]}},"_links":{"self":[{"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/posts\/19209"}],"collection":[{"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/users\/109"}],"replies":[{"embeddable":true,"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/comments?post=19209"}],"version-history":[{"count":2,"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/posts\/19209\/revisions"}],"predecessor-version":[{"id":28091,"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/posts\/19209\/revisions\/28091"}],"wp:attachment":[{"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/media?parent=19209"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/categories?post=19209"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.saratoga.com\/living-well\/wp-json\/wp\/v2\/tags?post=19209"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}