Learn how you can easily avoid these 42 common mistakes in meta-analysis. Michael Borenstein, Ph.D. has worked, taught, conducted research, served the academic community by assisting on several review groups and advisory panels, developed statistical software programs, and published within the meta-analysis scientific area. Recently, I actively participated in a 3-day meta-analysis workshop led by him in Los Angeles, CA and since then have performed over 20 analyses.
Previously I read his first book, Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. (2009), Introduction to Meta-Analysis. West Sussex, United Kingdom, John Wiley & Sons, ISBN: 978-0-470-05724-7. Yet, I absolutely fell in love with his newest book, Borenstein M. (2019), Common Mistakes in Meta-Analysis and How to Avoid Them. Englewood, New Jersey, USA, Biostat Inc, ISBN: 978-1-7334367-1-7 (softcover). Hence, I want to review it for you.
Dr. Borenstein is acknowledged as a distinguished expert and is exceptionally positioned to author this landmark textbook. I have read the book and found it to be a critical addition to my library. It is helpful in expanding and developing my knowledge of meta-analysis, and especially in avoiding common mistakes when conducting systematic reviews of interventions. This is in its first edition (ISBN: 978-1-7334367-1-7 the softcover version) and available on Amazon.
The author has chosen to identify and explain 42 mistakes made when conducting these important analyses. He uses excellent, easily understood examples for you to apply and learn the principles of meta-analysis. The author explains why each is a mistake, the implications of the mistake, and how you can avoid the mistake.
For Those New To Meta-Analyses: You will never forget the history behind the term “forest plots” and will learn why you should refer to them as “forest plots” and not “Forest plots” even though you will see that term referenced in the literature.
If you are new to this area of statistical analysis, there is no better book for you to read at this early stage in your career. Don’t be afraid to be a beginner. It provides a historical perspective to help you understand where the industry was and where it is headed in terms of the evolution of meta-analysis when applied to social, behavioral, educational, medical, clinical, regulatory, or legal applications.
If you need to know the basics and wonder “Why do meta-analyses and what are the different models that can be used?” this book, starting with the preface and continuing through chapters 6 and 7, is for you. Before you get to page 42 you will learn what many don’t seem to know or remember; how to decide on the model to use upfront, prior to your analysis. Dr. Borenstein will give you a solid explanation to support that fundamental principle. Refer to chapter 4 to obtain websites for this book, the software program Comprehensive Meta-Analysis, upcoming workshops, and then register your email to receive updates to remain current and competent. Finally, review the glossary in chapter 15. Now you will be ready to actively participate in your first meta-analysis team meeting using your solid, fundamentally sound foundation.
For Those Seasoned Professionals: If you are a seasoned professional, grounded in the basics already, you will enjoy the following chapters with discussions regarding (1) issues and myths about statistical models, (2) heterogeneity, (3) mistakes related to significance testing, (4) publication bias, (5) mistakes in subgroup analyses, (6) features of the Comprehensive Meta-Analysis software program, and (7) how to correctly report the results of an analysis. Relevant examples are presented. Study them. For a more in-depth review, the author includes ten appendices that contain supplementary material which may be helpful in providing a more comprehensive understanding of your research question or problem.
Reflecting On My Experience Reading This Book: I personally found value in (1) knowing what the 42 mistakes were, (2) the easily understood Summary Sections the author provides, and (3) the “Putting It All Together” sections. He clarifies the issues that were previously ambiguous. The section on heterogeneity and prediction interval, an index of dispersion, was of enormous value and very practical to me. I learned what the “I squared” (I2) statistic correctly is and is not; it really doesn’t tell you how effects vary, despite the widespread misconception in the literature. It tells us what proportion of the variance in observed effects reflects variation in true effects, rather than sampling error. It does not tell us how much variation there is. Rely on the prediction interval instead. Finally, templates were provided to assist with reporting results that communicate clearly to my intended audience.
When the treatment effect (or effect size) is consistent from one study or observation to the next, meta-analysis can be used to identify this common effect. When the effect varies from one study or observation to the next, a meta-analysis may be used to identify the reason for the variation. For example, when you run the analysis, why does one population seem to not benefit and actually appear to be harmed, while another population has a small treatment effect, a third tends to have a modest effect, and finally some appear to have the largest treatment effect? Now it’s our opportunity to put those detective shoes on and set up strategies to identify, address or evaluate those differences in effects. Regulatory bodies, clinicians, and Notified Bodies will want to confirm that your product, for example, is working as intended in the approved population. Performing a meta-analysis is one way to assess product safety and performance.
In this life, few are given the responsibility or opportunity to change the world (or at least a piece of it where you live) by analyzing significant data correctly. Don’t model your comprehensive reviews and analyses on what others have published in the literature. Base it on the principles learned in this book. It is a 5-star, brilliantly written, expertly communicated reference that has received my highest recommendation. Avoid making the 43rd mistake by not having this as part of your library.
If you believe that decisions affecting people’s lives in areas of clinical, public health or policy settings should be informed by relevant research evidence using meta-analyses, then this review is dedicated to you. With the proper education, training and other easy to use tools that exist today, you too can be on the verge of changing the world where you live. How about your experiences? I’d love to hear more heartwarming stories from the analyses you have performed.
Key Websites For Your Continued Learning
https://meta-analysis-books.com/, www.cochrane.org/, https://www.meta-analysis-workshops.com/, and training.cochrane.org/.
David R Rutledge, Pharm.D. FCCP, FAHA
david.rutledge@globalstrategicsolutions.com
+1 (630) 846-0350 cell
Based in Silicon Valley, California, USA