Business Experiments with R (eBook)

B. D. McCullough (Autor)

eBook Download: EPUB

2021
John Wiley & Sons (Verlag)
978-1-119-68988-1 (ISBN)

Infos

BUSINESS EXPERIMENTS with R

A unique text that simplifies experimental business design and is dedicated to the R language

Business Experiments with R offers a guide to, and explores the fundamentals of experimental business designs. The book fills a gap in the literature to provide a text on the topic of business statistics that addresses issues such as small samples, lack of normality, and data confounding. The author-a noted expert on the topic-puts the focus on the A/B tests (and their variants) that are widely used in industry, but not typically covered in business statistics textbooks.

The text contains the tools needed to design and analyze two-treatment experiments (i.e., A/B tests) to answer business questions. The author highlights the strategic and technical issues involved in designing experiments that will truly affect organizations. The book then builds on the foundation in Part I and expands the multivariable testing. Since today's companies are using experiments to solve a broad range of problems, Business Experiments with R is an essential resource for any business student. This important text:

Presents the key ideas that business students need to know about experiments

Offers a series of examples, focusing on a specific business question

Helps develop the ability to frame ill-defined problems and determine what data and analysis would provide information about that problem

Written for students of general business, marketing, and business analytics, Business Experiments with R is an important text that helps to answer business questions by highlighting the strategic and technical issues involved in designing experiments that will truly affect organizations.

B. D. MCCULLOUGH, PHD, was a Professor in the Department of Decision Sciences & MIS, LeBow College of Business, Drexel University, Philadelphia, PA.

Inhalt

Preface xiii

Suggested courses using this book xv

Acknowledgments xix

1 Why Experiment? 1

2 Analyzing A/B Tests: Basics 49

3 Designing A/B Tests with Large Samples 107

4 Analyzing A/B Tests: Advanced Techniques 127

5 Designing Tests with Small Samples 189

6 Analyzing Designs via Regression 229

7 Two-Level Full Factorial Experiments 281

8 Two-Level Screening Designs 329

9 Custom Design of Experiments 357

10 Epilogue 397

Index 419

Textauszug

Preface

Rigorous experiments are rapidly becoming a critical tool for businesses. While experiments have been a popular tool in engineering for decades, driven largely by the quality movement of the 1980s, experiments are rapidly coming out of the engineering lab to be deployed more broadly across business operations. Companies today are using experiments to solve a broad range of problems from determining whether improved training results in better employee productivity and retention to gauging whether store renovations result in higher profitability, to determining which subject lines in a promotional email attract customer attention, to assessing which behavioral nudges are most likely to get utility users to shift energy consumption to nonpeak times, and to deciding which color to use for the checkout buttons on a website. Articles in the business press such as those by Davenport (2009), Anderson and Simester (2011), and Thomke and Manzi (2014) have made the case that experiments are a key component of “data‐driven” business and companies are taking note and incorporate experiments into their day‐to‐day operations. With this rapid adoption of experimentation, there is a need for a comprehensive resource for business leaders and analysts who want to use experiments to understand how business systems work and solve tactical business problems.

Yet, material on business experiments remains siloed within specific business functions. For example, there are dozens of books and articles on website testing that focus intently on experimentation with websites and never mention that the same basic ideas – randomization, measurement, and comparison – can be applied in other business domains. Our goal for this book is to step back from this highly tactical approach and give business students a broad, theoretically grounded understanding of the role of experiments in business, providing them with a practical foundation for designing and analyzing experiments that will be useful to them throughout their careers.

Of course, experiments have long been a foundational tool in engineering, physical sciences, psychology, medicine, and statistics, and excellent books on how to design and analyze experiments are available within each of these fields, yet none of these books provides the right coverage for business students. The literature on experiments in the physical sciences and engineering provides great depth on multivariable test design, yet often focuses on situations where the experimenter is studying a physical process in a lab setting, where everything can be measured and controlled and error can be reduced nearly to zero – conditions that seldom hold in business. The classic textbook on experimental design for engineers by Montgomery (2017) naturally assumes a level mathematical skill that is not common among business students. Psychologists like Campbell and Stanley (1963) focus more on experiments with human subjects, raising many of the same issues that come up in marketing and human resources experiments, yet they focus intently on using experiments to develop theory and so cover issues like construct validity that are less relevant in the types of tactical experiments that business uses to drive day‐to‐day decisions. Useful discussions of the practicalities of running experiments in the field have appeared recently in the economics literature championed by Harrison and List (2004) and Gerber and Green (2012), yet the experiments reported tend to focus on testing economic theory. In many ways, the types of tactical experiments that are valuable in business are much more similar to clinical trials and other biomedical experiments, where there is a tradition of tactical experiments designed to determine, “Should we give this treatment to patients?” and a well‐established culture of randomized controlled trials as the gold standard of evidence. While much of this work from other disciplines is relevant to business experiments, it doesn't provide students with concrete examples of how to apply these methods to answer important business questions.

This book has drawn from all of these literatures selecting the key ideas that business students need to know about experiments. This book approaches experimentation as a skill that one becomes better at over time, like playing a sport or an instrument, which requires equal measures of creativity and technical skill. To help students practice, the book is structured around a series of examples, focusing first on a specific business question, then turning to the design or analysis of an experiment that then will answer that question, and then building up the appropriate statistical methods. This way, the technical material is presented in context. This approach provides an ideal “second course in statistics,” where students can review material they have seen before (specifically two‐sample tests, power, ANOVA, confidence intervals, and regression) in the context of business problems. My hope is that students who work through this book will move beyond the mechanics and develop their abilities to frame ill‐defined problems, determine what data and analysis would provide information about that problem, and examine the evidence for or against a particular business decision.

For clarity in the use of pronouns, I have adopted the convention that the analyst is always female. Other persons are male.

Suggested Courses Using This Book

This book can be used both with general business students, such as MBA or executive education students, and with the growing number of master's programs in business analytics and marketing analytics.

The first four or five chapters (depending on instructor preference) provides students with the tools they need to design and analyze two‐treatment experiments (i.e. A/B tests) to answer business questions, focusing on the strategic and technical issues involved in designing experiments that will truly affect organizations. The book begins with a discussion of the importance of causal analysis in business and the risks of observational data analysis in Chapter 1. By design, the next chapter begins with the analysis of A/B tests, with a review of the statistical methods required for analyzing experiments, focusing on visualization and hypothesis testing. Once students understand how experiments are analyzed, in Chapter 3, they can move on to understanding issues involved in designing experiments including selecting treatments, measuring responses, planning sample sizes, and the importance of randomization for causal interpretation. Chapter 4 returns to the analysis of A/B tests, covering several advanced analysis techniques including analyzing tests with matched pairs, analyzing tests with more than two treatment groups, analyzing subgroups within an A/B test, and determining the minimal detectable effect for an experiment that has been completed. Chapter 5, which covers issues involved in the design of A/B tests when the number of units available for tests is small, as is the case when doing geo‐testing or testing across store locations. This coverage makes Part I ideal for a half‐semester graduate course for second‐year MBA students or strong undergraduate students. These chapters can be covered over five weeks, skipping some of the more technical material and allowing time for the students to apply the material to a project where they plan, execute, and analyze an A/B test.

Part II builds on the foundation in Part I, expanding to multivariable testing. Chapter 6 begins Part II with a review of the primary tool used to analyze multivariate tests: regression analysis. Next comes full and fractional factorial designs in Chapters 7 and 8, which builds students' understanding of the core issues in multivariate design: interactions and confounding. Fractional factorial designs are useful for building understanding. Next comes custom design, which is ideal for designing experiments in constrained business contexts. The development of software for optimal design has made this sophisticated tool accessible to users, and my goal for Chapter 9 is to provide students with a basic introduction to optimal design, while avoiding the algorithmic details that would be more appropriate to a “technical manual.” Parts I and II together form the core of a full‐semester course that is appropriate for more analytically oriented graduate students including MBA students concentrating in analytics and master's of business analytics students.

Not all experiments are technically complicated. There is a great deal of “low‐hanging fruit” in the business world that can be profitably plucked by persons whose knowledge of experimental design does not exceed 2k designs. The goal of this book is to enable students who have taken business statistics to pluck that fruit. In particular, after finishing the first half of this book, the student should be able to conduct an A/B test. After finishing the second half, the student should be able to design, execute, and analyze a 2k experiment. To this end, instructors should require students to complete a project where they design, execute, analyze, and report on an experiment. Students benefit greatly from the experience, which helps them see the challenges involved in designing a conclusive experiment and cement the ideas in the book by applying them in practice. A/B tests are easy to dream up, multivariable tests not so much. Hence, Box's helicopter experiment is very important. It is perhaps the only easily...

Produkt-Details

• Erscheint lt. Verlag: 26.3.2021
• Sprache: englisch
• Themenwelt: Mathematik / Informatik ► Mathematik ► Statistik
• Themenwelt: Mathematik / Informatik ► Mathematik ► Wahrscheinlichkeit / Kombinatorik
• Schlagworte: Business & Management • Computer Science • Database & Data Warehousing Technologies • Data Mining • Data Mining Statistics • Datenbanken u. Data Warehousing • Decision Sciences • Informatik • R (Programm) • Statistics • Statistik • Theorie • Theorie der Entscheidungsfindung • Wirtschaft u. Management
• ISBN-10: 1-119-68988-0 / 1119689880
• ISBN-13: 978-1-119-68988-1 / 9781119689881