Machine Learning (eBook)

a Concise Introduction

Steven W. Knox (Autor)

eBook Download: PDF

2018
John Wiley & Sons (Verlag)
978-1-119-43907-3 (ISBN)

Infos

AN INTRODUCTION TO MACHINE LEARNING THAT INCLUDES THE FUNDAMENTAL TECHNIQUES, METHODS, AND APPLICATIONS
PROSE Award Finalist 2019
Association of American Publishers Award for Professional and Scholarly Excellence

Machine Learning: a Concise Introduction offers a comprehensive introduction to the core concepts, approaches, and applications of machine learning. The author-an expert in the field-presents fundamental ideas, terminology, and techniques for solving applied problems in classification, regression, clustering, density estimation, and dimension reduction. The design principles behind the techniques are emphasized, including the bias-variance trade-off and its influence on the design of ensemble methods. Understanding these principles leads to more flexible and successful applications. Machine Learning: a Concise Introduction also includes methods for optimization, risk estimation, and model selection- essential elements of most applied projects. This important resource:

• Illustrates many classification methods with a single, running example, highlighting similarities and differences between methods
• Presents R source code which shows how to apply and interpret many of the techniques covered
• Includes many thoughtful exercises as an integral part of the text, with an appendix of selected solutions
• Contains useful information for effectively communicating with clients

A volume in the popular Wiley Series in Probability and Statistics, Machine Learning: a Concise Introduction offers the practical information needed for an understanding of the methods and application of machine learning.

STEVEN W. KNOX holds a Ph.D. in Mathematics from the University of Illinois and an M.S. in Statistics from Carnegie Mellon University. He has over twenty years' experience in using Machine Learning, Statistics, and Mathematics to solve real-world problems. He currently serves as Technical Director of Mathematics Research and Senior Advocate for Data Science at the National Security Agency.

STEVEN W. KNOX holds a Ph.D. in Mathematics from the University of Illinois and an M.S. in Statistics from Carnegie Mellon University. He has over twenty years' experience in using Machine Learning, Statistics, and Mathematics to solve real-world problems. He currently serves as Technical Director of Mathematics Research and Senior Advocate for Data Science at the National Security Agency.

Inhalt

Machine Learning: a Concise Introduction 3
Contents 7
Preface 13
Organization—How to Use This Book 15
Acknowledgments 19
About the Companion Website 21
1 Introduction—Examples from Real Life 23
2 The Problem of Learning 25
2.1 Domain 26
2.2 Range 26
2.3 Data 26
2.4 Loss 28
2.5 Risk 30
2.6 The Reality of the Unknown Function 34
2.7 Training and Selection of Models, and Purposes of Learning 34
2.8 Notation 35
3 Regression 37
3.1 General Framework 38
3.2 Loss 39
3.3 Estimating the Model Parameters 39
3.4 Properties of Fitted Values 41
3.5 Estimating the Variance 44
3.6 A Normality Assumption 45
3.7 Computation 46
3.8 Categorical Features 47
3.9 Feature Transformations, Expansions, and Interactions 49
3.10 Variations in Linear Regression 50
3.11 Nonparametric Regression 54
4 Survey of Classification Techniques 55
4.1 The Bayes Classifier 56
4.2 Introduction to Classifiers 59
4.3 A Running Example 60
4.4 Likelihood Methods 62
4.4.1 Quadratic Discriminant Analysis 63
4.4.2 Linear Discriminant Analysis 65
4.4.3 Gaussian Mixture Models 67
4.4.4 Kernel Density Estimation 69
4.4.5 Histograms 73
4.4.6 The Naive Bayes Classifier 76
4.5 Prototype Methods 76
4.5.1 k-Nearest-Neighbor 77
4.5.2 Condensed k-Nearest-Neighbor 78
4.5.3 Nearest-Cluster 78
4.5.4 Learning Vector Quantization 80
4.6 Logistic Regression 81
4.7 Neural Networks 84
4.7.1 Activation Functions 84
4.7.2 Neurons 86
4.7.3 Neural Networks 87
4.7.4 Logistic Regression and Neural Networks 95
4.8 Classification Trees 96
4.8.1 Classification of Data by Leaves (Terminal Nodes) 96
4.8.2 Impurity of Nodes and Trees 97
4.8.3 Growing Trees 98
4.8.4 Pruning Trees 101
4.8.5 Regression Trees 103
4.9 Support Vector Machines 103
4.9.1 Support Vector Machine Classifiers 103
4.9.2 Kernelization 110
4.9.3 Proximal Support Vector Machine Classifiers 114
4.10 Postscript: Example Problem Revisited 115
5 Bias–Variance Trade-off 119
5.1 Squared-Error Loss 120
5.2 Arbitrary Loss 123
6 Combining Classifiers 129
6.1 Ensembles 129
6.2 Ensemble Design 132
6.3 Bootstrap Aggregation (Bagging) 134
6.4 Bumping 137
6.5 Random Forests 138
6.6 Boosting 140
6.7 Arcing 143
6.8 Stacking and Mixture of Experts 143
7 Risk Estimation and Model Selection 149
7.1 Risk Estimation via Training Data 150
7.2 Risk Estimation via Validation or Test Data 150
7.2.1 Training, Validation, and Test Data 150
7.2.2 Risk Estimation 151
7.2.3 Size of Training, Validation, and Test Sets 152
7.2.4 Testing Hypotheses About Risk 153
7.2.5 Example of Use of Training, Validation, and Test Sets 154
7.3 Cross-Validation 155
7.4 Improvements on Cross-Validation 157
7.5 Out-of-Bag Risk Estimation 159
7.6 Akaike’s Information Criterion 160
7.7 Schwartz’s Bayesian Information Criterion 160
7.8 Rissanen’s Minimum Description Length Criterion 161
7.9 R2 and Adjusted R2 162
7.10 Stepwise Model Selection 163
7.11 Occam’s Razor 164
8 Consistency 165
8.1 Convergence of Sequences of Random Variables 166
8.2 Consistency for Parameter Estimation 166
8.3 Consistency for Prediction 167
8.4 There Are Consistent and Universally Consistent Classifiers 167
8.5 Convergence to Asymptopia Is Not Uniform and May Be Slow 169
9 Clustering 171
9.1 Gaussian Mixture Models 172
9.2 k-Means 172
9.3 Clustering by Mode-Hunting in a Density Estimate 173
9.4 Using Classifiers to Cluster 174
9.5 Dissimilarity 175
9.6 k-Medoids 175
9.7 Agglomerative Hierarchical Clustering 176
9.8 Divisive Hierarchical Clustering 177
9.9 How Many Clusters Are There? Interpretation of Clustering 177
9.10 An Impossibility Theorem 179
10 Optimization 181
10.1 Quasi-Newton Methods 182
10.1.1 Newton’s Method for Finding Zeros 182
10.1.2 Newton’s Method for Optimization 183
10.1.3 Gradient Descent 183
10.1.4 The BFGS Algorithm 184
10.1.5 Modifications to Quasi-Newton Methods 184
10.1.6 Gradients for Logistic Regression and Neural Networks 185
10.2 The Nelder–Mead Algorithm 188
10.3 Simulated Annealing 190
10.4 Genetic Algorithms 190
10.5 Particle Swarm Optimization 191
10.6 General Remarks on Optimization 192
10.6.1 Imperfectly Known Objective Functions 192
10.6.2 Objective Functions Which Are Sums 193
10.6.3 Optimization from Multiple Starting Points 194
10.7 The Expectation-Maximization Algorithm 195
10.7.1 The General Algorithm 195
10.7.2 EM Climbs the Marginal Likelihood of the Observations 195
10.7.3 Example—Fitting a Gaussian Mixture Model Via EM 198
10.7.4 Example—The Expectation Step 199
10.7.5 Example—The Maximization Step 200
11 High-Dimensional Data 201
11.1 The Curse of Dimensionality 202
11.2 Two Running Examples 209
11.2.1 Example 1: Equilateral Simplex 209
11.2.2 Example 2: Text 209
11.3 Reducing Dimension While Preserving Information 212
11.3.1 The Geometry of Means and Covariances of Real Features 212
11.3.2 Principal Component Analysis 214
11.3.3 Working in “Dissimilarity Space” 215
11.3.4 Linear Multidimensional Scaling 217
11.3.5 The Singular Value Decomposition and Low-Rank Approximation 219
11.3.6 Stress-Minimizing Multidimensional Scaling 221
11.3.7 Projection Pursuit 221
11.3.8 Feature Selection 223
11.3.9 Clustering 224
11.3.10 Manifold Learning 224
11.3.11 Autoencoders 227
11.4 Model Regularization 231
11.4.1 Duality and the Geometry of Parameter Penalization 234
11.4.2 Parameter Penalization as Prior Information 235
12 Communication with Clients 239
12.1 Binary Classification and Hypothesis Testing 240
12.2 Terminology for Binary Decisions 241
12.3 ROC Curves 241
12.4 One-Dimensional Measures of Performance 246
12.5 Confusion Matrices 247
12.6 Multiple Testing 248
12.6.1 Control the Familywise Error 248
12.6.2 Control the False Discovery Rate 249
12.7 Expert Systems 250
13 Current Challenges in Machine Learning 253
13.1 Streaming Data 253
13.2 Distributed Data 253
13.3 Semi-supervised Learning 254
13.4 Active Learning 254
13.5 Feature Construction via Deep Neural Networks 255
13.6 Transfer Learning 255
13.7 Interpretability of Complex Models 255
14 R Source Code 257
14.1 Author’s Biases 258
14.2 Libraries 258
14.3 The Running Example (Section 4.3) 259
14.4 The Bayes Classifier (Section 4.1) 263
14.5 Quadratic Discriminant Analysis (Section 4.4.1) 265
14.6 Linear Discriminant Analysis (Section 4.4.2) 265
14.7 Gaussian Mixture Models (Section 4.4.3) 266
14.8 Kernel Density Estimation (Section 4.4.4) 267
14.9 Histograms (Section 4.4.5) 270
14.10 The Naive Bayes Classifier (Section 4.4.6) 275
14.11 k-Nearest-Neighbor (Section 4.5.1) 277
14.12 Learning Vector Quantization (Section 4.5.4) 279
14.13 Logistic Regression (Section 4.6) 281
14.14 Neural Networks (Section 4.7) 282
14.15 Classification Trees (Section 4.8) 285
14.16 Support Vector Machines (Section 4.9) 289
14.17 Bootstrap Aggregation (Section 6.3) 294
14.18 Boosting (Section 6.6) 296
14.19 Arcing (Section 6.7) 297
14.20 Random Forests (Section 6.5) 297
Appendix A List of Symbols 299
Appendix B Solutions to Selected Exercises 301
Appendix C Converting Between Normal Parameters and Level-Curve Ellipsoids 321
C.1 Parameters to Axes 322
C.2 Axes to Parameters 322
Appendix D Training Data and Fitted Parameters 323
D.1 Training Data 323
D.2 Fitted Model Parameters 324
D.2.1 Quadratic and Linear Discriminant Analysis 324
D.2.2 Logistic Regression 325
D.2.3 Neural Network 325
D.2.4 Classification Tree 325
References 327
Index 337
EULA 343

Produkt-Details

• Erscheint lt. Verlag: 8.3.2018
• Reihe/Serie: Wiley Series in Probability and Statistics
• Reihe/Serie: Wiley Series in Probability and Statistics
• Reihe/Serie: Wiley Series in Probability and Statistics
• Sprache: englisch
• Themenwelt: Informatik ► Theorie / Studium ► Künstliche Intelligenz / Robotik
• Themenwelt: Mathematik / Informatik ► Mathematik ► Statistik
• Themenwelt: Mathematik / Informatik ► Mathematik ► Wahrscheinlichkeit / Kombinatorik
• Schlagworte: activation functions of machine learning • categorical features of machine learning • clustering by mode hunting of machine learning • Computer Science • consistency of parameter estimation of machine learning • convergence of sequences of random variables • cross validation in machine learning • Data Mining • Data Mining & Knowledge Discovery • Data Mining Statistics • Data Mining u. Knowledge Discovery • Electrical & Electronics Engineering • Elektrotechnik u. Elektronik • estimating the model parameters of machine learning • examples of machine learning • framework for machine learning • Gaussian mixture models of machine learning • Guide to machine learning • Informatik • logistic regression of machine learning • Mustererkennung • neural networks of machine learning • Pattern Analysis • properties of fitted values of machine learning • risk estimation of machine learning • Statistics • Statistik • stepwise model selection and machine learning • techniques of machine learning • testing hypotheses about risk in machine learning • using classifiers to cluster of machine learning
• ISBN-10: 1-119-43907-8 / 1119439078
• ISBN-13: 978-1-119-43907-3 / 9781119439073