Python for Probability, Statistics, and Machine Learning (eBook)

José Unpingco (Autor)

eBook Download: PDF

2016 | 1. Auflage
XV, 288 Seiten
Springer-Verlag
978-3-319-30717-6 (ISBN)

Infos

This book covers the key ideas that link probability, statistics, and machine learning illustrated using Python modules in these areas.  The entire text, including all the figures and numerical results, is reproducible using the Python codes and their associated Jupyter/IPython notebooks, which are provided as supplementary downloads. The author develops key intuitions in machine learning by working meaningful examples using multiple analytical methods and Python codes, thereby connecting theoretical concepts to concrete implementations. Modern Python modules like Pandas, Sympy, and Scikit-learn are applied to simulate and visualize important machine learning concepts like the bias/variance trade-off, cross-validation, and regularization. Many abstract mathematical ideas, such as convergence in probability theory, are developed and illustrated with numerical examples.  This book is suitable for anyone with an undergraduate-level exposure to probability, statistics, or machine learning and with rudimentary knowledge of Python programming.

Dr. José Unpingco completed his PhD from the University of California, San Diego in 1998 and has since worked in industry as an engineer, consultant, and instructor on a wide-variety of advanced data processing and analysis topics, with deep experience in multiple machine learning technologies. He was the onsite technical director for large-scale Signal and Image Processing for the Department of Defense (DoD) where he also spearheaded the DoD-wide adoption of scientific Python. As the primary scientific Python instructor for the DoD, he has taught Python to over 600 scientists and engineers. Dr. Unpingco is currently the Technical Director for Data Science for a non-profit Medical Research Organization in San Diego, California.

Inhalt

Preface 6
Acknowledgments 7
Contents 8
Notation 12
About the Author 14
1 Getting Started with Scientific Python 15
1.1 Installation and Setup 17
1.2 Numpy 18
1.2.1 Numpy Arrays and Memory 20
1.2.2 Numpy Matrices 23
1.2.3 Numpy Broadcasting 24
1.2.4 Numpy Masked Arrays 26
1.2.5 Numpy Optimizations and Prospectus 26
1.3 Matplotlib 27
1.3.1 Alternatives to Matplotlib 29
1.3.2 Extensions to Matplotlib 30
1.4 IPython 30
1.4.1 IPython Notebook 32
1.5 Scipy 34
1.6 Pandas 35
1.6.1 Series 35
1.6.2 Dataframe 37
1.7 Sympy 39
1.8 Interfacing with Compiled Libraries 41
1.9 Integrated Development Environments 42
1.10 Quick Guide to Performance and Parallel Programming 43
1.11 Other Resources 46
References 46
2 Probability 48
2.1 Introduction 48
2.1.1 Understanding Probability Density 49
2.1.2 Random Variables 50
2.1.3 Continuous Random Variables 55
2.1.4 Transformation of Variables Beyond Calculus 58
2.1.5 Independent Random Variables 60
2.1.6 Classic Broken Rod Example 62
2.2 Projection Methods 63
2.2.1 Weighted Distance 66
2.3 Conditional Expectation as Projection 67
2.3.1 Appendix 73
2.4 Conditional Expectation and Mean Squared Error 73
2.5 Worked Examples of Conditional Expectation and Mean Square Error Optimization 77
2.5.1 Example 77
2.5.2 Example 81
2.5.3 Example 83
2.5.4 Example 86
2.5.5 Example 87
2.5.6 Example 90
2.6 Information Entropy 91
2.6.1 Information Theory Concepts 92
2.6.2 Properties of Information Entropy 94
2.6.3 Kullback-Leibler Divergence 95
2.7 Moment Generating Functions 96
2.8 Monte Carlo Sampling Methods 100
2.8.1 Inverse CDF Method for Discrete Variables 101
2.8.2 Inverse CDF Method for Continuous Variables 103
2.8.3 Rejection Method 105
2.9 Useful Inequalities 108
2.9.1 Markov's Inequality 109
2.9.2 Chebyshev's Inequality 110
2.9.3 Hoeffding's Inequality 111
References 112
3 Statistics 114
3.1 Introduction 114
3.2 Python Modules for Statistics 115
3.2.1 Scipy Statistics Module 115
3.2.2 Sympy Statistics Module 116
3.2.3 Other Python Modules for Statistics 117
3.3 Types of Convergence 117
3.3.1 Almost Sure Convergence 118
3.3.2 Convergence in Probability 120
3.3.3 Convergence in Distribution 122
3.3.4 Limit Theorems 123
3.4 Estimation Using Maximum Likelihood 124
3.4.1 Setting Up the Coin Flipping Experiment 126
3.4.2 Delta Method 136
3.5 Hypothesis Testing and P-Values 138
3.5.1 Back to the Coin Flipping Example 139
3.5.2 Receiver Operating Characteristic 143
3.5.3 P-Values 145
3.5.4 Test Statistics 146
3.5.5 Testing Multiple Hypotheses 153
3.6 Confidence Intervals 154
3.7 Linear Regression 157
3.7.1 Extensions to Multiple Covariates 167
3.8 Maximum A-Posteriori 171
3.9 Robust Statistics 177
3.10 Bootstrapping 184
3.10.1 Parametric Bootstrap 188
3.11 Gauss Markov 189
3.12 Nonparametric Methods 193
3.12.1 Kernel Density Estimation 193
3.12.2 Kernel Smoothing 196
3.12.3 Nonparametric Regression Estimators 201
3.12.4 Nearest Neighbors Regression 202
3.12.5 Kernel Regression 206
3.12.6 Curse of Dimensionality 207
References 209
4 Machine Learning 210
4.1 Introduction 210
4.2 Python Machine Learning Modules 210
4.3 Theory of Learning 214
4.3.1 Introduction to Theory of Machine Learning 216
4.3.2 Theory of Generalization 220
4.3.3 Worked Example for Generalization/Approximation Complexity 222
4.3.4 Cross-Validation 228
4.3.5 Bias and Variance 232
4.3.6 Learning Noise 235
4.4 Decision Trees 238
4.4.1 Random Forests 245
4.5 Logistic Regression 247
4.5.1 Generalized Linear Models 252
4.6 Regularization 253
4.6.1 Ridge Regression 257
4.6.2 Lasso 261
4.7 Support Vector Machines 263
4.7.1 Kernel Tricks 266
4.8 Dimensionality Reduction 269
4.8.1 Independent Component Analysis 273
4.9 Clustering 277
4.10 Ensemble Methods 281
4.10.1 Bagging 281
4.10.2 Boosting 284
References 286
Index 287

Produkt-Details

• Erscheint lt. Verlag: 16.3.2016
• Zusatzinfo: XV, 276 p. 221 illus., 7 illus. in color.
• Verlagsort: Cham
• Sprache: englisch
• Themenwelt: Mathematik / Informatik ► Informatik ► Datenbanken
• Themenwelt: Mathematik / Informatik ► Mathematik ► Statistik
• Themenwelt: Technik ► Elektrotechnik / Energietechnik
• Schlagworte: IPython Notebooks • machine learning • probability and statistics • Python Toolchain • Scientific Python
• ISBN-10: 3-319-30717-7 / 3319307177
• ISBN-13: 978-3-319-30717-6 / 9783319307176