Classification, Parameter Estimation and State Estimation (eBook)

An Engineering Approach Using MATLAB

Bangjun Lei, Guangzhu Xu, Ming Feng, Yaobin Zou, Ferdinand van der Heijden, Dick de Ridder, David M. J. Tax (Autoren)

eBook Download: EPUB

2017 | 2. Auflage
John Wiley & Sons (Verlag)
978-1-119-15245-3 (ISBN)

Infos

A practical introduction to intelligent computer vision theory, design, implementation, and technology

The past decade has witnessed epic growth in image processing and intelligent computer vision technology. Advancements in machine learning methods-especially among adaboost varieties and particle filtering methods-have made machine learning in intelligent computer vision more accurate and reliable than ever before. The need for expert coverage of the state of the art in this burgeoning field has never been greater, and this book satisfies that need. Fully updated and extensively revised, this 2nd Edition of the popular guide provides designers, data analysts, researchers and advanced post-graduates with a fundamental yet wholly practical introduction to intelligent computer vision. The authors walk you through the basics of computer vision, past and present, and they explore the more subtle intricacies of intelligent computer vision, with an emphasis on intelligent measurement systems. Using many timely, real-world examples, they explain and vividly demonstrate the latest developments in image and video processing techniques and technologies for machine learning in computer vision systems, including: 

• PRTools5 software for MATLAB-especially the latest representation and generalization software toolbox for PRTools5
• Machine learning applications for computer vision, with detailed discussions of contemporary state estimation techniques vs older content of particle filter methods
• The latest techniques for classification and supervised learning, with an emphasis on Neural Network, Genetic State Estimation and other particle filter and AI state estimation methods
• All new coverage of the Adaboost and its implementation in PRTools5.

A valuable working resource for professionals and an excellent introduction for advanced-level students, this 2nd Edition features a wealth of illustrative examples, ranging from basic techniques to advanced intelligent computer vision system implementations. Additional examples and tutorials, as well as a question and solution forum, can be found on a companion website.

Authors:
Professor Bangjun Lei, Dr. Guangzhu Xu, Dr. Ming Feng, Dr. Yaobin Zou, Dr. F. van der Heijden, Professor Dick de Ridder, Dr. David M. J. Tax

Autor

Professor Bangjun Lei, Dr. Guangzhu Xu, and Dr. Ming Feng are with The Institute of Intelligent Vision and Image Information, China Three Gorges University, China. Professor Yaobin Zou is an associate professor at China Three Gorges University. Dr. Ferdinand van der Heijden, Ph.D., is on the faculty of theDepartment of Signals and Systems, University of Twente, Netherlands. Professor Dick de Ridder is Professor at the Bioinformatics lab at Wageningen University, Netherlands. Professor David M. J. Tax, is a researcher with the Pattern Recognition laboratory, Delft University of Technology.

Inhalt

Preface xi

About the Companion Website xv

Introduction 1

1.1 The Scope of the Book 2

1.2 Engineering 10

1.3 The Organization of the Book 12

1.4 Changes from First Edition 14

1.5 References 15

PRTools Introduction 17

2.1 Motivation 17

2.2 Essential Concepts 18

2.3 PRTools Organization Structure and Implementation 22

2.4 Some Details about PRTools 26

2.5 Selected Bibliography 42

Detection and Classification 43

3.1 Bayesian Classification 46

3.2 Rejection 62

3.3 Detection:The Two-Class Case 66

3.4 Selected Bibliography 74

Exercises 74

Parameter Estimation 77

4.1 Bayesian Estimation 79

4.2 Performance Estimators 94

4.3 Data Fitting 100

4.4 Overview of the Family of Estimators 110

4.5 Selected Bibliography 111

Exercises 112

State Estimation 115

5.1 A General Framework for Online Estimation 117

5.2 Infinite Discrete-Time State Variables 125

5.3 Finite Discrete-Time State Variables 147

5.4 Mixed States and the Particle Filter 163

5.5 Genetic State Estimation 170

5.6 State Estimation in Practice 183

5.7 Selected Bibliography 201

Exercises 204

Supervised Learning 207

6.1 Training Sets 208

6.2 Parametric Learning 210

6.3 Non-parametric Learning 217

6.4 Adaptive Boosting - Adaboost 245

6.5 Convolutional Neural Networks (CNNs) 249

6.6 Empirical Evaluation 252

6.7 Selected Bibliography 257

Exercises 257

Feature Extraction and Selection 259

7.1 Criteria for Selection and Extraction 261

7.2 Feature Selection 272

7.3 Linear Feature Extraction 288

7.4 References 300

Exercises 300

Unsupervised Learning 303

8.1 Feature Reduction 304

8.2 Clustering 320

8.3 References 345

Exercises 346

Worked Out Examples 349

9.1 Example on Image Classification with PRTools 349

9.2 Boston Housing Classification Problem 361

9.3 Time-of-Flight Estimation of an Acoustic Tone Burst 372

9.4 Online Level Estimation in a Hydraulic System 392

9.5 References 406

Appendix A: Topics Selected from Functional Analysis 407

Appendix B: Topics Selected from Linear Algebra and Matrix Theory 421

Appendix C: Probability Theory 437

Appendix D: Discrete-Time Dynamic Systems 453

Index 459

Textauszug

1
Introduction

Engineering disciplines are those fields of research and development that attempt to create products and systems operating in, and dealing with, the real world. The number of disciplines is large, as is the range of scales that they typically operate in: from the very small scale of nanotechnology up to very large scales that span whole regions, for example water management systems, electric power distribution systems or even global systems (e.g. the global positioning system, GPS). The level of advancement in the fields also varies wildly, from emerging techniques (again, nanotechnology) to trusted techniques that have been applied for centuries (architecture, hydraulic works). Nonetheless, the disciplines share one important aspect: engineering aims at designing and manufacturing systems that interface with the world around them.

Systems designed by engineers are often meant to influence their environment: to manipulate it, to move it, to stabilize it, to please it, and so on. To enable such actuation, these systems need information, for example values of physical quantities describing their environments and possibly also describing themselves. Two types of information sources are available: prior knowledge and empirical knowledge. The latter is knowledge obtained by sensorial observation. Prior knowledge is the knowledge that was already there before a given observation became available (this does not imply that prior knowledge is obtained without any observation). The combination of prior knowledge and empirical knowledge leads to posterior knowledge.

The sensory subsystem of a system produces measurement signals. These signals carry the empirical knowledge. Often, the direct usage of these signals is not possible, or is inefficient. This can have several causes:

• The information in the signals is not represented in an explicit way. It is often hidden and only available in an indirect, encoded, form.
• Measurement signals always come with noise and other hard-to-predict disturbances.
• The information brought forth by posterior knowledge is more accurate and more complete than information brought forth by empirical knowledge alone. Hence, measurement signals should be used in combination with prior knowledge.

Measurement signals need processing in order to suppress the noise and to disclose the information required for the task at hand.

1.1 The Scope of the Book

In a sense, classification and estimation deal with the same problem: given the measurement signals from the environment, how can the information that is needed for a system to operate in the real world be inferred? In other words, how should the measurements from a sensory system be processed in order to bring maximal information in an explicit and usable form? This is the main topic of this book.

Good processing of the measurement signals is possible only if some knowledge and understanding of the environment and the sensory system is present. Modelling certain aspects of that environment – like objects, physical processes or events – is a necessary task for the engineer. However, straightforward modelling is not always possible. Although the physical sciences provide ever deeper insight into nature, some systems are still only partially understood; just think of the weather. Even if systems are well understood, modelling them exhaustively may be beyond our current capabilities (i.e. computer power) or beyond the scope of the application. In such cases, approximate general models, but adapted to the system at hand, can be applied. The development of such models is also a topic of this book.

1.1.1 Classification

The title of the book already indicates the three main subtopics it will cover: classification, parameter estimation and state estimation. In classification, one tries to assign a class label to an object, a physical process or an event. Figure 1.1 illustrates the concept. In a speeding detector, the sensors are a radar speed detector and a high-resolution camera, placed in a box beside a road. When the radar detects a car approaching at too high a velocity (a parameter estimation problem), the camera is signalled to acquire an image of the car. The system should then recognize the licence plate, so that the driver of the car can be fined for the speeding violation. The system should be robust to differences in car model, illumination, weather circumstances, etc., so some pre-processing is necessary: locating the licence plate in the image, segmenting the individual characters and converting it into a binary image. The problem then breaks down to a number of individual classification problems. For each of the locations on the license plate, the input consists of a binary image of a character, normalized for size, skew/rotation and intensity. The desired output is the label of the true character, that is one of ‘A’, ‘B’,…, ‘Z’, ‘0’,…, ‘9’.

Figure 1.1 Licence plate recognition: a classification problem with noisy measurements.

Detection is a special case of classification. Here, only two class labels are available, for example ‘yes’ and ‘no’. An example is a quality control system that approves the products of a manufacturer or refuses them. A second problem closely related to classification is identification: the act of proving that an object-under-test and a second object that is previously seen are the same. Usually, there is a large database of previously seen objects to choose from. An example is biometric identification, for example fingerprint recognition or face recognition. A third problem that can be solved by classification-like techniques is retrieval from a database, for example finding an image in an image database by specifying image features.

1.1.2 Parameter Estimation

In parameter estimation, one tries to derive a parametric description for an object, a physical process or an event. For example, in a beacon-based position measurement system (Figure 1.2), the goal is to find the position of an object, for example a ship or a mobile robot. In the two-dimensional case, two beacons with known reference positions suffice. The sensory system provides two measurements: the distances from the beacons to the object, r1 and r2. Since the position of the object involves two parameters, the estimation seems to boil down to solving two equations with two unknowns. However, the situation is more complex because measurements always come with uncertainties. Usually, the application not only requires an estimate of the parameters but also an assessment of the uncertainty of that estimate. The situation is even more complicated because some prior knowledge about the position must be used to resolve the ambiguity of the solution. The prior knowledge can also be used to reduce the uncertainty of the final estimate.

Figure 1.2 Position measurement: a parameter estimation problem handling uncertainties.

In order to improve the accuracy of the estimate the engineer can increase the number of (independent) measurements to obtain an overdetermined system of equations. In order to reduce the cost of the sensory system, the engineer can also decrease the number of measurements, leaving us with fewer measurements than parameters. The system of equations is then underdetermined, but estimation is still possible if enough prior knowledge exists or if the parameters are related to each other (possibly in a statistical sense). In either case, the engineer is interested in the uncertainty of the estimate.

1.1.3 State Estimation

In state estimation, one tries to do either of the following – either assigning a class label or deriving a parametric (real-valued) description – but for processes that vary in time or space. There is a fundamental difference between the problems of classification and parameter estimation, on the one hand, and state estimation, on the other hand. This is the ordering in time (or space) in state estimation, which is absent from classification and parameter estimation. When no ordering in the data is assumed, the data can be processed in any order. In time series, ordering in time is essential for the process. This results in a fundamental difference in the treatment of the data.

In the discrete case, the states have discrete values (classes or labels) that are usually drawn from a finite set. An example of such a set is the alarm stages in a safety system (e.g. ‘safe’, ‘pre-alarm’, ‘red alert’, etc.). Other examples of discrete state estimation are speech recognition, printed or handwritten text recognition and the recognition of the operating modes of a machine.

An example of real-valued state estimation is the water management system of a region. Using a few level sensors and an adequate dynamical model of the water system, a state estimator is able to assess the water levels even at locations without level sensors. Short-term prediction of the levels is also possible. Figure 1.3 gives a view of a simple water management system of a single canal consisting of three linearly connected compartments. The compartments are filled by the precipitation in the surroundings of the canal. This occurs randomly but with a seasonal influence. The canal drains its water into a...

Produkt-Details

• Erscheint lt. Verlag: 17.3.2017
• Sprache: englisch
• Themenwelt: Mathematik / Informatik ► Mathematik ► Angewandte Mathematik
• Themenwelt: Naturwissenschaften ► Physik / Astronomie ► Mechanik
• Themenwelt: Technik ► Elektrotechnik / Energietechnik
• Themenwelt: Technik ► Maschinenbau
• Schlagworte: AdaBoost • adaboost machine learning in intelligent computer vision • advanced computer vision theory • advances in intelligent computer vision • AI state estimation methods for intelligent computer vision • classification and supervised learning in intelligent computer vision • computer vision • computer vision basics • computer vision theory and practice • Electrical & Electronics Engineering • Elektrotechnik u. Elektronik • FPGA and video processing • genetic state estimation in intelligent computer vision • intelligent computer system design • intelligent computer systems design and implementation • intelligent computer vision • intelligent computer vision design examples • intelligent computer vision examples • intelligent computer vision system design • intelligent computer vision systems development • intelligent computer vision technology • latent support vector machines in intelligent computer vision • latent SVM for intelligent computer vision • machine learning • machine learning in computer vision systems • Mustererkennung • Neural Networks in intelligent computer vision • particle filtering • particle filtering methods in computer vision systems • Pattern Analysis • pattern recognition in intelligent computer vision systems • PRTools5 software for MATLAB • PRTools5 toolbox • Signal Processing • Signalverarbeitung • state-of-the-art computer vision technology • struct support vector machines in intelligent computer vision • struct SVM for intelligent computer vision • Support Vector Machines in intelligent computer vision • SVM for intelligent computer vision • video processing with field programmable gate arrays
• ISBN-10: 1-119-15245-3 / 1119152453
• ISBN-13: 978-1-119-15245-3 / 9781119152453