Systems Engineering Neural Networks (eBook)

Alessandro Migliaccio, Giovanni Iannone (Autoren)

eBook Download: EPUB

2023
John Wiley & Sons (Verlag)
9781119902010 (ISBN)

Infos

SYSTEMS ENGINEERING NEURAL NETWORKS

A complete and authoritative discussion of systems engineering and neural networks

In Systems Engineering Neural Networks, a team of distinguished researchers deliver a thorough exploration of the fundamental concepts underpinning the creation and improvement of neural networks with a systems engineering mindset. In the book, you'll find a general theoretical discussion of both systems engineering and neural networks accompanied by coverage of relevant and specific topics, from deep learning fundamentals to sport business applications.

Readers will discover in-depth examples derived from many years of engineering experience, a comprehensive glossary with links to further reading, and supplementary online content. The authors have also included a variety of applications programmed in both Python 3 and Microsoft Excel.

The book provides:

• A thorough introduction to neural networks, introduced as key element of complex systems
• Practical discussions of systems engineering and forecasting, complexity theory and optimization and how these techniques can be used to support applications outside of the traditional AI domains
• Comprehensive explorations of input and output, hidden layers, and bias in neural networks, as well as activation functions, cost functions, and back-propagation
• Guidelines for software development incorporating neural networks with a systems engineering methodology

Perfect for students and professionals eager to incorporate machine learning techniques into their products and processes, Systems Engineering Neural Networks will also earn a place in the libraries of managers and researchers working in areas involving neural networks.

Alessandro Migliaccio is a certified systems engineer and member of the INCOSE Artificial Intelligence Working Group. He is a graduate of the Delft University of Technology in Space Engineering, USA, and has second level master's degree in Robotics and Intelligent Systems.

Giovanni Iannone is a mechanical engineer and a graduate of the University of Naples Federico II. Second level master's degree in Systems Engineering at Missouri University of Science and Technology, USA. He has been an active member of INCOSE for several years.

Inhalt

ABOUT THE AUTHORS

ACKNOWLEDGEMENTS 7

HOW TO READ THIS BOOK 8

Part I 9

1 A BRIEF INTRODUCTION 9

THE SYSTEMS ENGINEERING APPROACH TO ARTIFICIAL INTELLIGENCE (AI) 14

SOURCES 18

CHAPTER SUMMARY 18

QUESTIONS 19

2 DEFINING A NEURAL NETWORK 20

BIOLOGICAL NETWORKS 22

FROM BIOLOGY TO MATHEMATICS 24

WE CAME A FULL CIRCLE 25

THE MODEL OF McCULLOCH-PITTS 25

THE ARTIFICIAL NEURON OF ROSENBLATT 26

FINAL REMARKS 33

SOURCES 35

CHAPTER SUMMARY 36

QUESTIONS 37

3 ENGINEERING NEURAL NETWORKS 38

A BRIEF RECAP ON SYSTEMS ENGINEERING 40

THE KEYSTONE: SE4AI AND AI4SE 41

ENGINEERING COMPLEXITY 41

THE SPORT SYSTEM 45

ENGINEERING A SPORT CLUB 51

OPTIMISATION 52

AN EXAMPLE OF DECISION MAKING 56

FUTURISM AND FORESIGHT 60

QUALITATIVE TO QUANTITATIVE 61

FUZZY THINKING 64

IT IS ALL IN THE TOOLS 74

SOURCES 77

CHAPTER SUMMARY 77

QUESTIONS 78

Part II 79

4 SYSTEMS THINKING FOR SOFTWARE DEVELOPMENT 79

PROGRAMMING LANGUAGES 82

ONE MORE THING: SOFTWARE ENGINEERING 94

CHAPTER SUMMARY 101

QUESTIONS 102

SOURCES 102

5 PRACTICE MAKES PERFECT 103

EXAMPLE 1: COSINE FUNCTION 105

EXAMPLE 2: CORROSION ON A METAL STRUCTURE 112

EXAMPLE 3: DEFINING ROLES OF ATHLETES 127

EXAMPLE 4: ATHLETE'S PERFORMANCE 134

EXAMPLE 5: TEAM PERFORMANCE 142

A human-defined-system 142

Human Factors 143

The sport team as system of interest 144

Impact of Human Error on Sports Team Performance 145

EXAMPLE 6: TREND PREDICTION 156

EXAMPLE 7: SYMPLEX AND GAME THEORY 163

EXAMPLE 8: SORTING MACHINE FOR LEGO® BRICKS 168

Part III 174

6 INPUT/OUTPUT, HIDDEN LAYER AND BIAS 174

INPUT/OUTPUT 175

HIDDEN LAYER 180

BIAS 184

FINAL REMARKS 186

CHAPTER SUMMARY 187

QUESTIONS 188

7 ACTIVATION FUNCTION 189

TYPES OF ACTIVATION FUNCTIONS 191

ACTIVATION FUNCTION DERIVATIVES 194

ACTIVATION FUNCTIONS RESPONSE TO W AND b VARIABLES 200

FINAL REMARKS 202

CHAPTER SUMMARY 204

QUESTIONS 205

SOURCES 205

8 COST FUNCTION, BACK-PROPAGATION AND OTHER ITERATIVE METHODS 206

WHAT IS THE DIFFERENCE BETWEEN LOSS AND COST? 209

TRAINING THE NEURAL NETWORK 212

BACK-PROPAGATION (BP) 214

ONE MORE THING: GRADIENT METHOD AND CONJUGATE GRADIENT METHOD 218

ONE MORE THING: NEWTON'S METHOD 221

CHAPTER SUMMARY 223

QUESTIONS 224

SOURCES 224

9 CONCLUSIONS AND FUTURE DEVELOPMENTS 225

GLOSSARY AND INSIGHTS 233

Textauszug

1
A Brief Introduction

I see it all perfectly; there are two possible situations – one can either do this or that. My honest opinion and my friendly advice is this: do it or do not do it – you will regret both.

Søren Kierkegaard

From the Ancient Greeks through the Renaissance, and until our present day, human beings have always tried to give meaning to the reality surrounding them. This effort was not based on tradition or myth, but on the human rational ability to describe reality through the laws of mathematics.

When writing a scientific text and trying to give reality a meaning by applying a mathematical model, we cannot ignore certain philosophical concepts. On the contrary, we must find inspiration in the opinions of the great thinkers of the past. We will only examine a few postulates, but you can rest assured that many more are available and extensively explained in the literature. We take advantage of the knowledge that was made available to us by such human talents.

Our calculus teachers would never stop saying that numbers have to be interpreted, understood, and explained organically. Thanks to numbers we can define an object, an event26, or a physical phenomenon. Why is there a need to interpret numbers?

According to Pythagoras, numbers are the primordial elements from which reality is derived. The latter can be inferred through a strict mathematical and geometric sequence. The qualitative and contemplative elements coexist with the quantitative one. Each number is associated with a shape containing elements which allows them to stay together in a harmonized and neat manner. Therefore, if we base our interpretation of the world on its numerical and harmonious nature, we can come to understand it starting from its measurements. Are numbers all we need to understand the world? What is your idea of the reality surrounding us?

It is not easy to have an idea and expand upon it – we could find it difficult, for example, to distinguish true from false and zero from one. Once its traits are defined – zero or one, true or false – an idea is absolute and unalterable, therefore we can associate it to reality.

To paraphrase the words of Plato and inferring his theories from his dialogs – we apologize in advance to our fellow philosophers and teachers of philosophy – the Idea exists outside of our mind. It is detectable only by our intellect and not by our perception, the latter being not sufficient to understand reality. We can simplify by saying that the idea is similar to a standard of judgment. We have access to the knowledge of things only if we have ideas (See Figure 1.1).

Ideas act as measurements to evaluate the tangible reality, and they do not reside in our imagination. The idea, as an objective entity, is not to be confused with opinion, which is instead subjective. As we know in physics and mathematics, we have to measure the phenomena we observe daily with certainty and precision. The idea is a model (or archetype) correlated to our empirical world – we should only try to imitate or duplicate this model. We will see later on how this model acts as an absolute reference for our implementation.

Figure 1.1 The human being becomes the standard for all things.

If we wanted to make a measurement of a particular event and then attribute to that event a meaning, we could start with the concept of an idea, in an absolute sense, as an essential reference. On the other hand, if the measurement is associated with a judgment, then we cannot know whether there is an absolute rule to discriminate that event. Surely we can trace back through experience to the general rule governing an event.

Therefore, as we have mentioned earlier in this chapter, human beings have always felt the need to explain the worldly reality they live in and, might we add, it could not be any other way. Man's senses can be accepted as an important source of knowledge.

However, would we as human beings be able to understand the world based on rules established by our rationality? The Vitruvian Man, famous work by Leonardo da Vinci, conveys a model of a human body that is analyzed and measured through mathematical and geometric tools. The human being becomes the standard for all things, therefore humanity as a whole gains full awareness. Man2, put at the center of the world, becomes the symbol of a better future.

We now have all the elements to start writing about mathematical models, which can also be referred to as rational and variable structures, integrated in logical processes. These structures are based on the concepts of Number, Idea and Human, which we have introduced earlier.

The ultimate meaning of an event is seen as a reliable reference, and we aim toward it. A systematic and methodical approach to the analysis of an event – as we will see in Chapter 3 – will help us interpret it. Its modeling can take us to more reliable, though not absolute, conclusions.

When applied to varied events in our lives, the use of mathematical models can help the reader to better interpret certain dynamics that are part of our daily life. The examples in this book are relevant to those aspects that are often difficult to decipher due to their complex nature. Processes such as decisional ones can be understood via computational models used in the examples provided.

The German physicist W.K. Heisenberg affirmed that concepts of probability apply to all cognitive processes. Human beings would not be able to reach a perfect understanding of a physical phenomenon because the observer is not able to determine how they are interfering with the observed object.

To build a mathematical model we need to define a prerequisite that establishes its efficacy: this is uncertainty (Figure 1.2). Uncertainty obviously plays a vital role in the development of complex models, as they require a reliable mathematical form to describe a real problem. We would be ignoring our reality, and our understanding of it, if we ignored the element of uncertainty. It is easily understandable that closed-form expressions offer a description of simple and less uncertain realities, which is more accurate.

But what if reality is instead complex, uncertain, and difficult to interpret?

Figure 1.2 Number, Idea, Human, Uncertainty…what else?

Now, this is our objective: let us develop a tool that operates through estimates and aims at minimizing the possibility of error so that we can rely on a result as close to objectivity as possible.

How can we achieve that?

Below are two images, at first glance remarkably similar, depicting two different animals: a cheetah and a leopard (Figure 1.3). Would you be able to tell them apart and say which is which? How can we define the two animals if we do not have an extensive knowledge of zoology? The obvious solution, not to be dismissed from the start, would be to ask an expert and have him explain to us how to tell them apart. We can obtain an accurate solution to the problem if we make use of specific knowledge to define a series of characteristics. The solution, as a result of data processing, will be more accurate if we can count on all the applicable variables of the problem and if we can relate these to the characteristics of the animal depicted below.

Figure 1.3 Cheetah or Leopard? Could you tell them apart?

Source: Image by Jonathan Reichel from Pixabay.

The approach we take in this book is a common application of linear and non-linear algebraic combinations – these somehow describe the various interactions happening in our brain when it is prompted to find a solution to a problem. The individual outcome of these efforts can be defined as the union of elements and variables working together to perform a specific function.

The reader will find a brief description of neural networks, as a calculation methodology, and some cases, taking the opportunity at the same time to exemplify the “system approach” used throughout the book. It is important to clarify that the learning phase is essential for the neural network to work as expected. A network can detect the trends in the data regardless of how they fluctuate, based on the exact behavior of all the involved variables. The scope of machine learning3 as a discipline is to find a correlation between historic data and present (and future) data; when the correlation is found, the network detects it and uses it as the basis of its prediction32.

Even a distracted reader can see that predictions are valid only if future data trends align with past ones. In other words, if we think of an unchangeable, static and frozen reality, we could set in stone all the algorithms that work successfully. Our reality is constantly changing, but we have to start somewhere – won't you agree? Therefore, let us build the basis of our models and then create an algorithm that can guarantee a certain degree of accuracy in analyzing specific dynamics, and also reduce uncertainty34 as much as possible. In fact, the continuous changes of events can be managed by adapting and improving the algorithms.

Let us take the human brain as a reference to explain this better – after all, we have always been told that our brain is the biggest computer ever created.

Pieces of information move inside a neural circuit of two or more neurons, thanks to a communication process based on chemicals and electrical impulses that is repeated billions of times. Any piece...

Produkt-Details

• Erscheint lt. Verlag: 10.1.2023
• Sprache: englisch
• Themenwelt: Technik ► Elektrotechnik / Energietechnik
• Schlagworte: activation functions • AI • AI bias • Artificial Intelligence • artificial neurons • back-propagation • Bias in AI • Bias in Artificial Intelligence • Communication Technology - Networks • Complexity theory • cost functions • Deep learning • Electrical & Electronics Engineering • Elektrotechnik u. Elektronik • hidden layers • Kommunikationsnetze • machine learning • Neural networks • Neuronale Netze • optimization parameters • software development • Systems Engineering & Management • Systemtechnik u. -management
• ISBN-13: 9781119902010 / 9781119902010