Open Source Intelligence Investigation (eBook)
XIV, 304 Seiten
Springer International Publishing (Verlag)
978-3-319-47671-1 (ISBN)
One of the most important aspects for a successful police operation is the ability for the police to obtain timely, reliable and actionable intelligence related to the investigation or incident at hand. Open Source Intelligence (OSINT) provides an invaluable avenue to access and collect such information in addition to traditional investigative techniques and information sources. This book offers an authoritative and accessible guide on how to conduct Open Source Intelligence investigations from data collection to analysis to the design and vetting of OSINT tools. In its pages the reader will find a comprehensive view into the newest methods for OSINT analytics and visualizations in combination with real-life case studies to showcase the application as well as the challenges of OSINT investigations across domains. Examples of OSINT range from information posted on social media as one of the most openly available means of accessing and gathering Open Source Intelligence to location data, OSINT obtained from the darkweb to combinations of OSINT with real-time analytical capabilities and closed sources. In addition it provides guidance on legal and ethical considerations making it relevant reading for practitioners as well as academics and students with a view to obtain thorough, first-hand knowledge from serving experts in the field.
Babak Akhgar is Professor of Informatics and Director of CENTRIC (Centre of excellence in terrorism, resilience, intelligence and organised crime research). Babak has more than 100 referred publications on security, OSINT, counter terrorism and cybercrime. He is principal investigator and technical lead in several multimillion Euros international security initiatives. He has co-edited two books on intelligence management - Intelligence Management: Knowledge Driven Frameworks for Combating Terrorism and Organised Crime, and Strategic Intelligence Management: National Security Imperatives and Information and Communications Technologies. Prof Akhgar latest books are titled Cyber Crime and Cyber Terrorism, An investigators handbook and Application of Big Data for National Security are published by Elsevier. In addition to his scholarly activities; Prof Akhgar is trustee of Police National Legal Database (PNLD), board member of European Organisation for Security and member of SAS UK academic board.
Petra Saskia Bayerl is Associate Professor of Technology and Organizational Behaviour at Rotterdam School of Management, Erasmus University, Netherlands and Program Director of Technology at the Centre of Excellence in Public Safety Management (CESAM). She is also Visiting Research Fellow at the Centre of Excellence for Terrorism, Resilience, Intelligence and Organised Crime Research (CENTRIC) at Sheffield Hallam University, U.K. She holds master degrees in psychology, linguistics and organizational dynamics from universities in Germany and the USA and a Ph.D. from Technical University in Delft, Netherlands. Her current research interests lay at the intersection of human-computer interaction, organizational communication, and organizational change with a special focus on the impact of technological innovations and public safety.
Fraser Sampson is Chief Executive and Solicitor of the Office of Police and Crime Commissioner West Yorkshire Police. Before taking up the role he was Chief Executive and Solicitor of the West Yorkshire Police Authority and in 2008 he was the first Executive Director of the Civil Nuclear Police Authority created by the Energy Act 2004. While in commercial legal practice Fraser represented police officers in a number of high profile cases arising out of employment and disciplinary matters and acted for Humberside Police officers in the Bichard Inquiry. Fraser has written and edited a number of leading books on police law including the best selling Blackstone's Police Manuals and the Routledge Companion to UK Counter-Terrorism (2012). He is on the editorial board of the Oxford Journal Policing: a journal of policy and practice. He is an advisor to the board at the Centre for Criminal Justice Studies at the University of Leeds and is an associate of the Scottish Institute for Policing Research. Fraser is a CEDR trained mediator and a Weinstein Fellow of the JAMS Foundation for Mediation in California.
Babak Akhgar is Professor of Informatics and Director of CENTRIC (Centre of excellence in terrorism, resilience, intelligence and organised crime research). Babak has more than 100 referred publications on security, OSINT, counter terrorism and cybercrime. He is principal investigator and technical lead in several multimillion Euros international security initiatives. He has co-edited two books on intelligence management – Intelligence Management: Knowledge Driven Frameworks for Combating Terrorism and Organised Crime, and Strategic Intelligence Management: National Security Imperatives and Information and Communications Technologies. Prof Akhgar latest books are titled Cyber Crime and Cyber Terrorism, An investigators handbook and Application of Big Data for National Security are published by Elsevier. In addition to his scholarly activities; Prof Akhgar is trustee of Police National Legal Database (PNLD), board member of European Organisation for Security and member of SAS UK academic board.Petra Saskia Bayerl is Associate Professor of Technology and Organizational Behaviour at Rotterdam School of Management, Erasmus University, Netherlands and Program Director of Technology at the Centre of Excellence in Public Safety Management (CESAM). She is also Visiting Research Fellow at the Centre of Excellence for Terrorism, Resilience, Intelligence and Organised Crime Research (CENTRIC) at Sheffield Hallam University, U.K. She holds master degrees in psychology, linguistics and organizational dynamics from universities in Germany and the USA and a Ph.D. from Technical University in Delft, Netherlands. Her current research interests lay at the intersection of human-computer interaction, organizational communication, and organizational change with a special focus on the impact of technological innovations and public safety.Fraser Sampson is Chief Executive and Solicitor of the Office of Police and Crime Commissioner West Yorkshire Police. Before taking up the role he was Chief Executive and Solicitor of the West Yorkshire Police Authority and in 2008 he was the first Executive Director of the Civil Nuclear Police Authority created by the Energy Act 2004. While in commercial legal practice Fraser represented police officers in a number of high profile cases arising out of employment and disciplinary matters and acted for Humberside Police officers in the Bichard Inquiry. Fraser has written and edited a number of leading books on police law including the best selling Blackstone’s Police Manuals and the Routledge Companion to UK Counter-Terrorism (2012). He is on the editorial board of the Oxford Journal Policing: a journal of policy and practice. He is an advisor to the board at the Centre for Criminal Justice Studies at the University of Leeds and is an associate of the Scottish Institute for Policing Research. Fraser is a CEDR trained mediator and a Weinstein Fellow of the JAMS Foundation for Mediation in California.
Preface 6
Acknowledgements 8
Contents 9
Editors and Contributors 11
Introduction 15
1 OSINT as an Integral Part of the National Security Apparatus 16
Abstract 16
1.1 Introduction 16
1.2 OSINT and Counter Terrorism Strategy 17
1.3 The CENTRIC OSINT Hub 21
1.4 Concluding Remarks 22
References 22
2 Open Source Intelligence and the Protection of National Security 23
Abstract 23
2.1 Introduction 23
2.2 From Threat to Threat 24
2.3 Online Radicalisation 26
2.4 Counter Measures 28
2.5 Conclusions 30
References 31
3 Police Use of Open Source Intelligence: The Longer Arm of Law 32
Abstract 32
3.1 Introduction 32
3.2 Understanding Intelligence in Policing 33
3.3 Intelligence Collection Disciplines 35
3.4 Characteristics of Open Source Intelligence 35
3.5 Modelling Open Source Intelligence 39
3.6 Conclusions 41
References 42
4 OSINT as Part of the Strategic National Security Landscape 43
Abstract 43
4.1 Introduction 43
4.2 Understanding the Strategic Landscape into Which OSINT Must Be Applied 44
4.3 Understanding the Intelligence Cycle in Which OSINT Must Exist and the Wider Intelligence Mix in Which It Must Integrate 47
4.3.1 Understanding the Application of OSINT in Operational Decision Making 52
4.3.2 UK Government Intelligence: Its Nature, Collection, Assessment and Use 53
4.4 How Might an Overarching Information Governance Architecture Support OSINT for Decision Making Within the Wider Intelligence Mix and Cycle? 58
4.5 Summary 63
References 64
5 Taking Stock of Subjective Narratives Surrounding Modern OSINT 66
Abstract 66
5.1 Introduction 66
5.2 Contextual Background 67
5.3 Lack of Public Clarity 68
5.4 Opposing Narratives 69
5.5 Independent Reviews 71
5.6 Conclusion 72
References 73
Methods, Tools and Techiques 75
6 Acquisition and Preparation of Data for OSINT Investigations 76
Abstract 76
6.1 Introduction 76
6.2 Reasons and Strategies for Data Collection 78
6.3 Data Types and Sources 80
6.3.1 Structured and Unstructured Data 80
6.3.2 Where and How to Obtain Open Source Data 80
6.3.2.1 Supporting Manual Searches 81
6.3.2.2 Web Crawling and Spiders 81
6.3.2.3 Web Metadata 83
6.3.2.4 APIs 83
6.3.2.5 Open Data 84
6.3.2.6 Social Media 84
6.3.2.7 Traditional Media 87
6.3.2.8 RSS 87
6.3.2.9 Grey Literature 88
6.3.2.10 Paid Data and Consented Data 88
6.3.2.11 Data on the Deep and Dark Web 89
6.4 Information Extraction 90
6.4.1 Natural Language Processing 90
6.4.1.1 Main Body Extraction 91
6.4.1.2 Entity Extraction 93
6.4.2 Modelling 94
6.4.2.1 Entity Relation Modelling 94
6.4.3 Feedback Loops 94
6.4.4 Validation Processes 95
6.4.5 Disinformation and Malicious Intent 95
6.4.6 Software Tools for Data Collection and Preparation 96
6.5 Privacy and Ethical Issues 97
6.5.1 Privacy by Design 97
6.5.2 Being Polite Online 98
6.5.2.1 Monitor Web Crawls and Respecting robots.txt 98
6.5.2.2 Keeping to API Limits 98
6.6 Conclusion 99
References 99
7 Analysis, Interpretation and Validation of Open Source Data 101
Abstract 101
7.1 Introduction 101
7.2 Types of Data Analysis 102
7.2.1 Textual Analysis 102
7.2.1.1 Text Processing 102
7.2.1.2 Word Sense Disambiguation 103
7.2.1.3 Sentiment Analysis 104
7.2.2 Aggregation 105
7.2.2.1 Document Clustering 105
7.2.3 Connecting the Dots 106
7.2.3.1 Network Analysis 107
7.2.3.2 Co-occurrence Networks 108
7.3 Location Resolution 109
7.3.1 Geocoding 110
7.3.2 Reverse Geocoding 110
7.4 Validating Open Source Information 111
7.4.1 Methods for Assigning Priority 112
7.4.2 Approaches for Recognising Credibility 113
7.4.3 Methods for Identifying Corroboration 114
7.5 Conclusion 114
References 115
8 OSINT and the Dark Web 117
Abstract 117
8.1 Introduction 117
8.2 Dark Web 120
8.2.1 Darknets on the Dark Web 120
8.2.2 Dark Web Size 124
8.2.3 Dark Web Content 124
8.3 OSINT on the Dark Web 126
8.3.1 Landscape of Dark Web Activities of Investigative Interest 126
8.3.2 Challenges Faced by LEAs on the Dark Web 128
8.4 OSINT Techniques on the Dark Web 129
8.4.1 Crawling 130
8.4.2 Search Engines 131
8.4.3 Traffic Analysis and de-Anonymization 132
8.5 Case Study: HME-Related Information on the Dark Web 133
8.5.1 Methodology 134
8.5.2 Experimental Evaluation 135
8.6 Conclusions 136
References 137
9 Fusion of OSINT and Non-OSINT Data 139
Abstract 139
9.1 Introduction 139
9.2 OSINT Data 140
9.2.1 Geographical Data 140
9.2.2 Statistical Data 141
9.2.3 Electoral Register 141
9.2.4 Court Records 142
9.2.5 Social Media 142
9.2.6 Blogging Platforms 142
9.2.7 Search Engines 143
9.2.8 Internet Archive 144
9.2.9 Freedom of Information 144
9.3 Non-OSINT Data 144
9.3.1 Criminal Records 145
9.3.2 Financial Records 146
9.3.3 Telecommunication Records 147
9.3.4 Medical Records 148
9.3.5 Imagery, Sensors and Video Data 149
9.4 Fusion Opportunities 149
9.4.1 Targeted Search 150
9.4.2 Validation of Other ‘INTs’ 150
9.4.3 Filling in the Missing Links 150
9.4.3.1 Identity Matching 151
9.4.3.2 Enhanced Social Network Creation 152
9.4.4 Environmental Scanning 153
9.4.5 Predictive Policing 154
9.4.6 Situational Awareness During Major Events 155
9.4.7 Identification and Tracking of Foreign Fighters 156
9.4.8 Child Sexual Exploitation 156
9.5 Conclusions 157
References 157
10 Tools for OSINT-Based Investigations 159
Abstract 159
10.1 Introduction 159
10.1.1 Effective Cyber-Risk Management 160
10.2 Key Assessment Themes 161
10.2.1 Security 161
10.2.1.1 Privacy 161
10.2.1.2 Protecting Against Malware 162
10.2.1.3 Unnecessary Bundled Software 162
10.2.1.4 Cloud-Based Services 162
10.2.2 Reliability 163
10.2.2.1 Code Quality 163
10.2.2.2 Open Formats and Standards 163
10.2.3 Legality 164
10.2.3.1 Licensing 164
10.2.3.2 Authorities 165
10.3 Completing a Tool Review 165
10.4 Assessment Framework 166
10.4.1 Document Information 167
10.4.2 Supplier Assessment 168
10.4.3 External Assessments 168
10.4.4 Practitioner’s Assessment 169
10.5 Conclusion 170
References 171
11 Fluidity and Rigour: Addressing the Design Considerations for OSINT Tools and Processes 172
Abstract 172
11.1 Introduction 172
11.2 Intelligence Analysis 175
11.3 What Do We Design? 177
11.4 Designing for Fluidity and Rigour 180
11.4.1 Fluidity as a Design Concept for OSINT Investigations 182
11.4.2 Rigour as a Design Concept for OSINT Investigations 184
11.5 Conclusions: Guidance for Designing Analysts’ Tools 187
Acknowledgments 188
References 188
Pratical Application and Cases 191
12 A New Age of Open Source Investigation: International Examples 192
Abstract 192
12.1 Introduction 192
12.2 Conclusion 198
References 199
13 Use Cases and Best Practices for LEAs 200
Abstract 200
13.1 Introduction 200
13.2 OSINT in an Increasingly Digital World 201
13.3 OSINT Best Practices for LEAs 203
13.3.1 Absolutes 203
13.3.2 Exploitables 203
13.3.3 Information Auditing 205
13.3.4 Strategic Data Acquisition 205
13.3.5 OSINT Pitfalls 206
13.3.5.1 Leakage 206
13.3.5.2 Anonymization 206
13.3.5.3 Crowd-Sourcing and Vigilantism 207
13.3.5.4 Corrupting the Chain of Evidence 207
13.3.5.5 Source Validation 208
13.4 LEA Usage of OSINT in Investigations: Case Examples 208
13.4.1 Exploiting Friendships in an Armed Robbery Case 208
13.4.2 Locating Wanted People Through Social Media 209
13.4.3 Locating a Sex Offender 210
13.4.4 Proactive Investigation Following a Terrorist Attack 211
13.5 Going Undercover on Social Media 212
13.6 Conclusions 212
References 213
14 OSINT in the Context of Cyber-Security 215
Abstract 215
14.1 Introduction 215
14.2 The Importance of OSINT with a View on Cyber Security 218
14.3 Cyber Threats: Terminology and Classification 219
14.4 Cyber-Crime Investigations 220
14.4.1 Approaches, Methods and Techniques 220
14.4.2 Detection and Prevention of Cyber Threats 223
14.5 Conclusions 229
References 229
15 Combatting Cybercrime and Sexual Exploitation of Children: An Open Source Toolkit 234
Abstract 234
15.1 Introduction 234
15.2 The Extended Impact of Cybercrime 235
15.3 Tools for Law Enforcement 237
15.4 The Role of OSINT 238
15.5 The UINFC2 Approach 240
15.5.1 Citizen Reporting Form 240
15.5.2 LEA/HOTLINE UINFC2 Platform 242
15.6 Concluding Remarks 248
Acknowledgments 249
References 249
16 Identifying Illegal Cartel Activities from Open Sources 251
Abstract 251
16.1 Introduction 252
16.2 The Principles 254
16.2.1 The Definition of a Cartel 254
16.2.2 The Sources of Information 255
16.2.2.1 Government Procurement Records 257
16.2.2.2 Company Registry 257
16.2.2.3 Legal Databases 257
16.2.2.4 Other Open-Source Intelligence (OSINT) sources 258
16.2.3 Cartel Patterns 258
16.2.4 Security Models 260
16.2.4.1 Negative Security Models and Supervised Learning 260
16.2.4.2 Positive Security Models and Unsupervised Learning 261
16.3 Data Acquisition from Open Sources 261
16.3.1 The Architecture 261
16.3.2 Entity Extraction 262
16.3.3 Filtering Out Suspicious Items in the Fusion Centre 262
16.3.4 Feature Engineering 264
16.3.5 Fitted Parameters of Economic Models 265
16.3.6 Network Science and Visualization 265
16.4 Machine Learning Methodologies 266
16.4.1 Evaluation of Predictive Methods 267
16.4.2 Logistic Regression 268
16.4.3 Decision Trees 269
16.4.4 Boosting 269
16.5 Conclusion and Further Work 270
References 271
Legal Considerations 274
17 Legal Considerations for Using Open Source Intelligence in the Context of Cybercrime and Cyberterrorism 275
Abstract 275
17.1 Introduction 275
17.2 Citizens’ Perceptions and Human Rights 276
17.3 Investigatory Powers 277
17.3.1 Existing and Proposed Powers 278
17.3.2 (Un)Lawful Practices 279
17.4 Data Protection 280
17.4.1 The Legislation 280
17.4.2 Further Considerations 282
17.5 Data Acquisition 283
17.6 Rules of Evidence 283
17.6.1 Seizing Digital Evidence 284
17.7 Unused Material 284
17.8 Different Jurisdictions 285
17.9 Overcoming Problems 286
17.9.1 Europol 286
17.9.2 Joint Investigation Teams 286
17.9.3 Eurojust 287
17.9.4 CEPOL 287
17.9.5 Interpol 288
17.10 Summary 288
17.11 Conclusion 290
References 291
18 Following the Breadcrumbs: Using Open Source Intelligence as Evidence in Criminal Proceedings 293
Abstract 293
18.1 Introduction 293
18.2 What Is the Difference Between Intelligence and Evidence? 294
18.3 Practical Issues 296
18.4 Legal Framework 296
18.5 European Convention on Human Rights 297
18.6 Uses of OSINT as Evidence 299
18.7 Conclusion 300
References 300
| Erscheint lt. Verlag | 1.1.2017 |
|---|---|
| Reihe/Serie | Advanced Sciences and Technologies for Security Applications | Advanced Sciences and Technologies for Security Applications |
| Zusatzinfo | XIV, 304 p. 40 illus., 35 illus. in color. |
| Verlagsort | Cham |
| Sprache | englisch |
| Themenwelt | Mathematik / Informatik ► Informatik ► Datenbanken |
| Recht / Steuern ► EU / Internationales Recht | |
| Sozialwissenschaften ► Politik / Verwaltung | |
| Technik ► Bauwesen | |
| Schlagworte | cyber security • dark Web • Deep Web/Dark web • Homeland Security • Intelligence collected from publicly available sources • Intelligence derived from public information • National open source enterprise • Open source intelligence techniques • OSCINT • OSINT against organised crime • OSINT Investigation • security intelligence • Social Media Analytics • Terrorism and transnational crime |
| ISBN-10 | 3-319-47671-8 / 3319476718 |
| ISBN-13 | 978-3-319-47671-1 / 9783319476711 |
| Informationen gemäß Produktsicherheitsverordnung (GPSR) | |
| Haben Sie eine Frage zum Produkt? |
PDF (Wasserzeichen)DRM: Digitales Wasserzeichen
Dieses eBook enthält ein digitales Wasserzeichen und ist damit für Sie personalisiert. Bei einer missbräuchlichen Weitergabe des eBooks an Dritte ist eine Rückverfolgung an die Quelle möglich.
Dateiformat: PDF (Portable Document Format)
Mit einem festen Seitenlayout eignet sich die PDF besonders für Fachbücher mit Spalten, Tabellen und Abbildungen. Eine PDF kann auf fast allen Geräten angezeigt werden, ist aber für kleine Displays (Smartphone, eReader) nur eingeschränkt geeignet.
Systemvoraussetzungen:
PC/Mac: Mit einem PC oder Mac können Sie dieses eBook lesen. Sie benötigen dafür einen PDF-Viewer - z.B. den Adobe Reader oder Adobe Digital Editions.
eReader: Dieses eBook kann mit (fast) allen eBook-Readern gelesen werden. Mit dem amazon-Kindle ist es aber nicht kompatibel.
Smartphone/Tablet: Egal ob Apple oder Android, dieses eBook können Sie lesen. Sie benötigen dafür einen PDF-Viewer - z.B. die kostenlose Adobe Digital Editions-App.
Buying eBooks from abroad
For tax law reasons we can sell eBooks just within Germany and Switzerland. Regrettably we cannot fulfill eBook-orders from other countries.
aus dem Bereich
