Wiley Handbook of Cognition and Assessment (eBook)

Frameworks, Methodologies, and Applications

Jacqueline P. Leighton, Andre A. Rupp (Herausgeber)

eBook Download: PDF

2016
Wiley (Verlag)
978-1-118-95659-5 (ISBN)

Infos

This state-of-the-art resource brings together the most innovative scholars and thinkers in the field of testing to capture the changing conceptual, methodological, and applied landscape of cognitively-grounded educational assessments. 

• Offers a methodologically-rigorous review of cognitive and learning sciences models for testing purposes, as well as the latest statistical and technological know-how for designing, scoring, and interpreting results
• Written by an international team of contributors at the cutting-edge of cognitive psychology and educational measurement under the editorship of a research director at the Educational Testing Service and an esteemed professor of educational psychology at the University of Alberta as well as supported by an expert advisory board
• Covers conceptual frameworks, modern methodologies, and applied topics, in a style and at a level of technical detail that will appeal to a wide range of readers from both applied and scientific backgrounds
• Considers emerging topics in cognitively-grounded assessment, including applications of emerging socio-cognitive models, cognitive models for human and automated scoring, and various innovative virtual performance assessments

André A. Rupp is Research Director at Educational Testing Service (ETS) in Princeton, NJ, where he works with teams that conduct comprehensive evaluation work for mature and emerging automated scoring systems. His research has focused on applications of principled assessment design frameworks in innovative assessment contexts as well as translating the statistical complexities of diagnostic measurement models into practical guidelines for applied specialists. Through dissemination and professional development efforts he is deeply dedicated to helping interdisciplinary teams navigate the complicated trade-offs between scientific, educational, political, and financial drivers of decision-making in order to help shape best methodological practices for evidentiary reasoning for complex assessment design and deployment lifecycles. He is co-author of Diagnostic Measurement: Theory, Methods, and Applications (2010).

Jacqueline P. Leighton is Professor and Chair of Educational Psychology at the University of Alberta, Canada. She is past Director of the University of Alberta's Centre for Research in Applied Measurement and Evaluation (CRAME). As a registered psychologist with the College of Alberta Psychologists, her research is focused on measuring the cognitive and socio-emotional processes underlying learning and assessment outcomes, including cognitive diagnostic assessment and feedback delivery and uptake. She has published in a variety of educational measurement journals and is past editor of Educational Measurement: Issues and Practice. She is co-author of The Learning Sciences in Educational Assessment (2011) and Cognitive Diagnostic Assessment for Education: Theory and Applications (2007) and co-editor of The Nature of Reasoning (2004).

Inhalt

Title Page 5
Copyright Page 6
Contents 9
Notes on Contributors 11
Foreword 21
Acknowledgements 23
Chapter 1 Introduction to Handbook 25
Motivation for Handbook 25
Handbook Structure 29
Closing Words 34
Part I Frameworks 37
Chapter 2 The Role of Theories of Learning and Cognition in Assessment Design and Development 39
A Brief History of Evaluation Criteria for Theories of Learning and Cognition 41
Principled Assessment Design as an Evaluative Lens for Theories of Learning and Cognition 42
Principled Assessment Design Evaluation Criteria for Theories of Learning and Cognition 46
Interface of Evaluation Criteria with Validity 57
Conclusion 60
References 61
Chapter 3 Principled Approaches to Assessment Design, Development, and Implementation 65
Motivations for Principled Approaches 67
Overviews of Principled Approaches 69
Elements of Principled Approaches 75
Use of the Elements in Principled Approaches 84
Discussion 89
Acknowledgments 94
Notes 95
References 95
Chapter 4 Developing and Validating Cognitive Models in Assessment 99
Definition of Cognitive Models 99
Model Development and Validation Methods 101
Offline Evidence: Methods and Approaches 103
Conclusion 120
References 120
Chapter 5 An Integrative Framework for Construct Validity 126
Unified Framework for Construct Validity 128
Example: Test Form Development with Generated Items 135
Conclusion 145
References 145
Chapter 6 The Role of Cognitive Models in Automatic Item Generation 148
Introduction 148
The Relentless Demand for Test Items 149
Automatic Item Generation: A Three?Step Method 151
Identifying Content for Item Generation Using Cognitive Modeling 153
Evaluating the Cognitive Models Used in Automatic Item Generation 162
Summary and Directions for Future Research 165
Acknowledgments 167
References 167
Chapter 7 Social Models of Learning and Assessment 170
Introduction 170
Defining “Social” Models of Learning 171
Examples of Assessment Informed by Social Models of Learning 175
Example 1: Carbon: Transformations in Matter and Energy (Carbon TIME) 175
Example 2: Contingent Pedagogies 179
Example 3: Data Modeling 184
Comparing the Examples: Key Sources of Variation 188
Looking to the Future: Additional Considerations for Assessment from a Sociocultural Perspective 190
References 191
Chapter 8 Socio-emotional and Self-management Variables in Learning and Assessment 198
Introduction 198
Importance of Social, Emotional, and Self?Management Variables in School and in Life: Correlational Studies 200
A Social, Emotional, and Self?Management Framework 202
Role of Socio?Emotional, Attitudinal, and Self?Management Variables on Summative and Formative Test Scores 210
Assessing Socio-Emotional, Attitudinal, and Self-Management Variables 213
Conclusions and Future Directions 214
References 215
Chapter 9 Understanding and Improving Accessibility for Special Populations 222
Accessibility of Educational Assessments 223
Cognitive Models of Learning and Performance 229
Putting It All Together: Assessing Accessibility for Individual Students 242
Conclusion 245
References 246
Chapter 10 Automated Scoring with Validity in Mind 250
An Overview of Automated Scoring Applications 251
Automated Scoring with Validity in Mind 253
Design Decisions in Automated Scoring 255
Discussion 263
Acknowledgments 264
References 264
Part II Methodologies 271
Chapter 11 Explanatory Item Response Models 273
Explanatory Item Response Models: An Approach to Cognitive Assessment 273
EIRM Framework and Modeling Principles 275
Data Set and Research Questions 276
Software and Model Specification 278
Example 1: Test Construction and Item Generation 279
Example 2: Diagnostic Assessment 284
Example 3: Correction for Confounding 286
General Discussion 288
Acknowledgments 289
References 289
Chapter 12 Longitudinal Models for Repeated Measures Data 291
Reading Comprehension Data 294
Latent Growth Curve Models 295
Growth Mixture Models 305
Conclusions 315
References 317
Chapter 13 Diagnostic Classification Models 321
Technical Foundations for DCMs 323
Evaluating Cognitive Theories through DCM Criticism and Refinement 330
Design Considerations for Diagnostic Assessments 342
Concluding Remarks 346
References 348
Chapter 14 Bayesian Networks 352
Evidentiary Motivations and Logical Bases 353
Statistical Definition and Estimation 356
Building and Using a BN 368
Understanding Models and Results across Communities 371
Conclusion 373
References 373
Chapter 15 The Rule Space and Attribute Hierarchy Methods 378
Test Design with the RSM and the AHM 379
Statistical Pattern Classifications with the RSM and the AHM 382
Demonstration of RSM and AHM Using a Real Data Example 389
Summary and Future Directions 396
References 399
Chapter 16 Educational Data Mining and Learning Analytics 403
Emerging Complex Assessment and Learning Environments 406
Case Studies in Assessment of Cognition via EDM 408
Conclusions 415
References 416
Part III Applications 421
Chapter 17 Large-Scale Standards-Based Assessments of Educational Achievement 423
Challenges of Using Cognitive Models for Large-Scale Assessments 426
Examples of Best Practices from Current Testing Programs 429
Conclusions and Recommendations 443
References 448
Chapter 18 Educational Survey Assessments 451
Introduction 451
Building Cognitive Models 453
The Role of Cognitively-based Assessment in ESAs 455
Definition and Guiding Examples 457
Three Methodological Perspectives 460
Summary, Trends and Upcoming Development 466
References 467
Chapter 19 Professional Certification and Licensure Examinations 470
Overview of Certification and Licensure 470
Assessment Frameworks for Certification and Licensure 473
Meeting Practical Certification and Licensure Assessment Challenges 484
Conclusions 490
Notes 491
References 492
Chapter 20 The In-Task Assessment Framework for Behavioral Data 496
Introduction 496
Evidence-Centered Design 499
In-task Assessment Framework 501
Example I-TAF Instantiations 509
Leveraging the I-TAF Framework to Instantiate ECD Models 522
Discussion 526
Acknowledgments 527
References 528
Chapter 21 Digital Assessment Environments for Scientific Inquiry Practices 532
Foundations for the Design of the Inq-ITS System 534
Case Studies with Think?Aloud Components 540
The Inq-ITS System 544
Summary 550
References 551
Chapter 22 Assessing and Supporting Hard-to-Measure Constructs in Video Games 559
Hard-to-measure Constructs and Why They are Hard to Measure 562
Well-designed Games as Vehicles for Assessing and Supporting These Constructs 564
Evidence-centered Design and Stealth Assessment in Well-designed Games 565
Examples of Stealth Assessments of Hard-to-Measure Constructs in Two Games 569
Validation of In?Game Measures 579
Discussion 581
Acknowledgments 582
References 582
Chapter 23 Conversation-Based Assessment 587
Performance-Based and Conversation-Based Assessments 589
An Augmented Taxonomy for Assessment Tasks 591
An Illustration of Conversation-Based Assessments 594
Discussion 600
References 602
Chapter 24 Conclusion to Handbook 604
Starting Points for Reflection 604
Challenges for Cognition and Assessment 605
Final Words 610
References 611
Glossary 612
Index 627
EULA 648

Produkt-Details

• Erscheint lt. Verlag: 21.11.2016
• Reihe/Serie: Wiley Handbooks in Education
• Reihe/Serie: Wiley Handbooks in Education
• Reihe/Serie: Wiley Handbooks in Education
• Sprache: englisch
• Themenwelt: Geisteswissenschaften ► Psychologie ► Entwicklungspsychologie
• Themenwelt: Geisteswissenschaften ► Psychologie ► Pädagogische Psychologie
• Themenwelt: Sozialwissenschaften ► Pädagogik ► Allgemeines / Lexika
• Themenwelt: Sozialwissenschaften ► Soziologie ► Empirische Sozialforschung
• Schlagworte: Assessment • Assessment, Evaluation & Research (Higher Education) • Bildungswesen • Cognition • Cognitive Psychology • Design • Education • Educational assessment • educational measurement • Educational Testing • ETS • Framework • Handbook • Hochschulen / Qualitätskontrolle, Evaluierung • Hochschulen / Qualitätskontrolle, Evaluierung • Kognitive Psychologie • Learning Sciences • Measurement • Methodology • Psychologie • Psychology • Psychometrics • standardized tests • Statistics • Testing
• ISBN-10: 1-118-95659-1 / 1118956591
• ISBN-13: 978-1-118-95659-5 / 9781118956595