Department of Education Vote 6 To be Appropriated by Vote R 10,857,872,000 Statutory Amount R 739,000 Responsible MEC ... xPublic Service Code of Remuneration (CORE)
Report CopyRight/DMCA Form For : National Assessments Of Educational Achievement
Developing Tests,and Questionnaires,for a National. Assessment of,Educational,Achievement,National Assessments of Educational Achievement. Developing Tests,and Questionnaires,for a National. Assessment of,Educational,Achievement,Prue Anderson and George Morgan. Vincent Greaney and,Thomas Kellaghan Series Editors. 2008 The International Bank for Reconstruction and Development The World. 1818 H Street NW,Washington DC 20433,Telephone 202 473 1000. Internet www worldbank org,E mail feedback worldbank org. All rights reserved,1 2 3 4 11 10 09 08, This volume is a product of the staff of the International Bank for Reconstruction. and Development The World Bank The findings interpretations and conclu. sions expressed in this volume do not necessarily reflect the views of the Executive. Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this. work The boundaries colors denominations and other information shown on any. map in this work do not imply any judgement on the part of The World Bank. concerning the legal status of any territory or the endorsement or acceptance of. such boundaries,Rights and Permissions, The material in this publication is copyrighted Copying and or transmitting. portions or all of this work without permission may be a violation of applicable. law The International Bank for Reconstruction and Development The World. Bank encourages dissemination of its work and will normally grant permission to. reproduce portions of the work promptly, For permission to photocopy or reprint any part of this work please send a. request with complete information to the Copyright Clearance Center Inc 222. Rosewood Drive Danvers MA 01923 USA telephone 978 750 8400 fax. 978 750 4470 Internet www copyright com, All other queries on rights and licenses including subsidiary rights should be. addressed to the Office of the Publisher The World Bank 1818 H Street NW. Washington DC 20433 USA fax 202 522 2422 e mail pubrights world. Cover design Naylor Design Washington DC,ISBN 978 0 8213 7497 9. eISBN 978 0 8213 7498 6,DOI 10 1596 978 0 8213 7497 9. Library of Congress Cataloging in Publication Data. Anderson Prue 1954, Developing tests and questionnaires for a national assessment of educational. achievement Prue Anderson George Morgan, p cm National assessment of educational achievement volume 2. Includes bibliographical references and index, ISBN 978 0 8213 7497 9 alk paper ISBN 978 0 8213 7498 6. 1 Educational tests and measurements United States 2 Educational evalua. tion United States I Morgan George 1945 II Title,LB3051 A715 2008. 371 26 1 dc22,2008002684,PREFACE xi,ABOUT THE AUTHORS AND EDITORS xiii. ACKNOWLEDGMENTS xv,ABBREVIATIONS xvii,PART 1 CONSTRUCTING ACHIEVEMENT TESTS. CHAPTER 1 INTRODUCTION 3,CHAPTER 2 DEVELOPING AN ASSESSMENT FRAMEWORK 9. The Test Blueprint or Table of Specifications 10,Validity 16. Test Language 16,Item Format 17,Student Population to Be Assessed 24. Reporting Results 24,Contexts 25,CHAPTER 3 ITEM WRITING 27. Item Difficulty 29,Item Bias 30,Stimulus Material 30. Item Format 33,Practice Items 45,vi CONTENTS,Item Layout and Design 46. The Item Writing Team 52,Item Panels 55,Other Reviewers 57. Tracking Items 58,CHAPTER 4 PRETESTING ITEMS 61,Designing the Pretest Form 64. Printing and Proofreading the Pretest 68,Implementing the Pretest 70. Scoring the Pretest 71,Reliability 76,CHAPTER 5 SELECTING TEST ITEMS 79. CHAPTER 6 PRODUCING THE FINAL TEST 85,Designing the Final Test 85. Printing and Proofreading 88,CHAPTER 7 HAND SCORING TEST ITEMS 91. PART 2 CONSTRUCTING QUESTIONNAIRES 97,CHAPTER 8 DESIGNING QUESTIONNAIRES 99. Questionnaire Content 102,Questionnaire Blueprint 105. Questionnaire Items 105,Item Format 108,Language of the Questionnaire 109. Respondents 109,Questionnaire Administration 110,Data Analysis Plan 111. CHAPTER 9 ITEM WRITING FOR QUESTIONNAIRES 113,Questions 114. Statements 114,Response Categories 115,Managing Sensitive Issues 117. Questionnaire Layout 117,Reviewing Questionnaires 119. CHAPTER 10 CODING QUESTIONNAIRE RESPONSES 121,Preparing Questionnaires for Data Entry 123. Coding Missing or Ambiguous Responses 123,CONTENTS vii. CHAPTER 11 MATCHING QUESTIONNAIRES AND TEST DATA 125. Student Questionnaires 125,Parent Questionnaires 126. Teacher and Head Teacher Questionnaires 126,PART 3 DESIGNING A MANUAL FOR TEST. ADMINISTRATION 127,CHAPTER 12 THE TEST ADMINISTRATORS MANUAL 129. Contents of the Manual 130,Use of the Manual 131,Features of a Manual 132. How Much Detail Is Necessary 132,Practice Questions 134. Tryout 134,Review 137,CHAPTER 13 THE TEST ADMINISTRATOR 139. Choice of Test Administrator 139,Following Instructions 140. Quality Assurance 142,Administrator s Checklist 142. CHAPTER 14 INFORMING SCHOOLS ABOUT THE,NATIONAL ASSESSMENT 145. APPENDIX A GLOSSARY 147,APPENDIX B FURTHER READING 153. APPENDIX C EXAMPLES OF TEST AND QUESTIONNAIRE,ITEMS AND ADMINISTRATION MANUALS. 2 1 Papua New Guinea Mathematics Curriculum 11,2 2 New Zealand English Curriculum 11. 2 3 Examples of Multiple Choice Items 18, 2 4 Example of a Closed Constructed Response Item 19. 2 5 Examples of Open Ended Short Response Items 19. viii CONTENTS,2 6 Example of an Essay Prompt 20,3 1 Example of Irrelevant Stimulus Material 32. 3 2 Example of an Item with Inaccurate or,Misleading Information 33. 3 3 Example of a Multiple Choice Item 34,3 4 Punctuation in Complete Sentences 35. 3 5 Punctuation in a List 35,3 6 Minimizing Reading 36. 3 7 Item with a Negative Stem 36,3 8 Poorly Paired Distractors 37. 3 9 Dealing with Pairs in Distractors 37, 3 10 Confusing Open Ended Item with Unclear Directions 40. 3 11 Good Example of a Closed Constructed Response Item 41. 3 12 Item with Partial Credit 42,3 13 Example of an Open Ended Response Item. with Scoring Guide 43,3 14 Example of a Closed Constructed Item. with Scoring Guide 44,3 15 Using Pictures to Reduce Words 48. 3 16 Keeping Images Simple 49,3 17 Labeling Graphs Clearly 49. 3 18 Labeling Maps Clearly 50,3 19 Leaving Space in the Stimulus Material 51. 3 20 Example of Style Sheet for Item Writers 54,4 1 Example of an Item in Multiple Choice. and Open Ended Format 71, 4 2 Example of a Data Entry Sheet for a Pretest 73. 6 1 Example of a Test Cover Page 86, 8 1 Attitudes and Values Questionnaire Blueprint 106. 9 1 Poor Alignment of Boxes and Response Categories 118. 9 2 Better Alignment of Boxes and Response Categories 118. 10 1 Grayscale Coding Example 122, 10 2 Example Treating Items as Separate Categories. for Data Entry 122,12 1 Administration Manual Instructions 133. 12 2 Information for Teachers and Principals 134,12 3 Administration of Practice Items 135. 13 1 Administration Checklist An Example from the Philippines 143. CONTENTS ix,1 1 National Assessment Organizational Chart 4. 1 2 Overview of National Assessment Activities 5,4 1 Example of Circular Linking of Items 65. 4 2 Model for Vertically Linking Items 66,C 1 Guide to CD Materials on Tests Questionnaires. and Manuals 158, 1 1 National Assessment Stages in Test Development and. Questionnaire Design 6, 2 1 Blueprint for a Middle Primary Mathematics Test 13. 2 2 TIMSS Mathematics Blueprint Grades 3 and 4 14, 2 3 Papua New Guinea Mathematics Content Blueprint 15. 2 4 Advantages and Limitations of Item Formats 23, 2 5 Papua New Guinea Mathematics Test Item Formats 24. 4 1 Link Items in Two Reading Units 67, 4 2 Part of a Spreadsheet for Tracking Items across Forms 68. 5 1 Example of Output from Analysis of a,Multiple Choice Item 80. 5 2 Example of Output from Analysis of an,Open Ended Partial Credit Item 81. 8 1 Components of Questionnaire Development 101, 8 2 Functions of Reading in an International Study. Weights Used to Create Two New Variables,Reading for a Utilitarian Purpose and Reading. for Enjoyment 107, The quality of any educational assessment exercise depends on the. quality of the instruments used In fact if these instruments are poorly. designed the assessment can be a waste of time and money Developing. Tests and Questionnaires for a National Assessment of Educational. Achievement the second of five books in the National Assessments of. Educational Achievement series describes how to develop technically. robust instruments for a national assessment of educational achieve. ment with a particular focus on carrying out this task in developing. countries Volume 1 in the series describes the key purposes and fea. tures of national assessments and is mainly aimed at policy makers and. decision makers in education This second book and most of the subse. quent books in the series provide step by step details on the design. implementation analysis and reporting of a national assessment and. are intended primarily for national assessment teams. Developing Tests and Questionnaires for a National Assessment of. Educational Achievement addresses the design of two types of data. collection instruments student achievement tests and background. questionnaires Part 1 covers the development of an assessment. framework and a test blueprint item writing pretesting and final. test layout Part 2 delineates comparable stages and activities in the. construction of background questionnaires which are used to gather. information from students teachers head teachers or parents on vari. ables that might help explain differences in student performance on. xii PREFACE, the achievement test Part 3 describes how to design a manual for. test administration to help ensure that all students take the test un. der standardized conditions The compact disc CD that accompa. nies this book offers examples of well designed test items question. naire items and administration manuals drawn from national and. international assessments and is meant to showcase the variety of. ways in which assessment teams have approached the design of. these instruments, Volume 3 in the series focuses on practical issues to be addressed. in implementing a large scale national assessment program including. logistics sampling and data cleaning and management Volume 4. deals with how to generate data on items and test scores and how to. relate the test scores to other educational factors Finally volume 5. covers how to write reports that are based on the national assessment. findings and how to use the results to improve the quality of educa. tional policy making, As readers make their way through this volume it should become. evident that the development of assessment instruments is a complex. and time consuming exercise that requires considerable knowledge. skill and resources At the same time experience has shown that the. payoff from well designed instruments can be substantial in terms of. the quality of the information provided on levels of student achieve. ment and on school and nonschool factors that might help raise those. achievement levels Good quality instruments can increase the confi. dence of policy makers and other stakeholders in the findings They. also can increase the likelihood that policy makers will use the results. of the national assessment to develop sound plans and programs. designed to enhance educational quality If the test and questionnaire. results achieve these outcomes they will more than justify the time. and effort involved in their development,Marguerite Clarke. Senior Education Specialist,The World Bank,ABOUT THE AUTHORS. AND EDITORS, Prue Anderson is a senior research fellow at the Australian Council. for Educational Research She has developed reading assessment. materials for system level testing programs for primary and lower. secondary students She has worked on educational monitoring. programs in Australia Brunei Darussalam Papua New Guinea and. the Philippines Currently she is project manager of the Interna. tional Schools Assessment program Other areas of professional. interest include a mapping assessment data against curriculum. outcome statements and frameworks and b measuring social. outcomes of schooling, George Morgan is an educational consultant He was a senior research. fellow in the Measurement Division and head of the Mathematics. and Science Test Development Group at the Australian Council for. Educational Research for almost 30 years He has developed mathe. matics and science curriculum and assessment materials across all. educational levels and has worked on large scale testing programs. More recently he has been closely involved with assessment projects. in Cambodia East Timor the Lao People s Democratic Republic. Papua New Guinea and Samoa,xiv ABOUT THE AUTHORS AND EDITORS. Vincent Greaney is an educational consultant He was lead education. specialist at the World Bank and worked in a range of countries in. Africa Asia and the Middle East A former teacher a research fellow. at the Educational Research Centre at St Patrick s College Dublin. and a visiting Fulbright professor at Western Michigan University. Kalamazoo he is a member of the International Reading Association s. Reading Hall of Fame Areas of interest include assessment teacher. education reading and promotion of social cohesion through text. book reform, Thomas Kellaghan is director of the Educational Research Centre at. St Patrick s College Dublin and is a fellow of the International. Academy of Education He has worked at the University of Ibadan in. Nigeria and at the Queen s University in Belfast Areas of research. interest include assessments and examinations educational disadvan. tage and home school relationships He served as president of the. International Association for Educational Assessment from 1997 to. 2001 He has worked on assessment issues in Africa Asia Latin.
30-Mar-2020 3 Views 31 Pages
Department of Education Vote 6 To be Appropriated by Vote R 10,857,872,000 Statutory Amount R 739,000 Responsible MEC ... xPublic Service Code of Remuneration (CORE)
30-Mar-2020 4 Views 45 Pages
Foot massage ($100) - 10 points for every lesson - total 30 Hands massage ($200) - 10 points for every lesson - total 30 Doing foot massage to Alice - 2 points (1 if is partly finished) Doing foot massage to mom - 1 point Doing back massage at the pool - 5 points Sensual massage - needs 100 points of Massage skill to unlock it (fastes way - 2 ...
30-Mar-2020 3 Views 51 Pages
terms of the Code of Remuneration (CORE). The DoD uses the Personnel Management Code (PMC) for uniform members. In spite of this change, skilled members have seen little
30-Mar-2020 5 Views 13 Pages
Harvard FAS Space Type Classification Standard Notes Assignable 100 Classroom Facilities 110 Classroom 1100 Classroom Scheduled instruction not tied to a specific subject or discipline (by equipment in the room or configuration of the space). ? Generic FICM Def. 1101 Classroom Movable Tablet Arm Chairs Scheduled instruction space equipped with movable chairs such as tablet arm chairs. [Note ...
30-Mar-2020 5 Views 61 Pages
HABERMAS'SRETREATFROMHERMENEUTICS' SYSTEMSINTEGRATION, SOCIALINTEGRATION ANDTHECRISIS OFLEGITIMATION Dieter Misgeld A. Introduction Theories endeavouring to articulate what a society is and whyand how people are organized in social forms cannot merely proceed, as if the very questions raised by them had never been articulated in the society itself. Societal members enjoy and ...
30-Mar-2020 3 Views 8 Pages
EMERGENCY KIT Poems for Strange Times edited by jo SHAPCOTTand MATTHEW SWEENEY S faberandfaber. Contents Introduction, xv ... Looking Up SUJATA BHATT, 238 Knot SUSAN ...
30-Mar-2020 3 Views 17 Pages
State of Alaska Department of Administration Division of General Services HOW TO DO BUSINESS WITH THE STATE OF ALASKA REVISED MARCH 2016 PREPARED BY THE DIVISION OF ...
30-Mar-2020 4 Views 14 Pages
changed blood bank practices in light of the COVID-19 pandemic. Questions include whether hospitals have changed transfusion medicine practices, such as whe ther indication thresholds for red blood cells and platelets have been lowered, whether prophylactic platelet transfusions have been discontinued, and similar questions. The survey, to be ...
30-Mar-2020 3 Views 12 Pages
involved with Youth for Christ, a global outreach program for high school students, volunteering her time by serving in the office and singing in the choir. Lois took her faith seriously and received a strong spiritual foundation at the Camp Street Brethren Church where her family attended. Even as she grew and availed
30-Mar-2020 7 Views 51 Pages
forestry in terms of education and managerial organizations in the forest development. MODULE I FORESTS AND FORESTRY History of management of forests in India Emerging trends of Forest Management in India, Modern tools and techniques in Forestry, Different schemes/missions on Forest protection and Conservation.