Item Dev slide show HSE cadre 0512161

50 %
50 %
Information about Item Dev slide show HSE cadre 0512161

Published on January 14, 2008

Author: Rainero


Item Development Training for the Kentucky Core Content Test :  Item Development Training for the Kentucky Core Content Test Adapted from WestEd Training for the Kentucky Department of Education June 2005 Clarion Hotel~Louisville Slide2:  Rhonda L. Sims, Director Division of Assessment Support Office of Assessment and Accountability 502-564-4394 Setting the Context:  Setting the Context State Assessment vs Continuous Classroom Assessment Changes Beginning in 2007 New Test Design Depth of Knowledge Addition Refined Core Content for Assessment Test Development Process:  Test Development Process 1 For reading, Bias Committee review of pretest passages 2 Pretest item development by CACs 3 Editorial review of pretest items by WestEd 4 Bias Committee review of pretest items 5 Review and selection of pretest items for forms by KDE and WestEd 6 Layout and production of forms 7 Field test 8 Live administration Your Role as a CAC Member:  Your Role as a CAC Member You are here representing your subject and grade level. You are here representing the entire Commonwealth of Kentucky. You need to consider the diversity of Kentucky when developing items. You must maintain confidentiality of item content. Key Resources for Development:  Key Resources for Development Academic Expectations* Program of Studies Core Content for Assessment* Test Blueprint* Kentucky Performance Level Descriptions Content resources * These three play a key role in item development. Test Item Development:  General item-writing guidelines General guidelines regarding bias/sensitivity issues Guidelines for developing multiple-choice items Guidelines for Developing open-response items Test Item Development General Item-Writing Guidelines:  Alignment to Standards The match to Core Content for Assessment is essential to the validity of the KCCT. Content Relevancy Make sure the item assesses important knowledge or skills identified by standards in the Kentucky Core Content for Assessment. General Item-Writing Guidelines Sample Items:  Sample Items The Preamble and the Articles of Confederation were written in which year? A. 1697 B. 1765 C. 1787 D. 1865 Slide10:  Which document describes the purpose of the U.S. Constitution? A. Preamble B. Bill of Rights C. Articles of Confederation D. Declaration of Independence General Item-Writing Guidelines:  General Item-Writing Guidelines Grade-Level Appropriateness Make sure the item reflects the Content Standards and Objectives at the appropriate grade level. Basic vs. Technical Vocabulary Use simple, basic vocabulary instead of technical vocabulary unless you are assessing the students’ knowledge of the meaning of the technical word/phrase. Sample Item:  Science, Grade 4 When you plant a seed, the roots grow downward. This is called geotropism. Which factor is responsible for geotropism? vs. When you plant a seed, the roots grow downward, and the stem grows upward. Which factor is responsible for the roots growing downward? Sample Item Slide13:  Grade Appropriateness Vocabulary Use grade-appropriate vocabulary as much as possible. Essential Information Only General Item-Writing Guidelines General Item-Writing Guidelines:  General Item-Writing Guidelines Clear Correct and Understandable Graphics Include clear, correct, easily understood graphics as required by the item. Bias and Sensitivity Consider Bias and Sensitivity Guidelines when drafting items. Slide15:  General Guidelines Regarding Bias/Sensitivity Issues Equal Opportunity and Access - The content and language of the items should provide all students with a fair opportunity to demonstrate what they know, regardless of their race ethnicity gender religion disability socioeconomic status, or region in which they live. General Guidelines Regarding Bias/Sensitivity Issues:  General Guidelines Regarding Bias/Sensitivity Issues Appropriate Portrayal of Groups - The subjects, issues, and/or themes addressed by the items must be approached in a way that does not demean, offend, or inaccurately portray any religious, ethnic, cultural, gender, or social group. General Guidelines Regarding Bias/Sensitivity Issues:  General Guidelines Regarding Bias/Sensitivity Issues Protection of Privacy - The content of the items cannot intrude on the privacy of the values or beliefs of students or their families. Slide18:  General Guidelines Regarding Bias/Sensitivity Issues Bias should be avoided in: item stems and prompts Response options and item directions graphics reading selections The emphasis should be on developing items that reflect academic instruction versus life experiences. Guidelines for Handling Sensitive Issues in Assessment Development:  Guidelines for Handling Sensitive Issues in Assessment Development Issues to be avoided:  Child Abuse  Suicide  Incest  Questioning Parental Authority  Rape  Occult  Sex/Sexuality  Divorce  Sexual Preference or Orientation Slide20:  Guidelines for Handling Sensitive Issues in Assessment Development Issues to be used only in a broad context:  Birth Control  Abortion  Sexually Transmitted Diseases  Witches/Witchcraft (Historical Context)  Religious Holidays Guidelines for Handling Sensitive Issues in Assessment Development:  Guidelines for Handling Sensitive Issues in Assessment Development  Death  Family Issues  Guns/Gun Control  Drugs/Alcohol/Tobacco  Homelessness  Murder  Religion  Pregnancy  Creation/Evolution  Violence  Racism/Sexism/Ageism  Use of Animals/Animal Rights Issues which may or may not be suitable: Slide22:  Guidelines for the Development of Multiple-Choice Items Advantages of Multiple-Choice Items:  Advantages of Multiple-Choice Items Can be used to measure a wide variety of learning outcomes Permit wide sampling and broad coverage of a content domain Are reliable and efficient to score Can provide useful diagnostic information about the learning of individual students or groups of students Disadvantages of Multiple-Choice Items:  Disadvantages of Multiple-Choice Items Multiple-choice items are difficult to write. Multiple-choice items are not well suited for measuring certain types of skills (e.g., the ability to organize and express ideas in writing). Performance on multiple-choice items can be influenced by student characteristics unrelated to the subject of measurement, such as reading ability and “test-wiseness.” Multiple-Choice Item Guidelines:  Multiple-Choice Item Guidelines Be sure there is only one right answer. State the item stems in positive terms (if possible). Avoid using negatives in both the item stem and the answer choices, double negatives are confusing. Slide26:  Multiple-Choice Item Guidelines Present a single, clear, complete problem or question in the stem of the item. Avoid the use of absolute terms in (always, never, all, none, only). Multiple-Choice Item Guidelines:  Multiple-Choice Item Guidelines Whenever possible, avoid answer choices that are mutually exclusive opposites (e.g., fiction/nonfiction, living/non-living). When such opposites are used, a student’s chance of getting the item correct becomes 1 in 2 versus 1 in 4. When writing response options…:  When writing response options… Make the incorrect response options plausible. Develop parallel response options. Options should be parallel with respect to content, structure and length. When writing response options…:  When writing response options… Common misconceptions or errors of students are good response options to include in an item. Use other concepts or terms from the grade-appropriate Core Content for Assessment. When writing response options…:  When writing response options… Avoid humorous or nonsensical response options. Avoid using the options “all of the above” and “none of the above”. When writing response options…:  When writing response options… Logically order the response options. Numbers should be listed in ascending or descending order. Unless testing sequence of events from a passage, list options in the order in which they appear. Slide32:  Training Examples for Multiple-Choice Items Slide33:  Do all four response options make sense? Each response option should make sense both grammatically and with respect to content. Slide34:  In order to grow and flourish, pumpkins need lots of A. milk, eggs, and molasses. B. sun, water, and space.* C. gourds, melons, and cucumbers. D. cream, eggs, and lots of sugar and spices. Slide35:  Is the vocabulary used in the stem and response options grade-level appropriate? It is very important that the wording used in the stem and response options be grade-level appropriate, otherwise the item may be assessing a student’s knowledge of vocabulary rather than content knowledge. Slide36:  What is the one way heat can move from one object to another? A. acceleration B. insulation C. adaptation D. conduction* Slide37:  Is there one – and only one – right answer? Each item must have only one right answer. Slide38:  A. in the kitchen B. on the cutting boards C. on the stove-top burners* D. in hot ovens According to the passage, where do most home accidents occur? Slide39:  Is there any inadvertent cueing going on in the item? Sometimes items are written in such a way that one of the response options contains an important word or phrase from the stem. Slide40:  A. Gothic. B. modern. C. Romanesque.* D. Byzantine. A style of architecture that uses the Roman arch, thick walls, and dimly lit interiors is called Slide41:  Does one response option stand out from the others in any way? Some items are written so that one of the response options clearly stands out from the other response options. There are many ways that a response option may stand out from the other options. One way is if one of the options begins with a different word than the other three options. Slide42:  A. for reconciliation of faith and reason. B. that the ends justify the means.* C. for the rise of the common man. D. for allowing freedom of thought. In The Prince, Renaissance author, Machiavelli, instructs the ruling monarchy in methods of statesmanship and argues Slide43:  Another way that an option can stand out is in length. If an option is much longer or shorter than the other three options, it stands out. Slide44:  A. increase the frog population but be harmful to the grass in the area.* B. be harmful to both frogs and grass. C. not affect the plants and animals in the area. D. cause most animals to leave the area. Some students want to build a pond near their schoolyard. This change will MOST LIKELY Slide45:  An option can stand out because it is the only option that contains a negative or a negative connotation. Response options need to be balanced with respect to the use of negatives, so that one option does not stand out. Slide46:  Which is a reason Mike went to the park? He wanted to play with his friends. He did not want to go to school. He liked playing on the swings. He hoped to find his notebook. Slide47:  Which is a reason Mike went to the park? He hoped to find his notebook. He did not want to go to school. He wanted to play with his friends. He did not want to stay home alone. Slide48:  Janice can best be described as angry. mean. rude. caring. Slide49:  Janice can best be described as angry. caring. mean. happy. Slide50:  Is the item format an appropriate one? Items should not be written in certain formats. For example, “fill in the blank format” should be avoided, as should the analogy format. Analogies in particular, are very difficult for many students. Slide51:  Color is to art as _________ is to music. A. melody B. rhythm C. dynamics D. timbre* Slide52:  Is there any extraneous information in the stem? The stem of an item should only include information that is essential to answering the item. For example, in the item below, the first sentence is not essential to the item. Slide53:  Corn is one of the largest crops grown in the United States. There are 2,500,000,000 bushels grown in the U.S. each year. If corn production is decreased by 50% for each year, what will be the U.S. corn production in 4 years? Slide54:  Does the item stem include all information essential to answering the item? Just as some items include too much information, other items include too little. It is very important for an item stem to include all of the information necessary to answer the item or necessary for the answer to be correct. Slide55:  To conduct an experiment Mike pulls marbles from a bag. Each time he pulls a marble he returns it to the bag. He does this 10 times. His results are shown in the table below. There are 100 marbles in the bag. Theoretically, how many marbles are in the bag. Red / Blue A. 60 red, 40 blue B. 50 red, 50 blue C. 70 red, 30 blue* D. 30 red, 70 blue Slide56:  Activity Review of Multiple Choice Slide57:  Rumor Control Open Response Slide58:  OAA’s Favorite OR Myths Restating the question is mandatory and will earn the student one point. Answers must be in paragraph form OR a graphic organizer should be done on the response page. Three or more examples must always be given. Doing more than required will score a 4. Scorers only have 30 seconds to score each response. Released items are BAD items that have been thrown out of the test. Slide59:  Guidelines for the Development of Open-Response Items Advantages of Open-Response Items:  Advantages of Open-Response Items Open-response items allow for more depth of knowledge to be demonstrated than in multiple-choice items. Students can be asked to demonstrate more complex cognitive behaviors such as comparing, relating, analyzing, inferring, concluding, predicting, generalizing, solving and/or applying. Disadvantages of Open-Response Items:  Disadvantages of Open-Response Items Open-response items are more difficult and more expensive to score. Because of their cost, the relative number of open-response items is significantly less than multiple choice. Effectiveness of open-response items is based on the scoring guide and answer information provided. Developing Open-Response Items:  Developing Open-Response Items Establishing the Prompt Designing Directions Establishing the Prompt…:  Establishing the Prompt… What is the Core Content to be assessed? What specific prompt is appropriate to provide context for the students to demonstrate their knowledge? Is the prompt age-appropriate? Does it include extraneous or misleading information? Even if the item has a graphic, there still must be a prompt (1 or 2 sentences) that describes or provides information related to the graphic and/or item directions. Designing Directions…:  Designing Directions… Do the directions clearly reflect knowledge and/or skills from the Core Content? Do the directions specify what students must do? Do the directions provide students the opportunity to reflect Proficient performance? Are the directions clearly formulated? Is there a clear link between the prompt and the directions? Is the wording clear and focused? Is the task achievable? Is it answerable within one page? Make sure that the task is actually achievable.:  Make sure that the task is actually achievable. If the item asks students to read a passage or examine a graphic and then “give three ways” or “explain two reasons” based on the material given, make sure there are three ways or two reasons found in the material provided. Slide66:  As a general rule, if students are asked to generate a certain number of ideas on their own without benefit of a passage or graphic, then there should be at least twice that number of possible answers. You will be asked to supply the possible answers. Make sure the directions are complete and specify what is desired from the students. :  Make sure the directions are complete and specify what is desired from the students. If you want the students to provide examples, the directions to the students should tell them to provide examples and how many. If you want the students to identify information, do not ask them to discuss, describe or explain. Suggestions for Item Format:  Suggestions for Item Format Present the prompt in paragraph form. Use bullets to emphasize the details in the prompt. If the students are required to respond to multiple parts of a question, label each part separately (a, b, c). Slide69:  The number is divisible by three but not by 9. The ones digit is the sum of the hundreds digit and the tens digit.” Show why 153 cannot be Martin’s number. Find all the numbers that match Martin’s clues. Show all your work. Write one more clue that would limit the answer in part b to one and only one correct number. Sample Item Martin said, “I am thinking of a whole number between 100 and 300. Types of Open-Response Items:  Types of Open-Response Items Scaffolded Questions Single Dimension/Component Student Choice: Topics/Options Provided Response to Provided Information Scaffolded:  Scaffolded Sequences increasingly more difficult/complex tasks Success on one part would likely mean there was success on previous parts Multipart (simple to complex) Mathematics Example:  Mathematics Example Corina was investigating information about natural wonders of the world. She found that Mt. Everest is the highest mountain in the world. It is 29,028 feet ABOVE sea level. She found that the Marianas Trench in the Pacific Ocean is the lowest point on Earth. It is 35,840 feet BELOW sea level. Mathematics Example (cont.):  Mathematics Example (cont.) a. If Corina could throw a rock from the top of Mt. Everest to the bottom of the Marianas Trench, how many feet would the rock fall? b. Draw a diagram and explain your answer for part a. Single Dimension/Component:  Single Dimension/Component Straightforward Question Draw a conclusion or take a position, then support it Explain a phenomenon or describe procedures Social Studies Example:  Social Studies Example Many Kentucky cities are located near large rivers. Describe three important advantages that the rivers provide these cities. Explain why each advantage you described in part a is important. Student Choice: Topic/Options Provided:  Student Choice: Topic/Options Provided Choose from the provided options Complete directions as specified Science Example:  Science Example Some of Earth’s materials are listed below: Soil Water Gases of the atmosphere Choose TWO materials from the list. Explain how a PLANT uses each of these materials to live. Choose TWO materials from the list. Explain how an ANIMAL uses each of these materials to live. Response To Provided Information:  Response To Provided Information Response to provided information such as data, readings, graphics Reading Example:  Reading Example In the story “First Light,” Matthew woke up in another time period, the 1850s. Describe FOUR things Matthew discovered that were different from what he was used to in his present life. Explain how each of those differences affected him. Use information from the story to support your answer. When drafting open-response items…:  When drafting open-response items… Draft items on scratch paper. Brainstorm examples of information that would be included in an acceptable response. Exchange items and have someone else answer it. Compare response to what you expected. Evaluate the response to be sure that it is possible to distinguish four levels of response (0 – 4, Blank). When drafting open-response items…:  When drafting open-response items… 6. Revise item as needed. 7. Draft on chart paper. 8. Critique each item as a group—for the match to Core Content, grade-level appropriateness, vocabulary, opportunity to demonstrate proficient performance and whether answerable within the one-page limit. Slide82:  Training Examples for Open-Response Items Slide83:  Is the item specific enough? Sometimes items are written that are not specific enough. Such items are usually difficult to score and unfair to students. The item shown below is a good example of an item that lacks specificity and would probably pose a problem for students. Slide84:  Name an insect that you think Mike should catch and keep in the jar. Draw a picture of the jar with all the features that will be necessary for the insect to live for a few weeks. Make sure that you labeled all these features. After Mike has placed his insect in the jar that you you have drawn, what must he do to make sure that the insect lives. When Mike’s class was studying insects, the teacher asked each student to catch an insect and keep it alive for a few weeks in a plastic jar. Slide85:  Does the item stem inadvertently cue students to one or more answers? Just as with multiple choice items, it is important to make sure that the stem of an open-response item does not cue students to the answer(s). Slide86:  In the article, the author writes, “Pumpkins are more than just symbols of autumn holidays. They are thousands of years old and the stars of folklore, fairy tales, and famous feasts.” Describe three ways people used or are now using pumpkins. Explain why each way was or is important. Slide87:  Activity Review of Open Response

Add a comment

Related presentations

Related pages

Othello Guest Hse • - Tuugo

★ Othello Guest Hse ★ ★ Guest Houses ... We have detected that you are in . Don’t you want to visit ?
Read more

OTTO Online-Shop | Mode, Möbel & Technik | Trends 2015

OTTO Online-Shop » Modetrends 2015 Kleidung Schuhe Möbel Wohnen Technik SALE Ratenkauf ✔ ...
Read more

Photo Centre

Order by Item Number; Order Status; My Account. Account Information; Address Book; ... film reels, slides or prints to DVD. Visit a Costco Photo Centre for ...
Read more

Nitish Bharadwaj - Wikipedia, the free encyclopedia

Currently Nitish judged a Marathi dance reality show on ETV Marathi Jallosh ... (1992 batch) from Madhya Pradesh cadre, in 2009. [15] Filmography
Read more

Procurement Process - Purchasing Insight for Purchase to ...

Includes a procurement process flowchart. Home; What we do; ... show me the money; ... An item of software to publish information on the company intranet ...
Read more


August 26, 2015 Agfa-Gevaert publiceert resultaten voor het tweede kwartaal van 2015 - Gereglementeerde informatie; August 26, ...
Read more