Published on December 17, 2007
Creating a Culture of Assessment: Creating a Culture of Assessment Judy Ashcroft Associate Vice President and Director, DIIA Archie Holmes Associate Professor, Electrical & Computer Engineering Pamela Brochhausen Instructional Design Technology Consultant, DIIA Slide2: Assessment is an integral part of learning, not something apart from a theory of learning. Students are fairly and effectively assessed when the professor has a coherent and cohesive theory of learning. Slide3: Plan your assessment as you plan your instruction. http://www.utexas.edu/academic/diia/assessment/iar/ Teaching as we were taught: Teaching as we were taught Follow examples of our best teachers Reflect on what works and what doesn’t Ask advice of others in our discipline Sometimes have a “knack” for it • Find it mystical, inexplicable, private, and personal Pedagogy, andragogy, learning theories, science of teaching, cognitive science?: Pedagogy, andragogy, learning theories, science of teaching, cognitive science? Difficult to define learning, so evaluation of learning is complex. “Edu-speak” jargon difficult to follow. Little time to learn another discipline. Slide6: Assessing as we were assessed doesn’t work in the age of accountability. Two paradigms of assessment: Two paradigms of assessment Importance of learning to assess writing: Importance of learning to assess writing Oral nature of classroom instruction replaced by writing for both professor and students in online environment. Important to all disciplines and is a life skill for citizenship in a democracy. Tips for effective assessment:: Tips for effective assessment: Know your students Assess in multiple ways Assess for consistent performance and improvement Clearly state expectations including following the Honor Code Creating a Culture of Assessment: Myths and Reality*: Creating a Culture of Assessment: Myths and Reality* Archie Holmes ECE Department * At least my version of myths and reality… Talking Points: Talking Points What does assessment mean How do I effectively do assessment with minimal effort What traps can I fall into Classic Definition of Assess: Classic Definition of Assess To determine the value, significance, or extent of; appraise Does this definition apply in education? I will argue that it DOES with some conditions… Myth #1: Myth #1 By evaluating out-of-the-classroom assignments, I am doing assessment Redefining Assessment: Redefining Assessment Assessment must: closely connect, in time, the actions of students and the review by the instructor Assessment and Homework: Assessment and Homework In my opinion, homework is a lousy assessment instrument Questions that cannot be answered about homework Did each student do it individually? How much help did the student receive? How long did the homework take the student to do? (Partial) Myth #2: (Partial) Myth #2 Having students do problems in the classroom (individually or in groups) provide assessment This is known as active learning The Value of Active Learning: The Value of Active Learning Active learning is a great tool and provide a number of benefits The classroom is dynamic Students play a role in their own learning Action of students is closely tied to review/feedback from instructor Why is Active Learning a Partial Myth?: Why is Active Learning a Partial Myth? Active learning can give you a feel for the “average” learning in the class The opinions of a few students are used to determine this average The students who tend to participate in class have certain characteristics that may not be representative of the class (Partial) Myth #3: (Partial) Myth #3 When students who are struggling do not come to office hours, the instructor has done something to convince them not to come. View from the Professor Perspective: View from the Professor Perspective No questions from the class during the lecture Students must have a good understanding of the material No one comes to office hours or students come to office hours who understand the material Every student can do the HW well Why don’t students come to office hours: Why don’t students come to office hours They have had a bad experience in the past Even after multiple good experiences, one bad experience can discourage a student Their cost/benefit calculation means that they are ignoring your class How to get Students to Come to Office Hours: How to get Students to Come to Office Hours It is my opinion that if a student perceives you care about their learning, they “may” come to office hours How do you create this? Admit when you make mistakes (and you will) Be willing to say “I do not know” and respond at the beginning of the next class day Redefining Assessment Again: Redefining Assessment Again Assessment must: closely connect, in time, the actions of students and the review by the instructor provide an accurate and quantifiable profile of student skills and abilities Information must be available so that instruction can be adjusted based upon the data How do I do meaningful assessment: How do I do meaningful assessment Use online tools to collect information OCA is a good feedback tool after class Tie it to the course grade by giving all students the same grade based upon total class participation Blackboard provides quizzes Use pre-class quizzes to gather students know/understand before they come to class. Use post-class quizzes to reinforce classroom instruction How do I do meaningful assessment con’t: How do I do meaningful assessment con’t Make students provide the assessment Having students make their own exam problems is a great way to measure understanding Using Ongoing Course Assessment (OCA) : Using Ongoing Course Assessment (OCA) https://web.austin.utexas.edu/diia/oca/ Pamela Brochhausen firstname.lastname@example.org Objectives: Objectives Present background on the tool Demonstrate the three ways to use the tool Discuss best strategies to keep in mind when creating a survey Background: Background Student Government and Educational Policy Committee requested tool in 2003 with following features: All student answers are anonymous. It is also impossible to link student answers across questions. Instructors have control over what questions are asked. Overview: Overview Three ways to use the tool 1) Use a pre-made survey template 2) Customize a template 3) Create own questions First way to use the tool: First way to use the tool Use a pre-made survey template Second way to use the tool: Second way to use the tool Design own survey template using questions from the question bank Third way to use the tool: Third way to use the tool Create your own questions Results: Results Can be downloaded into Word or Excel Tips for effective use: Tips for effective use Have clear goals before creating survey. http://www.utexas.edu/academic/diia/assessment/iar/how_to/technology/about_oca.php Determine purpose of survey and how you will use the results Base questions on course objectives, instructor needs, department needs. Make each question count. Fewer than 20 questions best due to fatigue factor. Expect lower response rate than paper surveys. Encourage participation: Introduce in class (remind about anonymity). Send reminders. Send personalized email through Blackboard. Slide43: Sample student emails Tips for effective use cont.: Tips for effective use cont. Discuss results in class. More likely that students will participate in surveys in future. Examples: Find out student background and their purpose for taking the course. Get input of intended change to course before you proceed. Do pre- and post-test if you make a change to the class.