Assessment

63 %
38 %
Information about Assessment

Published on October 8, 2008

Author: rgibson

Source: slideshare.net

Robert Gibson, Ed.D. eLearning Institute October 9, 2008 Emporia State University Effective Online Assessment Strategies

Is Cheating a Problem on Your Campus? Research suggests it may be a very big problem… perhaps bigger than you realize Effective Online Assessment Strategies

How to Cheat Online Why do students cheat? Everybody does it “ If I don’t cheat, my grade suffers” “ Unrealistic” demands by faculty Opportunity presents itself Effective Online Assessment Strategies

Why do students cheat?

Everybody does it

“ If I don’t cheat, my grade suffers”

“ Unrealistic” demands by faculty

Opportunity presents itself

How to Cheat Online In a recent survey of high school students: 80% of high-achieving students admitted to cheating at least once 50% said they did not believe cheating was necessarily wrong 95% of the cheaters said they’ve never been caught According to Duke’s Center for Academic Integrity , 75% of college students confess to cheating at least once Technology makes it easy. Students use the Internet, pagers, and cell phones to cheat http://chronicle.com/free/2008/09/4705n.htm http://tinyurl.com/4jler4 Using “Slugs” Effective Online Assessment Strategies

In a recent survey of high school students:

80% of high-achieving students admitted to cheating at least once

50% said they did not believe cheating was necessarily wrong

95% of the cheaters said they’ve never been caught

According to Duke’s Center for Academic Integrity , 75% of college students confess to cheating at least once

Technology makes it easy. Students use the Internet, pagers, and cell phones to cheat

http://chronicle.com/free/2008/09/4705n.htm

http://tinyurl.com/4jler4

Using “Slugs”

How to Cheat Online In a study of 500 middle and high school students, Rutgers management Professor Don McCabe found that more than two thirds said working with classmates didn't constitute cheating, and half thought it was fine for parents to do their homework Males report significantly higher incidences of cheating than do females No difference between freshmen and sophomores No difference between full- and part-time students Residential college students perceive cheating to be more socially acceptable than do off-campus students Effective Online Assessment Strategies

In a study of 500 middle and high school students, Rutgers management Professor Don McCabe found that more than two thirds said working with classmates didn't constitute cheating, and half thought it was fine for parents to do their homework

Males report significantly higher incidences of cheating than do females

No difference between freshmen and sophomores

No difference between full- and part-time students

Residential college students perceive cheating to be more socially acceptable than do off-campus students

How to Cheat Online Business majors tend to score lower on measures of moral development and reasoning than some other majors (particularly when compared to liberal arts students) Cheating behaviors are a function of situational factors that appear to be either independent of or are at least poorly correlated to the level of moral development Data suggest that instructors who repeatedly use the same examination each semester encourage cheating. Thornton & Peterson (2000) Effective Online Assessment Strategies

Business majors tend to score lower on measures of moral development and reasoning than some other majors (particularly when compared to liberal arts students)

Cheating behaviors are a function of situational factors that appear to be either independent of or are at least poorly correlated to the level of moral development

Data suggest that instructors who repeatedly use the same examination each semester encourage cheating. Thornton & Peterson (2000)

How to Cheat Online Blogging Made Easy with Wordpress … and the faculty?

How to Cheat Online A 25-question survey was completed by 742 faculty: Faculty do not perceive academic dishonesty to be a serious problem! Faculty claimed to be familiar with current policy and procedure and were unconcerned with implementation Most faculty did not regularly follow institutional policy; rather, handled incidents of cheating and plagiarism on an individual basis. They believed responsibility for reducing academic dishonesty lay primarily with students and individual faculty Effective Online Assessment Strategies

A 25-question survey was completed by 742 faculty:

Faculty do not perceive academic dishonesty to be a serious problem!

Faculty claimed to be familiar with current policy and procedure and were unconcerned with implementation

Most faculty did not regularly follow institutional policy; rather, handled incidents of cheating and plagiarism on an individual basis. They believed responsibility for reducing academic dishonesty lay primarily with students and individual faculty

How to Cheat Online 86% of faculty have suspected and 65% have been certain of academic dishonesty in their classrooms 800 faculty at 16 institutions surveyed: 40% indicated they never reported cheating 54% indicated they seldom reported cheating 6% indicated they often reported cheating Concerns regarding potential legal action or lengthy student appeals dissuaded many faculty from reporting this problem Effective Online Assessment Strategies

86% of faculty have suspected and 65% have been certain of academic dishonesty in their classrooms

800 faculty at 16 institutions surveyed:

40% indicated they never reported cheating

54% indicated they seldom reported cheating

6% indicated they often reported cheating

Concerns regarding potential legal action or lengthy student appeals dissuaded many faculty from reporting this problem

Continuum for Cheating Excuses Blaming the Technology Dishonest Collaboration Exploiting the Technology Hacking the Technology Low High Effective Online Assessment Strategies

Excuses

Blaming the Technology

Dishonest Collaboration

Exploiting the Technology

Hacking the Technology

Low

High

Continuum for Cheating Excuses, excuses… Exploit unclear course expectations Exploit faculty disorganization Prevent Excuses Include clear statements on syllabus (consider syllabus quiz) Indicate or link to institution policies Establish or link to institution honor code Demonstrate honesty Effective Online Assessment Strategies

Excuses, excuses…

Exploit unclear course expectations

Exploit faculty disorganization

Prevent Excuses

Include clear statements on syllabus (consider syllabus quiz)

Indicate or link to institution policies

Establish or link to institution honor code

Demonstrate honesty

Continuum for Cheating Blaming the Technology… “ Virus” claim “ Computer crashed” claim “ Quiz did not submit” claim Prevent Excuses Include clear statements on syllabus Shift responsibility to students (consider E-test helper guide) Provide links to Helpdesk, tutorials, helper guides Effective Online Assessment Strategies

Blaming the Technology…

“ Virus” claim

“ Computer crashed” claim

“ Quiz did not submit” claim

Prevent Excuses

Include clear statements on syllabus

Shift responsibility to students (consider E-test helper guide)

Provide links to Helpdesk, tutorials, helper guides

Continuum for Cheating Dishonest Collaboration… Print the test Split the work Recycle papers Prevent Dishonest Collaboration Consider no-print/no-copy script (Princeton) Use random question pools Use assessment timers Use SafeAssign (plagiarism detection) Utilize proctors for high-stakes, online assessments http://www.ncta-testing.org/cctc/ Effective Online Assessment Strategies

Dishonest Collaboration…

Print the test

Split the work

Recycle papers

Prevent Dishonest Collaboration

Consider no-print/no-copy script (Princeton)

Use random question pools

Use assessment timers

Use SafeAssign (plagiarism detection)

Utilize proctors for high-stakes, online assessments

http://www.ncta-testing.org/cctc/

Continuum for Cheating Exploiting the Technology… False identity Print and distribute assessments Utilize slugs and other proxies Paper mills Prevent Exploitation Consider using systems that defeat printing and copying Consider using secure testing software (SecureExam) Use SafeAssign (plagiarism detection) Do not place high-stakes assessments online Effective Online Assessment Strategies

Exploiting the Technology…

False identity

Print and distribute assessments

Utilize slugs and other proxies

Paper mills

Prevent Exploitation

Consider using systems that defeat printing and copying

Consider using secure testing software (SecureExam)

Use SafeAssign (plagiarism detection)

Do not place high-stakes assessments online

Continuum for Cheating Hacking the Technology… Packet-sniff passwords Exploit predictable passwords Attempt to gain access to the server itself Prevent Hacking http://www.birthindex.com Use strong passwords Avoid unsecured networks Don’t rely on online grade book and backup often Effective Online Assessment Strategies

Hacking the Technology…

Packet-sniff passwords

Exploit predictable passwords

Attempt to gain access to the server itself

Prevent Hacking

http://www.birthindex.com

Use strong passwords

Avoid unsecured networks

Don’t rely on online grade book and backup often

Designing Objective Questions : Primer Effective Online Assessment Strategies

Objective Question Mistakes Multiple-multiple choice (cognitive load is redirected to understanding the question-choice permutations) Example: When writing a learning objective, you should always include: An action verb A criterion measure The tools used A and B B and C B, C, and E. Effective Online Assessment Strategies

Multiple-multiple choice (cognitive load is redirected to understanding the question-choice permutations)

Example: When writing a learning objective, you should always include:

An action verb

A criterion measure

The tools used

A and B

B and C

B, C, and E.

Objective Question Mistakes The long correct option (cues the correct choice) Example: Most beginning book writers will seek out: Writing software Peer review An agent who can represent them to a publisher Good advice Effective Online Assessment Strategies

The long correct option (cues the correct choice)

Example: Most beginning book writers will seek out:

Writing software

Peer review

An agent who can represent them to a publisher

Good advice

Objective Question Mistakes Excessively long item choices (assesses reading comprehension as much as subject matter knowledge, and it’s difficult to remember the choices) Example: Used hypodermic needles should be disposed by: Bagging them in a leak-proof plastic bag and calling Environmental Health and Safety for Disposal. Dropping all the disposables into plastic containers free of punctures or warps and then calling Environmental Health and Safety. Collecting the disposables into a large metal bin that is leak-proof and free of punctures, and then calling Environmental Health and Safety Carrying the sharp objects to a Satellite Accumulation Area and storing them into a leak-proof plastic tub, and then calling Environmental Health and Safety. Effective Online Assessment Strategies

Excessively long item choices (assesses reading comprehension as much as subject matter knowledge, and it’s difficult to remember the choices)

Example: Used hypodermic needles should be disposed by:

Bagging them in a leak-proof plastic bag and calling Environmental Health and Safety for Disposal.

Dropping all the disposables into plastic containers free of punctures or warps and then calling Environmental Health and Safety.

Collecting the disposables into a large metal bin that is leak-proof and free of punctures, and then calling Environmental Health and Safety

Carrying the sharp objects to a Satellite Accumulation Area and storing them into a leak-proof plastic tub, and then calling Environmental Health and Safety.

Objective Question Mistakes Ridiculous or obvious foils (Humorous and an anxiety reducer. However, using them basically makes your multiple-choice test item a true-false item) Example: Who is best known for his/her contributions to mathematics? Paris Hilton Alfred North Whitehead Maya Angelou Sponge Bob Effective Online Assessment Strategies

Ridiculous or obvious foils (Humorous and an anxiety reducer. However, using them basically makes your multiple-choice test item a true-false item)

Example: Who is best known for his/her contributions to mathematics?

Paris Hilton

Alfred North Whitehead

Maya Angelou

Sponge Bob

Objective Question Mistakes Foils without parallel construction. (Items using the same grammatical structure. Cognitive resources directed to understanding the question and not applying knowledge) Example : A lazy coach is a manager who: Provides positive feedback but avoids negative feedback. Provides negative feedback but no positive feedback. When an annual review for the employee comes up, he gives feedback, but never at any other time. A manager who is the type of person who never provides any feedback. Effective Online Assessment Strategies

Foils without parallel construction. (Items using the same grammatical structure. Cognitive resources directed to understanding the question and not applying knowledge)

Example : A lazy coach is a manager who:

Provides positive feedback but avoids negative feedback.

Provides negative feedback but no positive feedback.

When an annual review for the employee comes up, he gives feedback, but never at any other time.

A manager who is the type of person who never provides any feedback.

Objective Question Mistakes “ All of the above” or “none of the above” choices (Avoid these foils unless you are attempting to remind the student that all/none of these apply. If the student knows one of the options to be true, it reduces her choices to that one or the all of the above choice.) Example : Coaching is best done: During the annual review. During the semiannual review. When a coachable moment presents itself. All of the above. None of the above. Effective Online Assessment Strategies

“ All of the above” or “none of the above” choices (Avoid these foils unless you are attempting to remind the student that all/none of these apply. If the student knows one of the options to be true, it reduces her choices to that one or the all of the above choice.)

Example : Coaching is best done:

During the annual review.

During the semiannual review.

When a coachable moment presents itself.

All of the above.

None of the above.

Objective Question Mistakes Negative questions (Readers overlook the not word in the question. Avoid these items when possible, especially if using a negative in the stem.) If negatives must be used, capitalize, underscore, bold, or otherwise highlight Example : Which of the following is not recommended for the victim of an accident: Not leaving the victim unattended. Not moving the victim. Notifying the authorities Not talking to the victim. Effective Online Assessment Strategies

Negative questions (Readers overlook the not word in the question. Avoid these items when possible, especially if using a negative in the stem.) If negatives must be used, capitalize, underscore, bold, or otherwise highlight

Example : Which of the following is not recommended for the victim of an accident:

Not leaving the victim unattended.

Not moving the victim.

Notifying the authorities

Not talking to the victim.

Place as much of the question in the stem as possible, rather than duplicating material in each of the options Avoid using ALWAYS and NEVER in the stem. Test-wise students are likely to rule out such universal statements from consideration Research supports using from three to five foils Avoid clues to the correct answer in the grammatical construction of the question Objective Question Mistakes Effective Online Assessment Strategies

Place as much of the question in the stem as possible, rather than duplicating material in each of the options

Avoid using ALWAYS and NEVER in the stem. Test-wise students are likely to rule out such universal statements from consideration

Research supports using from three to five foils

Avoid clues to the correct answer in the grammatical construction of the question

Multiple-Choice-True-False Questions. Create two true and two false items, and add reasons why the item is true or false. Example : The Colorado Supreme Court banned the use of cell phones in elevators: True, it was deemed a safety hazard. True, it was judged a public nuisance False, because it violated freedom of speech. False, there was a hung jury Test Item Alternatives Effective Online Assessment Strategies

Multiple-Choice-True-False Questions.

Create two true and two false items, and add reasons why the item is true or false.

Example : The Colorado Supreme Court banned the use of cell phones in elevators:

True, it was deemed a safety hazard.

True, it was judged a public nuisance

False, because it violated freedom of speech.

False, there was a hung jury

Scenario-based . Smaller than a case study but longer than a standard multiple-choice stem, scenarios put the student into situations that they must solve. Example : You have just given Ariel her performance review. You ask her to sign her evaluation ( Meets Expectations ), but she believes she is underrated. What should you do? Give Ariel the option of signing or being suspended for 30 days. Note and date her disagreement in writing, and have her sign it. Set several performance based 30-day goals that will upgrade your evaluation if she achieves them. Verbally acknowledge her right to disagree, and indicate that you are obligated to submit the evaluation anyway. Test Item Alternatives Effective Online Assessment Strategies

Scenario-based . Smaller than a case study but longer than a standard multiple-choice stem, scenarios put the student into situations that they must solve.

Example : You have just given Ariel her performance review. You ask her to sign her evaluation ( Meets Expectations ), but she believes she is underrated. What should you do?

Give Ariel the option of signing or being suspended for 30 days.

Note and date her disagreement in writing, and have her sign it.

Set several performance based 30-day goals that will upgrade your evaluation if she achieves them.

Verbally acknowledge her right to disagree, and indicate that you are obligated to submit the evaluation anyway.

Testlets ( Context-Dependent Item Sets ) A test problem or scenario is presented and a set of related test items are asked. Example : Mark is a blood lab technician who has worked at the University Health Sciences Center for the last five years. Mark has just found out he has HIV.  Is Mark covered by ADA? a. Yes, HIV is covered by ADA b. Yes, as long as he is taking medicine for HIV. c. No, only AIDS is covered by ADA, not HIV. d. No, neither HIV nor AIDS are covered by ADA. Test Item Alternatives Effective Online Assessment Strategies

Testlets ( Context-Dependent Item Sets ) A test problem or scenario is presented and a set of related test items are asked.

Example : Mark is a blood lab technician who has worked at the University Health Sciences Center for the last five years. Mark has just found out he has HIV.  Is Mark covered by ADA?

a. Yes, HIV is covered by ADA

b. Yes, as long as he is taking medicine for HIV.

c. No, only AIDS is covered by ADA, not HIV.

d. No, neither HIV nor AIDS are covered by ADA.

Continued… Must the university allow Mark to continue to work with blood since he has HIV? a. No, the university is required to reassign Mark to a position where he is not working in a medical facility. b. No, the university is required to reassign Mark to a position where he is not working with blood. c. Yes, unless Mark's HIV posses a demonstrable safety risk to him or others, the university must continue to allow Mark to work with blood. d. Yes, the university cannot make HIV employees switch positions. Test Item Alternatives Effective Online Assessment Strategies

Continued…

Must the university allow Mark to continue to work with blood since he has HIV?

a. No, the university is required to reassign Mark to a position where he is not working in a medical facility.

b. No, the university is required to reassign Mark to a position where he is not working with blood.

c. Yes, unless Mark's HIV posses a demonstrable safety risk to him or others, the university must continue to allow Mark to work with blood.

d. Yes, the university cannot make HIV employees switch positions.

Alternate-Choice. A two-option MC question that works better with high achieving examinees (grad students and majors) who might readily eliminate several obvious foils in a four-item MC test. Best used to generate more items than you would with a standard four item MC test. Example: Which is a proper closing line for an autumn haiku? a. An island song Like a floating river Rain Rain Fall Fall b. An island song Like a floating river With falling rain a ‘quiver. Test Item Alternatives Effective Online Assessment Strategies

Alternate-Choice. A two-option MC question that works better with high achieving examinees (grad students and majors) who might readily eliminate several obvious foils in a four-item MC test. Best used to generate more items than you would with a standard four item MC test.

Example: Which is a proper closing line for an autumn haiku?

a. An island song

Like a floating river

Rain Rain Fall Fall

b. An island song

Like a floating river

With falling rain a ‘quiver.

Alternative Forms of Assessment Self Assessment Reflective Journal Writing Discussions/Blogs/Wikis Peer Assessment Electronic Portfolios (Assess.) Case Studies Role Playing/Simulation/Games Teach-Back/Oral Projects Group Projects Native Blackboard Tool Self-and-Peer Assessment Baylor Journal/Learning Objects Discussion Board/LO Self-and-Peer Assessment No native Blackboard tool Student Group Tool Virtual Classroom or chat Virtual Classroom or chat Student Group Tool Effective Online Assessment Strategies

Self Assessment

Reflective Journal Writing

Discussions/Blogs/Wikis

Peer Assessment

Electronic Portfolios (Assess.)

Case Studies

Role Playing/Simulation/Games

Teach-Back/Oral Projects

Group Projects

Contact and Conclusion Dr. Robert Gibson Technology and Computing Services Emporia State University [email_address] http://rob-gibson.net 32 Effective Online Assessment Strategies

Dr. Robert Gibson

Technology and Computing Services

Emporia State University

[email_address]

http://rob-gibson.net

Add a comment

Related presentations

Related pages

Assessment – Wikipedia

Assessment (englisch für „Bewertung, Beurteilung, Einschätzung“) steht für: Assessment (Psychologie), Erfassung psychologischer und anderer Merkmale ...
Read more

dict.cc | assessment | Wörterbuch Englisch-Deutsch

Übersetzung für assessment im Englisch-Deutsch-Wörterbuch dict.cc.
Read more

Assessment-Center – Wikipedia

Ein Assessment-Center (AC) (engl. assessment „Beurteilung“) ist das Gremium in einem Personalauswahlverfahren, das unter mehreren Bewerbern diejenigen ...
Read more

Assessment Center - Auswahlverfahren - Vorbereitung

Assessment Center in allen Berufsgruppen und Positionen. Wir erklären Assessment Center, Auswahlverfahren und Co und geben Tipps zur Vorbereitung
Read more

Assessment Center | Infos und Tipps für AC

Was passiert im Assessment-Center und was erwartet dich? Alle Informationen und Hilfe, wie ein Assessment Center abläuft, findest du auf ABSOLVENTA.
Read more

dict.cc Wörterbuch :: assessment :: Deutsch-Englisch ...

Englisch-Deutsch-Übersetzung für assessment im Online-Wörterbuch dict.cc (Deutschwörterbuch).
Read more

Assessment - definition of assessment by The Free Dictionary

as·sess·ment (ə-sĕs′mənt) n. 1. The act of assessing; appraisal. 2. An amount assessed, as for taxation. assessment (əˈsɛsmənt) n 1. the act of ...
Read more

assessment - Englisch ⇔ Deutsch Wörterbuch - leo.org

Das Sprachangebot für Englisch-Deutsch: Wörterbuch mit Übersetzungen, Flexionstabellen und Audio, interaktivem Forum und Trainer für flexibles Lernen.
Read more

Welcome to Assessment.com, Home of MAPP Career Test

Online career test, free results. Career aptitude test or career assessment test. Free results, get career ideas, find the right job for you!
Read more

it-assessment.de - Strategie

Unternehmen, die am schnellsten auf Veränderungen reagieren, beginnen die Märkte zu dominieren und werden auf Dauer überleben. In der Konsequenz wird ...
Read more