Appraising my Teaching Skills using MSF

33 %
67 %
Information about Appraising my Teaching Skills using MSF

Published on July 1, 2009

Author: drcolinmitchell


Appraising my ‘IMPROMPTU WARD-BASED Teaching’ Dr Colin Mitchell MSc Geriatric Medicine June 2009

Choosing a Topic I know some of my strengths as a teacher: Can explain complex concepts Keen to teach And I’m aware of my weaknesses Prone to long-windedness Can over-elaborate I already know to concentrate on the known weaknesses But are there any issues that I’m not fully aware of?

Is this right? Previous 360o feedback ‘Can be intimidating’ Really? Jars with my self-image Could interfere with a conducive learning environment

The MSF - Keeping it simple Aim to focus on the learning environment Is the environment conducive to learning? Relaxed / Intimidating Is some pressure a good thing? Further focus on one skill: ward-based teaching (WBT) Multiple exposures, multiple sources of feedback Just F1s

Assessing Educational Environment DREEM - Dundee Ready Educational Environment Measure* 50 Questions – agree / disagree Based on instruments in use for 40 years Designed by 80+ experts using the Delphi process Statistically validated to focus on reliable / discriminatory elements Used worldwide to assess medical school teaching (PHEEM) *Roff et al, 1997. “Development and validation of the DREEM. Medical Teacher 19:4, 205-209

Refining the Quantitative Principles of choosing the items*: Relevant Clear, concise, focused and simple Some balance of +ve and –ve (DREEM 82% +ve) Chose 7 from DREEM (slightly adapted) Added 4 of my own Some concordance, not repetition 8 +ve, 3 -ve *Adapted from Berk (2006), 13 Strategies to Measure College Teaching. Stylus: Virginia

Choosing the scale DREEM uses 5-point bipolar ‘intensity’ anchors: Strongly Agree Agree Uncertain Disagree Strongly disagree Simple but allows a breadth of response Still sufficiently granular & simple Balanced (✔) and has a midpoint (?✔)

Defining the Qualitative Opportunity to explore particular issues more fully Correlate with quantitative themes

Defining the Qualitative Opportunity to explore particular issues more fully Correlate with quantitative themes

Making it anonymous - online MSF in UK training – anonymous. In industry – 97% anonymous “Maintaining the anonymity of the rater’s feedback is key to the process. Just as a tie to compensation can lead raters to soften their ratings, so can a lack of anonymity”* “…when peer assessment is used in a high stakes setting, it results in inflated estimates of performance... There is no way to completely avoid this issue but it may help to ensure the anonymity of evaluators”** Online forms preserve anonymity - This can be made clear in the MSF Rogers E et al, 2002. “Improving the Payoff from 360-Degree Feedback”. Human Resource Development. 25: 44-54 **Norcini J, 2003. “Peer assessment of competence”. Medical Education. 37(6): 539-543

Validation Cannot empirically validate without significant resources Using 7 items from the DREEM – pre-validated My own questions must be logically validated Representative pilot Sent to 5 SHOs. 4 replied with comments (unstructured) Minor adjustments made to wording, stressing anonymity


Rater Scores (% of Maximum)

Question Scores (Fractions of 100%)

Scoring For negative aspects the score is reversed Therefore: Higher score is a good thing

Academic Perceptions

Perception of the Teacher

Teaching Atmosphere

Free Responses (1) What are the strengths of my ward-based teaching? When a situation arises you take time to discuss the case (23/44*) Enthusiastic and approachable… make[s] the learner think the problem through in a logical manner (30/40) …instead of just giving an answer, you get us to think through the problem and help us work out the solution ourselves (24/44) *Commenter’s overall score for quantitative elements shown in parentheses

Free Responses (2) What do you like LEAST about my style of ward-based teaching? (Only 5/11 replied) Sometimes assume too much which can be uncomfortable (20/40) Sometimes can go into a lot of depth for too long (26/44) … on some occasions it would be easier to get an answer, especially when it gets busy, but I would not learn as well… (24/44) Sometimes it becomes a little didactic (25/44)

Free Responses (3) How does the pressure of on-the-spot quizzing affect your learning (+vely or –vely)? (In the US, “Pimping”generally viewed positively*) 8/11 were generally positive, 2 didn’t answer Only negative response: Too pressurizing and can be intimidating and lead to not wanting to ask for teaching (29/44) *Wear D et al, 2005. “Pimping: perspectives of 4th year medical students”. Teaching and Learning in Medicine. 17(2) 184-191

Free Responses (4) Every teacher can improve his skills – how would you suggest I improve my ward-based teaching? Be a bit more aware of when people have got lost in your explanations (20/40) You are a good teacher and have a lot of knowledge, but could improve by talking more succinctly and not rambling (26/44) There is very little to improve on, your teaching is excellent (25/44) I think sometimes you loose [sic] the balance and focus too much on teaching rather than seeing the patients… and thus even though your teaching is good, you are not setting a good example for juniors (28/40)


Robust? Some basic statistical analysis of average total score: Median = average (59%, around a 2.5) 95% CI: ±19% Some observations can be made: Lower scorers more likely to give a critique in the weaknesses question Poor correlation between ridicule / intimidating / relaxed questions. The lowest scorer commented: “the relaxed manner of teaching is definitely a good approach” He/she also disagreed with the ‘relaxed atmosphere’ item (?)

Findings – My Teaching Quantitative results Overall positive (59% compares with DREEM results) I can be intimidating Yet the atmosphere is generally relaxed (=3rd highest scoring item) 3 comments specifically mentioned friendly / approachable Themes emerging from qualitative results Rambling / long-windedness Correlates with ‘teacher-centred’ Few negative comments about atmosphere Only one comment directly mentioned ‘intimidating’ or synonyms. Also note comment about assuming too much = ‘uncomfortable’ This rater also agreed with both the intimidate and ridicule questions

Findings – The Process Getting good feedback is more difficult than it seems Despite: Limiting scope Using validated questions Piloting Anonymous data entry The results didn’t tell me much I didn’t already know Partially corroborated ‘intimidating’ problem Confirmed ‘long-winded’ problem

assimilation Borg Cube by Martin Teufel -

How I see myself Image from Top Gun - Paramount Pictures.

How I see myself

YOU BOY! What I need to avoid: Val-de-ree! Val-de ha ha ha ha ha ha!

Further Actions I may not be intimidating to most… But clearly a few may find me so How do I know who they are? What do I do about it? Design my own feedback forms for my teaching sessions Add demographics Push for free-text responses Don’t be afraid of constructive criticism Criticism is worst when I don’t know how to fix the problem

Questions? Links: Email: Twitter: @drcolinmitchell

Add a comment

Related presentations

Related pages

Appraising my Teaching Skills using MSF - Education

Assessment = Improved Teaching and Learning: Using Rubrics to Measure Information Literacy Skills
Read more

Appraising | LinkedIn

Appraising my Teaching Skills using MSF. 1,624 Views. MUNIF. Appraising performance. 6,468 Views. linkedin_share. Appraising the requirements to develop ...
Read more

Msf | LinkedIn

Appraising my Teaching Skills using MSF. 1,624 Views. m_fields. MSF Macro Presentation Abridged 8.12. 211 Views. CrosstownTO. Finch MSF EA Consultation ...
Read more

About MSF - Motorcycle Safety Foundation

Will MSF market my product in its Online Store? ... About MSF. What is MSF? ... As a teacher, you need good communication skills.
Read more

Teaching critical appraisal skills for Nursing research ☆

Evidence-based practice is a ... key aspects of published papers using structured ... the teaching of research skills for nursing and health ...
Read more

SOCIAL WORK SKILLS - McGraw-Hill Education

SOCIAL WORK SKILLS A PRACTICE HANDBOOK ... Tanya Sellars for their patience in addressing my queries. ... tion [needs to be] paid to the teaching of social ...
Read more

Guiding Classroom Instruction Through Performance Assessment

Guiding Classroom Instruction Through Performance ... and skills. Brain research and best teaching practices indicate the importance of using prior ...
Read more

Resume Guide for Teachers final revised - DePaul | The ...

Resume Guide for Teachers ... classroom management skills and teaching styles that make you ... using teacher’s curriculum to raise her grade from a C ...
Read more

Achieving Excellence through Continuing Professional ...

hone teaching skills˜and ... Continuing professional development opportunities ... Documents in My CPD Portfolio Create a CPD Portfolio using some of ...
Read more