A Practical Method for Courseware Evaluation

50 %
50 %
Information about A Practical Method for Courseware Evaluation

Published on January 18, 2009

Author: ademoor

Source: slideshare.net

Description

A. de Moor (2007). A Practical Method for Courseware Evaluation. In Proc. of the 2nd International Conference on the Pragmatic Web (PragWeb 2007), Tilburg, the Netherlands, October 22-23, 2007. ACM International Conference Proceedings Series, Vol. 280, pp.57-63.

A Practical Method for Courseware Evaluation Aldo de Moor CommunitySense PragWeb 2007

The Pragmatic Web of what? Communal effective use of Information Functionalities

Communal effective use of

Information

Functionalities

Courseware and the Pragmatic Web Pragmatic Web = context How to evaluate web functionalities in their context of use? Courseware provides an interesting domain technological environments consisting of multiple functionality components, together offering a complete system of info/comm services required for supporting course needs Functionality evaluation needed Too much (costly) functionality Gaps between required and available functionality Conflicting functionality requirements

Pragmatic Web = context

How to evaluate web functionalities in their context of use?

Courseware provides an interesting domain

technological environments consisting of multiple functionality components, together offering a complete system of info/comm services required for supporting course needs

Functionality evaluation needed

Too much (costly) functionality

Gaps between required and available functionality

Conflicting functionality requirements

Functionality Functionality A set of functions and their specified properties that satisfy stated or implied needs Levels of granularity Systems Courseware environments Tools Blackboard Modules Announcements Functions Post announcement

Functionality

A set of functions and their specified properties that satisfy stated or implied needs

Levels of granularity

Systems

Courseware environments

Tools

Blackboard

Modules

Announcements

Functions

Post announcement

A context model of courseware evaluation

A practical courseware evaluation method Portfolio methods Scores weighed by infrastructure/architecture of organization. Bedell’s method for ICT investment selection Functionalities scored on both effectiveness and importance for the activities to be supported. Practical method Simplification of Bedell No higher-order analyses Actors (users in their roles) provide , interpret and decide upon scores Context: courseware evaluation: Actors: students, software manager Tool system level: module Two questions How well are course activities supported by various functionality components? To what extent are the functionality modules used?

Portfolio methods

Scores weighed by infrastructure/architecture of organization.

Bedell’s method for ICT investment selection

Functionalities scored on both effectiveness and importance for the activities to be supported.

Practical method

Simplification of Bedell

No higher-order analyses

Actors (users in their roles) provide , interpret and decide upon scores

Context: courseware evaluation:

Actors: students, software manager

Tool system level: module

Two questions

How well are course activities supported by various functionality components?

To what extent are the functionality modules used?

Scores Elements I(a) = importance of activity E.g. I(Information Collection) = 9 I(f,a) = importance of functionality f in supporting activity a E.g. I(Virtual Chat, Information Collection) = 4 Q(f,a) = quality of functionality in supporting an activity E.g. Q(File Transfer, Submission of Results) = 8

Elements

I(a) = importance of activity

E.g. I(Information Collection) = 9

I(f,a) = importance of functionality f in supporting activity a

E.g. I(Virtual Chat, Information Collection) = 4

Q(f,a) = quality of functionality in supporting an activity

E.g. Q(File Transfer, Submission of Results) = 8

Activity and functionality scores Activity scores  I(f i ,a) * Q(f i ,a), for all functionalities 1..i. Usefulness of the combined technologies for a particular activity Relevant for technology users (lectures, students) Functionality scores  I(a j ) * I(f,a j ) * Q(f,a j ), for all activities 1..j Usefulness of a particular functionality component for the combined activities Relevant for technology developers/maintainers

Activity scores

 I(f i ,a) * Q(f i ,a), for all functionalities 1..i.

Usefulness of the combined technologies for a particular activity

Relevant for technology users (lectures, students)

Functionality scores

 I(a j ) * I(f,a j ) * Q(f,a j ), for all activities 1..j

Usefulness of a particular functionality component for the combined activities

Relevant for technology developers/maintainers

Experiment: evaluating group assignment functionality Two courseware tools: Blackboard, CourseFlow Goal: making group assignments Four activities, 11 functionality modules Actors: 2 nd year Information Management students, software manager 2002: 62 students, 16 groups 2003: 46 students, 12 groups Questions Quality of tools for various group assignment activities? Usefulness of various functionality modules?

Two courseware tools: Blackboard, CourseFlow

Goal: making group assignments

Four activities, 11 functionality modules

Actors: 2 nd year Information Management students, software manager

2002: 62 students, 16 groups

2003: 46 students, 12 groups

Questions

Quality of tools for various group assignment activities?

Usefulness of various functionality modules?

Activity scores 2002/2003

Functionality scores 2002/2003

Activity Scores BB/CF (2003)

Functionality Scores BB/CF (2003)

Conclusion Practical method, shown to be useful for initial courseware functionality selection Context-bound, in toto, ex-post evaluation Simple measures useful for quickscan and discussion purposes Open source can perform just as well “ E-learning organization”: continuous evolution / evaluation needed

Practical method, shown to be useful for initial courseware functionality selection

Context-bound, in toto, ex-post evaluation

Simple measures useful for quickscan and discussion purposes

Open source can perform just as well

“ E-learning organization”: continuous evolution / evaluation needed

Add a comment

Related presentations

Related pages

A practical method for courseware evaluation

As more courseware becomes available, choosing the right functionality for a particular e-learning community is becoming more problematic. Systematic ...
Read more

A Practical Method for Courseware Evaluation

A Practical Method for Courseware Evaluation Aldo de Moor CommunitySense Cavaleriestraat 2 5017 ET Tilburg, the Netherlands +31-13-4564126 ademoor@ ...
Read more

CiteSeerX — A practical method for courseware evaluation

BibTeX @INPROCEEDINGS{Moor07apractical, author = {Aldo De Moor}, title = {A practical method for courseware evaluation}, booktitle = {In the Second ...
Read more

A Practical Method for Courseware Evaluation - Technology

1. A Practical Method for Courseware Evaluation Aldo de Moor CommunitySense PragWeb 2007 ; 2 ...
Read more

A practical method for courseware evaluation (PDF Download ...

As more courseware becomes available, choosing the right functionality for a particular e-learning community is becoming more problematic. Systematic ...
Read more

An Evaluation Instrument for Hypermedia Courseware

... social and practical ... Evaluation instrument, Hypermedia courseware. ... evaluation is a usability engineering method ...
Read more

Evaluation criterion for computer-based training courseware

The need for a practical method for accessing ... to the development of a new approach to the evaluation of CBT courseware. The method, ...
Read more

Evaluation practical guide - Heriot-Watt University

Implementing Learning Technology: ... Learning Tech. Evaluation Methods Evaluation: A practical ... The evaluation of the courseware within the ...
Read more