Short Case Study: Evaluation using course and contribution statistics


Context

Course statistics are logged automatically in most VLE/learning management systems and can give a crude picture of student engagement in terms of log-in frequencies and the number of visits to specific areas of a course, such as content areas (folders or items) and tools that you have set up within your module site.  Tracking can also be enabled for specific content items to observe how widely items are accessed by students. The statistics will reveal general patterns of student engagement and activity within the module site such as how often students visited the site and the range of resources that they have used. However, statistics are less useful in providing information on levels of engagement with these resources (e.g. time on task in completing study activities).

Contribution statistics can also be generated by the VLE to assess student engagement for evaluation purposes. The number of discussion forum posts made by an individual could be compared with the total number of visits to the forum or views of other posts to judge the overall level of student engagement with a discussion activity. You can also consult contribution statistics for wiki tools – focusing on the number of edits made and percentage breakdown of individual contributions to a group wiki report. Again, these statistics need to be treated with caution, as they will not reflect the complete picture of student learning – i.e. what takes place outside the formal learning environment – the informal learning processes such as drafting and discussion which may be taking place using alternative (student-controlled) tools, if the wiki is viewed by students as the formal publishing location for the activity.

Case Study Overviews

Environment and Health (Environment)

Course statistics were used to track student activity for the VLE site for this third year undergraduate module. Student log-in patterns were varied across the module, with peaks of high activity recorded at the time of practical sessions and deadlines for assessed coursework. 33% of hits occurred on Mondays – the day before the weekly lecture. The statistics tracked student access to content areas within the site, providing an insight into usage patterns. The group work area attracted the most hits, with 76% of the total (9347 hits). The practical area accounted for 14.5% of hits (1781 hits) and the course materials for 5% (630 hits).

Evolutionary Ecology (Biology)

This screenshot of an undergraduate Biology module shows how contribution statistics can be generated through the use of a group wiki tool – presenting the number of page saves (frequency and percentage of saves by the individual for the group report as a whole), and the total number of lines that have been modified by the individual in the report. These statistics can give a rough indication of an individual’s contribution, although they will not reflect the interaction and informal learning that shapes the group’s use of this tool. The quality of the individual’s learning contribution is therefore harder to assess, although there is the page history option to track the changes that have been made to the report.

Screenshot of wiki participation statistics

Some conclusions can be drawn from the data on the nature of group activity, focusing on the distribution of roles within the group. We can gather from the data whether the final editing of reports has been a shared task between group members or one assigned to a particular individual. We may need other evidence though to judge the extent of the contributions made by individuals to the drafting of the report & research of findings, if these activities have not been captured through the use of the wiki.

Screenshot of some of the conclusions drawn from wiki statistics

Next Steps

Case study last updated: December 2008

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s