Using Digital Footprints to Improve Student Retention

I recently received a query asking the team whether there are any facilities within Learning Central that can track student online engagement.  This is a good question and a very useful function, particularly for distance and e-learning courses. It can also can prove useful over the Summer holiday, so I thought for the blog this week it might be useful to look into this further.

The Digital Footprint

Most online sites and applications these days have background functions that record every interaction and mouse click that a user makes into ‘log files’.  These are stored on the host provider site and can be used for a variety of purposes including fault finding and fixing of sites and applications, audience demographic statistics for advertising, or for just tracking website success for administrators.  It is these log files that make up a substantial part of what is known as a users’ digital footprint and are used most extensively in the commercial sector to record where you’ve been and sometimes to even predict (or influence) where you are going!  Even Cardiff University uses it for things like our corporate website to track audience demographics.

Once you have this data collected in log files, it needs to be processed and reported on for it to be of any use, and for this we use web analytics.  Even if you have never come across such a facility before, most people seem to have heard of Google Analytics.  This is Google’s service which can be attached to virtually any open access website that records and reports web traffic information, including approximate geographical location, what browser software they have used, the operating system they are using  (e.g. Windows, MacOS, Linux, Android, iOS), the amount of times they have accessed certain pages or used certain functions, and what time of day and for how long they access resources.

It seems to be a little known fact that Learning Central has analytics tools of its own which make it possible to obtain massively extensive raw copies from log files as well as (more usefully) concise user reports that are geared toward learner progress and engagement, rather than just displaying pages of mouse clicks.

Using Digital Footprints for Education

‘Student Engagement Analytics’, or ‘Student Performance Tracking’ is the process of using logs files to report on students’ interactions, progress and engagement with online course content. These reports can then be used to flag when students are either not logging in, performing below average in assessments, or not engaging with online content.

In Learning Central there are two main facilities that can be used to interpret these massive log files into very basic statistics tables for reporting purposes.  These are:

Performance Dashboard –  a very quick overview table of every user on the module/course (including staff), their last access date/time, days since a user last accessed the module/course, a review status (not commonly used in Cardiff), adaptive release status (which allows you to check what a particular user can or can’t see online), how many discussion board posts added (useful if you have formative or summative online discussion tasks), retention centre status (which is explained in the second point below) and a convenient link directly to their Grade Centre statistics.

Retention Centre – a tool that can alert staff when student activity, or lack of it, occurs. It is based on certain criteria that indicates the student could be ‘at risk’ from disengaging with your course.  As the name suggests it is designed to improve student retention by giving staff an easy way to see graphically who hasn’t logged in, hasn’t participated in online activities, achieved below average grades or missed deadlines. The value for these criteria, or ‘rules’ as they are called in Learning Central, are set to default values by the software, but can be adjusted to suit your requirements, depending on the level of engagement you would like/expect for your material and assessments.  It is these defined rules that are used in the Performance Dashboard view mentioned above and they highlight how many of the criteria students fulfil.  The more Retention Centre rules reached, the greater the chance the student is disengaging from a course.

Both of these facilities should be available to you on all of your modules/courses in Learning Central as long as you have sufficient administrative privileges, that is, teaching assistant or instructor, but what then are the overall benefits of using them?

Advantages of using Student Engagement Analytics tools in Learning Central:

• Allows staff to easily keep track of students who aren’t engaging with content online.
• Allows staff to easily track under-performance amongst students.
• Gives a quick overview of student assessments online.
• Enables staff to contact students directly through the retention centre notification feature.
• Logs all communication between staff and students in case of academic appeals.
• Allows for extra rules to be created to highlight above average performance.
• Provides monitoring flags that allow staff to monitor ‘students of concern’ on the Retention Centre homepage.

Disadvantages of using Student Engagement Analytics tools in Learning Central:

• Activity engagement is only tracked against the average within the cohort, so if the overall activity is low within a module because content is not engaging enough, this will not be identified without other carefully written rules.
• It does not take inclusiveness or accessibility into account, both in terms of Disability Discrimination Act compliance as well as access to technology.
• It can only be used as a indicator, rather than as a definitive tool, since set criteria may be too strict or not strict enough.

So as you can see there are both benefits and issues with using these tools, but the positives far outweigh the negatives, especially if used correctly. A follow up post about the practicalities and best practices of using both the Module Reports and the Retention Centre will be posted soon.  For now I have included links below to the official Blackboard instructions on how to use them. I have also included some links specifically looking at Student Engagement Analytics for improving the student experience, as well as JISC information and a case study from Leeds University.

ELTT can also run training sessions on these facilities for you and your colleagues if required. Please email ELTT@cardiff.ac.uk for further information.

Further reading:

• Article on student experience enhancement using Analytics;
• JISC information on student engagement analytics;
• Case study from Leeds on analytics;
• Guide to how to use the Learning Central Performance Dashboard;
• Guide to how to use the Learning Central Retention Centre.