This article on analytical framework is written, keeping in mind learning management type of architecture for all types of technology based learning. If any other technologies are used, the analytical framework will not change.
And although this note mainly applies for on-line courses, the same analytical framework may be used for classroom teaching as well, the only thing is that the data captured will have to be entered physically in the system for the back-end program to generate reports, etc.
The idea of using any learning analytics is important because of the demand of skilling millions to be job worthy, quickly and easily, and only by automating and capturing the teaching-learning process will all stakeholders monitor what is going on!
A back-end programming can be done to monitor various pre-determined parameters and criteria; then reports can be generated and sent to all stake holders. From these reports stake holders can use an analytical framework for evaluating variables, such as student/teacher traffic, academic progress of the learners, exam scores, the academic delivery, feedbacks, actions taken on feedbacks, access to resources by users, etc. The variables list can be changed and will depend on what the business/mission requires to monitor.
Purpose of an Analytical Framework
An analytical framework is a statistical method to measure set criteria and scope of the business of teaching-learning, for example academic usage or academic delivery to end-users. The use of an analytical framework is generally for all stake holders. It is a visual way of linking concepts and assumptions taken at the beginning of a process, with evidences of results. When such a framework is applied during a mid-course, it helps to garner logical flaws in the systems and processes, so that corrective steps may be taken. It is a sort of a feedback to all stake holders regarding what is occurring in the system.
A Sample Flow Chart
The following gives a sample (schematic) flow chart of how a course or a subject delivered, and what are the nodes for monitoring. From here a list of variables can be identified by person responsible, which can go into the analytical framework.
In the flow chart, a box has been coloured:
There will be important questions that will flow out of this box and should reflect in the analytical framework. These questions could be:
- Is teaching occurring?
- Is learning happening?
- Do the test/exam score reflect anything about the process of teaching-learning?
- How is content delivered? Which across various modes (video, WLC, eLearning, etc.) is better assimilated by students?
How are these issues to be resolved? Here are some possible solutions:
- Create rubrics that map learning objectives to learning outcomes (which will embed an assessment matrix too). These need to be weighted, depending on the curriculum topic weights.
- Learning outcomes can be assessed by formative/summative assessments
- Monitor time spent by students/teachers on each topic/subject, time spent in accessing resources.
- There could be a difference in assimilation across various content modes, across various user groups, etc. These can be programmed to monitor too.
List of Variables in an Analytical Framework
A possible list of variables that may be monitored in an analytical framework given below:
- Who using the course work, and their roles
- Course/subject attendance, and time duration
- How many assignments have been submitted?
- How many attempts to pass an assessment
- Time spent completing a subject/course
- Are there any instructional issues from analysis of student feedback + scores
- Do students like a particular form of content, for example, video lectures?
- Have all course completion criteria been met? If yes, then issue of a course completion certificate? Allow further courses?
- What does an automated analysis report indicate? Has it changed with time?
- How many times a content/curriculum needed to be upgraded to meet learning objectives/outcomes
Dashboards can give a visual representation of how teaching-learning process is occurring. It is an important part of any analytical framework. Each stakeholder can have their own dashboards.
A student’s dashboard for a particular course/subject may capture the following variables:
- Date wise time spent on the course
- Date wise assignments submitted
- Formative Assessment – Date taken, no. of attempts, scores
- Summative Assessment: Date taken, score
- Feedbacks given
- Date wise time spent with an coach/tutor/lectures
A teacher’s or course owner’s dashboard for a particular course may capture the following variables:
- Number of students registered for a course
- Date wise time spent on the course by each student (can be made more granular if required)
- Date wise assignments submitted by each student
- Which mode of content is preferred by the students?
- Assessments: Date taken, score by each student, is there any improvement?
- Have learning objectives/outcomes been met?
- Date and time wise live coaching/lectures to students
- Student report card
- Student feedbacks and actions taken. Has this changed with time?
An Admin’s dashboard may capture (role-wise) the following variables:
- Course or subject titles
- Number of teachers teaching each course/subject
- Number of students registered for a course/subject
- List of course and content owners
- Date and time wise website traffic
- Which mode of content is preferred by the students?
- Number of requests for lectures/coaching, etc. Reasons for the same.
- Date wise feedback analysis reports, has it improved with time?
- Monitor performance of teachers, have they managed to deliver the course objectives?
- Monitor performance of students, have they fulfilled course completion criteria? If yes, then issue of a course completion certificate? Should they be allowed further courses?
(a) from Khan Academy website:
(b) Tata Interactive System’s RFID Course