Tagged: W25A23

Week 25 – Activity 23: Deploying a vision of learning analytics

Timing: 4 hours

  • Select two implementations of learning analytics from the list below.
    • Case Study 1A: The Open University, UK: Data Wranglers (pp. 131–3).
    • Case Study 1B: The OU Strategic Analytics Investment Programme (pp. 133–7).
    • Case Study 2: The University of Technology, Sydney, Australia (pp. 138–41).
    • Cluster 1 – focused on student retention (pp. 19–21).
    • Cluster 2 – focused on understanding teaching and learning processes (pp. 19–21).
    • An example from your personal experience or from your reading on this subject.
  • Read about the implementations and, with the help of the ROMA framework (see Figure 3 of ‘Setting learning analytics in context’), make notes on how the visions that underpinned these impacted on the implementation of learning analytics.
  • You may find that the vision is not clearly defined. If this is the case, state it as clearly as you can, based on the information that you have.
  • In the list above, the options from Ferguson et al. (2015) are already aligned with the ROMA framework. The options from Colvin et al. (2015) are not single examples but clusters of examples. Looking at these examples may help you to become aware of elements that are not clearly represented in the ROMA framework.
    • I added point 19. Streamlining and improved data understanding/usability to be added to the end of the cycle as a consideration. 
  • In the discussion forum, or in OU Live, compare your findings with those of people who have selected different examples. Which vision aligns best with your view of how learning analytics should be used?

Answers:

Q1. Make notes on how the visions that underpinned these impacted on the implementation of learning analytics

A1:

For Case Study 1A, the ROMA framework was directly linked to, as per figure 3. Slight changes in the language used and steps taken were used in the articles summary. Within asterisk, the actual ROMA step is defined by myself in the table below (from my point of view).

Screen Shot 2016-07-29 at 13.51.56

Vision

Impact (related to *ROMA*)

Case Study 1A

1. Using the volume of educational data more effectively. Policy objectives were defined (ROMA 1)

*ROMA 1*

2. Develop a group of staff with expertise in the individual faculty contexts. Policy objectives were defined (ROMA 1)

*ROMA 1*

3. Set up a system for collating, synthesising, and reporting on the available data. Policy objectives were defined (ROMA 1)

*ROMA 1*

4. produce reports at regular intervals Policy objectives were defined (ROMA 1)

*ROMA 1*

5. build strong relationships with the faculties. Policy objectives were defined (ROMA 1)

*ROMA 1*

6. analyse and influence teaching and learning practice. Map the context (ROMA 2)

*ROMA 1*

7. Senior management in each faculty (responsible for learning, teaching and/or curriculum development, curriculum developers, those responsible for data gathering and curation, and general senior management. Stakeholders were identified (Roma 3)

*ROMA 2*

8. Key focus on curriculum development and quality enhancement. Learning analytics purposes were identified (ROMA 4)

*ROMA 3*

9. Integrate available data with completion rates, pas rates. Learning analytics purposes were identified (ROMA 4)

*ROMA 4*

10. Conduct extensive consultation and feedback regarding implementation. Strategy development (ROMA 5)

*ROMA 5*

11. Conduct early pilot work. Strategy development (ROMA 5)

*ROMA 5*

12. Decide on an implementation plan, and dates for review. Strategy development (ROMA 5)

*ROMA 4*

13. Provide/achieve content analysis. Capacity analysis, Human Resources developed (Roma 6)

*ROMA 6*

14. Develop a full understanding of the faculty teaching and learning context. Capacity analysis, Human Resources developed (Roma 6)

*ROMA 6*

15. Deployment of new technical tools (data management software, etc.) Capacity analysis, Human Resources developed (Roma 6)

*ROMA 6*

16. Develop an understanding and appreciation of what the data could show, as well as an awareness of how to access it without the mediation of a Data Wrangler. Capacity analysis, Human Resources developed (Roma 6)

*ROMA 3*

17. Build in feedback from stokeholds into the delivery of reports. A monitoring and learning system was developed (Roma 7)

*ROMA 6*

18. Gather feedback from key stakeholders from evaluation exercises. A monitoring and learning system was developed (Roma 7)

*ROMA 6*

19. Undertake reviews with the aim of streamlining. Streamlining the process & Improve understanding of data usability(Non-ROMA). < Self-added

Case Study 1B

1. To use an apply information strategically (through specific indicators) to retain students and enable them to progress and achieve their study goals. *Roma 1*
2. (Macro) Aggregate information about the student learning experience at an institutional level in order to inform strategic priorities that will improve student retention and progression. *Roma 4*
3. (Micro) Make use of analytics to drive short, medium, and long term interventions. *Roma 4*
4. Stakeholders make use of integrated analytics to inform interventions designed to improve outcomes. *Roma 3*
5. Evaluate the evidence base for factors that drive student success (post initial intervention data collection) *Roma 3*
6. Develop models that ensure key stakeholders can implement appropriate support interventions for both short- and long-term benefits. *Roma 3*
7. To use an analysis of current student performance to identify priority areas for action, both in terms of change to the curriculum and learning design, and in terms of interventions with the students most at risk. *Roma 4*
8. Develop a common methodology to evaluate the relative value of interventions through measuring the resulting student behaviours and improvements (to inform future SS experience) *Roma 4*
9. Create near real-time data visualisations around key performance measure. *Roma 6*
10. Triangulation of different data sources, to help in identifying patterns that influence success in a given context. *Roma 6*
11. Create an ethics policy that details what data is being collected and its ethical uses. *Roma 6*
12. Develop machine-learning-based predictive modelling systems. I.e. Will a student hand in his/her assignment based on online activity? *Roma 3*
13. Improve feedback methods from students (shift from end of module assessment to in-module assessment) to improve reaction times to issues. *Roma 4*
14. Measure the success/impact of learning designs through the systematic collection of data. *Roma 5*
15.Create an CoP ‘evidence hub’ focused on the progression of first year students to second year. *Roma 6*
16. Development of ‘small data’ student tools, to allow students to monitor their own progress, visually, to make informed study choices. *Roma 6*

Q2: Which vision aligns best with your view of how learning analytics should be used?

A2:

Vision two. As there was a more defined focus on improving student progress. Rather than the random ‘Data Wrangling’ discussed in the first case study.

Advertisements