Week 24 – Activity 19: Reflecting on the ROMA framework

Timing: 4 hours

  • Start by reading Sections 1–4 (pp. 121–30) of this paper:

    The paper makes links between the introduction of learning analytics, the TEL Complex and the ROMA framework, so it provides a way of reviewing work you have carried out this week. You will see that Macfadyen and Dawson, who wrote the paper you read in Activity 17, are co-authors of this paper. The 2012 paper by Macfadyen and Dawson identified problems with the implementation of learning analytics; this paper proposes a way of dealing with those problems.

  • In your learning journal, take the problems and barriers you identified and listed in the previous two activities and map them to the steps of the ROMA framework. You could do this in the form of a list, or you could use presentation software, such as PowerPoint or Keynote, to produce visualisations of the links between steps and issues.
  • Make notes of your reflections on this mapping. Did any of the issues you identified in previous exercises map clearly to steps in the framework? Did the framework prompt you to view some of the problems in new ways?
  • In the discussion forum, or in OU Live, discuss ways in which the ROMA framework would help you to find ways to deal with the issues you have identified.

Answers:

The ROMA model consists of the following seven-steps. The numbers alongside will be added to the list of problems and barriers. A full definition of what each of the seven steps entails, can be found in the paper.

1. Define a clear set of overarching policy objectives

2. Map the context

3. Identify the key stakeholders

4. Identify learning analytics purposes

5. Develop a strategy.

6. Analyse capacity; develop human resources

7. Develop a monitoring and learning system (evaluation)

 

Problems and barriers identified in previous activities:

Interpretation and Observability of Analytics

3 (esp. related to interpretation), 7 (regarding development of a learning system)

Institutional Resistance

1, 2, 3, 7 (if changes are needed to better accommodate these)

Lack of Clear Goal(s)

1, 2 (esp. evidence to convince), 4,

Investment in Student Outcomes

Senior Staff Reluctance and Poorly Understood Objectives/Aims

1, 2 (esp. evidence to convince), 3, 4 (purposes for their involvement)

Underdeveloped Ed Tech Sector

4, 5, 6 (to identify), 7 (in further improving this area)

Workload

1, 2, 5, 7 (if changes are needed to better accommodate these)

Lack of Support From Individuals

1, 3, 7 (if changes are needed to better accommodate these)

Institutional Culture

1, 2, 3

Funding and Revenue Generation

3,

Pedagogic Res Community 

1, 2, 5, 7 (if changes are needed to better accommodate these)

Student Community

1, 2,

Ecology of Practices

1, 2, 7 (if changes are needed to better accommodate these)

 

Further barriers identified in this paper:

Unwillingness to act on findings outside of own research area

2,

individual preferences for qualitative or quantitative approaches

2, 4,

basing decisions on anecdote rather than on research

2 (esp. the evidence), 4, 5,

the different forms of discourse used by researches and decision makers

4,

unfamiliarity with statistical methods on the part of the decision makers

2 (esp. links), 4,

different expectations around communication between researches and those responsible for implementation

1, 2 esp. (political context), 3

different levels of engagement with the research

3,

different expectations about the role and purpose of educational research

1, 2 esp. (political context), 3, 4,

Ethical Concerns

5, 7 (if changes are needed to better accommodate these)

Lack of Skills Available

6 (identification of these)

 

Consider:

1. Did any of the issues you identified in previous exercises map clearly to steps in the framework?

Many of them almost did. More problems and barriers were added whilst reading the paper. The points that would benefit the most (or most accurately) are:

– Funding an revenue generation, matched with considering the stakeholders (3).

– Senior Staff Reluctance and Poorly Understood Objectives/Aims can be combated through extensive explanation of the process and consultation. 

– Lack of clear goals can be combated in a similar way to the above point.

2. Did the framework prompt you to view some of the problems in new ways?

It prompted me to add new problems/barriers. Also, the cyclical cycle is very important, as ultimately we will need to evaluate and adjust, not only based on failures, but also on our findings. I.e. if analytic A is very successful, what does this mean for analytic B?

“Evaluation processes are important, not only to track progress, make any necessary adjustments and assess the effectiveness of the approach, but also to learn lessons from the future” (pg.130)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s