Week 24 – Activity 19: Reflecting on the ROMA framework

Timing: 4 hours

  • Start by reading Sections 1–4 (pp. 121–30) of this paper:

    The paper makes links between the introduction of learning analytics, the TEL Complex and the ROMA framework, so it provides a way of reviewing work you have carried out this week. You will see that Macfadyen and Dawson, who wrote the paper you read in Activity 17, are co-authors of this paper. The 2012 paper by Macfadyen and Dawson identified problems with the implementation of learning analytics; this paper proposes a way of dealing with those problems.

  • In your learning journal, take the problems and barriers you identified and listed in the previous two activities and map them to the steps of the ROMA framework. You could do this in the form of a list, or you could use presentation software, such as PowerPoint or Keynote, to produce visualisations of the links between steps and issues.
  • Make notes of your reflections on this mapping. Did any of the issues you identified in previous exercises map clearly to steps in the framework? Did the framework prompt you to view some of the problems in new ways?
  • In the discussion forum, or in OU Live, discuss ways in which the ROMA framework would help you to find ways to deal with the issues you have identified.

Answers:

The ROMA model consists of the following seven-steps. The numbers alongside will be added to the list of problems and barriers. A full definition of what each of the seven steps entails, can be found in the paper.

1. Define a clear set of overarching policy objectives

2. Map the context

3. Identify the key stakeholders

4. Identify learning analytics purposes

5. Develop a strategy.

6. Analyse capacity; develop human resources

7. Develop a monitoring and learning system (evaluation)

 

Problems and barriers identified in previous activities:

Interpretation and Observability of Analytics

3 (esp. related to interpretation), 7 (regarding development of a learning system)

Institutional Resistance

1, 2, 3, 7 (if changes are needed to better accommodate these)

Lack of Clear Goal(s)

1, 2 (esp. evidence to convince), 4,

Investment in Student Outcomes

Senior Staff Reluctance and Poorly Understood Objectives/Aims

1, 2 (esp. evidence to convince), 3, 4 (purposes for their involvement)

Underdeveloped Ed Tech Sector

4, 5, 6 (to identify), 7 (in further improving this area)

Workload

1, 2, 5, 7 (if changes are needed to better accommodate these)

Lack of Support From Individuals

1, 3, 7 (if changes are needed to better accommodate these)

Institutional Culture

1, 2, 3

Funding and Revenue Generation

3,

Pedagogic Res Community 

1, 2, 5, 7 (if changes are needed to better accommodate these)

Student Community

1, 2,

Ecology of Practices

1, 2, 7 (if changes are needed to better accommodate these)

 

Further barriers identified in this paper:

Unwillingness to act on findings outside of own research area

2,

individual preferences for qualitative or quantitative approaches

2, 4,

basing decisions on anecdote rather than on research

2 (esp. the evidence), 4, 5,

the different forms of discourse used by researches and decision makers

4,

unfamiliarity with statistical methods on the part of the decision makers

2 (esp. links), 4,

different expectations around communication between researches and those responsible for implementation

1, 2 esp. (political context), 3

different levels of engagement with the research

3,

different expectations about the role and purpose of educational research

1, 2 esp. (political context), 3, 4,

Ethical Concerns

5, 7 (if changes are needed to better accommodate these)

Lack of Skills Available

6 (identification of these)

 

Consider:

1. Did any of the issues you identified in previous exercises map clearly to steps in the framework?

Many of them almost did. More problems and barriers were added whilst reading the paper. The points that would benefit the most (or most accurately) are:

– Funding an revenue generation, matched with considering the stakeholders (3).

– Senior Staff Reluctance and Poorly Understood Objectives/Aims can be combated through extensive explanation of the process and consultation. 

– Lack of clear goals can be combated in a similar way to the above point.

2. Did the framework prompt you to view some of the problems in new ways?

It prompted me to add new problems/barriers. Also, the cyclical cycle is very important, as ultimately we will need to evaluate and adjust, not only based on failures, but also on our findings. I.e. if analytic A is very successful, what does this mean for analytic B?

“Evaluation processes are important, not only to track progress, make any necessary adjustments and assess the effectiveness of the approach, but also to learn lessons from the future” (pg.130)

Week 24 – Activity 18: Considering the TEL Complex

Timing: 4 hours

The reading features many quotes and highlights. Good for TMA. 

Scenario

Imagine that an educational institution you know well has asked you to help develop a plan for rolling out learning analytics, either across the institution or across one section of the institution. The management team’s vision is that, in the future, they will be able to claim on their website that ‘learning and teaching at this institution are supported by learning analytics’. At this stage the management team is not sure what would be involved in implementing a change of this sort, so they need an outline of the changes and developments that might be required.

  • Start by reading the introduction to the report below, and Section 6, ‘The process of TEL innovation’.
  • Use the text in Figure 1: The Beyond Prototypes model of the TEL Complex as headings. Under each heading create a series of notes about what the institution would have to take into account.

Answer:

Policy Context:

Student Community

“The communities associated with these different sets of stakeholders [teachers, students, etc.] often have different sets of values, perspectives, objectives and above all, expertise.”

In my organisation this speaks to the parents of children, or the friends of adults, who might well have other ideas of education and practice. It is not only about the student, but also about satisfying the customers networks.

Ecology of Practices

“The strong community present within the TEL Complex constitutes a major challenge for TEL innovation, and in many cases exhibits super-stability, meaning that change is extremely difficult to achieve. In particular, current expectations of teachers and students affect the adoption of TEL innovations.”

In my organisation, I would include managers into the decision process. Ultimately the decision lies with them. There is also a slight variance in what teachers ‘on the ground’ see, and managers expect or assume. This disconnect can result in any large scale changes that would change the system, being seen as unfavourable.

Environment:

Pedagogic Res Community

“Pedagogy comprises an extremely complex and distinctive process which involved both student and teacher engagement, delivering a set of education services by means of specific channels.”

Consideration would need to be given as to how this relationship between student and teacher might change, given the potential unequal distribution of learning analytics. I.e. with the teacher holding information about the learner. Ethical considerations would need to come into account.

An open approach would be needed in the institution (Phoenix) to share with adult learners why certain actions are taking place. It would be challenging to assure that all students receive a similar level of teacher/with similar tasks. Especially in group contexts.

Revenue Generation:

Technical Communities and Teacher Communities (combined for ease)

will need to take into account the ecology surrounding the practices decided.

“Current practices are not easily altered; they are at the core of super-stability in the overall educational system.”

Both student and teacher practices need to be considered, and how these will be affected by changes in TEL. Training, apprenticeships, and experience is required. The changes cannot be made in isolation, i.e. likely a shift in TEL approaches in my organisation will result in shifts in other pedagogical aspects of the classroom.

Tech Context

“They are the technological elements that are used to support the pedagogy with the aim of achieving a vision that is concerned with enhancing learning in a specified way.”

Financial concerns might arise within my institution, due to new training and tools being required. Extensive decision making processes, as to ascertain what analytics will be deployed and how, will need to be agreed upon.

Technical

“These complex interdependencies make it difficult to get ant one element to work or make a difference by itself without consideration of the whole.”

“Ultimate success depends on the totality of the configuration or bundle, rather than on any single component.” Furthermore, often existing technologies come together to form new innovations. Which of the existing technologies in our institutions, such as the NOS, or the ETS, could be used in this manner?

As mentioned above, any changes to the system of teaching would surely result in changes being seen or needed in other areas. I.e. the changes to e-books over standard paper textbooks, which bring enhancements such as advanced annotation, however will limit the ways a textbook can be used in a  teaching context and could result in external concerns, such as distractions.

Funding:

“In the case of TEL, if policy dictates that funding for TEL is subsumed within general educational budgets or within special project funding, then competition with regular demands to the time-limited nature of project funding can work against long-term sustainability and adequacy of support. This is important, because complex innovations typically require decades for effective diffusion.”

Within my organisation, such consideration are important for much the same reasons as given above. Furthermore, employee wage demands based on workload increases could also play a role. This funding might not come from external sources, and could be bared on management to control.

  • In your notes, consider these questions:
    1. Which groups and factors fall under each heading?
    2. How are these groups likely to feel about the management vision identified above (would you recommend changing the wording)?
    3. Which barriers will need to be overcome?
    4. What changes would have to be made in order to achieve the vision?

Answers:

1. Done.

2. How would the following feel about the management vision:

Student Community

Changes not only to students syllabus and teaching style will need to be made, but student’s wider networks, such as friends and family, will need to be considered. How can the changes be conveyed to those paying, for example?

It is likely that student who have been with us for a longer period of time, will exert more confidence in changes.

Pedagogic Res Community

Changes in one aspect can overflow into others. It is important that in my institution we do not assume that TEL innovations/implementation occur in isolation. How will students, teachers, and management be affected by, for example, the instruction of analytics?

Important quote: “[A research group is] lulled into thinking that when the have a successful pilot the next step will be easy. This next step is the hardest step of all. When they go to schools with their piece of kit and their wonderful technology [they fail because] other factors such as curriculum, professional development, sustainability and appropriateness are not taken into consideration.” (pg. 32).

Senior staff would likely show reluctance to change, especially if it require new workloads. The vision of how such changes will benefit the organisation need to be clear.

Technical Communities

Training, and general experience is needed. Both for students and teachers. Extra budgets or time would need to be allocated to assist the transition.

Again, the benefits need to be clear for all stakeholders, as to move away from personal cost-benefit analysis, and a broader consideration of positive effects.

Teacher community

The influences of potential challenges and changes need to be made clear. Training and general fascination will be needed. Concerns in my organisation would relate to the division of labor, and in particular, if hierarchies emerge. A concern could be that younger employees are more accustomed to the technologies, making older employees less welcoming of changes that might see their skills becoming outdated.

  • In a blog post, or in your learning journal, write a short, informal report for the management team about the changes that would need to be made to the ecology of practices, and the technical context at the institution, in order to introduce learning analytics successfully.

Answers:

  • Take into account:
    • access to resources and training
      • The project would need to focus on design-based learning (pg. 34 summary), to take into account all stakeholders concerns and wishes.
      • Due to the nature of analytics or other TEL innovations, on ground support would be needed for training. This is especially true for senior staff.
      • Consideration of the cost-benefit analysis concerns (where individuals might reject the innovation due to time and work constraints vs. understood benefits) will require training to be a very important step in explanation to staff of what is occurring and WHY.
    • what the major barriers are likely to be
      • The vision is rarely achieved from the offset, and instead “emerges and evolves through exploration, through networking and through the active engagement in research, development and educational practice.”
      • “Engagement [with students and teachers during the design process] may also be a necessary condition to properly understand the ecology of practices that will be the context for any particular TEL innovation.”
      • Major barriers are likely to occur in the following sections above:
        • Student community/Ecology of practices – as these are the customers, their satisfaction and understanding of the new process is paramount. How it would affect them positively needs to be considered. The support of those paying, be it students themselves or parents/other needs to be considered.
        • Teaching community – for the reasons given above, teachers need to be on board with changes and willing to sacrifice some of the status quo.
        • Technical – training will be a large component, and would likely need to continue for an extended period of time. Senior staff in particular need to be considered. Furthermore, for newcomers into the company, training schedules will need to be defined to consider how easy it will be to have them catch up with developments and why they have occurred.
        • Funding – in my organisation this will occur internally. Concerns over staff workload will recur in relation to wages.
    • who would be responsible for dealing with problems.
      • Likely a team-leader will be selected, and a team constructed to deal with all aspects of the project/innovation. This leader will need to pay attention to all factors above, with particular concern to the spin-offs of a change affecting all sectors of the institution.

    If you conclude that it is not possible to achieve the management vision, make it clear why this is, and how your conclusion is influenced by the ecology of practices and the technical context.

  • Share your conclusions, or a link to your blog post, in the forum. Discuss there, or in OU Live, whether you think your chosen institution will be able to claim in the future that ‘learning and teaching at this institution are supported by learning analytics’.

Week 24 – Activity 17: Why analytics may be ignored 

Timing: 2 hours

  • Start by reading the paragraph headed ‘Outcomes of participant observation’ (p. 157) and then read from the heading ‘Why numbers are not enough’ (p. 159) to the end of the paper.
  • Dawson and Macfadyen group the reasons for lack of uptake under two headings: ‘Perceived attributes of an innovation’ and ‘The realities of university culture’. In a blog post, or in your learning journal, note the reasons they identify for lack of uptake, and choose your own headings to group them under.
  • In the discussion forum, or in OU Live, discuss the headings you have selected. Can you agree on a common set of headings? Do any of these groups of reasons stand out as more important than the others?

Answer:

Copied directly from the text. Cite before use.

Institutional Culture

– We suggest here that this may be the result of lack of attention to institutional culture within higher education.
– lack of understanding of the degree to which individuals and cultures resist innovation and change.
– lack of understanding of approaches to motivating social and cultural change.
– Although social systems such as educational institutions do evolve and change over time, they are inherently resistant to change and designed to neutralize the impact of attempts to bring about change.

Lack of Support From Individuals

– no vision or plan will emerge or be embraced without the support of faculty and staff.
– an individual’s reaction to change reflects their cognitive evaluation of the way in which a new event or context will affect their personal wellbeing
– individuals will assess it situationally for its “relative advantage.
– They will assess it for “compatibility”: the degree to which it is consistent with existing practice and values, and with needs of potential adopters.
– They will assess it for “complexity”: the degree to which it is perceived to be difficult to understand or to use.

Workload

– Faculty may view the introduction of technologies into teaching as a time-consuming imposition
– Activities may be perceived as being antithetical to the current institutional culture.
– time-commitment needed for quality instructional design

Poorly Understood

– Potential for learning technologies to enhance teaching and learning may be poorly understood and incongruent with individual perceptions and beliefs surrounding good teaching practice.
– Faculty may worry that spending time on technology will actually hamper their career due to poor evaluations of teaching.

Underdeveloped Ed Tech Sector
– academic culture still rewards faculty for verifiable teaching expertise.
– current lack of standardized methods of assessment of online teaching expertise.
– cooperative nature of effective team-based course development mean that incentives are often very low for faculty to invest time in working with technology.

Senior Staff Reluctance

– members of the senior administration participating in committees charged with LMS review and selection—are typically senior faculty members rather than professional managers.
– assessing the degree to which any change will burden themselves and their colleagues with the need to learn how to use complex new tools, and/or the need to redesign change their teaching habits and practices, without offering any appreciable advantage or reward.

Investment in Student Outcomes

– Information technology managers and staff similarly are most likely to assess proposals for new technology innovations from the perspective of workload and technical compatibility with existing systems, and have an even smaller investment in student learning outcomes.

Lack of Clear Goal(s)

– absence of a strategic goal or vision (and of any clear incentives to strive towards such a strategic vision), analytic data reporting on current LMS data have little motivating power.

Institutional Resistance

– institutional resistance” is found in the very culture of academic institutions
– consensus governance (rather than industrial-style hierarchical management)
– faculty control over the major goal activities (teaching and research)
– an organizational culture that supports change by adding resources rather than by strategically reallocating resources
– a curriculum structure that makes false (though some would argue, necessary) assumptions about learner homogeneity
– any direct interference in faculty democracy is not welcome.

Interpretation and Observability of Analytics

– Not used used to highlight progress and room for growth against a backdrop of institutional targets and vision—and if participants are committed to the vision and motivated to achieve it
– Interpretation remains critical
– Greater attention is needed to the accessibility and presentation of analytics processes and findings so that learning analytics discoveries also have the capacity to surprise and compel, and thus motivate behavioural change
– to date, efforts to mine educational data have been hampered by the lack of data mining tools that are easy for non-experts to use.
–  Poor integration of data mining tools with e-learning systems; and by a lack of standardization of data and models so that tools remain useful only for specific courses/frameworks.
– Collectively, these difficulties make analytics data difficult for non-specialists to generate (and generate in meaningful context), to visualize in compelling ways, or to understand, limiting their observability and decreasing their impact.

Week 23 – Activity 16: Socialised learning analytics

Timing: 2 hours

Very good and information paper. Lots of notes made.

  • Finish reading the article and make notes on the different types of socialised learning analytic and how they might be implemented.

All information below is from the paper. Much of it is copied directly.

Social Learning Network analytics: Social network analysis investigates ties, relations, roles, and network formations. Social learning network analysis is concerned with how these are developed and maintained to support learning. Particular uses include: As social network analysis is developed and refined in the context of social learning, it has the potential to be combined with other types of social learning analytic in order to define what counts as a learning tie and thus to identify interactions which promote the learning process. It also has the potential to be extended in order take more account of socio-material networks, identifying and, where appropriate, strengthening and developing indirect relationships between people which are characterised by the ways in which they interact with the same ‘objects of knowledge’ [20].

Discourse analytics: The ties between learners in a network are typically established or strengthened by their use of dialogue. These interactions can be studied using the various forms of discourse analysis that offer ways of understanding the large amounts of text generated in online courses and conferences. Educational success and failure have been related to the quality of learners’ educational dialogue [24]. Social learning discourse analytics can be employed to analyse, and potentially to influence, dialogue quality. Particular uses include: A related approach to social learning discourse analytics employs a structured deliberation/argument mapping platform to study what learners are paying attention to, what they focus on, which viewpoints they take up, how learning topics are distributed amongst participants, how learners are linked by semantic relationships such as support and challenge, and how learners react to different ideas and contributions

Content analytics: The various methods used to examine, index, and filter online media assets for learners. The analytics may be used to provide recommendations of resources tailored to the needs of an individual or a group of learners. Particular uses include: through tags, and possible other technology to extract details from images and other content to create links; iSpot.

Disposition analytics: These dispositions can be used to render visible the complex mixture of experience, motivation, and intelligences that make up an individual’s capacity for lifelong learning and influence responses to learning opportunities. Particular uses include: Drawing learners’ attention to the importance of relationships and interdependence as one of the seven key learning dispositions. Secondly, they can be used to suppose learners as they reflect on their ways of perceiving, processing and reacting to learning interactions. Finally, they play a central role in an extended mentoring relationship.

Context analytics: Students may be learning alone, in a network, in an affinity group, in communities of inquiry, communities of interest, or communities of practice. Context analytics are the analytic tools that expose, make use of, or seek to understand these context. Particular uses include: (an example extract) Rebecca is on the edge of a community, and dispositions analysis shows that she is currently working on her collaboration skills, then a context-focused recommendation might suggest that she could join a teamwork skills group and use analytics visualizations to monitor her position within the group. Several weeks later, she might be prompted to reflect on her collaboration skills and to rate the group. She might receive this prompt directly from the system, or the system could recommend her teacher, mentor or group leader to engage with her and to make the recommendation.

 

  • Consider the mock-ups of different learning analytics that are presented in Figures 1–5 of the paper. If you had to prioritise the development of one of these for use on the H817 module website, which would it be, and why?

Social learning network analytics. Especially at a masters level, and indeed at PhD level, it would be very useful to know of others who are doing similar work, and whom I could contact for additional information of collaboration. This could include a degree of content analysis as well, to define what I wish to investigate and offer relationships. 

  • Share your opinion in the discussion forum and comment on your colleagues’ opinions.

Week 23 – Activity 15: Citation networks

Timing: 4 hours

This activity focuses on an article that you are likely to find difficult to understand unless you have previous experience of network analytics. It was written for one of the Learning Analytics and Knowledge (LAK) conferences by some of the leaders in the field.

  • Now turn to the practical implications of the paper (Section 4.3). Read this, and expand your notes in the form of a bullet-point list. Note which findings are linked to a practical implication. At this stage, you will have gained a sense of what the paper is about, although you have only read a couple of sections.
    • the development of curriculum in the growing number of academic programs that include learning analytics as a topic; (practical – expansion of the application of LA)
    • promotion of under-represented groups and research methods to the learning community; (practical – expanding the research types and sister fields)
    • fostering the development of empirical work and decreased reliance on founding, overview and conceptual papers; and
    • improved connection to sister organisation such as the International Educational Data Mining Society. (practical – combining experience)
  • Note what the authors aimed to do in the paper, and what their main findings were.
    • An evaluation of the current state the field of learning analytics through analysis of articles and citations occurring in the LAK conferences.
    • A citation analysis and structured mapping aimed to identify the emergence of trends and disciplinary hierarchies that are influencing the development of the field to date.
    • An evaluation of the current state of learning analytics. 
  • Begin by looking at the abstract of this paper:
  • Note whether you think this is an example of learning analytics, according to the definition you produced in Week 21 of this module, or whether it is simply about learning analytics.
    • My definition: Learning analytics concerns the process of measuring and collecting learning data from a variety of student activity, with the aim of using said data to improve and/or alter the student learning process.
    • Noting the bolded section, this would fit more as being about learning analytics, as it is research based and no student based. 
  • Look at the figures and tables in the paper. Some require specialist knowledge, but others are clearer to non-experts. Figure 4 shows the disciplinary background of learning analytics researchers. Table 1 identifies the ten most-cited papers in the field.
  • In the forum, suggest why it is that these most-cited papers have similar numbers of citations in the learning analytics literature, but very different citation rates in a wider context (the Google Scholar citation counts shown on the right of Table 1).
    • Possibly due to the papers cross applicability. For example, if a paper has implications for other research areas, or provides a useful summary of an issue, it will be picked up by those in other fields in referencing and discussion. Whilst, within the LAK community, the paper might only provide a few cases of important information for their work. 
  • Now read carefully through Sections 1 and 2 of the paper, and use all you have read to produce a two-page summary of the article.

    the strategic application of analytics [16] to inform practice has not been extensive within the education sector.

    The outcome of limited systemic analytics activity is a predominance of research that is founded on the extraction of readily available data such as those drawn from learning management systems (LMS), student information systems (SIS), and basic demographics and student grades.

    However, these commonly bivariate analyses are t h e “low hanging fruit” in terms of the overall potential for analytics to redefine and shape education praxis.

    For example, while it is helpful to note that students who regularly log into a LMS may perform better than their less active peers, this information is not suitable for developing a focused response to poor performing students.

    LMS or SIS data can be a useful proxy for seeing a part of a problem, but it is insufficient to serve as a model for intervention that is based on the current state of learning sciences.

    Learning analytics to date has served to identify a condition, but has not advanced to deal with t h e learning challenges in a more nuanced and integrated manner.

    For instance, the field draws on assorted theory and methodologies from disciplines as diverse as education, psychology, philosophy, sociology, linguistics, learning sciences, statistics, machine learning/artificial intelligence and computer science.

    An evaluation of the current state of learning analytics provides numerous benefits for the development of the field, including:

    •  a foundation for future research through the acknowledgement of past research activities;
    •  assistance for grant-making agencies by identifying promising research areas that align with regional and national education goals;
    •  identify disciplines that are under-represented and require more strategic and targeted support and funding opportunities;
    •  identify gaps in research for researchers and students; and
    •  improve the integration between theory and practice by identifying connections between researchers and papers.
    • To address these questions this paper explores through the lens of structured mapping and citation networks the research domains and relationships that have extensively contributed to the field to date.

     

    To address these questions this paper explores through the lens of structured mapping and citation networks the research domains and relationships that have extensively contributed to the field to date.

    In undertaking a mapping and review of the collaborations that have evolved in the field, researchers and practitioners can identify the cliques and sub-culture that define the broader learning analytics community.

    Citation profiles and analyses are central measures adopted in these practices in order to determine research impact.

    As Waltman et al., [20] argued, highly cited papers are not always indicative of impactful research. However, as the authors further noted, on average this premise does tend to hold true. As such, it is reasonable to assume that high citation rates do reflect a certain level of quality [23].

    Current citation databases such as Elsevier’s Scopus, Google Scholar, web of science and others have made the extraction of co-citations and identification of co-author networks more accessible.

    The aim for this paper is to provide a mapping of the learning analytics research community to identify the:

    •  prominent papers referred to in the research;
    •  dominant disciplines and methodologies adopted in learning analytics; and
    • diversity of research paper genres that comprise learning analytics (e.g., opinion papers, reviews, conceptual, empirical research, etc.).

  • Write an entry in your learning journal or blog about your experience of reading a paper in this way. What do you think you gained or lost by missing out some sections?

I found that the general context was lost. I was picking up the surface, but not really able to think critically about what was being said.

Week 23 – Activity 14: Visualising social networks

Timing: 4 hours

Part 1

  • Read this short paper that provides an introduction to social network visualisations and to the SNAPP tool:
  • In your learning journal, or blog, make a note of the things that can be revealed by a network diagram of students’ discussions. The authors identify six – you may be able to think of more.
  • Use these as headings and, under each one, note how this information could be used to support learning and/or teaching in a group.
  • Also note any potential problems, and how these might be addressed.

Part 2

The figures in this paper show some typical visualisations of forum discussions. The paper discusses what these reveal about the discussions.

  • Try creating your own social network diagram. Take a recent thread in the tutor group forum, which includes six or more postings, and sketch it as a network diagram. Note who appears to be central to the discussion.
  • Now take another discussion thread and add it to the same diagram. Drawing by hand, you probably won’t be able to include any further threads in your diagram, unless they are very short.
  • Reflect on what is represented in this diagram and on what is missing.
    1. Does the diagram change your understanding of the tutor group and of your role within it?
    2. Could you use this diagram to make recommendations that might improve learning in the forum or would you need more information?
    3. In what ways is the diagram misleading?
  • Share your conclusions in the forum. If possible, add a scan or a photograph of your network diagram.
  • Discuss situations in your local educational environment, or in one you know well, in which a social network diagram might be used to improve learning and/or teaching.

Answers:

What can be revealed by a network diagram of students’ discussions:

  • identify disconnected (at risk) students;

SS who do not interact with forums, especially on key topics, could be contacted individually.

  • (continuation from above) monitor lone students

Students who are not receiving interaction from others could be at risk of feeling excluded, and as research has shown (in Dawson et al. ,2010), tend to have higher attrition rates, and also evaluation courses/tutors poorly.

  • identify key information brokers within a class;

Comparing a key information broker’s grades with his/her contributions could be valuable, as he/she could be sharing information that is not accurate if it goes unchallenged. A tutor might need to reply directly to him/her in threads to mediate the discussion.

Furthermore, as from Dawson et al. (2010), we can identify if a ‘waggon wheel’ type discussion is occurring, and evaluate if this is desirable given the timing of the discussion (early discussions in a course tend to take this formation, but it is less desirable generally).

  • identify potentially high and low performing students so teachers can better plan learning interventions;

Tutors/teachers needn’t intervene with high performing students, saving them time to focus on students who do not engage with forum activity.

  • indicate the extent to which a learning community is developing within a class;

Courses normally aim for a class to grow closer together and feel more comfortable with sharing ideas as it progresses. Using SNAPP a tutor could monitor if this is indeed occurring. If not, an intervention to prop up the discussion might help, and forums can be further monitored to see the effect.

  • provide a “before and after” snapshot of the various interactions occurring pre and post learning interventions. (This diagrammatic representation is also a useful indicator of reflective teaching practice e.g. through integration with teaching portfolio artefacts);

Similar to above, though a tutor could also monitor if all interventions were positive or negative, and learn from such. For example, tutor involvement can also result in breaking up a discussion as some students shy away.

  • monitor individual student contributions

A tutor could monitor a particular student’s interactions over time. These could be combined with course grades. If their is an effect such as grades lowering that coincides with lower engagement, the tutor could point out such to the student.

Diagram:

IMG_0251.jpg

  1. Does the diagram change your understanding of the tutor group and of your role within it?
    • Slightly. It shows the layout of the discussion as being more evenly spread than what appeared when looking only at the forum. I feel as though the indentation in the forums made my mind think of those with direct replies as carrying more clout than replies. This should not be the case, which the diagram represents.
    • In both cases, A was the initiator of the discussion, however it is evident that both G and K had equally important roles in the discussion.
  2. Could you use this diagram to make recommendations that might improve learning in the forum or would you need more information?
    • It could do, though the context of the discussion is important. In the case of the two threads used, they were both suggestions from A for group projects. As such, seeing two spinoff discussions but generally a good trend where discussion came back to the thread starter is a positive sign. However, if the purpose of the thread was for general discussion, we might wish to have seen a wider spread in communication between, for example K and G.
  3. In what ways is the diagram misleading?
    • The diagram doesn’t accurately portray the purpose of the thread. This is fundamental to interpreting the discussion.
    • P’s role was only an intervention, not a contribution to encourage more participation.

– Discuss situations in your local educational environment, or in one you know well, in which a social network diagram might be used to improve learning and/or teaching.

  • Evaluation meetings
  • General team meetings
  • Email and communication later communication adherence.

Week 23 – Activity 13: Why do we need social learning analytics?

Timing: 4 hours

Part 1

Part 2

Imagine you work in an educational institution that is considering making more use of learning analytics. Staff already have some understanding of learning analytics, but don’t know anything about social learning analytics. You have been tasked with addressing this lack of knowledge.

  • Prepare slides for a ten-minute presentation that can be used to explain to staff what social learning analytics are, why they are becoming more important and the benefits they may offer. Use suitable software, such as PowerPoint, Keynote, Prezi or something else you know well.
  • In the time available, you will be able to include 5-10 slides, so select points and examples that are key to understanding social analytics.
  • In the forum, share your slides or a link to your presentation, together with a brief description of your target audience.
  • Consider the presentations put together by other learners and discuss why those key points have been selected.

Answer: