Harper, Green, & Fernandez-Toro: "Evaluating the integration of JingĀ® screencasts in feedback on written assignments"

Harper, F., Green, H., & Fernandez-Toro, M. (2012). Evaluating the integration of Jing® screencasts in feedback on written assignments. In 2012 15th International Conference on Interactive Collaborative Learning (ICL) (pp. 1–7). Presented at the 2012 15th International Conference on Interactive Collaborative Learning (ICL). doi:10.1109/ICL.2012.6402092

Introduction

Looking for ways to enhance tutor feedback/improve engagement via distance learning. Terms: "feed back and feed forward."

Usually receive fb for written assignments via review features in Word. Speaking assignments = written summary with audio file.

Tested feedback on foreign language instruction (beg Sp.; low. interm. Germ; upper interm. Sp). Used Jing.

Research Background

Defines/explains screencasts. Cites previous studies of audio/video feedback for commentary:

  • [4] Johanson, R. Re-thinking the Red Ink: Audio Feedback in the ESL Writing Classroom. Texas Papers in Foreign Language Education. 1999; 4(1): 31-38
  • [5] Rotheram, R. Using an MP3 recorder to give feedback on student assignments. Educational Developments The Magazine of the Staff and Educational Development Association Ltd (SEDA) 2007; 8(2).
  • [6] Nortcliffe, A. and Middleton, A. Audio Feedback for the iPod Generation International Conference on Engineering Education, Coimbra, Portugal 3rd -7th September 2007.
  • [7] Lunt, T. and Curran, J. “Are you listening please?” The advantages of electronic audio feedback compared to written feedback. Assessment & Evaluation in Higher Education, 2010; 35(7): 759–769.
  • [8] Merry, S. & Orsmond, P. Students’ Attitudes to and Usage of Academic Feedback Provided Via Audio Files. Bioscience Education E-Journal 2008; 11 (3). http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-3.aspx (Accesssed 01/06/2012).
  • [9] Crook, A., Park, J., Lawson, C., Lundqvist, K., Drinkwater, R. and Walsh, J. (2010) ASSET: Moving Forward Through Feedback, JISC Final Report, JISC. http://www.reading.ac.uk/web/FILES/asset/ASSET_final_report.pdf (Accessed 01/06/2012)

Claim that from these studies, following benefits emerge:

  • more engaging due to var. of voice/expression
  • easier to understand (students can prioritize)
  • more depth (teachers can say more)
  • more personalized experience
  • increases sense of tutor presence
  • less daunting than f2f (1-2)

Cites Nortcliffe and Middletown-- students performed marginally worse with only summative audio feedback than those receiving short written comments.

Refers to other studies. Audio comments-- Ice et al. "They reported extremely high satisfaction levels, improved conveyance of nuance, enhanced learning community interactions and better retention of content. They also found via an analysis of final projects that students had incorporated learning from previous audio comments three times more frequently than from written comments." 

Source: Ice, P., Curtis, R., Phillips, P., & Wells, J. Using asynchronous audio feedback to enhance teaching presence and student sense of community. Journal of Asynchronous Learning Networks, 2007; 11(2): 3-25.

Perceptions: better able to follow, made instructor thought processes clear, recognition that encouragement and explanation rather than scolding/ criticism. Students more willing to follow up. Thompson and Lee identified a need for instruction for students on how to follow up.

Aims of the Project

  • identify issues with Jing functionality
  • investigate range of approaches taken by tutors to providing feedback
  • investigate whether different approaches in diff lang/levels and according to student competence
  • analyze nature of feedback provided in terms of the criteria addressed/depth of feedback related to strengths and weaknesses
  • evaluate perception of students of the use of the tool
  • evaluate perception of tutors of the use of the tool.

Research Methodology

blend self-reported and observed evidence: feedback questionnaires/interviews; analysis of the feedback. FL instruction. Deliberately did not provided detailed guidance on how to provide feedback.

Tutors completed online questionnaire, combining quant analysis of responses using Likert with free text to investigate:

  • user friendliness of Jing, learning curve
  • how tutors used it/ what they'd like to see it used for
  • whether they noticed diff in student response to feedback
  • what sort of fb could be provided (comp w/trad written)/ whether they belieed Jing enhanced
  • adv v disadv/ whether they would recommend Jing.

Students invited to complete online questionnaire investigating:

  • how they rated Jing fb and written fb
  • whether they would recommend Jing fb
  • how tutor used Jing for for commentary
  • how they felt about receiving fb in this form/ how interacted with fb
  • adv/disadv
  • whether they felt that Jing fb enhanced trad written fb.

5 students -- follow up phone interview 4-6 mos after

fb tutors provided also examined-- terms of orientation, performance areas of focus, depth of feedback.

Outcomes

Approaches to feedback

Tutor approaches: common to select one paragraph and focus on it. Various approaches: highlighting errors first then starting recording; identifying and and correcting as moving through; correcting before recording, then explaining; showing corrected/uncorrected version side by side. Some tutors group errors in paragraph.

Small number recorded showing whole script-- corrected errors first then provided overview of categories of errors.

Two tutors (german) generic recordings to which students directed if necessary.

Note:

A comparative analysis of all three modes of feedback provided by a sample of 4 tutors on a single assignment was conducted, using the FACT analysis tool [18]. This is an evaluation instrument that indicates the ‘profile’ of a piece of feedback in terms of the depth of a tutor’s comments about the strengths of the work and the depth of the comments focusing on its weaknesses. The results of this exploratory analysis show some differences in the ways in which individual tutors use the three media available to them: annotations on the written script, electronic summary form, and Jing® recording. For example, two of the tutors used Jing® exclusively for correcting every language error within a selected extract, whereas on the written script a number of errors had been indicated only. Another tutor also used Jing® in order to identify content-related strengths, give specific examples of what the student had done well, and explain why these constituted strengths. This level of depth in relation to strengths did not occur within any of the other media in any of the cases examined.


Covers differences by language (German, Sp, etc.)

Tutor response

tutors unanimous-- Jing offered greater depth for fb than written; believed would have more impact on students, would be more personal, one even noted accessibility. tutors also felt students would get the praise portion more. Appreciated 5 minute restriction. 8/9 would recommend Jing

Student response

clearer explanations/improved retention

advantages of multi-sensory approach. Made feel that work valued. Many had revisited "their screencasts."

Conclusion

suggests compelling feedback on effectiveness of feedback. Students felt greater affective engagement. Many tutors have adopted for other uses.

Further Research

"Further research will need to look into the impact of different approaches to the use of Jing® for commenting on students’ work, the efficacy of generic recordings and the feasibility of extending the use of Jing® on a larger scale. Whilst students report that they find the feedback memorable and that they have understood explanations, our study did not seek to obtain objective evidence in the form of a comparison between the impact of written and Jing® feedback on future work. This would also be a productive line of research."