Category: Musings

Evaluating the UX of Instructor Feedback: An Exploratory Analysis

Posted on by Andrea Beaudin

Using UX for Instructional Design

As instructional designers, we often focus on the design: the appearance, the structure and navigation, and the resources. However, the learning process also requires feedback. Feedback is, essentially, the response to the learner question, “am I doing it right?”

For teachers of writing, providing feedback is often an emotionally- and time-intensive process. Feedback for writers is more than marking grammar and syntax; it’s responding to critical thinking, encouraging development, and providing strategies.

And many instructors question whether students review the feedback they’ve created.

As a writing instructor, I wanted to know how learners react to and act upon instructor commentary. My dissertation employs UX research strategies to explore usability and learner satisfaction. 

If you have a few hours and want to read my dissertation, the dissertation PDF is available for viewing. If you only have an hour or so, you may want to view the PDF of the presentation (with speakers’ notes) .

Screenshot of presentation title “Evaluating the UX of Instructor Feedback: An Exploratory Analysis”

Findings

How are instructors currently providing feedback to students (what media, content, and style)?
The majority of instructors (national and English 2311) who participated in the study reported they are using digital written commentary or handwritten comments for student feedback. Few use audio, video, or other methods at all, and when they do, it is sometimes or rarely. From the instructor-evaluated submissions collected as part of this study, English 2311 instructors noted design issues most frequently,10 though higher- and lower-order concerns are typically addressed equally. Instructor comments ranged from one-word corrections to brief explanations. Individual styles varied as far as tone, with generally neutral or positive comments.
What experiences with and expectations for feedback do students have?
Students who participated in the survey and in user testing have mostly received handwritten feedback, with some experience with digital embedded text and conferencing. They have little to no experience with audio or video response. This population prefers handwritten comments to all other media, though they consider digital embedded text and instructor conferencing to be nearly as effective.
How do students typically interact with feedback?
The study revealed that Blackboard’s Review Submission History screen was not a familiar interface for participants, so it is difficult to ascertain whether the access to and subsequent interaction with feedback was “typical.” As would be expected, test participants moved sequentially through their instructor’s comments. Many considered the individual comments to be a “to do list” for correction, but in prioritizing their to do lists, they tended to make choices based on their analysis of audience needs and expectations. Test participants found the (often perceived) instructor’s tone to be a major factor in how they received the feedback. Comments that were interpreted as attacking or negative elicited defensive, even resistant, responses from students.
How might the medium of the feedback impact its usability (how effective, efficient, engaging, error tolerant, and easy to learn it is for students), and why?
The findings from this study indicate that Blackboard—the medium used by English 2311 instructors—impacted the usability of feedback for test participants. Difficulty with access and navigation affected efficiency, effectiveness, and even engagement.

Takeaways

Best practices based on analysis of findings:

  • Whether offering criticism or praise, be direct and specific, and provide the rationale for the evaluation.
  • Be aware that tone can be misunderstood.
  • Consider integrating multiple media options for response.
  • Anticipate the potential practical impediments to achieving pedagogical goals and implement strategies that allow students to overcome these impediments.

Trying Out H5P to Create Interactive Video

Posted on by Andrea Beaudin

My first foray working with H5P to create interactive video. I’ve used other applications (PlayPosit, Articulate360, Captivate), but I find that H5P does most of what I want easily and, considering the plugin is free, affordably.

If you’re wondering about the video itself, it’s a shortened version of the one I linked in When Done Is Better Than Perfect. I wanted to test out how H5P would work with changing some of the “Your Turn” reflective breaks in the video.

It won’t quite do what I want for all situations, as it doesn’t have (that I can tell) an option for learners to write brief answers. Only multiple choice/ fill-in-the-blanks type (quantitative) options are available for input. But it does have some neat features with the multiple choice, such as a message for the “correct” answer, which I repurposed to provide suggestions (see “Evaluating Your Lesson’s Purpose” at 2:00). 

That being said, H5P offers a lot of options overall. I’ve only worked with the video interactivity, and for that, I was generally pleased. Adding bookmarks was a breeze. Inserting hyperlinks could be done in a few clicks. Even better, each option had a tutorial link in the tool dialog box. I only needed to refer to the tutorial once, but it was great to have the resource hyperlinked and ready for me if I had questions..

H5P Pros

For the most part, the interface is intuitive and quite easy to use. Definite positives include:

  • ease of adding bookmarks (chapters) to a video
  • ability to overlay images
  • ease of adding URLs or hotspots
  • several options for users to check/ test their knowledge
  • cost—it’s a free plugin that works with many CMS installations

H5P Cons

The limitations I’ve noticed (but I have only just started playing with the technology):

  • large videos may hang indefinitely while uploading (this might be a constraint of my CMS). I ended up using a YouTube video instead.
  • there’s no way for learners to include even brief responses; any input on the user’s part is strictly defined
  • time stamps are only to the second, not milliseconds, which can be irritating if the video is very tightly sequenced
  • if you want to use it in a commercial LMS like Blackboard or Canvas, you’ll need both institutional permissions and an enterprise license 

Overall Rating: Try It Out! 

I definitely want to play with H5P more. It only took a few minutes for me to be comfortable with the interface and work on developing interactivity. If you are comfortable tinkering with tech, it’s definitely worth the time to take a test drive. 

I’ll be doing some troubleshooting to find out if there are any workarounds for some of the limitations I’ve encountered. If you have any insights, feel free to let me know.

.

When Done Is Better Than Perfect

Posted on by Andrea Beaudin

When I was in graduate school, I often heard the axiom “the best dissertation is a done dissertation.”

As a pragmatic individual, I acknowledged the wisdom and practicality of that statement. As a detail-oriented perfectionist, I didn’t want to do anything but the absolute best.

I’m still a detail-oriented perfectionist, but I have learned to better balance my perfectionism with pragmatism. That means that I do my best, acknowledging that

  • Time is a finite resource,
  • Complications will arise, and yet
  • Deadlines must be met.

I was reflecting upon this as I was preparing a module introduction for a class that I am teaching. It was a presentation on presentations (very meta). My perfectionism was at extreme levels because the presentation was for a graduate course for future professionals, and I believed that the presentation had to be a perfect model of the genre.

I storyboarded. I began shooting footage (of myself, which is not my happy place). I did screen captures of processes to include. I designed a theme.

Then life intervened. Although the wildfires on the West Coast were nearly 1,000 miles away, the sun in Lubbock started taking on an ominous red halo. Soon after, the air quality went from “Good” to  “Poor,” to “Unhealthy,” coming dangerously close to “Hazardous.” 

What does that have to do with video production? A lot more than you might think.

The air quality affected my voice to the extent that there was a marked difference between earlier voiceovers and more recent ones. And no amount of allergy meds and eye drops would make me look less itchy and red.

But deadlines had to be met. So I revised my original presentation plan drastically, which meant I couldn’t do the “live” presentation with the eye contact and body language that I had wanted to use as part of my communication strategy. I also had to accept that I couldn’t re-record every voice over for consistency. 

The video is in no way perfect, but it is done. And a done presentation is a better resource for my students than no presentation.

Perhaps that’s a more important lesson for my students than any of the concepts I covered.

In any case, if you are curious, the video, with all of its imperfections, is linked below. The presentation is for experienced presenters who know the general best practices, such as limiting slide text, using only appropriate and necessary graphics, and pacing. For this learner population, I framed the discussion as a learner-centered approach that influences technology and design choices, including guidance for designing for accessibility.