Investigating the impact of including videos or still images in computer-based academic listening comprehension tests

Visual materials are central to second language listening (L2), yet their use in L2
listening assessment is very limited. Disagreement about the listening abilities that
should be included in L2 listening construct and the absence of empirical evidence about
the effects of visual materials on performance in L2 listening tests led to disagreement
about their use in L2 listening tests. Moreover, previous research did not explore how
test takers interact with visual materials other than video texts. The present research
attempts to contribute to the existing literature by exploring test takers’ viewing patterns
using eye-tracking technology with video and still photo texts during L2 academic
listening tests. In addition, cued retrospective reports are employed to extract test takers’
perceptions about the two types of visual materials and shedding light on the underlying
cognitive processes they employed. Mixed-research method based on triangulation
design was used to investigate test takers’ (n = 30) performance in the video, still photo, and
audio texts, in addition to recording their viewing patterns, and their reported
perceptions about the visual materials. The results revealed that test takers’ performance
in the video texts was superior to both still photos and audio texts, with statistically
significant differences to audio texts. Cued retrospective report data showed higher
helpful perceptions by test takers related to video texts with a strong correlation to their
scores, while still photo texts were perceived with higher distractedness. Eye tracking
data partially coincided with the rest of the results, with two out of three measures�Fixation counts and Total dwell time- found higher with video texts. Implications of the study are that visual materials, especially video texts, should be considered in L2 listening tests as they present better representations of the target language use domain, which requires reconsidering the current L2 listening construct.