Investigating the impact of including videos or still images in computer-based academic listening comprehension tests

Visual materials are central to second language listening (L2), yet their use in L2 listening assessment is very limited. Disagreement about the listening abilities that should be included in L2 listening construct and the absence of empirical evidence about the effects of visual materials on performance in L2 listening tests led to disagreement about their use in L2 listening tests. Moreover, previous research did not explore how test takers interact with visual materials other than video texts. The present research attempts to contribute to the existing literature by exploring test takers’ viewing patterns using eye-tracking technology with video and still photo texts during L2 academic listening tests. In addition, cued retrospective reports are employed to extract test takers’ perceptions about the two types of visual materials and shedding light on the underlying cognitive processes they employed. Mixed-research method based on triangulation design was used to investigate test takers’ (n = 30) performance in the video, still photo, and audio texts, in addition to recording their viewing patterns, and their reported perceptions about the visual materials. The results revealed that test takers’ performance in the video texts was superior to both still photos and audio texts, with statistically significant differences to audio texts. Cued retrospective report data showed higher helpful perceptions by test takers related to video texts with a strong correlation to their scores, while still photo texts were perceived with higher distractedness. Eye tracking data partially coincided with the rest of the results, with two out of three measures�Fixation counts and Total dwell time- found higher with video texts. Implications of the study are that visual materials, especially video texts, should be considered in L2 listening tests as they present better representations of the target language use domain, which requires reconsidering the current L2 listening construct.