Covert eye-tracking: an innovative method to investigate compliance with instructions

The present study introduces a covert eye-tracking procedure as an innovative approach to investigate the adequacy of research paradigms used in psychology. In light of the ongoing debate regarding ego depletion, the frequently used “attention-control video task” was chosen to illustrate the method. Most participants did not guess that their eyes had been monitored, but some participants had to be excluded due to poor tracking ratio. The eye-tracking data revealed that the attention-control instructions had a significant impact on the number of fixations, revisits, fixation durations, and proportion of long fixation durations on the AOIs (all BF10 > 18.2). However, number of fixations and proportions of long fixation durations did not mediate cognitive performance. The results illustrate the promise of covert eye-tracking methodology to assess task compliance, as well as adding to the current discussion regarding whether the difficulties of replicating “ego depletion” may be in part due to poor task compliance in the video task.

Dataset

Participants were first given an information sheet and consent form, and demographic questionnaire. Next, they were positioned at 65-cm distance to the screen, which was fitted with an SMI RED 250 screen-based mobile eye-tracker, and told they were doing a brief concentration task, adapted from the SAM lab’s instructions (B. Lassetter, personal communication) and which served as a covert procedure for calibrating the eye tracker. Participants’ eye movements were recorded at 60 Hz, using a 5-point calibration procedure. Participants were instructed to face their monitor comfortably while the experimenter would measure the distance between their head and the screen. They were told to keep their head still and maintain the distance while they looked at the screen. Participants were then told that a circle would appear, move across the screen, and pause once in a while. They were instructed to count in their heads the number of times the circle paused and to report this number to the experimenter at the end of the concentration task. By following the circle with their eyes, unbeknownst to them, participants were actually providing the data required to calibrate the eye-tracker. If the calibration was noisy (that is, accuracy > 1°), the experimenter would repeat the procedure. If the calibration failed (i.e., it was not possible to get an accurate validation) after five attempts, the participant was thanked, debriefed, and dismissed (verbatim instructions to participants can be found on OSF, in the protocol document available at https://osf.io/xxaz6/).