Frontiers in Human Neuroscience (Jul 2021)

A Deep Learning-Based Approach to Video-Based Eye Tracking for Human Psychophysics

  • Niklas Zdarsky,
  • Stefan Treue,
  • Stefan Treue,
  • Stefan Treue,
  • Stefan Treue,
  • Moein Esghaei

DOI
https://doi.org/10.3389/fnhum.2021.685830
Journal volume & issue
Vol. 15

Abstract

Read online

Real-time gaze tracking provides crucial input to psychophysics studies and neuromarketing applications. Many of the modern eye-tracking solutions are expensive mainly due to the high-end processing hardware specialized for processing infrared-camera pictures. Here, we introduce a deep learning-based approach which uses the video frames of low-cost web cameras. Using DeepLabCut (DLC), an open-source toolbox for extracting points of interest from videos, we obtained facial landmarks critical to gaze location and estimated the point of gaze on a computer screen via a shallow neural network. Tested for three extreme poses, this architecture reached a median error of about one degree of visual angle. Our results contribute to the growing field of deep-learning approaches to eye-tracking, laying the foundation for further investigation by researchers in psychophysics or neuromarketing.

Keywords