short-paper

Exploring simple neural network architectures for eye movement classification

Authors:

Jonas Goltz,

Michael Grossberg,

Ronak EtemadpourAuthors Info & Claims

ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications

Article No.: 4, Pages 1 - 5

https://doi.org/10.1145/3314111.3319813

Published: 25 June 2019 Publication History

Get Access

Abstract

Analysis of eye-gaze is a critical tool for studying human-computer interaction and visualization. Yet eye tracking systems only report eye-gaze on the scene by producing large volumes of coordinate time series data. To be able to use this data, we must first extract salient events such as eye fixations, saccades, and post-saccadic oscillations (PSO). Manually extracting these events is time-consuming, labor-intensive and subject to variability. In this paper, we present and evaluate simple and fast automatic solutions for eye-gaze analysis based on supervised learning. Similar to some recent studies, we developed different simple neural networks demonstrating that feature learning produces superior results in identifying events from sequences of gaze coordinates. We do not apply any ad-hoc post-processing, thus creating a fully automated end-to-end algorithms that perform as good as current state-of-the-art architectures. Once trained they are fast enough to be run in a near real time setting.

References

[1]

Richard Andersson, Linnea Larsson, Kenneth Holmqvist, Martin Stridh, and Marcus Nyström. 2017. One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. 49, 2 (2017), 616--637.

Google Scholar

[2]

A Terry Bahill and Allan Brockenbrough. 1981. Variability and development of a normative data base for saccadic eye movements. 21, 1 (1981), 11.

Google Scholar

[3]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. 9, 8 (1997), 1735--1780.

Digital Library

Google Scholar

[4]

Ignace T. C. Hooge, Diederick C. Niehorster, Marcus Nyström, Richard Andersson, and Roy S. Hessels. 2018. Is human classification by experienced untrained observers a gold standard in fixation detection? 50, 5 (2018), 1864--1881.

Google Scholar

[5]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. (2015), 11.

Digital Library

Google Scholar

[6]

Marcus Nyström. 2016. Dataset. (2016). https://www.humlab.lu.se/en/person/MarcusNystrom/

Google Scholar

[7]

Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research. 124, 3 (1998), 372.

Google Scholar

[8]

Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the symposium on Eye tracking research & applications - ETRA '00. ACM Press, 71--78.

Digital Library

Google Scholar

[9]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15 (2014), 30. http://jmlr.org/papers/v15/srivastava14a.html

Digital Library

Google Scholar

[10]

Mikhail Startsev, Ioannis Agtzidis, and Michael Dorr. 2018. 1D CNN with BLSTM for automated classification of fixations, saccades, and smooth pursuits. (2018).

Google Scholar

[11]

Raimondas Zemblys. 2016. Eye-movement event detection meets machine learning. 20, 1 (2016).

Google Scholar

[12]

R. Zemblys, K. Holmqvist, D. Wang, F.B. Mulvey, J.B. Pelz, and S. Simpson. 2015. Modeling of settings for event detection algorithms based on noise level in eye tracking data. 8 (2015).

Google Scholar

[13]

Raimondas Zemblys, Diederick C. Niehorster, and Kenneth Holmqvist. 2018. gazeNet: End-to-end eye-movement event detection with deep neural networks. (2018).

Google Scholar

[14]

Raimondas Zemblys, Diederick C. Niehorster, Oleg Komogortsev, and Kenneth Holmqvist. 2017. Using machine learning to detect events in eye-tracking data. 50, 1 (2017), 160--181.

Google Scholar

Cited By

View all

Khan SPachori R(2023)Automated Eye Movement Classification Based on EMG of EOM Signals Using FBSE-EWT TechniqueIEEE Transactions on Human-Machine Systems10.1109/THMS.2023.323811353:2(346-356)Online publication date: Apr-2023
https://doi.org/10.1109/THMS.2023.3238113
Fristiana AAlfarozi SPermanasari AWibirama S(2023)Improving Deep Learning-Based Eye Movements Classification Using Bayesian Optimization2023 IEEE International Biomedical Instrumentation and Technology Conference (IBITeC)10.1109/IBITeC59006.2023.10390966(197-202)Online publication date: 9-Nov-2023
https://doi.org/10.1109/IBITeC59006.2023.10390966
Novák JMasner JBenda PŠimek PMerunka V(2023)Eye Tracking, Usability, and User Experience: A Systematic ReviewInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222160040:17(4484-4500)Online publication date: 18-Jun-2023
https://doi.org/10.1080/10447318.2023.2221600
Show More Cited By

Index Terms

Exploring simple neural network architectures for eye movement classification
1. Applied computing
  1. Life and medical sciences
    1. Bioinformatics
2. General and reference
  1. Cross-computing tools and techniques
    1. Evaluation

Recommendations

Hierarchical HMM for Eye Movement Classification
Computer Vision – ECCV 2020 Workshops
Abstract
In this work, we tackle the problem of ternary eye movement classification, which aims to separate fixations, saccades and smooth pursuits from the raw eye positional data. The efficient classification of these different types of eye movements ...
Video Saliency Prediction via Deep Eye Movement Learning
MMAsia '21: Proceedings of the 3rd ACM International Conference on Multimedia in Asia

Existing methods often utilize temporal motion information and spatial layout information in video to predict video saliency. However, the fixations are not always consistent with the moving object of interest, because human eye fixations are ...
Real time eye movement identification protocol
CHI EA '10: CHI '10 Extended Abstracts on Human Factors in Computing Systems

This paper introduces a Real Time Eye Movement Identification (REMI) protocol designed to address challenges related to the implementation of the eye-gaze guided computer interfaces. The REMI protocol provides the framework for 1) eye position data ...

Comments

Information & Contributors

Information

Published In

ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications

June 2019

623 pages

ISBN:9781450367097

DOI:10.1145/3314111

Conference Chairs:
Krzysztof Krejtz
SWPS University, Poland
,
Bonita Sharif
University of Nebraska-Lincoln

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ETRA '19

Sponsor:

ETRA '19: 2019 Symposium on Eye Tracking Research and Applications

June 25 - 28, 2019

Colorado, Denver

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
440
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)8

Reflects downloads up to 25 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Khan SPachori R(2023)Automated Eye Movement Classification Based on EMG of EOM Signals Using FBSE-EWT TechniqueIEEE Transactions on Human-Machine Systems10.1109/THMS.2023.323811353:2(346-356)Online publication date: Apr-2023
https://doi.org/10.1109/THMS.2023.3238113
Fristiana AAlfarozi SPermanasari AWibirama S(2023)Improving Deep Learning-Based Eye Movements Classification Using Bayesian Optimization2023 IEEE International Biomedical Instrumentation and Technology Conference (IBITeC)10.1109/IBITeC59006.2023.10390966(197-202)Online publication date: 9-Nov-2023
https://doi.org/10.1109/IBITeC59006.2023.10390966
Novák JMasner JBenda PŠimek PMerunka V(2023)Eye Tracking, Usability, and User Experience: A Systematic ReviewInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222160040:17(4484-4500)Online publication date: 18-Jun-2023
https://doi.org/10.1080/10447318.2023.2221600
Startsev MZemblys R(2022)Evaluating Eye Movement Event Detection: A Review of the State of the ArtBehavior Research Methods10.3758/s13428-021-01763-755:4(1653-1714)Online publication date: 17-Jun-2022
https://doi.org/10.3758/s13428-021-01763-7
Fikri MPutra IWibirama S(2022)Object Selection Using LSTM Networks for Spontaneous Gaze-Based Interaction2022 14th International Conference on Information Technology and Electrical Engineering (ICITEE)10.1109/ICITEE56407.2022.9954104(1-5)Online publication date: 18-Oct-2022
https://doi.org/10.1109/ICITEE56407.2022.9954104
Fikri MSantosa PWibirama S(2021)A Review on Opportunities and Challenges of Machine Learning and Deep Learning for Eye Movements Classification2021 IEEE International Biomedical Instrumentation and Technology Conference (IBITeC)10.1109/IBITeC53045.2021.9649434(65-70)Online publication date: 20-Oct-2021
https://doi.org/10.1109/IBITeC53045.2021.9649434
Xi RGao BXia X(2021)Real-time Feedback System of Funding Data Flow Based on Data Tracking and Classification2021 Fifth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)10.1109/I-SMAC52330.2021.9640661(657-661)Online publication date: 11-Nov-2021
https://doi.org/10.1109/I-SMAC52330.2021.9640661

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Hierarchical HMM for Eye Movement Classification

Video Saliency Prediction via Deep Eye Movement Learning

Real time eye movement identification protocol