Human Computer Interaction

Research Projects

Project teaser image

500,000 images closer to eyelid and pupil segmentation

We propose a fully convolutional neural networkfor pupil and eyelid segmentation as well as eyelid landmark and pupil ellipsis regression. The network is jointly trained using the Log loss forsegmentation and L1 loss for landmark and ellipsis regression. The ap-plication of the proposed network is the offline processing and creationof datasets. Which can be used to train resource-saving and real-timemachine learning algorithms such as random forests. In addition, we willprovide the worlds largest eye images dataset with more than 500,000images.

Project teaser image

55 Rides

A dataset of 55 rides of naturalistic driving. Annotated via with appearance based eye-tracking and driver attention rating.

Project teaser image

Blink Detection

A blink detection algorithm on eye images tailored towards head-mounted eye-trackers.

Project teaser image

Eye Movements Identification

Approaches for segmentation and synthesis of eye-tracking data using different neural networks and machine learning approaches.

Project teaser image

Eye labeling tool

Ground truth data is an important prerequisite for the development and evaluation of many algorithms in the area of computer vision, especially when these are based on convolutional neural networks or other machine learning approaches that unfold their power mostly by supervised learning. This learning relies on ground truth data, which is laborious, tedious, and error prone for humans to generate. In this paper, we contribute a labeling tool (EyeLad) specifically designed for remote eye-tracking data to enable researchers to leverage machine learning based approaches in this field, which is of great interest for the automotive, medical, and human-computer interaction applications. The tool is multi platform and supports a variety of state-of-theart detection and tracking algorithms, including eye detection, pupil detection, and eyelid coarse positioning.

Project teaser image

EyeRecToo

In this paper, we introduce EyeRecToo, an open-source software for real-time pervasive head-mounted eye-tracking. Out of the box, EyeRecToo offers multiple real-time state-of-the-art pupil detection and gaze estimation methods, which can be easily replaced by user implemented algorithms if desired. A novel calibration method that allows users to calibrate the system without the assistance of a human supervisor is also integrated.

Project teaser image

Eyetrace

Eyetrace is a tool for analysis of eye-tracking data. It has the approach to bunch a variety of different evaluation methods for a large share of eye trackers supporting scientific work and medical diagnosis. To allow EyeTrace to be compatible to different eye trackers, an additional tool called Eyetrace Butler is used. The Eyetrace Butler performs a data preprocessing and conversion for analysis with Eyetrace. It provides plugins for different eye trackers and converts their data into a format that can be imported and used by Eyetrace.

Project teaser image

Intelligent Surgical Microscope

Head-mounted eye tracking offers remarkable opportunities for research and applications regarding pervasive health monitoring, mental state inference, and human computer interaction in dynamic scenarios. Although a plethora of software for the acquisition of eye-tracking data exists, they often exhibit critical issues when pervasive eye tracking is considered, e.g., closed source, costly eye tracker hardware dependencies, and requiring a human supervisor for calibration. In this paper, we introduce EyeRecToo, an open-source software for real-time pervasive head-mounted eye-tracking. Out of the box, EyeRecToo offers multiple real-time state-of-the-art pupil detection and gaze estimation methods, which can be easily replaced by user implemented algorithms if desired. A novel calibration method that allows users to calibrate the system without the assistance of a human supervisor is also integrated. Moreover, this software supports multiple head-mounted eye-tracking hardware, records eye and scene videos, and stores pupil and gaze information, which are also available as a real-time stream. Thus, EyeRecToo serves as a framework to quickly enable pervasive eye-tracking research and applications.

Project teaser image

Neural networks for optical vector and eye ball parameter estimation

In this work we evaluate neural networks, support vector machinesand decision trees for the regression of the center of the eyeballand the optical vector based on the pupil ellipse. In the evaluationwe analyze single ellipses as well as window-based approaches asinput. Comparisons are made regarding accuracy and runtime. Theevaluation gives an overview of the general expected accuracy withdifferent models and amounts of input ellipses. A simulator wasimplemented for the generation of the training and evaluation data.For a visual evaluation and to push the state of the art in opticalvector estimation, the best model was applied to real data. Thisreal data came from public data sets in which the ellipse is alreadyannotated by an algorithm. The optical vectors on real data and thegenerator are made publicly available.

Project teaser image

Robust Pupil Detection and Gaze Estimation

The reliable estimation of the pupil position in eye images is perhaps the most important prerequisite in gaze-based HMI applications. While there are many approaches that enable accurate pupil tracking under laboratory conditions, tracking the pupil in real-world images is highly challenging due to changes in illumination, reflections on glasses or on the eyeball, off-axis camera position, contact lenses, and many more.

Project teaser image

Scanpath Comparison

Our eye movements are driven by a continuous trade-off between the need for detailed examination of objects of interest and the necessity to keep an overview of our surrounding. In consequence, behavioral patterns that are characteristic for our actions and their planning are typically manifested in the way we move our eyes to interact with our environment. Identifying such patterns from individual eye movement measurements is however highly challenging.

Project teaser image

The applicability of Cycle GANs for pupil and eyelid segmentation, datageneration and image refinement

We evaluated Generative Adversarial Networks(GAN) for eyelid and pupil area segmentation, data gener-ation, and image refinement. While the segmentation GANperforms the desired task, the others serve as supportiveNetworks. The trained data generation GAN does not re-quire simulated data to increase the dataset, it simply usesexisting data and creates subsets. The purpose of the re-finement GAN, in contrast, is to simplify manual annota-tion by removing noise and occlusion in an image withoutchanging the eye structure and pupil position. In addition100,000 pupil and eyelid segmentations are made publiclyavailable for images from the labeled pupils in the wild dataset.

Project teaser image

Vishnoo - A Visual Search Examination Tool

Vishnoo (Visual Search Examination Tool) is an integrated framework that combines configurable search tasks with gaze tracking capabilities, thus enabling the analysis of both, the visual field and the visual attention.