Automatisierter Scanpath Vergleich

chair/team/thomas-kueblerchair/team/enkelejda-kasneciMy research concerns all aspects of eye-tracking, from the design of a physical recording device, the necessary image processing steps via classical computer vision as well as DNNs, to the high-level interpretation of recorded gaze sequences through machine learning. I work on assessing data quality, visualization and gaze analysis tools in various applications in the medical, educational, automotive and art historian fields. With my Spin-Off Look! we develop eye-tracking solutions for automotive applications as well as in-vehicle head-mounted devices for driving schools and instructor teaching. Our algorithms for the registration and analysis of gaze data are well suited to infer cognitive processes such as attention and vigilance solely from the movement of the eyes - and are therefore an excellent complementary factor to traditional measures such as perclos, head movements or gaze-on-road. This adds to robustness and sensitivity of driver monitoring systems by enabling the vehicle to sense the current attentional state of the driver. Driving instructors are enabled by our head-mounted eye-tracking devices to see the streets through the eyes of their students. That way they can provide efficient feedback and speed up the learning process by making students aware of the importance of correct visual exploration of their surroundings. {% aside %}
{% endaside %} {% box %} ## Research Interests * Gaze-based driver assistance and monitoring systems * Computational models of human gaze behavior * Algorithms for the comparison of exploratory gaze sequences * Eye-tracking data quality in real-world applications * Algorithms and tools for the analysis and visualization of eye-tracking data * Head-mounted eye-tracking hardware {% endbox %} {% box %} Enkelejda Kasneci is a Professor of Computer Science at the University of Tübingen, Germany, where she leads since December 2019 the Human-Computer Interaction Lab. As a BOSCH scholar, she received her M.Sc. degree in Computer Science from the University of Stuttgart in 2007. In 2013, she received her PhD in Computer Science from the University of Tübingen. For her PhD research, she was awarded the research prize of the Federation Südwestmetall in 2014. From 2013 to 2015, she was a postdoctoral researcher and a Margarete-von-Wrangell Fellow at the University of Tübingen. She established her research and teaching activities at the University of Tübingen from 2015 to 2019 as an assistant professor and head of the Perception Engineering Lab. Her research evolves around the application of machine learning for intelligent and perceptual human-computer interaction. She serves as academic editor for PlosOne and as a TPC member and reviewer for several major conferences and journals in the areas of intelligent and multimodal human-computer interaction, computer vision, and cognitive computing. {% endbox %} {% box %} ## Research Interests * Multimodal Human-Computer Interaction * Eye-tracking Methods and Applications, especially eye tracking in the wild * Eye movements and VR/AR * Cognitive Computing * Applied Machine Learning {% endbox %} {% box %} ## Selected Invited taks (since 2015) * 2021: Invited talk at the IEEE VR 2021 Workshop “Towards a roadmap for privacy and security research for mixed reality applications” * 2021: Invited talk at CHI 2021 Workshop on Eye Movements as an Interface to Cognitive State * 2021: Invited Keynote at Adobe Research, Cambridge, MA, USA * 2020: GPRC/VMV 2020: Enhancing user models through visual scanpath analysis * 2020: ECCV 2020 OpenEyes: Eye Gaze in AR, VR and in the wild * 2020: Closing Keynote at Augmented Human, AH 2020 * 2019: Machine Learning in Education, FernUni Hagen * 2019: Keynote at the ACM Symposium on Eye Tracking Research and Applications, ETRA 2019 * 2019: Opening Keynote European Conference on Eye Movements, ECEM 2019 * 2018: Keynote on Cognitive Interfaces, Ada-Lovelace-Festival, Berlin * 2018: Global Female Leaders Summit, Berlin * 2017: Hub.Berlin * 2017: Ada-Lovelace-Festival, Berlin * 2017: Eye-Tracking während des Fahrens, [OCG Jahresopening 2017, Vienna](http://www.ocg.at/de/jahresopening2017) * 2016: It’s in your eyes – How eye tracking will shape our future, [Ada-Lovelace-Festival, Berlin](http://wiwo.konferenz.de/ada/en/speaker-2016/prof-dr-enkelejda-kasneci/) * 2016: Maschinelles Lernen und Eye-Tracking-Technologie zur Erforschung der Mechanismen der visuellen Wahrnehmung, [INFORMATIK2016, Klagenfurt](http://www.informatik2016.de/1273.html) {% endbox %} {% box %} ## Scholarships, Awards and Administrative Functions * Since 10/2020: Dean of studies (Studiendekanin), [Department of Computer Science](https://uni-tuebingen.de/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/fachbereich/) * Since 2020: [Member of the Cyber Valley](https://cyber-valley.de/de/people) * 2020 Ruf auf eine W3 Professur an der TU Chemnitz (abgelehnt) * Since 2019 Full member of DFG [Cluster of Excellence](https://uni-tuebingen.de/en/research/core-research/cluster-of-excellence-machine-learning/people/team/cluster-members/#c1023408) - Machine Learning of Science * since 2016 [Associated Member of the Learning, Educational Achievement, and Life Course Development (LEAD), Graduate School and Research Network](https://www.uni-tuebingen.de/en/research/core-research/lead-graduate-school-and-research-network/members.html) * since 2016 [Junior Fellow of the German Informatics Society (Gesellschaft für Informatik e.V. (GI))](https://www.gi.de/aktuelles/meldungen/detailansicht/article/gi-junior-fellows-ernannt.html) * since 2016 Fast Track program of the Robert Bosch Stiftung (Excellence and Leadership Skills for Outstanding Women in Science) * 2016 NVIDIA Hardware Grant * 2014 - 2015 Margarete-von-Wrangell-Fellowship * 2014 [Südwestmetall-Förderpreis 2014 für Nachwuchswissenschaftler](https://www.suedwestmetall.de/swm/web.nsf/id/li_sweb9hzgnz.html?Open&lang=DE) * 2001-2007 Scholarship of the Robert-Bosch-Stiftung {% endbox %}

Menschen können ihre Aufmerksamkeit gezielt lenken. Eine Methode diese Aufmerksamkeitszuweisung aufzuzeichnen sind Eye-Tracking Aufnahmen: Objekte, die momentan interessant sind, werden mit den Augen fixiert. Oftmals ist es nicht nur interessant, wohin einzelne Individuen ihre Aufmerksamkeit lenken, sondern auch ein Vergleich zwischen Individuen oder verschiedenen Zeitpunkten. Hierzu werden diese auf einfache Elemente reduziert: Fixationen und Sakkaden (schnelle Augenbewegungen). Die zeitliche Abfolge und räumliche Position von Fixationen und Sakkaden nennt man Scanpath. Aktuelle Vergleichsmetoden basieren auf zwei verschiedenen Ansätzen:

  • Umwandlung des Scanpaths in eine String-Repräsentation und Anwenden bioinformatischer String-Alignment Techniken.
  • Vergleich der Scanpaths als Vektorpfad.

Beide Methoden zeichne sich durch massive Einschränkungen und sehr begrenzte Anwendbarkeit aus: Sie funktionieren generell nur bei einfachen Scanpaths und unbewegtem Betrachter gut. Deshalb ist die Analyse von Hand noch immer sehr verbreitet, obwohl hierdurch Objektivität verloren geht und viel Zeit nötig ist.

Aufgabenstellung

Ziel dieser Arbeit ist die Verbesserung bestehender Scanpath Vergleichsmethoden. Dies beinhaltet zum einen das Finden geeigneter Distanzmetriken, zum anderen die Anwendung von Pattern-Recognition Verfahren. Das Erkennen und Wiederfinden kurzer Wiederholter Muster (Schulterblick oder Blick in den Rückspiegel beim Autofahren) könnte die momentane Verfahren deutlich verbessern.

alt "Automatisierter Scanpath Vergleich"
PDF