System śledzenia punktu fiksacji wzroku na monitorze komputerowym System for eye gaze tracking on computer screen Mariusz Kurkowski Rafał Rybacki 21.03.2008r.

Current project conceptions Five IR LEDs idea LED switching hardware Eye-tracker project Parts of this presentation Current project conceptions Five IR LEDs idea LED switching hardware Image recording device Image processing

Eye gaze tracker Analog camera Digital USB camera + hardware LED sync Leading project conceptions Analog camera Digital USB camera + hardware LED sync Digital USB camera + software LED sync

System śledzenia punktu fiksacji wzroku na monitorze komputerowym KAMERA ANALOGOWA /ANALOGUE CAMER - Mgr Maciej Kulesza Prototyp urządzenia pozwalającego na oświetlanie gałki ocznej człowieka z wykorzystaniem promieniowania podczerwonego i akwizycję obrazów, w celu ich dalszego przetwarzania / Prototype of the device letting for lighting the eyeball of the man with using the infrared radiation and the canvassing of images, to the purpose of further processing

Eye gaze point is estimation method 5 LEDs algorithm Find the pupil area Find the center of the pupil The eye gaze point on screen is placed similarly to center point of the pupil between four glint points

Eye gaze point is estimation method 5 LEDs algorithm

System śledzenia punktu fiksacji wzroku na monitorze komputerowym HOW IT WORKS obraz „jasny” (4 LED) bright image „ciemny” (1 LED) dark image różnicowy, do określenia p. fiksacji wzroku differencing, for determining the eye gaze point

top of the screen left part of the screen

left-top corner left-bottom corner

center of the screen

System śledzenia punktu fiksacji wzroku na monitorze komputerowym -punk fiksacji jest zawsze w wielokącie tworzonym przez uzyskane z odbić punkty referencyjne / point of the fixation is always in the polygon -uproszczenie obliczeń związane z wyliczaniem geometrycznych zależności pomiędzy okiem, kamerą i monitorem / simplifying calculations -wykorzystanie zależności położeń odbić promieni LED na powierzchni tęczówki i środka źrenicy w wielokącie, a ich odzwierciedleniami na ekranie. / using relations of reflections of LED rays from the relation on the surface of the iris and the centre of the pupil in the polygon,and theirs reflections on the screen

LEDs controller Programmable LED switching using LPT port Features: -driven by data pins of LPT port (easy to implement) powered by USB power source (5V) output current intensity– 90 mA (22.5 mA per LED)

Image recording device Logitech 9000 PRO USB camera Camera features: -1440x960@15fps Possible difficulties: -IR-pass filter thickness (~0.35mm) cachting the autofocus wide angle of the camera lens software LED sync the cam IR camera shot

Image recording device Logitech 9000 PRO USB camera Camera features: -1440x960@15fps Possible difficulties: -IR-pass filter thickness (~0.35mm) cachting the autofocus wide angle of the camera lens software LED sync the cam std camera shot

Hardware vs software LEDs switching Two different conceptions Hardware switching: 100% synchronization Software switching : easy to implement, switching controlled

Frames delay measurement Logitech 9000 PRO USB camera Frames delay causes the possibility that LEDs will be switched asynchronusly to image acquisition on camera CCD matrix. system slowdown

Frames delay measurement Crucial for proper software switching Frames delay causes the possibility that LEDs will be switched asynchronusly to image acquisition on camera CCD matrix. Frames delay measurement results Mean delay between image acquisition and the interception by software: 134.38 ms (frame period – 84 ms) Standard deviation: 38 ms Frames lost: 3.9% @12FPS Frames lost with delay prediction alghoritm: 2.8%

Implementation environment Microsoft Visual Studio C++ Warstwa software / Software: Oprogramowanie kamery oraz całego systemu stworzone w środowisku Microsoft Visual Studio 2005 Microsoft Visual Studio as the environment of developement Oprogramowanie w języku C++ C++ environment Niezbędne operacje graficzne oparte o silnik biblioteki OpenCV graphical operations based at OpenCV library

Thank you for your attention IEEE: Non-contact Eye Gaze Tracking System by Mapping of Corneal Reflections Dziękujemy za uwagę Thank you for your attention