advertisement

GaZIR: Gaze-based Zooming Interface for Image Retrieval

67 %
33 %
advertisement
Information about GaZIR: Gaze-based Zooming Interface for Image Retrieval
Technology

Published on February 13, 2014

Author: franzonadiman

Source: slideshare.net

advertisement

Intelligent Multimodal Interaction 2014 Francesco Bonadiman Craig Kershaw GaZIR “Gaze-based Zooming Interface for Image Retrieval” László Kozma, Arto Klami, Samuel Kaski Helsinki Institute for Information Technology HIIT

What is GaZIR? ● Gaze-based interface for searching and browsing images ○ Collecting information from what the user would do naturally via eye-tracking (implicit feedback) ● The user can zoom-in and out ○ Focusing on the centre or the borders ○ Allowing Image retrieval

Put a Ring on it! ● Consists of 3 Rings of images ○ Each consecutive ring shows the next set of relevant images based on information gathered from the previous ring ● Better for predicting image relevancy ○ Avoids users scanning images row-by-row, as with grid-based layouts

Rings

Eye Tracking ● Eye-movements tracking user’s pupils ○ Fixation of >120ms → Relevant image ● 3 main advantages of Eye-Tracking ○ Effortlessness, user only needs to look at the images ○ “I-will-know-it-when-I-see-it” search problems ○ Hands are not needed → Motor disabilities

Similar Work ● Only preliminary studies ● Oyekoya et al. ○ Simple retrieval → relevance from viewing time ● Klami et al. ○ More complex predictions ○ Only measure isolated predictions ○ Artificial setup

… and GaZIR? ● GaZIR → combines two approaches ○ More sophisticated relevance predictor ○ Real retrieval search engine ○ Gaze-based interaction designed interface

Aim of the research paper ● To provide a user interface that is more fluid and natural with searching ● Test whether it is feasible to construct and if it works in practice with pre-existing CBIR search engines ○ Designed to work with any CBIR engine

Data collection ● Simplifications were made ○ ○ ○ ○ user was only expected to zoom inwards not allowed to reset the process images only retrieved when zooming-in mouse wheel used for zooming (no eye control) ● Training data collected to create a model ○ show images closer to users’ expectations

Experiment 1 ● 6 different users ● Each of them performing 6 search tasks ○ look into the MirFlickr database ○ search images matching the category description ○ indicate which ones were relevant ● On average around 120 images ○ eye movement over 4300 user-task-image instances

Experiment 2 ● 3 average users from the previous ● 6 new search tasks: ○ 2 with the gaze-based relevance predictor ○ 2 with a dummy interface ○ 2 same interface + explicit feedback (mouse click) ● Performances measurement: ○ by counting the proportion of relevant images

Results ● Prediction accuracy > random for all users ○ confirms “relevance through eye movements” ● Huge differences between the users ○ ○ ○ ○ ○ due to different tasks or use of the system for some prediction accuracy → excellent for others → slightly better than random explicit feedback (mouse click) → the best predicted feedback → comparable for 50% of tasks

Contribution ● Distinction “false positive and negative” ○ former: look similar to the relevant but miss details ○ latter: images (too) easy to recognize as relevant ● Promising results → further experiments ● GaZIR is concluded to be ○ “first attempt of building a sophisticated image retrieval interface utilizing implicit gaze information”

Thank you Any questions?

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

GaZIR: Gaze-based zooming interface for image retrieval ...

Page 1. GaZIR: Gaze-based Zooming Interface for Image Retrieval László Kozma laszlo.kozma@tkk.fi Arto Klami arto.klami@tkk.fi Samuel Kaski samuel.kaski@ ...
Read more

GaZIR: Gaze-based Zooming Interface for Image Retrieval

GaZIR: Gaze-based Zooming Interface for Image Retrieval László Kozma Department of Information and Computer Science Helsinki University of Technology
Read more

GaZIR: gaze-based zooming interface for image retrieval (2009)

BibTeX @INPROCEEDINGS{Kozma09gazir:gaze-based, author = {László Kozma and Arto Klami and Samuel Kaski}, title = {GaZIR: gaze-based zooming interface for ...
Read more

GaZIR - dl.acm.org

GaZIR: gaze-based zooming interface for image retrieval. Full Text: PDF: Authors: ... A Novel Image Retrieval System with Real-Time Eye Tracking, ...
Read more

GaZIR: gaze-based zooming interface for image retrieval ...

... gaze-based zooming interface for image retrieval. ... GaZIR: Gaze-based Zooming Interface for Image ... images by zooming in the interface, ...
Read more

Report on eye-movement-based user interface studies

Report on eye-movement-based user interface ... GaZIR: Gaze-based Zooming Interface for Image ... based zooming interface for image retrieval.
Read more

Arto Klami: Publications, bio, bibliography, etc ...

... (2009): GaZIR: gaze-based zooming interface for image retrieval. In: Proceedings of the 2009 International Conference on Multimodal Interfaces ...
Read more

László Kozma - Aalto

László Kozma, Arto Klami, Samuel Kaski: GaZIR: Gaze-based Zooming Interface for Image Retrieval, ... A Proactive Interface for Image Retrieval, ...
Read more

Samuel Kaski: Publications, bio, bibliography, etc ...

Samuel Kaski: Publications, bio, bibliography, ... (2009): GaZIR: gaze-based zooming interface for image ... and Development in Information Retrieval ...
Read more