Mobile Interaction Based on Human Gesture Analysis

100 %
0 %
Information about Mobile Interaction Based on Human Gesture Analysis

Published on June 12, 2008

Author: tjvguerreiro

Source: slideshare.net

Description

Details in:
http://m-accessibility.blogspot.com

This presentation is focused on mnemonical body shortcuts (check http://immi.inesc-id.pt/~tjvg/ for Publications on the subject). In the presentation we detail information we can obtain from an accelerometer and how it can be used to improve mobile device interaction.

The presentation was performed at ISHF 2007.

Mobile Interaction Based on Human Gesture Analysis Ricardo Gamboa Tiago Guerreiro, Joaquim Jorge {rjssg,tjvg,jaj}@immi.inesc-id.pt Hugo Gamboa [email_address]

Actual Interaction Successful & Appropriate?

Screen and Keypad size are Limited

Multi-Task Graphical-based devices

Slow and Visually Demanding Interaction Mobility Issues!

Possible solution Key Shortcuts

Memory Issues – Where is each shortcut?

Other Solution Voice shortcuts

Voice Recognition Issues

Low Social Acceptance

Proposed Solution Gestural Interface!

Gestures ease Communication

Task Analysis - User observation Actual panorama on mobile usage and shortcutting habits. Results proved: Mobile Interaction is “keystroke consuming” Shortcuts are ineffective.

Propposed Approach – Mnemonical Body Shortcuts “ Phone in Silence Mode”

Propposed Approach – Mnemonical Body Shortcuts “ Phone your Girlfriend”

Related Work RFID Accelerometers EMG Cameras Touch Screens

RFID Prototype Pocket LOOX 720 ACG RF RFID reader RFID Tags

Evaluation

Mnemonical Body Shortcuts Evaluation Mouth Hand Chest Head Wrist Eye Finger Ear SMS 10 1 6 Call 3 1 12 Contacts 3 5 2 1 Clock 10 1 Photos 2 8 Calculator 3 Mp3 2 Agenda 1 3 1 Alarm-clock 2 2 2 3

20 Users 5 chosen Applications 20 Shortcuts Key Shortcuts Vs Mnemonics 94% Recognition Rate

Accelerometer ADXL 330 MEMS Bioplux4 Device

Why Accelerometers?

Accelerometer Applicable in Contextual & Explicit Human Motion

Contextual Interaction

Stopped Holding Picked Walking Running Stopped Movement Analysis - Amplitude Holding Time (s) Acc (g) Amp = √( x^2+y^2+z^2 )

Fall Detection

Explicit Interaction Explicit Interaction

Mnemonical Body Shortcuts X Y

Mnemonical Body Shortcuts Position – Y axis Acceleration Calibrated Acceleration Thresholded

Mnemonical Body Shortcuts Position – Y axis ∫ dt ∫ dt Velocity Position

Mnemonical Body Shortcuts Position + Final Rotation

Mnemonical Body Shortcuts Evaluation – Default Gestures TOTAL RECOGNITION 82% Mouth - 85% Chest – 85% Navel – 90% Shoulder - 75% Neck – 100% Ear - 60% Head – 85% Leg – 80% Wrist – 85% Eye – 75% 10 users – 5 Gestures – No training – 20 Recognitions each

Mnemonical Body Shortcuts Evaluation – Trainable Gestures TOTAL RECOGNITION 71% 10 users – 5 Gestures – 5x Train for each gesture – 20 Recognitions Leg - 85% Mouth – 94% Navel – 64% Neck – 95% Ear - 55% 5 most common gestures

Mnemonical Body Shortcuts Discussion Default Gestures Good Recognition Rate. Limited to 10 pre-defined gestures. Users have to learn the gestures. Treinable gestures Lower Recognition Rate – Similar gestures are choosen. Position isn’t very effective to desambiguate gestures outside x,y plan. One training error spoils the recognition – outlier detection is needed.

Default Gestures

Good Recognition Rate.

Limited to 10 pre-defined gestures.

Users have to learn the gestures.

Treinable gestures

Lower Recognition Rate – Similar gestures are choosen.

Position isn’t very effective to desambiguate gestures outside x,y plan.

One training error spoils the recognition – outlier detection is needed.

Tilt – Angle Calculation Time (s/1000) Angle (degrees) X Y Z

Tilt - Centralization and Joining Y & Z X Y + Z Angle (degrees) - start position variation Time (s/1000)

Tilt - Thresholding X Y + Z Angle (degrees) - start position variation Time (s/1000)

Tilt - Decision LEFT RIGHT UP DOWN X Y + Z Time (s/1000) Angle (degrees)

Tilt Evaluation

Tilt Results - Recognition LEFT TILT 85% RIGHT TILT 94% UP TILT 75% DOWN TILT 93% TOTAL RECOGNITION 86%

Future Work Mobility tests on Mnemonical Body Shortcuts Try a Feature Based Algorithm Design a full prototype ( + feedback and shortcuts)

Mobility tests on Mnemonical Body Shortcuts

Try a Feature Based Algorithm

Design a full prototype

( + feedback and shortcuts)

Add a comment

Related presentations

Related pages

Mobile Interaction Based On Human Gesture Analysis (PDF ...

Conference: International Symposium on Measurement, Analysis and Modeling of Human Functions (ISHF 2007)
Read more

Mobile Interaction Based On Human Gesture Analysis | Tiago ...

We're trying Google Ads to subsidize server costs. If you are logged in, you won't see ads. Hover to learn more.
Read more

MMoobbbiiillleee n IIInnttteeerrraaaccctttiiiooonnn a ...

MOBILE INTERACTION BASED ON HUMAN GESTURE ANALYS IS Gamboa Ricardo, Guerreiro Tiago, Gamboa Hugo, Jorge Joaquim are common cultural gestures). Both ...
Read more

User-Defined Motion Gestures for Mobile Interaction

User-Defined Motion Gestures for Mobile Interaction ... the prior work on classifying human gestures ... physical gesture input as an add-on to voice-based
Read more

Gesture-Based Human-Computer Interaction and Simulation ...

Gesture-Based Human-Computer Interaction and Simulation - 7th International Gesture Workshop, GW 2007, Lisbon, Portugal, May 23-25, 2007, ...
Read more

Gesture-Based Human-Computer Interaction and Simulation ...

This book constitutes the thoroughly refereed post-proceedings of the 7th International Workshop on Gesture-Based Human-Computer Interaction ... mobile ...
Read more

Patent US9261979 - Gesture-based mobile interaction ...

Motion of a device is sensed using image data, and a gesture corresponding to the sensed motion of the device is recognized. Functionality of the device ...
Read more

Gesture recognition - Wikipedia, the free encyclopedia

Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms.
Read more

One-against-all frame differences based hand detection for ...

Hand gesture based human–computer ... Hand gesture based human–computer interaction with mobile phones ... the hand gesture analysis system ...
Read more