Quality of Sensory Experience (QuaSE)

29 %
71 %
Information about Quality of Sensory Experience (QuaSE)
Technology

Published on October 13, 2014

Author: christian.timmerer

Source: slideshare.net

Description

This presentation was given at the final QUALINET workshop in Delft and follows the general principle that there is a need for a scientific framework to capture, measure, quantify, judge, and explain the quality of (sensory) experience.

1. Quality of Sensory Experience (QuaSE) Chris8an Timmerer Alpen-­‐Adria-­‐Universität Klagenfurt (AAU) w Faculty of Technical Sciences (TEWI) w Department of Informa8on Technology (ITEC) w Mul8media Communica8on (MMC) w Sensory Experience Lab (SELab) blog.&mmerer.com w selab.itec.aau.at w dash.itec.aau.at chris&an.&mmerer@itec.aau.at QUALINET Final Workshop October 10, 2014 Acknowledgments. This work was supported in part by the European Commission in the context of the NoE INTERMEDIA (NoE 038419), the P2P-­‐Next project (FP7-­‐ICT-­‐216217), the ALICANTE project (FP7-­‐ICT-­‐248652), the SocialSensor project (FP7-­‐ICT-­‐287975), and the COST Ac&on IC1003 QUALINET.

2. Sensory Experience • Consump&on of mul&media content may s&mulate also other senses – Vision or hearing – Olfac&on, mechanorecep&on, thermocep&on, … • Annota&on with metadata providing so-­‐called sensory effects that steer appropriate devices capable of rendering these effects 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 2

3. Sensory Experience • Consump&on of mul&media content may s&mulate also other senses – Vision or hearing – Olfac&on, mechanorecep&on, thermocep&on, … • Annota&on with metadata providing so-­‐called sensory effects that steer appropriate devices capable of rendering these effects … giving her/him the sensa8on of being part of the par8cular mulsemedia ➪ worthwhile, informa8ve user experience 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 3

4. General Principle – Outline • General principle: there is a need for a scien8fic framework to capture, measure, quan,fy, judge, and explain the quality of (sensory) experience • Outline – [How to create, delivery, consume?] – How to capture and measure? – How to quan&fy? – How to judge and explain? 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 4

5. How to create, delivery, consume? • Sensory Effect Descrip&on Language (SEDL) – Basic building blocks to describe, e.g., light, wind, fog, vibra&on, scent – MPEG-­‐V Part 3, Sensory Informa&on: Effects, GroupOfEffects – Adopted MPEG-­‐21 DIA tools for adding &me informa&on (synchroniza&on) • Descrip&on conforming to SEDL :== Sensory Effect Metadata (SEM) – Can be associated to any kind of mul&media content (e.g., movies, music, Web sites, games) – Support to be included in file (MP4) and transport (M2TS) formats • Tool support for crea&ng (annota&on tools) and consump&on (players, Web plugins) ➜ selab.itec.aau.at • Devices: e.g., amBX (Ambient Experience) system + SDK, Gameskunk, Scentscape, etc. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 5

6. How to capture and measure? • Subjec&ve quality assessments – Methodology: based on standard methods – Test content: different genres, manually annotated (cf. QUALINET DB) • Experiment I – Aim: Demonstrate sensory effects as a vital tool for enhancing the quality of experience depending on the actual genre • Experiment II – Aim: inves&gate the rela&onship of the QoE to various video bit-­‐rates of mul&media contents annotated with sensory effects. – Subjec&ve quality gap between video resources annotated with and without sensory effects at different bit-­‐rates • [Experiment III] ambient lights & different color calcula&on seqngs • Experiment IV – Aim: inves&gate the enhancement of the QoE and how users’ emo&ons are elicited and influenced by Web videos annotated with and without sensory effects 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 6

7. Experiment II: Results Test Sequences Sequence Babylon A.D. Earth Duration 35s 21s Resolution 1280 x 544 1280 x 720 Motion High Low Nr. of Effects W: 7; V: 9 W: 8; V: 1 Bit-rates Kbit/s PSNR Kbit/s PSNR Low Quality 2154 38.93 2204 38.11 Medium Quality 3112 41.27 3171 40.65 High Quality 4044 42.95 4116 42.27 Highest Quality 6315 N/A 6701 N/A MOS vs. PSNR/bit-­‐rate for Earth. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 7

8. How to quan&fy? • Experiment V – Aim: towards a quality/u&lity model for QuaSE • S&muli with all combina&ons of sensory effects – Vibra&on higher impact than light & wind – Highest QoE with all effects present • General QuaSE model 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 8

9. How to judge and explain? • Experiment VI – Aim: understand QuaSE • Biosensor-­‐based QoE evalua&on system J. Donley, C. Ritz, M. Shujau, "Analysing the Quality of Experience of Mul&sensory Media from Measurements of Physiological Responses,” QoMEX2014, Singapore, Sep. 2014. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 9

10. How to judge and explain? • Experiment VII – Aim: understand QuaSE • EEG Correlates of Pleasant and Unpleasant Odor Percep&on E. Kroupi, A. Yazdani, J.-­‐M. Vesin, T. Ebrahimi, "EEG Correlates of Pleasant and Unpleasant Odor Percep&on," ACM TOMM, vol. 11, no. 1s, Sep. 2014. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 10

11. How to judge and explain? • Experiment VIII – Aim: understand QuaSE • Mul&ple-­‐Scent Enhanced Mul&media Synchroniza&on General temporal boundaries: -­‐10s to +15s are “in-­‐sync”, skew values beyond are “out-­‐of-­‐sync” N. Murray, B. Lee, Y. Qiao, and G.-­‐M. Muntean, "Mul&ple-­‐Scent Enhanced Mul&media Synchroniza&on," ACM TOMM, vol. 11, no. 1s, Sep. 2014. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 11

12. Conclusions • From the need for a scien8fic framework to capture, measure, quan,fy, judge, and explain the quality of experience • To … – How to create, delivery, consume? – How to capture and measure? – How to quan&fy? – How to judge and explain? • Open issues? ✔︎ ✔︎ ✔︎ ✔︎ Many! 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 12

13. Sensory Experience Lab hyp://selab.itec.aau.at/ Sozware and Services Standardiza&on Publica&ons Media Funding 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 13

14. References • Markus Waltl, Chris&an Timmerer, Hermann Hellwagner, “A Test-­‐Bed for Quality of Mul8media Experience Evalua8on of Sensory Effects”, Proceedings of the First Interna&onal Workshop on Quality of Mul&media Experience (QoMEX 2009), San Diego, USA, July 29-­‐31, 2009. • C. Timmerer, J. Gelissen, M. Waltl, H. Hellwagner, “Interfacing with Virtual Worlds”, Proceedings of the NEM Summit 2009, Saint-­‐Malo, France, September 28-­‐30, 2009. • M. Waltl, C. Timmerer H. Hellwagner, “Increasing the User Experience of Mul8media Presenta8ons with Sensory Effects”, Proceedings of the 11th Interna&onal Workshop on Image Analysis for Mul&media Interac&ve Services (WIAMIS’10), Desenzano del Garda, Italy, April 12-­‐14, 2010. • M. Waltl, C. Timmerer, H. Hellwagner, “Improving the Quality of Mul8media Experience through Sensory Effects”, Proceedings of the 2nd Interna&onal Workshop on Quality of Mul&media Experience (QoMEX2010), Trondheim, Norway, June 21-­‐23, 2010. • C. Timmerer, M. Waltl, B. Rainer, H. Hellwagner, “Assessing the Quality of Sensory Experience for Mul8media Presenta8ons”, Signal Processing: Image Communica8on, 2012. • M. Waltl, C. Timmerer, B. Rainer, H. Hellwagner, ”Sensory Effects for Ambient Experiences in the World Wide Web”, Mul&media Tools and Applica&ons (MTAP), 2012. • B. Rainer, M. Waltl, Eva Cheng, Muawiyath Shujau, C. Timmerer, Stephen Davis, Ian Burney, Chris&an Ritz, H. Hellwagner, ”Inves8ga8ng the Impact of Sensory Effects on the Quality of Experience and Emo8onal Response in Web Videos”, Proceedings of 4th Int’l. Workshop on Quality of Mul&media Experience (QoMEX2012), 2012. • M. Waltl, C. Timmerer, B. Rainer, H. Hellwagner, “Sensory Effect Dataset and Test Setups”, Proceedings of 4th Int’l. Workshop on Quality of Mul&media Experience (QoMEX2012), 2012. • C. Timmerer, B. Rainer, M. Waltl, “A U8lity Model for Sensory Experience”, In Proceedings of the 5th Interna&onal Workshop on Quality of Mul&media Experience (QoMEX2013), 2013. • J. Donley, C. Ritz, M. Shujau, “Analysing the Quality of Experience of Mul8sensory Media from Measurements of Physiological Responses,” In Proceedings of the 5th Interna&onal Workshop on Quality of Mul&media Experience (QoMEX2014), Singapore, 2014. • E. Kroupi, A. Yazdani, J.-­‐M. Vesin, T. Ebrahimi, "EEG Correlates of Pleasant and Unpleasant Odor Percep8on," ACM TOMM, vol. 11, no. 1s, Sep. 2014. • N. Murray, B. Lee, Y. Qiao, and G.-­‐M. Muntean, "Mul8ple-­‐Scent Enhanced Mul8media Synchroniza8on," ACM TOMM, vol. 11, no. 1s, Sep. 2014. • G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. "Mulsemedia: State of the Art, Perspec8ves, and Challenges," ACM TOMM, vol. 11, no. 1s, Sep. 2014. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 14

15. Thank you for your ayen&on ... ques&ons, comments, etc. are welcome … Assoc-­‐Prof. Dipl.-­‐Ing. Dr. Chris&an Timmerer Klagenfurt University, Department of Informa&on Technology (ITEC) Universitätsstrasse 65-­‐67, A-­‐9020 Klagenfurt, AUSTRIA chris&an.&mmerer@itec.uni-­‐klu.ac.at hyp://research.&mmerer.com/ Tel: +43/463/2700 3621 Fax: +43/463/2700 3699 © Copyright: Chris.an Timmerer 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 15

16. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 16

17. 2014/10/10 Chris&an Timmerer, Alpen-­‐Adria-­‐Universität, Austria 17

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Quality of Sensory Experience (QuaSE) - Technology

1. Quality of Sensory Experience (QuaSE) Chris8an Timmerer Alpen-­‐Adria-­‐Universität Klagenfurt (AAU) w Faculty ...
Read more

Towards Quality of Sensory Experience in Multimedia

Towards Quality of Sensory Experience in Multimedia He Xu ... immersion as described earlier in the Quality of Sensory Experience (QuaSE).
Read more

Sensory Experience: Quality of Experience Beyond Audio ...

... {Timmerer2014_QuASE, language ... Markus and Rainer, Benjamin and Murray, Niall}, title = {Sensory Experience: Quality of Experience Beyond ...
Read more

Towards Quality of Sensory Experience in Multimedia - COST

Towards Quality of Sensory Experience in Multimedia He Xu He.Xu@epfl.ch Fernando Pereira Fernando.Pereira@lx.it.pt ... Multi-sensory media ...
Read more

Sensory Experience: Quality of Experience Beyond Audio ...

Sensory Experience: Quality of Experience Beyond ... concept of Sensory Experience which aims to ... to the quality of the user experience.
Read more

A Generic Utility Model Representing the Quality of ...

A Generic Utility Model Representing the Quality of Sensory Experience. Full ... a utility model representing the Quality of Sensory Experience (QuaSE) ...
Read more

A Generic Utility Model Representing the Quality of ...

A Generic Utility Model Representing the Quality of Sensory Experience on ResearchGate, the professional network for scientists.
Read more

Sensory Experience | LinkedIn

View 2851 Sensory Experience posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn.
Read more