Published on February 27, 2014
Using Augmented Reality to Create Empathic Experiences Mark Billinghurst firstname.lastname@example.org The HIT Lab NZ, University of Canterbury February 27th 2014
Courtesy Matt Rettig, CMU
Processing Power Adapt Experience Operate
Beyond the Desktop
Intelligent User Interfaces AI + HCI: User Interface involving some elements of Artificial Intelligence Computer having model of user/domain First IUI Conference in 1997 Readings in IUI (Wahster 1998)
Microsoft Clippy (1997) MS Office Intelligent User Interface
Intelligent User Interfaces
Multiple Intelligences Frames of Mind: The Theory of Multiple Intelligences Howard Gardner (1983)
Emotional Intelligence Emotional Intelligence - Why it can matter more than IQ Goldman (1995) Identify, assess, and control the emotions of oneself, of others, and of groups
Foundations of Emotional Intelligence
Empathy vs. Intelligence Intelligence: the power of one’s brain, divided into many categories and used in numerous ways. Empathy: the power of one’s heart, expressing one’s true emotions with themselves, those around them, and their own world.
IQ vs EQ?
Mirror Neurons Neuron that fires both when an animal acts and it observes the same action performed by another Giacomo Rizzolatti, Univ. of Palma (1980s/90s)
Empathic Computing 1. Computing systems that can understand your feelings and emotions 2. Computing systems that help you better understand the feelings of others
Affective Computing Ros Picard – MIT Media Lab http://affect.media.mit.edu
Appliances That Make You Happy Jun Rekimoto – Univ. Tokyo Smile detection + smart devices
Can we develop interfaces that allow us to be more empathetic to others?
Empathy Computing Requirements Basic Requirements Making the technology transparent Empathy Definition Seeing with the eyes of another Hearing with the ears of another Feeling with the heart of another
Augmented Reality 1977 2008
Using AR for Empathy Augmented Reality can: Remove technology barriers Enhance communication Change perspective Share experiences Enhance interaction in real world
Communication Seams Communication Space Task Space Technology introduces artificial seams in the communication (eg separate real and virtual space)
Removing Barriers: Shared Space Face to Face interaction, Tangible AR metaphor - ~3,000 users (Siggraph 1999) Easy collaboration with strangers Users acted same as if handling real objects Billinghurst, M., Poupyrev, I., Kato, H., & May, R. (2000). Mixing realities in shared space: An augmented reality interface for collaborative computing. In Multimedia and Expo, 2000. ICME 2000. 2000 IEEE International Conference on (Vol. 3, pp. 1641-1644).
Enhancing Face to Face Communication AR Pad Handheld AR device AR shows viewpoints Users collaborate easier Show communication cues Virtual Viewpoint Visualization Mogilev, D., Kiyokawa, K., Billinghurst, M., & Pair, J. (2002, April). AR Pad: An interface for face-to-face AR collaboration. In CHI'02 extended abstracts on Human factors in computing systems (pp. 654-655).
Changing Perspective CamNet (1992) British Telecom Wearable Teleconferencing audio, video Remote collaboration Sends task space video Similar CMU study (1996) cut performance time in half
WACL: Remote Expert Collaboration Wearable Camera/Laser Pointer Independent pointer control Remote panorama view
WACL: Remote Expert Collaboration Remote Expert View Panorama viewing, annotation, image capture Kurata, T., Sakata, N., Kourogi, M., Kuzuoka, H., & Billinghurst, M. (2004, October). Remote collaboration using a shoulder-worn active camera/laser. In Wearable Computers, 2004. ISWC 2004. Eighth International Symposium on (Vol. 1, pp. 62-69).
View Through Google Glass Always available peripheral information display Combining computing, communications and content capture
Ego-Vision Collaboration Google Glass camera + processing + display + connectivity
Current Collaboration on Glass First person remote conferencing/hangouts Limitations Single POV, no spatial cues, no annotations, etc
Sharing Space: Social Panoramas Capture and share social spaces in real time Enable remote people to feel like they’re with you
Key Technology Google Glass Capture live image panorama (compass + camera) Capture spatial audio, live video Remote device (desktop, tablet) Immersive viewing, live annotation
Capturing Space: Real World Capture Hands free AR Portable scene capture (color + depth) Projector/Kinect combo, Remote controlled pan/tilt Remote expert annotation interface
Remote Expert View
Capturing Behaviours 3 Gear Systems Kinect/Primesense Sensor Two hand tracking http://www.threegear.com
Skeleton Interaction + AR HMD AR View Viewpoint tracking Two hand input Skeleton interaction, occlusion
Ghostman Use AR to capture and overlay your actions into remote persons space Eg remote therapy
Looking to the Future What’s Next?
Brain to Brain Control Rajesh Rao, University of Washington First Brain to Brain control
Scaling Up Seeing actions of millions of users in the world Augmentation on city/country level
AR + Smart Sensors + Social Networks Track population at city scale (mobile networks) Match population data to external sensor data medical, environmental, etc Mine data to improve social services
Research Challenges How to convey emotion? How to measure empathy? Interface/interaction models? How to communicate emotion? Scaling up to city/country scale?
Harvard Grant Study $20 million, 75 years study 268 Harvard graduates 456 disadvantaged people Led by George Valliant What makes us happy? warmth of relationships throughout life have the greatest positive impact on "life satisfaction".
“The seventy-five years and twenty million dollars expended on the Grant Study points to a straightforward five-word conclusion: Happiness is love. Full stop.” George Valliant
Conclusions Empathic Computing Sharing what you see, hear and feel AR Enables Empathic Experiences Removing technology Changing perspective Sharing space/experience Many directions for future research
More Information • Mark Billinghurst – Email: email@example.com – Twitter: @marknb00 • Website – http://www.hitlabnz.org/
Using augmented reality to create empathic experiences on ResearchGate, the professional network for scientists.
Using Metaio Creator to Develop a Markerless Augmented Reality ... a Markerless Augmented Reality Experience ... Create Augmented Reality ...
Print materials spring to life with digital content using the world’s #1 augmented reality app. Layar. ... Try it yourself & create an amazing experience ...
These Augmented Reality Examples ... been mixed with Augmented Reality to create an enhanced experience! ... of using augmented reality applications ...
... Tool to create Augmented Reality applications. New immersive experiences can be produced by using D ... Augmented Reality experience is ready to ...
... thinking it may lead to... identification through your own experience, you are empathic. To make the concept of empathy more clear, I use the term ...
How To Create Augmented Reality Apps Tinkernut. ... Augmented Reality Tutorial No. 14: Augmented Reality using Unity3D and Vuforia (part 1) ...
I love that Instructable and now I am posting my own! Augmented Reality(AR) ... In Metaio Creator create a channel on ... Augmented Reality Using Unity3D ...