Hacking Reality: Browser-Based VR with HTML5

62 %
38 %
Information about Hacking Reality: Browser-Based VR with HTML5
Technology

Published on October 21, 2014

Author: auradeluxe

Source: slideshare.net

Description

Slides from fall 2014 HTML5 Dev Conf talk on Browser-Based VR

1. HACKING REALITY: HTML5 AND JAVASCRIPT FOR CROSS-PLATFORM VR TONY PARISI OCTOBER, 2014

2. ABOUT ME CREDS Co-creator, VRML and X3D http://www.tonyparisi.com 10/21/2014 CONTACT tony@vizi.gl skype: auradeluxe http://twitter.com/auradeluxe http://www.tonyparisi.com/ http://www.learningwebgl.com/ GET VIZI https://github.com/tparisi/Vizi GET THE BOOKS! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/ dp/1449362966 MEETUPS http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/ BOOK CODE https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications GET GLAM http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.vizi.gl/

3. THE PROMISED LAND! CONSUMER VR http://www.tonyparisi.com 10/21/2014

4. VR TODAY http://www.tonyparisi.com 10/21/2014 GIANT DOWNLOADS SILO EXPERIENCES CUMBERSOME NATIVE APPS AND INSTALLS PROPRIETARY PLATFORMS AND DEVELOPMENT STACKS VR DOTH BE HARD!

5. WHY I LOVE THE WEB  INSTANT PUBLISHING  INSTANT ACCESS TO INFORMATION  NO TOLLS  NOBODY CONTROLS IT  CULTURE OF COLLABORATION  VIEW SOURCE …THE WEB WILL NEVER CLOSE UP SHOP. image: Mark Surman http://commonspace.wordpress.com/2014/03/12/happybirthday/ http://www.tonyparisi.com 10/21/2014

6. THE THREE D’S OF THE WEB http://www.tonyparisi.com 10/21/2014  DEVELOPMENT  CROSS-PLATFORM  VENDOR-NEUTRAL  OPEN SOURCE  DEPLOYMENT  CLOUD  PUSH-BUTTON UPDATE AND PUBLISH  DISTRIBUTION AND DISCOVERY  EMBED IN OTHER PAGES  SHARE WITH A HYPERLINK  NO APP TO DOWNLOAD

7. VR AND THE WEB: TWO GREAT TASTES… ? http://www.tonyparisi.com 10/21/2014

8. WEB VR FAST, CHEAP, AND TOTALLY DEMOCRATIZED.  BROWSER-BASED VIRTUAL REALITY  WEBGL  CSS3  VR SUPPORT NOW IN BROWSER DEV BUILDS!!!  NO BIG APP DOWNLOADS AND INSTALLS!!! http://www.tonyparisi.com 10/21/2014  JUST ADD SMART PHONE  “SMARTVR” USING GOOGLE CARDBOARD $25 CHEAP!

9. AN EXAMPLE http://www.tonyparisi.com 10/21/2014 INFORMATION DIVING SHOWCASE http://mozvr.github.io/infodive/ POWERED BY FIREFOX BUILT WITH VIZI https://github.com/tparisi/Vizi

10. THE WEBVR API 1. QUERY FOR VR DEVICE(S) TO RENDER // polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */ http://www.tonyparisi.com 10/21/2014 var self = this; getVRDevices().then( gotVRDevices ); function gotVRDevices( devices ) { var vrHMD; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof HMDVRDevice ) { vrHMD = devices[i]; self._vrHMD = vrHMD; self.leftEyeTranslation = vrHMD.getEyeTranslation( "left" ); self.rightEyeTranslation = vrHMD.getEyeTranslation( "right" ); self.leftEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "left" ); self.rightEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "right" ); break; // We keep the first we encounter } } } get left/right eye (camera) positions get per-eye camera field of view; use WebGL to render scene from two cameras

11. THE WEBVR API 2. GO FULLSCREEN (VR MODE) fullscreen mode requires a DOM element http://www.tonyparisi.com 10/21/2014 var self = this; var renderer = this._renderer; var vrHMD = this._vrHMD; var canvas = renderer.domElement; var fullScreenChange = canvas.mozRequestFullScreen? 'mozfullscreenchange' : 'webkitfullscreenchange'; document.addEventListener( fullScreenChange, onFullScreenChanged, false ); function onFullScreenChanged() { if ( !document.mozFullScreenElement && !document.webkitFullScreenElement ) { self.setFullScreen( false ); } } if ( canvas.mozRequestFullScreen ) { canvas.mozRequestFullScreen( { vrDisplay: vrHMD } ); } else { canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } ); } handle exiting fullscreen mode request fullscreen mode for this VR device

12. THE WEBVR API 3. HEAD TRACKING query HMD device state http://www.tonyparisi.com 10/21/2014 // polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */ var self = this; getVRDevices().then( gotVRDevices ); function gotVRDevices( devices ) { var vrInput; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof PositionSensorVRDevice ) { vrInput = devices[i] self._vrInput = vrInput; break; // We keep the first we encounter } } } … // called once per tick from requestAnimationFrame() var update = function() { var vrState = this.getVRState(); if ( !vrState ) { return; } // vrState.hmd.rotation contains four floats, quaternion x,y,z,w setCameraRotation(vrState.hmd.rotation); }; get head-tracking VR device update camera to match HMD device orientation

13. WEBVR AND CARDBOARD  GOOGLE CARDBOARD SHOWCASE  Mobile Chrome http://g.co/chromevr  Desktop Chrome http://vr.chromeexperiments.com/  EVEN EASIER  RENDER WEBGL SIDE-BY-SIDE STEREO (NO NEED TO QUERY DEVICES)  USE BROWSER DEVICE ORIENTATION API FOR HEAD TRACKING http://www.tonyparisi.com 10/21/2014

14. WEBVR AND THREE.JS  THE MOST POPULAR WEBGL LIBRARY  http://threejs.org/  LATEST STABLE VERSION (r68) INCLUDES STEREO RENDERING AND HEAD TRACKING  RENDERING examples/js/effects/StereoEffect.js (Cardboard) examples/js/effects/VREffect.js (Rift)  HEAD TRACKING examples/js/controls/DeviceOrientationControls.js (Cardboard) examples/js/controls/VRControls.js (Rift) http://www.tonyparisi.com 10/21/2014

15. VIZI: A FRAMEWORK FOR WEBVR APPLICATIONS  GOAL: MAKE IT EASY TO QUICKLY BUILD INTERESTING 3D APPLICATIONS  ARCHITECTURE INSPIRED INSPIRED BY MODERN GAME ENGINE DESIGN  COMPONENT-BASED DESIGN – EASILY PLUG NEW FEATURES INTO OBJECTS  APPLICATION OBJECT – HANDLES EVENT LOOP, THREE.JS CREATION, PAGE RESIZE, ROUTING EVENTS TO OBJECTS  INTERACTIONS – MAKE IT EASY TO PUT MOUSE AND TOUCH INTERACTION ON OBJECTS  BEHAVIORS – PREBUILT BEHAVIORS AUTOMATICALLY ROTATE, MOVE ETC.  PRE-BUILT OBJECTS – FOR COMMONLY USED DESIGN PATTERNS  SIMILAR IN DESIGN TO UNITY3D  USES THREE.JS FOR ALL GRAPHICS  ENHANCES THREE.JS VR RENDERING AND CONTROLLERS http://www.tonyparisi.com 10/21/2014

16. OPEN TOOLS FOR CROSS-PLATFORM VR http://www.tonyparisi.com 10/21/2014 game engines/IDEs Goo Enginehttp://www.gootechnologies.com/ Verold http://verold.com/ Turbulenz https://turbulenz.com/ PlayCanvas http://www.playcanvas.com/ Artillery Engine https://artillery.com/ Sketchfab https://sketchfab.com/ Unreal https://www.unrealengine.com/ Unity http://unity3d.com/#unity-5 scene graph libraries/page frameworks Three.js http://threejs.org/ SceneJS http://scenejs.org/ BabylonJS http://www.babylonjs.com/ Vizi https://github.com/tparisi/Vizi Voodoo.js http://www.voodoojs.com/ PhiloGL http://www.senchalabs.org/philogl/ tQuery http://jeromeetienne.github.io/tquery/

17. PRO TOOLS FOR WEB VR Unreal 4 in WebGL https://www.youtube.com/watch?v=c2uNDlP4RiE#t=42 60FPS ported in 5 days Unreal native C++ engine -> JavaScript Emscripten + asm.js http://www.tonyparisi.com 10/21/2014 EMSCRIPTEN - THE COOLEST HACK EVER! https://github.com/kripken/ems cripten  CROSS-COMPILE C++ NATIVE CODE TO JAVASCRIPT  asm.js- LOW-LEVEL OPTIMIZED JAVASCRIPT  UNITY, UNREAL ENGINES USE THIS TO DEPLOY ON THE WEB  WATCH OUT: HUGE DOWNLOADS AND HARD TO DEBUG…. !

18. WEBVR AND CSS http://www.tonyparisi.com 10/21/2014 MOZILLA VR CSS SHOWCASE http://mozvr.github.io/vr-web-examples/domvr-birds/

19. WEBVR AND CSS http://www.tonyparisi.com 10/21/2014 <script type="text/javascript" src="js/vrutils.js"></script> <script> /* VR Headset and its associated orientation/position sensor */ var vrHMD, vrSensor; /* Element that will serve as our camera, moving the rest of the scene around */ var cssCamera = document.getElementById("camera"); /* the camera's position, as a css transform string. For right now, we want it just in the middle. */ var cssCameraPositionTransform = "translate3d(0, 0, 0)"; /* Request animation frame loop. */ function animate() { /* Acquire positional data from VR headset and convert into a transformation that can be applied to CSS. */ if (vrSensor !== undefined) { var state = vrSensor.getState(); var cssOrientationMatrix = cssMatrixFromOrientation(state.orientation, true); } /* Apply positional data to camera element */ cssCamera.style.transform = cssOrientationMatrix + " " + cssCameraPositionTransform; window.requestAnimationFrame(animate); } query HMD device state calculate “camera” orientation update “camera’s” CSS 3D transform

20. VR + ML A MARKUP LANGUAGE FOR THE METAVERSE? http://www.tonyparisi.com 10/21/2014  GLAM (GL AND MARKUP) - A DECLARATIVE LANGUAGE FOR 3D WEB CONTENT https://github.com/tparisi/glam/  DEFINE 3D SCENE CONTENT IN MARKUP; STYLE IT IN CSS THE MARKUP <glam> <renderer type="rift"></renderer> <scene> <controller type="rift"></controller> <background id="sb1" class="skybox"> </background> <group y ='1' z='-3'> <sphere class="bubble skybox”> </sphere> <sphere class="bubble skybox”> </sphere> </group> … THE CSS <style> .skybox { envmap-right:url(../images/Park2/posx.jpg); … } .bubble { radius:.5; shader-vertex:url(../shaders/fresnel.vs); shader-fragment:url(../shaders/fresnel.fs); shader-uniforms:mRefractionRatio f 1.02 mFresnelBias f 0.1 mFresnelPower f 2.0 mFresnelScale f 1.0 tCube t null; } #sb1 { background-type:box; } </style>

21. CHALLENGES  WEBVR ON OCULUS IS NOT READY FOR PRIME TIME  (THAT’S OK NEITHER IS OCULUS!)  LATENCY IS THE BIGGEST ISSUE – BROWSER NEEDS TO UN-THROTTLE AT 60FPS  BUT WE’RE GOOD TO GO ON CARDBOARD!  60FPS WORKS GREAT  NEARLY 2 BILLION VR DEVICES ALREADY DEPLOYED!  FRAMEWORKS AND TOOLS ARE TYPICALLY WEB-ISH: UNDER DEVELOPMENT, ALWAYS IN FLUX, PRETTY MUCH OUT OF CONTROL  BUT OPEN SOURCE SO WE CAN LIVE WITH IT http://www.tonyparisi.com 10/21/2014

22. LINKS  BROWSER DEV BUILDS Firefox http://blog.bitops.com/blog/2014/08/20/updated-firefox-vr-builds/ Chrome https://drive.google.com/folderview?id=0BzudLt22BqGRbW9WTHMtOWMzNjQ  MOZILLA NEXT STEPS FOR VR (THURSDAY 23 OCT 2014) https://air.mozilla.org/virtual-reality-the-web-next-steps/  SAN FRANCISCO WEBVR MEETUP http://www.meetup.com/Web-VR/  WEBVR MAILING LIST web-vr-discuss@mozilla.org  CARDBOARD VR SHOWCASE http://vr.chromeexperiments.com/ http://www.tonyparisi.com 10/21/2014

23. KEEP IN TOUCH CREDS Co-creator, VRML and X3D http://www.tonyparisi.com 10/21/2014 CONTACT tony@vizi.gl skype: auradeluxe http://twitter.com/auradeluxe http://www.tonyparisi.com/ http://www.learningwebgl.com/ GET VIZI https://github.com/tparisi/Vizi GET THE BOOKS! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/ dp/1449362966 MEETUPS http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/ BOOK CODE https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications GET GLAM http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.vizi.gl/

24. HACKING REALITY: HTML5 AND JAVASCRIPT FOR CROSS-PLATFORM VR TONY PARISI OCTOBER, 2014

#sb1 presentations

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

HTML5 Developer Conference - html5devconf.com

The HTML5 Developer Conference has grown to become the highest ... Virtual Reality & The Future of the ... Hacking Reality: Browser-Based VR in HTML5.
Read more

Browser Based | LinkedIn

... our new browser based scanning application, ... Hacking Reality: Browser-Based VR with HTML5. 6,197 Views. matkelly01. Browser-Based Digital Preservation.
Read more

Social hacking with GlobaLeaks - Technology - documents.mx

Social Hacking with GlobaLeaks Secure and anonymous Open Source Whistleblowing ... Hacking Reality: Browser-Based VR with HTML5
Read more

IOTAconf Internet of Things Conference

Internet of Things Conference Moscone Center, SF. Conference: October 20-21. Training: October 22, 23, 24. IoT; ... Hacking Reality: Browser-Based VR in HTML5.
Read more

Learning Virtual Reality - O'Reilly Media

Comments about oreilly Learning Virtual Reality: Great Job ! The book covers a lot of bases to get you started on almost every VR platform. PC,Mobile ...
Read more

Making schedule table about HTML5DevConf 2014 conference

/** * 3. You should run this script on Google script editor with Google SpreadSheet. */ function myFunction() { // this data was extracted by the `debugger ...
Read more

IOTAconf Internet of Things Conference

Internet of Things Conference ... Google Developer Advocate Kevin Nilson will talk about Chrome Sender App Development and HTML5 ... immersive VR a reality.
Read more