Lightspeed SIGGRAPH talk

50 %
50 %
Information about Lightspeed SIGGRAPH talk
Technology

Published on September 27, 2008

Author: jrk

Source: slideshare.net

Description

Talk slides from the SIGGRAPH 2007 presentation of the paper "The Lightspeed Automatic Interactive Lighting Preview System" by Ragan-Kelley, et al.

The Lightspeed Automatic Interactive Lighting Preview System Jonathan Ragan-Kelley / MIT CSAIL Charlie Kilpatrick / ILM Brian Smith / ILM Doug Epps / Tippett Studio Paul Green / MIT CSAIL Christophe Héry / ILM Frédo Durand / MIT CSAIL

Lighting Design ■ End of pipeline fixed geometry, viewpoint, material ■ Slow feedback 10-60 mins to render

Lighting Design another hour later... 1 ■ End of pipeline fixed geometry, viewpoint, material ■ Slow feedback 10-60 mins to render

Goal: Fast Lighting Preview Exploit redundancy between previews ■ geometry ■ view ■ material

High-Level Approach ■ Precompute deep-framebuffer cache e.g. ... normal position diffuse specular texture texture ■ Preview dynamically on GPU as the user specifies new light parameters

Prior Work

Render Caching Parameterized Ray Tracing [Séquin & Smyrl 1989] G-Buffer [Saito & Takahashi 1990] Fast Relighting Engine [Gershbein & Hanrahan 2000] ■ Precomputed buffer (normals, texture) with BRDF*light reevaluated at runtime ■ Shortcomings ■ Simplistic shading ■ No antialiasing, motion blur, transparency

Pixar’s Lpics [Pellacini et al. 2005] ■ Production shaders & lights (RenderMan) ■ Exploit programmable GPUs ■ No antialiasing, transparency ■ Requires extra shader-writing work ■ (final) RenderMan version + extra caching code ■ GPU preview version

Specializing Shaders [Guenter et al. 1995] ■ Given dynamic parameters, automatically split shader code into: ■ static ■ dynamic ■ Simple language no control flow ■ Simple shaders Plate 1: Test image. Plate 1: Test image. Plate 2: Image obtained by reshading image of Plate 1. This Plate 2: Image obtained by reshading image of Plate 1. This took 12 seconds unspecialized, 0.5 seconds specialized. took 12 seconds unspecialized, 0.5 seconds specialized. Bibliography Bibliography [1] Andersen, Lars Ole. Self-applicable C Program [1] Andersen, Lars Ole. Self-applicable C Program [9] Jones, Neil D., Carsten K. Gomard, and Peter Sestoft. [9] Jones, Neil D., Carsten K. Gomard, and Peter Sestoft. Specialization. Proceedings of the ACM SIGPLAN Specialization. Proceedings of the ACM SIGPLAN Partial Evaluation and Automatic Program Generation. Partial Evaluation and Automatic Program Generation. Workshop on Partial Evaluation and Semantics-Based Workshop on Partial Evaluation and Semantics-Based Prentice-Hall, 1993. Prentice-Hall, 1993. Program Manipulation (San Francisco, California, June 12- Program Manipulation (San Francisco, California, June 12- [10] Mogensen, Torben. The Application of Partial Evaluation to [10] Mogensen, Torben. The Application of Partial Evaluation to 20, 1992). Yale University technical report 20, 1992). Yale University technical report YALEU/DCS/RR-909, 1992, 54-61. Ray-Tracing. Master's thesis, DIKU, University of Ray-Tracing. Master's thesis, DIKU, University of YALEU/DCS/RR-909, 1992, 54-61. Copenhagen, Denmark, 1986. Copenhagen, Denmark, 1986.

Precomputed Radiance Transfer e Transfer for Real-Time Rendering requency Lighting Environments e.g. Sloan et al. 2002, Ng et al. 2003 Jan Kautz John Snyder x-Planck-Institut für Informatik Microsoft Research jnkautz@mpi-sb.mpg.de johnsny@microsoft.com use and hat cap- process, over the equency s global ect onto o actual ampling e rigidly ting and Figure 1: Precomputed, unshadowed irradiance from [34] (left) vs. our pherical precomputed transfer (right). The right model can be rendered at 129Hz Inappropriate for our application ntly on with self-shadowing and self-interreflection in any lighting environment. o a dot Glossy frequency lighting environments, using a low-order spherical ■ Large precomputation cost e further harmonic (SH) basis to represent such environments efficiently lighting without aliasing. The main idea is to represent how an object g points scatters this light onto itself or its neighboring space. ■ Largely ignore light functions m rigidly To describe our technique, assume initially we have a convex, ers. We diffuse object lit by an infinitely distant environment map. The roach. ■ Limitations on shading models object’s shaded “response” to its environment can be viewed as a chniques, transfer function, mapping incoming to outgoing radiance, which in this case simply performs a cosine-weighted integral. A more complex integral captures how a concave object shadows itself, where the integrand is multiplied by an additional transport factor tions are representing visibility along each direction. tunately, Our approach is to precompute for a given object the expensive environ- transport simulation required by complex transfer functions like 25], rad- shadowing. The resulting transfer functions are represented as a ple point dense set of vectors or matrices over its surface. Meanwhile, ndering. incident radiance need not be precomputed. The graphics hard- difficul- ware can dynamically sample incident radiance at a number of s of real points. Analytic models, such as skylight models [33] or simple

Our Design Goals 1. High-performance preview ■ Low-latency feedback ■ Fast initial precomputation 2. Seamless integration ■ Same input: RenderMan scene & shaders ■ Same output: high quality image 3. Ease of implementation & maintenance

System Overview

System Overview RenderMan scene & shaders

System Overview RenderMan cache & scene & preview shaders shaders automatic caching

System Overview RenderMan cache & preview preview image scene & shaders shaders relighting automatic engine caching (gpu)

System Overview RenderMan cache & preview preview image scene & shaders shaders relighting automatic engine caching (gpu) lighting parameters one-time precomputation iterative preview

System Overview RenderMan cache & preview preview image scene & shaders shaders relighting automatic engine caching (gpu) lighting parameters Automatic for real production scenes ‣ specialization & translation ‣ cache compression

System Overview RenderMan cache & preview preview image scene & shaders shaders relighting automatic engine caching (gpu) lighting parameters Automatic High fidelity for real production scenes antialiasing, motion blur, ‣ specialization & translation transparency ‣ cache compression ‣ indirect framebuffer

Automatic Caching RenderMan cache & preview preview image scene & shaders shaders relighting automatic engine caching (gpu) lighting parameters

Program Analysis

Program Analysis mySurface(color
c)
{ albedo
=
c*Texture(U,V); shade
=
Light(L)*BRDF(N,L); return
albedo*shade; }

Program Analysis mySurface(color
c)
{ dynamic albedo
=
c*Texture(U,V); shade
=
Light(L)*BRDF(N,L); return
albedo*shade; } what part of the code depends on the lighting parameter L?

Program Analysis mySurface(color
c)
{ dynamic albedo
=
c*Texture(U,V); shade
=
Light(L)*BRDF(N,L); static return
albedo*shade; } everything else is static

Program Analysis mySurface(color
c)
{ dynamic albedo
=
c*Texture(U,V); shade
=
Light(L)*BRDF(N,L); static return
albedo*shade; cache } values at the boundary are cached for reevaluation

Program Analysis mySurface(color
c)
{ albedo
=
c*Texture(U,V); shade
=
Light(L)*BRDF(N,L); return
albedo*shade; }

Program Analysis albedo shade mySurface(color
c)
{ albedo
=
c*Texture(U,V); shade
=
Light(L)*BRDF(N,L); return
albedo*shade; } c Texture Light BRDF U V L N

Deep Framebuffer Generation mySurface(color
c)
{ albedo
=
c*Texture(U,V); shade
=
Light(L)*BRDF(N,L); return
albedo*shade; }

Deep Framebuffer Generation caching shader mySurfaceCache(color
c)
{ albedo
=
c*Texture(U,V); cache(N); cache(albedo); }

Deep Framebuffer Generation N albedo final deep renderer framebuffer cache caching shader mySurfaceCache(color
c)
{ albedo
=
c*Texture(U,V); cache(N); cache(albedo); }

Deep Framebuffer Generation N albedo deep framebuffer cache caching shader

Deep Framebuffer Generation N albedo deep framebuffer cache caching shader mySurfaceDynamic(vector
L)
{ N
=
lookup(); albedo
=
lookup(); shade
=
Light(L)*BRDF(N,L); preview return
albedo*shade; shader }

Deep Framebuffer Generation N albedo deep framebuffer cache caching shader mySurfaceDynamic(vector
L)
{ N
=
lookup(); albedo
=
lookup(); shade
=
Light(L)*BRDF(N,L); preview return
albedo*shade; shader }

Deep Framebuffer Generation N albedo deep framebuffer cache caching shader rendered shade image preview shader

Automatic Caching

Automatic Caching RenderMan scene scene preprocessor RenderMan shaders

Automatic Caching RenderMan scene scene preprocessor program analysis RenderMan shaders

Automatic Caching RenderMan scene scene preprocessor caching shaders program analysis RenderMan shaders

Automatic Caching deep RenderMan framebuffer scene scene preprocessor cache x2 x1 final N P renderer caching shaders program analysis RenderMan shaders

Automatic Caching deep RenderMan framebuffer scene scene preprocessor cache x2 x1 final N P renderer caching shaders program analysis RenderMan dynamic shaders preview shaders

Automatic Caching challenge: cache size deep RenderMan framebuffer scene scene preprocessor cache x2 program analysis is final x1 N P conservative renderer caching shaders generates many cached terms program analysis (hundreds) RenderMan dynamic shaders preview shaders large caches (gigabytes)

Cache Compression deep RenderMan framebuffer scene scene preprocessor cache x2 x1 final N P renderer caching shaders Remove redundant program analysis values after caching RenderMan dynamic shaders preview shaders

Cache Compression deep RenderMan framebuffer scene scene preprocessor cache x2 x1 N final N cache P P renderer compression caching shaders Remove redundant program analysis values after caching RenderMan dynamic shaders preview shaders

Cache Compression deep RenderMan framebuffer scene scene preprocessor cache x2 x1 N final N cache P P renderer compression caching shaders Remove redundant program analysis values after caching RenderMan shaders dynamic preview → 5x compression shaders

System Overview deep RenderMan framebuffer scene scene preprocessor cache N P final cache renderer compr. comp. caching shaders program analysis RenderMan dynamic shaders preview shaders

System Overview deep RenderMan framebuffer scene scene preprocessor cache N P final cache renderer comp. caching shaders program analysis RenderMan dynamic shaders preview shaders

System Overview deep RenderMan framebuffer scene scene preprocessor cache N P final cache renderer comp. relighting caching shaders engine (gpu) program analysis preview image RenderMan dynamic shaders preview shaders lighting parameters

System Overview antialiasing? RenderMan scene scene preprocessor deep framebuffer cache transparency? N P final cache renderer comp. relighting caching shaders engine (gpu) program analysis preview image RenderMan dynamic shaders preview shaders lighting parameters

Classical Deep Framebuffer B A

Classical Deep Framebuffer B A

Classical Deep Framebuffer x2 deep x1 framebuffer N cache B P cache shade A Cache one sample per-pixel rendered Shade cache values to render image image

Classical Deep Framebuffer x2 deep x1 framebuffer N cache B P cache shade A How can we do antialiasing? rendered image

Classical Deep Framebuffer supersample image x2 x1 N B P cache shade A How can we do antialiasing? transparency?

Classical Deep Framebuffer x4 supersample x3 cache N B P cache shade A supersample shading How can we do antialiasing? transparency?

Classical Deep Framebuffer x4 x3 N B P ...? cache shade A How can we do antialiasing? ...? transparency?

Our Indirect Framebuffer ■ Decouple shading from visibility ■ only supersample visibility ‣ Do what RenderMan does ■ Compress samples based on static visibility

RenderMan / REYES B show *filtering* pixel color? A

RenderMan / REYES B b1 show *filtering* pixel color? b2 a1 A dice into micropolygons

RenderMan / REYES B b1 show *filtering* pixel color? b2 a1 A shade micropolygons

RenderMan / REYES B b1 show *filtering* pixel color? b2 a1 A rasterize micropolygons

RenderMan / REYES B b1 show *filtering* pixel color? b2 a1 A

RenderMan / REYES 1 2 B a1 b2 b1 show *filtering* 1 2 pixel color? b2 3 a 1 4 3 4 A a1 a1 pixel = a1+b2+a1+a1

RenderMan / REYES 1 2 B a1 b2 b1 show *filtering* 1 2 pixel color? b2 3 a 1 4 3 4 A a1 a1 pixel = a1+b2+a1+a1 4

RenderMan / REYES 1 2 B a1 b2 b1 show *filtering* 1 2 pixel color? b2 3 a 1 4 3 4 A a1 a1 pixel = a1+b2+a1+a1 = 0.75*a1+0.25*b2 4

RenderMan / REYES 1 2 B a1 b2 b1 show *filtering* 1 2 pixel color? b2 3 a 1 4 3 4 A a1 a1 pixel = ?

RenderMan / REYES B b1 b2 a1 A pixel = ?

RenderMan / REYES 1 B a1 b1 α=0.3 α=1.0 b1 0.3*a1+0.7*b1 1 b2 a1 A pixel = ?

RenderMan / REYES 1 2 B a1 b1 b2 α=0.3 α=1.0 α=1.0 b1 2 0.3*a1+0.7*b1 1.0*b2 1 b2 a1 A pixel = ?

RenderMan / REYES 1 2 B a1 b1 b2 α=0.3 α=1.0 α=1.0 b1 2 0.3*a1+0.7*b1 1.0*b2 1 b2 3 a 1 4 3 4 A a1 a1 b2 α=0.3 α=0.3 α=1.0 0.3*a1 0.3*a1+0.7*b1 pixel = ?

RenderMan / REYES 1 2 B a1 b1 b2 α=0.3 α=1.0 α=1.0 b1 2 0.3*a1+0.7*b1 1.0*b2 1 b2 3 a 1 4 3 4 A a1 a1 b2 α=0.3 α=0.3 α=1.0 0.3*a1 0.3*a1+0.7*b1 pixel = 0.225*a1+0.35*b1+0.25*b2

Indirect Framebuffer

Indirect Framebuffer b1 a1b2

Indirect Framebuffer b1 a1b2 x4 x4 deep x1 x3 x3 buffer N x2 x2 P P P a1 b1 b2 no longer image space (per-micropolygon)

Indirect Framebuffer b1 a1b2 x4 x4 deep x1 x3 x3 buffer N x2 x2 P P P a1 b1 b2 shading shade samples shaded like a conventional deep-framebuffer

Indirect Framebuffer b1 a1b2 indirect framebuffer x4 x4 deep x1 x3 x3 buffer N x2 x2 P P P a1 b1 b2 shading shade samples

Indirect Framebuffer b1 a1b2 indirect framebuffer x4 x4 deep x1 x3 x3 buffer 0.225 0.35 0.25 N x2 x2 P P P a1 b1 b2 shading shade samples densely stored in vertex array, on GPU

Indirect Framebuffer b1 a1b2 indirect framebuffer x4 x4 deep x1 x3 x3 buffer 0.225 0.35 0.25 N x2 x2 P P P a1 b1 b2 blend shading final shade samples pixel 4 subpixel samples, 2 transparent layers only 3 unique micropolygons

Indirect Framebuffer b1 a1b2 indirect framebuffer x4 x4 deep x1 x3 x3 buffer 0.225 0.35 0.25 N x2 x2 P P P a1 b1 b2 blend shading final shade samples pixel 4 subpixel samples, 2 transparent layers only 3 unique micropolygons all contributions linearized into a single weight

More samples, b1 Same cost a1b2 indirect framebuffer x4 x4 deep x1 x3 x3 buffer 0.242 0.331 0.245 N x2 x2 P P P a1 b1 b2 blend shading final shade samples pixel

More samples, b1 Same cost a1b2 indirect framebuffer x4 x4 deep x1 x3 x3 buffer 0.242 0.331 0.245 N x2 x2 P P P a1 b1 b2 blend shading final shade samples pixel

More samples, b1 Same cost a1b2 indirect framebuffer x4 x4 deep x1 x3 x3 buffer 0.242 0.331 0.245 N x2 x2 P P P a1 b1 b2 blend shading final shade samples pixel

Indirect Framebuffer Results antialiasing motion blur transparency identical to RenderMan

Indirect Framebuffer Results antialiasing motion blur transparency identical to RenderMan RenderMan 45M micropolygons 60M subpixel samples brute-force >100M samples Indirect framebuffer 10M deep fb +15M indirect fb 720x389, 64x supersampling

System Architecture

System Architecture RenderMan scene scene preprocessor RenderMan shaders

System Architecture RenderMan scene scene preprocessor program analysis RenderMan shaders

System Architecture RenderMan scene scene preprocessor caching shaders program preview analysis shaders RenderMan shaders

System Architecture RenderMan scene scene preprocessor final renderer caching shaders program preview analysis shaders RenderMan shaders

System Architecture RenderMan scene scene preprocessor deep fb indirect fb final cache x2 x1 renderer compr. N P caching shaders program preview analysis shaders RenderMan shaders

System Architecture RenderMan scene scene preprocessor relighting engine (GPU) deep fb indirect fb final cache x2 x1 renderer compr. N P caching shaders program preview analysis shaders RenderMan shaders

System Architecture RenderMan scene scene preprocessor relighting engine (GPU) deep fb indirect fb final cache x2 x1 renderer compr. N P caching shaders shade program preview analysis shaders RenderMan shaders

System Architecture RenderMan scene scene preprocessor relighting engine (GPU) deep fb indirect fb rendered image final cache x2 x1 renderer compr. N P caching shaders shade blend program preview analysis shaders RenderMan shaders

System Architecture RenderMan scene scene preprocessor relighting engine (GPU) deep fb indirect fb rendered image final cache x2 x1 renderer compr. N P caching shaders shade blend program preview analysis shaders lighting parameters RenderMan shaders one-time precomputation iterative preview

Additional Features ■ Shadows ■ Subsurface Scattering ■ Performance enhancement ■ Progressive Refinement ■ Light Caching … ■ Tiling

Results initial feedback: 0.05 sec full refinement: 0.7 sec Lightspeed 720x389 offline render: 5 mins

Results initial feedback: 0.05 sec full refinement: 0.7 sec RenderMan Lightspeed 720x389 offline render: 5 mins

Results

Results caching: 16 mins initial feedback: 0.11 sec full refinement: 2.7 sec 914x389 offline render: 59 mins 13x13 antialiasing 42 lights

Results caching: 16 mins initial feedback: 0.11 sec full refinement: 2.7 sec Lightspeed RenderMan 914x389 offline render: 59 mins 13x13 antialiasing 42 lights

Results caching: 16 mins initial feedback: 0.11 sec full refinement: 2.7 sec Lightspeed RenderMan 914x389 offline render: 59 mins 13x13 antialiasing 42 lights

Results

Results

Results

Results

Limitations ■ GPU programming model ■ Dynamic calls to external C code ■ Complex data structures ■ GPU limits (bandwidth, memory, registers) ■ Fully-accurate dynamic ray tracing ■ Unbounded dynamic loops ■ Additional features not yet implemented ■ Indirect diffuse ■ Deep shadows ■ Non-linear lights

Discussion ■ Automatic and manual specialization (Lpics) both have advantages ■ Manual specialization allows hand optimization ■ Compiler requires up-front R&D, never perfect (especially on the GPU) ■ Saves significant time in production ■ Some material parameters are editable ■ Cache compression is key to practical automatic specialization much simpler than fancy static analysis ■ Indirect Framebuffer is powerful, scalable

Summary ■ Interactive lighting preview milliseconds instead of hours ■ Automatic caching for our production scenes ■ Program analysis ■ Cache compression ■ Indirect framebuffer ■ Efficient antialiasing, motion blur, transparency ■ Progressive refinement ■ In use on current productions

Acknowledgments Inception: Pat Hanrahan, Ujval Kapasi Compilers: Alex Aiken, John Kodumal Tippett: Aaron Luk, Davey Wentworth ILM: Alan Trombla, Ed Hanway, Dan Goldman, Steve Sullivan, Paul Churchill Images: Michael Bay, Dan Piponi Money: NSF, NVIDIA, MSR, Sloan & Ford fellowships

Summary ■ Interactive lighting preview milliseconds instead of hours ■ Automatic caching for our production scenes ■ Program analysis ■ Cache compression ■ Indirect framebuffer ■ Efficient antialiasing, motion blur, transparency ■ Progressive refinement ■ In use on current productions

EXTRAS

Cache Compression deep framebuffer cache Remove redundant values after caching ■ non-varying same for many pixels ■ non-unique same within a pixel unique varying → 4-5x compression

Cache Compression shader dynamic varying unique (static analysis) (compressed) generic surface 402 145 97 metallic paint 450 150 97

Visibility Compression Indirect RenderMan Framebuffer scene res. samples shade subpix shade indirect robot 914x389 13x13 2.1M 32M 633k 1.6M robot 720x306 13x13 1.5M 21M 467k 3.8M (blur) pirate 640x376 4x4 2.5M 2.3M 327k 716k hairs 720x389 8x8 43M 58M 11M 17M

Add a comment

Related presentations

Related pages

lightspeed_siggraph_talk - Ace Recommendation Platform - 20

Related Contents; p343-guenter with the illumination sources and the viewpoint. Global illumination effects, such as specular and diffuse interreflectance ...
Read more

people.csail.mit.edu

people.csail.mit.edu
Read more

The Lightspeed Automatic Interactive Lighting Preview System

The Lightspeed Automatic Interactive Lighting Preview System ... (Proceedings of SIGGRAPH 2007) ... Talk: Keynote (33 MB) PDF (17 MB)
Read more

Jonathan Ragan-Kelley - HubSlide

Lightspeed SIGGRAPH talk Talk slides from the SIGGRAPH 2007 presentation of the pape... 6 months ago © HubSlides 2016. About Us; Help; Terms of Use ...
Read more

3dcgi.com - Lightspeed Design to Debut DepthQ HD 3D ...

Lightspeed Design to Debut DepthQ HD 3D Projector at Siggraph 2008. BELLEVUE, WA (August 5, 2008) – Lightspeed Design, Inc., a worldwide leader in stereo 3D
Read more

Siggraph | LinkedIn

View 1669 Siggraph posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn.
Read more

Lightspeed | LinkedIn

View 17225 Lightspeed posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn.
Read more

Affordable HD3D - DepthQ

Talk about immersive! Imagine playing video games or watching movies ten feet wide in HD 3D! The next ... Visit Lightspeed Design at Siggraph 2008, ...
Read more