Siggraph Precomputed Radiance Transfer: Theory and Practice

Transcription

1 Siggah 2005 Pecomuted Radiance Tansfe: Theoy and Pactice Summay Geneal model of shading and shadowing fo eal-time endeing. Basic adiance tansfe techniques, moe advanced techniques that incooate highe-fequency lighting and abitay BRDFs, the diffeences among these algoithms, and insights the esentes have gained woking in this aea. The couse includes imlementation details and a comlete theoetical deivation. Lectues Jan Kautz, Massachusetts Institute of Technology, kautz@gahics.csail.mit.edu Pete-Pike Sloan, Micosoft Cooation, sloan@windows.micosoft.com Jaakko Lehtinen Helsinki Univesity of Technology and Remedy Entetainment Ltd., jaakko@tml.hut.fi Abstact Inteactive endeing of ealistic objects unde geneal lighting models oses thee incial challenges. Handling comlex light tansot henomena like shadows, inteeflections, caustics and sub-suface scatteing is difficult to do in eal time. Integating these effects ove lage aea light souces comounds the difficulty, and finally eal objects have comlex satially-vaying BRDF s. Pecomuted Radiance Tansfe (PRT encasulates a family of techniques that atially addesses these challenges. PRT is an active of aea of eseach that has elevance to both the academic eseach community and actitiones of inteactive comute gahics. This technique and its vaiants ae being actively investigated in the game develoment community and thee is quite a lot of inteest due to the ecent aeaance of PRT techniques in games such as Halo 2. This couse coves these techniques, comaes them and discusses thei vaious stengths and weaknesses. A moe igoous deivation diectly fom the endeing equation is esented along with actical imlementation details, both of which ae geneally not included in technical aes. Afte intoducing the necessay foundation (endeing equation, basis functions, etc., we begin with simle PRT fo diffuse objects. We

2 continue with geneal PRT using the concet of tansfe matices, which allow fo abitay eflectance models. The ossible choices fo basis functions ae discussed as well. Diffeent light souce eesentations ae esented and comaed. Finally, we discuss actical issues with PRT, such as data comession, satial samling, nomal maing, ecomutation, and moe. By the end of the couse the audience will be able to ick the ight algoithm fo thei needs and will hoefully have gained some of the unublished insights the seakes have gained by woking in this aea. Peequisites Knowledge of shading algoithms and linea algeba is useful. Some knowledge of a lowlevel gahics API such as DiectX o OenGL. Level of difficulty Intemediate Intended audience Eveybody who is inteested in ealistic eal-time shading. The couse teaches what ealistic shading encomasses and how it can be achieved using ecomuted adiance tansfe. The couse is designed fo eole who aleady have some knowledge in ealtime endeing and want to lean a moe sohisticated shading technique. Syllabus 8:30 9:00 Intoduction (Jan Kautz * Motivation * Poblem Statement * Classification of Shading algoithms * Definitions (Rendeing Equation, Neuman Seies, Basis Functions, etc. 9:00 9:15 Diffuse Pecomuted Radiance Tansfe (Jan Kautz * Deivation fom the Rendeing Equation (Diect/Indiect Lighting * Rendeing Pieline * Pecomutation * Examles, Results, Limitations 9:15 10:15 Geneal Pecomuted Radiance Tansfe (Jaakko Lehtinen * Assumtions about Emissions - Envionment Ma - Paameteization using Finite Basis * Tansfeed Incident Radiance - The Radiance Flowing Though Points in Sace o Incident onto Sufaces * Tansfe Matix - Definition how to detemine tansfeed incident adiance fom the emission? - Deivation fom Rendeing Equation

3 * Outgoing Radiance - Glossy Objects: Integating tansfeed incident adiance against BRDF - Modeling Subsuface Scatteing * Basis Function Choices fo PRT - Light Basis (SH vs. diectional vs. Wavelets - BRDF Basis (SH vs. diectional vs. comact vs. secialized vs. hemisheical - Least squaes ojection on hemishee Beak 10:30 11:00 SH Light Reesentations (Jan Kautz * Simle Analytic Models (Cones / Shees * Envionment Ma Pojection * Satial Vaiation of Lighting - Iadiance Volumes - SH Gadients 11:00 11:30 Pactical PRT I (Pete-Pike Sloan * Comession - VQ, PCA, CPCA - Rendeing Imlications * Satial Samling 11:30 12:00 Pactical PRT II (Pete-Pike Sloan * Albedo Mas * Nomal Maing * GPU Simulato * PRT and DiectX 12:00 12:15 Conclusions/Summay (Pete-Pike Sloan * Comaison of Pesented Techniques * Field-Guide to PRT * Q&A

4 Couse esente infomation Jan Kautz, Massachusetts Institute of Technology, Cambidge, USA Jan is cuently a Post-Doc with the gahics gou at the Massachusetts Institute of Technology. He eceived his PhD fom the Max-Planck-Institut fü Infomatik, Saabücken, Gemany. His thesis was on eal-time shading and endeing. He is aticulaly inteested in the ealistic shading using gahics hadwae, about which he has ublished seveal aticles at vaious confeences including SIGGRAPH. Jan has been a tutoial seake on eal-time shading at Euogahics, SIGGRAPH, and othes. Pete-Pike Sloan, Micosoft Cooation, Redmond, USA Pete-Pike Sloan has been in the DiectX gou at Micosoft fo the ast two yeas. Pio to that he was a membe of the gahics gou in Micosoft Reseach, a staff membe at the Scientific Comuting and Imaging Gou at the Univesity of Utah and woked at both Evans and Sutheland and Paametic Technologies. He is inteested in most asects of comute gahics and most of his ublications ae available online at: htt://eseach.micosoft.com/~sloan Jaakko Lehtinen, Helsinki Univesity of Technology and Remedy Entetainment, Ltd., Helsinki, Finland Jaakko Lehtinen is a gaduate student in comute gahics at the Helsinki Univesity of Technology, whee he eceived his M.Sc. ecently. His eseach inteests san all asects of ealistic image synthesis. Jaakko s inteest in gahics stems fom woking on comute games at Remedy Entetainment, whee he has been esonsible fo design and imlementation of adiosity lighting tools and modeling softwae used in the oduction of the blockbuste hits Max Payne and Max Payne 2.

5 1

6 Pecomuted Radiance Tansfe: Theoy and Pactice 2

7 Intoduction Jan Kautz MIT 3

8 Intoduction We see hee an examle of a eal-wold scene which has a lot of visual comlexity and ichness. Geneating synthetic images that come close to this is an extemely challenging oblem. Having them animate and esond to a uses contol is even moe daunting. We will discuss some of the issues that need to be addessed to meet this challenge. 4

9 How do we get thee? Geometic Comlexity Mateial Comlexity Lighting Comlexity Tansot Comlexity Synegy Thee ae many tyes of scene comlexity which oeate individually and in synegy with each othe to geneate the visual comlexity of the esulting image. 5

10 Geometic Comlexity Real-wold scenes have a lot of lage-scale detail 6

11 Mateial Comlexity Models how light inteacts with a suface Assumes the stuctue of the mateial is below the visible scale 7

12 Mateial Comlexity Vaiations at a visible scale but not geomety Bum mas BTFs Medium-scale, o meso-scale, lies between invisible details which ae handled as mateials and lage details which ae handled as geomety. 8

13 Lighting Comlexity What kind of lighting envionment is an object in? Diectional/oint lights Diectional + ambient Smooth (low fequency lighting Comletely geneal 9

14 Lighting Comlexity What kind of lighting envionment is an object in? Diectional/oint lights Diectional + ambient Smooth (low fequency lighting Comletely geneal 10

15 Lighting Comlexity What kind of lighting envionment is an object in? Diectional/oint lights Diectional + ambient Smooth (low fequency lighting Comletely geneal 11

16 Lighting Comlexity What kind of lighting envionment is an object in? Diectional/oint lights Diectional + ambient Smooth (low fequency lighting Comletely geneal 12

17 Tansot Comlexity How light inteacts with objects/scene at a visible scale Shadows Inte-eflections Caustics Tanslucency (subsuface scatteing 13

18 Tansot Comlexity How light inteacts with objects/scene at a visible scale Shadows Inte-eflections Caustics Tanslucency (subsuface scatteing 14

19 Tansot Comlexity How light inteacts with objects/scene at a visible scale Shadows Inte-eflections Caustics Tanslucency (subsuface scatteing 15

20 Tansot Comlexity How light inteacts with objects/scene at a visible scale Shadows Inte-eflections Caustics Tanslucency (subsuface scatteing 16

21 Some of all of this Real scenes have all of these foms of comlexity High ealism on one is not necessaily inteesting without the othes Comlex mateials lit by single oint light Comlex lighting envionments on diffuse sufaces with no shadows 17

22 Goals Inteactively ende ealistic objects Challenges Comlex mateials Bushed metal, Weaved objects Geneal lighting envionments Aea instead of oint lights Allow to dynamically change the incident lighting Tansot effects Shadows, inte-eflections, subsuface scatteing The imay goal of this couse is to accuately ende objects in geneal lighting envionments at inteactive ates. Thee ae seveal challenges that must be ovecome to achieve this goal. Real objects have comlex and satially vaying mateial oeties so we need to suot geneal eflection models and handle satial vaiation. Geneal lighting envionments ae much moe comelling comaed to simle oint lighting models. Run time integation ove geneal aea lighting models is exensive with taditional techniques. We would like to suot comlex tansot effects soft shadows fom aea lights, inte-eflection, caustics and subsuface scatteing, while maintaining inteactive endeing ates. 18

23 PRT What is Pecomuted Radiance Tansfe? A way to shade objects unde diffeent illumination Includes diect & indiect lighting, caustics, subsuface scat. Any kind of tansot is ossible Illumination can come fom all diections (env-ma Usually assumed to be fa away Deending on imlementation, ossible estictions on lighting (e.g. low-fequency 19

24 PRT What does it not do? It does not do full global illumination fo abitay dynamic scenes Objects have to be static! Fist ideas on how to deal with animations ae aeaing Object-to-object inteaction is limited Some new algoithms coming out though 20

25 Examles Bust illuminated with a foest envionment. Note the soft shadows. 21

26 Examles Statue illuminated by two aea souces, one geen and one ed. Numbe and sizes of light souces don't occu any efomance enalty when PRT is used. 22

27 Examles Some moe examles. Note the nice soft shadows. 23

28 Examles PRT can also be used in volumes. 24

29 Couse Oveview 8:30 9:00 Intoduction (Jan Kautz 9:00 9:15 Diffuse Pecomuted Radiance Tansfe (Jan Kautz 9:15 10:15 Geneal Pecomuted Radiance Tansfe (Jaakko Lehtinen Beak 10:30 11:00 SH Light Reesentations (Jan Kautz 11:00 11:30 Pactical PRT I (Pete-Pike Sloan 11:30 12:00 Pactical PRT II (Pete-Pike Sloan 12:00 12:15 Conclusions/Summay (Pete-Pike Sloan 25

30 Oveview Intoduction Rendeing Equation Neumann Seies Shading Algoithms Basis Functions In this intoduction, we will ean about the Rendeing Equation, its Neumann exansion, diffeent shading algoithms, and basis functions. 26

31 Backgound Rendeing Inut: Geomety Mateial Lighting Outut: Image Loosely ut, endeing takes inut (geomety, mateials, lights and oduces an image. Thee ae many diffeent ways to comute an image given the inut. Pecomuted Radiance Tansfe is a technique to acceleate that ocess. PRT deals with the shading comutation (how much light is eflected fom evey visible oint. The most geneal fomulation fo that is the so-called Rendeing Equation. We will have a look at it in the following slides. 27

32 Backgound Rendeing Integate incident light * visibility * BRDF Emitte 1 Emitte 2 Object 28

33 Rendeing Equation L d L d f s d L s H s ds ( = e( + (, ( N ( Ω Given an object illuminated in a lighting envionment, the endeing equation models the equilibium of the flow of light in the scene. It can be used to detemine how light a visible oint eflects towads the viewe. We will walk though a hemisheical fomulation of this equation. 29

34 Rendeing Equation L d L d f s d L s H s ds ( = e( + (, ( N ( Ω Radiance leaving oint in diection d The desied quantity is the adiance leaving a oint on the object P in a given diection d. Radiance is the intensity of light fom a oint to a cetain diection. 30

35 Rendeing Equation L d L d f s d L s H s ds ( = e( + (, ( N ( Ω Radiance emitted fom oint in diection d The fist tem is the adiance emitted diectly fom the oint in the given diection. In ou wok we will assume that no objects emit light, they ae just lit by a distant lighting envionment (the souce adiance function. 31

36 Rendeing Equation L d L d f s d L s H s ds ( = e( + (, ( N ( Ω Integal ove diections s on the hemishee aound This is followed by an integal ove the hemishee aound the oint, whee s is used to denote a diection on this hemishee 32

37 Rendeing Equation L d L d f s d L s H s ds ( = e( + (, ( N ( Ω BRDF at oint evaluated fo incident diection s in outgoing diection d BRDF: defines look of mateial. 4D function: ( s d f The 1 st facto inside the integal is the BRDF of the suface at oint P, the BRDF is a 4D function that models what ecent of light fo some inut diection (s leaves in some outgoing diection (d. 33

38 Rendeing Equation L d L d f s d L s H s ds ( = e( + (, ( N ( Ω Radiance aiving at oint fom diection s (also LHS The second tem is the adiance aiving at oint P fom the diection S, note that this is also the vaiable we ae solving fo so this is an integal equation. 34

39 Rendeing Equation L d L d f s d L s H s ds ( = e( + (, ( N ( Ω Lambets law cosine between nomal and -s = dot(n, -s The final tem is the cosine tem that comes fom lambets law due to ojected aea. 35

40 Neumann Exansion L d = L d + L d + L ( 0( 1( Outgoing adiance exessed as infinite seies One convenient way to eason about the solution to this integal equation is by using a Neumann exansion of this exession, whee outgoing adiance is exessed as an infinite seies. 36

41 Neumann Exansion L d = L d + L d + L ( 0( 1( Diect lighting aiving at oint fom distant envionment The fist tem in this seies is the diect lighting aiving at oint P fom a distant lighting envionment the souce adiance envionment we efeed to ealie. 37

42 38 ( ( ( 0 1 L d L d L d = + + L Neumann Exansion Neumann Exansion Diect lighting aiving at oint fom distant envionment ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω This tem is an integal ove the hemishee at the oint.

43 Neumann Exansion L( d = L0( d + L1( d + L L ( d = f (, s d L ( s V ( s H 0 env N ( Ω s ds Souce Radiance distant lighting envionment One of the new factos in this exession is Lenv the souce adiance function we ae assuming that this is the only souce of light in the scene. It is assumed to be fa away, so thee is no deendence on the osition (same incident adiance at evey --- usually an envionment ma is used to eesent this distant lighting. This is just a conventional integal, the souce adiance function only exists inside the integal. 39

44 40 ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω Neumann Exansion Neumann Exansion Visibility function - binay ( ( ( 0 1 L d L d L d = + + L The next new facto is the visibility function this is a binay function that is 1 in a given diection if the oint can see the distant lighting envionment, and zeo othewise. L0 models how light that diectly eaches the suface contibutes to outgoing adiance.

45 Neumann Exansion L d = L d + L d + L ( 0( 1( All aths fom souce that take 1 bounce The next tem in the exansion models all aths fom the souce adiance function that each the given oint afte a single bounce and contibute to outgoing adiance in the given diection. 41

46 Neumann Exansion L d = L d + L d + L L d = f s d L s V s H s ds ( (, ( 1 ( ( 0( 1( ( ( 1 0 N Ω L 0 All aths fom souce that take 1 bounce This is also just a conventional integal whee the evious tem (L_0 is inside of the integal. 42

47 Neumann Exansion L d = f s d L s V s H s ds ( (, 1 ( 1 ( L d = L d + L d + L ( 0( 1( ( ( i i N Ω L i-1 All aths fom souce that take i bounces In geneal the ith bounce models how all of the enegy fom the evious bounce contibutes to outgoing adiance in the given diection. 43

48 Oveview Intoduction Rendeing Equation Neumann Seies Shading Algoithms Basis Functions 44

49 Shading Algoithms Real-time Rendeing: Aoximations to Rendeing Equation Diect lighting indiect lighting: ambient tem Point lights no aea lights Had shadows no soft shadows Distant envionment mas no neaby emittes In eal-time endeing, we usually make aoximation to the full endeing equation, as it is too exensive to comute on the fly. Common aoximations ae: - only diect lighting (no indiect illumination at all, a simle ambient tem can make u fo it somewhat - oint lights (aea lights cast soft shadows, which ae hade to comute, since the light souce needs to be samled - distant envionment mas (this means that the same incident light aives at all oints of an object, and so thee is no need to ecomute it 45

50 Shading Algoithms Lighting Comlexity full global light Envionment Maing Pecomuted Radiance Tansfe Pecomuted Radiance Tansfe single aea light [Gahics [Hadwae] Soft Shadows oint lights [Gahics [Hadwae] simle Had Shadows shadows [Daubet03] indiect Tansot Comlexity Thee ae vaious evious algoithms that deal with eal-time shading. The gah hee shows with what tye of lighting comlexity vesus tansot comlexity they can deal with. Pecomuted Radiance Tansfe fills the ga fo natual illumination, which can aive fom all diections (distant envionment ma. 46

51 Oveview Intoduction Rendeing Equation Neumann Seies Shading Algoithms Basis Functions 47

52 Basis Functions Functions can be eesented/aoximated using basis functions Simila to oints in sace in diffeent coodinate fames 48

53 Basis Functions Given some function f(t: Figues ae coutesy of Robin Geen Given basis functions B i (t: B 1 (t B 2 (t B 3 (t 49

54 Basis Functions Poject into the function sace: = c 1 = c 2 = c 3 Coefficients c i eesent function 50

55 Basis Functions Reconstuct oiginal function Weight each basis function with coefficient c = 1 c 2 = c 3 = 51

56 Basis Functions Reconstuct oiginal function Sum them all u N i= 1 c i B i ( x = In this case: aoximation Benefit: fewe numbes needed 52

57 Basis Functions Function: f (t (eal-valued Pojection: c = f ( t B i ( t dt : f ( t, B i ( t Ω i = Reconstuction: N R f ( t = c ib i ( t i Otimal fo N basis functions (least-squaes 53

58 54 Basis Functions Basis Functions Othogonal basis functions Othonomal basis functions Peviously shown ojection only holds fo othonomal basis functions! = = j i j i t B t B j i 0 0 (, ( = = j i j i t B t B j i 0 0 (, ( 1 (, ( = t B t B i i 1 (, ( = t B t B i i

59 Basis Functions Non-othogonal basis functions B ( t, B ( t =? i? j Pojection cannot be done diectly ~ Define dual basis Bk ( t : ~ B ( t, B ( t i k 1 i = k = 0 else 55

60 Basis Functions Dual Basis ~ Bk ( t Bk B i (t Is linea combination of basis functions Comute using following equations A A ij i j ~ = k ( t A 1 kj B j j = B ( t B ( t dt B ( t Matix is the Gam matix of the basis 56

61 Basis Functions Pojection into non-otho. basis with dual ~ c i = f ( t, B i ( t Reconstuction with non-othogonal basis N R f ( t = c ib i ( t i 57

62 Basis Functions Examle: Sheical Hamonics Othonomal basis functions ove the shee Simila to Fouie seies ( θ,ϕ m Y l i = 1 i = 2 i = 3 i = 4 i = 8 i = 12 i = 15 i = 19 58

63 Basis Functions Sheical Hamonics Recuence fomula fo SH (band l, mode m Y m l ( θ, ϕ m P l ( cosθ Hee olynomials 2K = 2K 0 Kl Pl m cos( mϕ Pl ( cosθ, m sin( mϕ Pl ( cosθ ( cosθ, m l m l 0 m > 0 m < 0 m = 0 ae the associated Legende, The exact definitions can be looked u on the web. If only the fist few bands ae used, it is moe efficient to use exlicit fomulas, which can be deived by using Male fo examle. Hee ae the exact definitions fo the fist 25 basis functions. Inut is a diection (in Catesian coodinates. The evaluated basis function is witten to ylm_aay[]. float x = di[0]; float y = di[1]; float z = di[2]; float x2, y2, z2; ylm_aay[0] = f; //l,m = 0,0 // 1/sqt(4i ylm_aay[1] = f * y; //1,-1 //sqt(3/4i ylm_aay[2] = f * z; //1,0 ylm_aay[3] = f * x; //1,1 x2 = x*x; y2 = y*y; z2 = z*z; ylm_aay[4] = f * x * y; //2,-2 // sqt(15/4i ylm_aay[5] = f * y * z; //2,-1 ylm_aay[6] = f * (3.f*z2-1.f ; //2,0 // sqt(5/16i ylm_aay[7] = f * x * z; //2, 1 ylm_aay[8] = f * (x2 - y2; //2,2 // sqt( 15/16i const float fy30const = f; //0.25f*sqt(7.f/M_PI; const float fy31const = f; //1.f/8.0f*sqt(42.f/M_PI; const float fy32const = f; //0.25f*sqt(105.f/M_PI; const float fy33const = f; //1.f/8.f*sqt(70.f/M_PI; ylm_aay[ 9] = fy33const*y*(3.f*x2 - y2; //3,-3 ylm_aay[10] = fy32const*2.f*x*y*z; // 3,-2 ylm_aay[11] = fy31const*y*(5.f*z2-1.f; // 3,-1 ylm_aay[12] = fy30const*z*(5.f*z2-3.f; // 3,0 ylm_aay[13] = fy31const*x*(5.f*z2-1.f; // 3,1 ylm_aay[14] = fy32const*z*(x2-y2; // 3,2 ylm_aay[15] = fy33const*x*(x2-3.f*y2; // 3,3 59

64 60 Basis Functions Sheical Hamonics Basis Functions Sheical Hamonics Examle definitions (in Catesian coods ( ( and so on zx Y x Y y x Y y x z Y z Y π π π π π = = = = =

65 Basis Functions Examle: Sheical Hamonics Reconstuction: N=4 N=9 N=25 N=26 2 oiginal Hee an examle envionment ma is ojected in sheical hamonics and then back (using 4, 9, 25, and 26^2 basis functions. 61

66 Basis Functions Examle: Haa Wavelets Coutesy Ren Ng Refeence 4096 coeffs. 100 coeffs. Altenatively, the Haa wavelet basis functions can be used (instead of SH. We will talk biefly about this late on. It should be noted, that wavelets ae good at eesenting all-fequency detail (e.g. with 100 coeffs the bight windows ae eesented well, the not so imotant dake aeas (floo is eesented with less accuacy. The blocky aeaance of the ojection with 100 coefficients isn't as bad as it might look, if the BRDF is elatively dull, i.e., the envionment will be blued eally heavily. In that case, the blocky aeaance does not matte. In case of a vey shiny BRDF, the blockiness will aea as blocky eflections. 62

67 Intoduction Questions? 63

68 64

69 Pecomuted Radiance Tansfe: Theoy and Pactice 65

70 Diffuse PRT Jan Kautz MIT 66

71 Diffuse PRT Goal: to shade a diffuse object using Pecomuted Radiance Tansfe Diffuse: Reflected light is view-indeendent Simlifies equations [Sloan02] 67

72 68 ( ( ( 0 1 L d L d L d = + + L Diffuse PRT Diffuse PRT Stat fom Neumann exansion and make simlifying assumtions ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω To deive PRT fo the diffuse case we ae going to stat with just the diect tem fom the Neumann exansion of the endeing equation and make seveal simlifying assumtions.

73 69 ( ( ( 0 1 L d L d L d = + + L Diffuse PRT Diffuse PRT Diffuse objects: light eflected equally in all diections view-indeendent ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω = s d s H s V s L L N env d ( ( ( 0 ( π ρ = s d s H s V s L L N env d ( ( ( 0 ( π ρ The bottom equation is the simlified fom. Fist, fo diffuse objects light is eflected equally in all diections, so outgoing adiance is indeendent of view diection.

74 70 = s d s H s V s L L N env d ( ( ( 0 ( π ρ = s d s H s V s L L N env d ( ( ( 0 ( π ρ ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ( ( ( 0 1 L d L d L d = + + L Diffuse PRT Diffuse PRT Diffuse objects: BRDF is a constant This also means the BRDF is just a constant (and indeendent of diection so it can be ulled out of the integal. Rho_d eesents the diffuse eflectivity of the suface, and is a numbe between 0 and 1. The divide by Pi enfoces enegy consevation.

75 71 = s d s H s V s L L N env d ( ( ( 0 ( π ρ = s d s H s V s L L N env d ( ( ( 0 ( π ρ ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ds s H s V s L d s f d L N env ( ( (, ( ( 0 = Ω ( ( ( 0 1 L d L d L d = + + L Diffuse PRT Diffuse PRT Assume: lighting comes fom infinity, indeendent of As befoe, we assume the souce adiance function is at infinity, this means we only need to concen ouselves with the diection s.

76 72 Diffuse PRT Diffuse PRT Visually: = s d s H s V s L L N env d ( ( ( 0 ( π ρ = s d s H s V s L L N env d ( ( ( 0 ( π ρ Incident Light Visibility Cosine Reflected Light

77 Visually Incident Light Visibility Integand Cosine Visually, we integate the oduct of thee functions (light, visibility, and cosine. 73

78 Visually Incident Light Visibility Integand Pecomute Cosine The main tick we ae going to use fo ecomuted adiance tansfe (PRT is to combine the visibility and the cosine into one function (cosine-weighted visibility o tansfe function, which we integate against the lighting. 74

79 Poblems Poblems emain: How to encode the sheical functions? How to quickly integate ove the shee? This is not useful e se. We still need to encode the two sheical functions (lighting, cosine-weighted visibility/tansfe function. Futhemoe, we need to efom the integation of the oduct of the two functions quickly. 75

80 Diffuse PRT L ( ρd 0 = Lenv( s V ( s H N ( s ds π L env( s l i y i ( s i Reesent lighting using basis function y i ( Now we ae going to aoximate the souce adiance function with its ojection into a set of basis functions on the shee (denoted y i ( in this equation. The l_i ae the ojection coefficients of a aticula lighting envionment. Fo didactic uoses we ae using iecewise constant basis functions. 76

81 77 Diffuse PRT Diffuse PRT Plug into equation. Since it's linea, we can move sum outside integal. Eveything within the integal can be ecomuted. Ω = s d s H s V s y l L N i i i d ( ( ( 0 ( π ρ Ω = s d s H s V s y l L N i i i d ( ( ( 0 ( π ρ Ω = s d s H s V s y l L N i i i d ( ( ( 0 ( π ρ Ω = s d s H s V s y l L N i i i d ( ( ( 0 ( π ρ We can lug this aoximation diectly into the eflected adiance equation. Maniulating this exession exloiting the fact that integation is a linea oeato (sum of integals = integal of sums, we can geneate the following equivalent exession. The imotant thing to note about the highlighted integal is that it is indeendent of the actual lighting envionment being used, so it can be ecomuted.

82 Diffuse PRT L ( 0 ρ d = l yi ( s V ( s H N ( s ds π i i Ω ρ L0 = lt π d 0 ( ( 0 L = lt 0 i i i i i i We call the ecomuted integals tansfe coefficients Outgoing adiance: just a dot-oduct! This integal eesents a tansfe coefficient it mas how diect lighting in basis function I becomes outgoing adiance at oint. The set of tansfe coefficients is a tansfe vecto that mas lighting into outgoing adiance. We can otionally fold the diffuse eflectivity into the tansfe vecto as well. 78

83 Diffuse PRT L d = L d + L d + L ( 0( 1( 0 1 ( = i( i + i + L L l t t i ( lt i i L = i We do this fo evey bounce and fold eveything into a final tansfe vecto A simila ocess can be used to model the othe bounces, so that a final vecto can be comuted and used to ma souce adiance to outgoing adiance at evey oint on the object. Outgoing adiance is then just the dot oduct of the lights ojection coefficients with the tansfe vecto. 79

84 Diffuse PRT Poject lighting Rotate light e object Looku t * = e ixel/vetex L out Comute integal = ( = L ( s V ( s max( n s,0 d s env This shows the endeing ocess. We oject the lighting into the basis (integal against basis functions. If the object is otated wt. to the lighting, we need to aly the invese otation to the lighting vecto (in case of SH, use otation matix. At un-time, we need to looku the tansfe vecto at evey ixel (o vetex, deending on imlementation. A (vetex/ixel-shade then comutes the dotoduct between the coefficient vectos. The esult of this comutation is the outgoing adiance at that oint. 80

85 PRT Results (using SH Unshadowed Shadowed 81

86 PRT Results (using SH Unshadowed Shadowed 82

87 PRT Results (using SH Unshadowed Shadowed 83

88 Rendeing Reminde: n L out ( = L i i T i Need lighting coefficient vecto: L = L ( s y ( s d s i env Comute evey fame (if lighting changes Pojection can e.g. be done using Monte- Calo integation, o on GPU i Rendeing is just the dot-oduct between the coefficient vectos of the light and the tansfe. The lighting coefficient vecto is comuted as the integal of the lighting against the basis functions (see slides about tansfe coefficient comutation. 84

89 Rendeing Wok that has to be done e-vetex is easy: // No colo bleeding, i.e. tansfe vecto is valid fo all 3 channels fo(j=0; j<numbevetices; ++j { // fo each vetex fo(i=0; i<numbecoeff; ++i { vetex[j].ed += Tcoeff[i] * lightingr[i]; // multily tansfe vetex[j].geen += Tcoeff[i] * lightingg[i]; // coefficients with vetex[j].blue += Tcoeff[i] * lightingb[i]; // lighting coeffs. } } Only shadows: indeendent of colo channels single tansfe vecto Inteeflections: colo bleeding 3 vectos Sofa, the tansfe coefficient could be single-channel only (given that the 3-channel albedo is multilied onto the esult late on. If thee ae inteeflections, colo bleeding will haen and the albedo cannot be factoed outside the ecomutation. This makes 3-channel tansfe vectos necessay, see next slide. 85

90 Rendeing In case of inteeflections (and colo bleeding: // Colo bleeding, need 3 tansfe vectos fo(j=0; j<numbevetices; ++j { // fo each vetex fo(i=0; i<numbecoeff; ++i { vetex[j].ed += TcoeffR[i] * lightingr[i]; // multily tansfe vetex[j].geen += TcoeffG[i] * lightingg[i]; // coefficients with vetex[j].blue += TcoeffB[i] * lightingb[i]; // lighting coeffs. } } 86

91 Pecomutation Integal ρ T = V ( s max( n s,0 y ( s d s i ρ π evaluated numeically with ay-tacing: N 1 4π ρ Ti = V ( s j max( n s j,0 y i ( s j N π j = 0 Diections s j need to be unifomly distibuted (e.g. andom Visibility V is detemined with ay-tacing V i The main question is how to evaluate the integal. We will evaluate it numeically using Monte-Calo integation. This basically means, that we geneate a andom (and unifom set of diections s_j, which we use to samle the integand. All the contibutions ae then summed u and weighted by 4*i/(#samles. The visibility V_( needs to be comuted at evey oint. The easiest way to do this, is to use ay-tacing Aisde: unifom andom diections can be geneated the following way. 1 Geneate andom oints in the 2D unit squae (x,y 2 These ae maed onto the shee with: theta = 2 accos(sqt(1-x hi = 2y*i 87

92 Pecomutation Visually... Basis 16 Basis 17 illuminate esult Basis Visual exlanation 2: This slide illustates the ecomutation fo diect lighting. Each image on the ight is geneated by lacing the head model into a lighting envionment that simly consists of the coesonding basis function (SH basis in this case illustated on the left. This just equies endeing softwae that can deal with negative lights. The esult is a satially vaying set of tansfe coefficients shown on the ight. To econstuct eflected adiance just comute a linea combination of the tansfe coefficient images scaled by the coesonding coefficient fo the lighting envionment. 88

93 Pecomutation Code // : cuent vetex/ixel osition // nomal: nomal at cuent osition // samle[j]: samle diection #j (unifomly distibuted // samle[j].di: diection // samle[j].shcoeff[i]: SH coefficient fo basis #i and di #j fo(j=0; j<numbesamles; ++j { double csn = dotpoduct(samle[j].di, nomal; if(csn > 0.0f { if(!selfshadow(, samle[j].di { // ae we self-shadowing? fo(i=0; i<numbecoeff; ++i { value = csn * samle[j].shcoeff[i]; // multily with SH coeff. esult[i] += albedo * value; // and albedo } } } } const double facto = 4.0*PI / numbesamles; // ds (fo unifom dis fo(i=0; i<numbecoeff; ++i Tcoeff[i] = esult[i] * facto; // esulting tansfe vec. Pseudo-code fo the ecomutation. The function selfshadow(, samle[j].di taces a ay fom osition in diection samle[j].di. It etuns tue if thee it hits the object, and false othewise. 89

94 Pecomutation Inteeflections Light can inteeflect fom ositions q onto s diect q s q indiect s q Object Not only shadows can be included into PRT, but also inteeflections. Light aiving at a oint q can be subsequently scatteed onto a oint. I.e. light aiving fom s_q can aive at, although thee is may be no diect ath (along s_q to (as in this examle. Note, that light is aiving fom infinity, so both shown diection s_q oiginate fom the same oint in infinity. 90

95 Pecomutation Inteeflections Light can inteeflect fom ositions q, whee thee is self-shadowing: out DS out L = L + L V s s n (1 ( max(,0 d s diect illumination Ω q ( s q (s light leaving fom towads invese visibility cosine q (s q (s q (s q (s q (s Moe fomally, we do not only have diect illumination L^DS (Diect Shadowed, but also light aiving fom diections s, whee thee is self-shadowing (i.e. 1-V_(s. The light aives fom ositions q, which ae the fist hit along s. 91

96 Pecomutation Inteeflections Pecomutation of tansfe vecto has to be changed An additional bounce b is comuted with T b, i whee ρ = T π T 0,i b 1 q, i (1 V Final tansfe vecto: ( s max( n s,0 ds is fom the ue shadow ass B 1 T, i = T b = 0 b, i To account fo inteeflections, the ecomutation has to be changed again. Each additional bounce b geneates a vecto T^b_{,i}, which is comuted as shown on the slide. Each of these additional tansfe vectos is fo a cetain bounce. To get the final tansfe vecto, they have to be added. Again, the un-time emains the same! 92

97 Inteeflections (using SH No Shadows/Inte Shadows Shadows+Inte This set of images shows the buddha model lit in the same lighting envionment, without shadows, with shadows and with shadows and inte eflections. 93

98 Choice of Basis Functions Citeia Want few coefficients, good quality No flickeing Peole have used: Sheical Hamonics Haa Wavelets Steeable basis functions 94

99 PRT Quality SH basis n=2 linea n=3 quadatic n=4 cubic n=5 quatic n=6 quintic n=26 n=26 windowed RT Quality of SH solution. 0 degee (oint light souce, 20 degee light souce, 40 degee light-souce. Light is blocked by a blocke casting a shadow onto the eceive lane. Diffeent ode of SH is shown (ode^2 = numbe of basis functions. Vey ight: exact solution. As stated befoe, lighting is assumed low-fequency, i.e. oint light doesn't wok well, but lage aea lights do! 95

100 PRT with Haa Wavelets Main diffeence to SH: Haa needs to ecomute and kee all lighting/ tansfe coefficients! Decide deending on lighting, which ones to use! (see ight Imlies (comessed stoage of all tansot coefficients (64*64*6 Not well-suited to hadwae endeing Coutesy Ren Ng As shown in the comaison on the ight, with moe coefficients, wavelets do much bette eesent the lighting than the SH (which show a lot of inging atifacts. Thee ae a few diffeences when using Haa instead of SH: 1 All tansfe coefficients need to be comuted and stoed! 2 Because which of the actual N coefficients ae used, is decided at un-time based on the lighting's most imotant N coefficients (N=100 seems sufficient. 3 This equies all tansfe coefficients to be stoed as well (can be comessed well, like lossy wavelet comessed images. 4 Since the coefficients to be used change at un-time, this is not well-suited to a GPU imlementation (but woks fine on CPU 96

101 Conclusions Pos: Fast, abitay dynamic lighting PRT: includes shadows and inteeflections Cons: Simle imlementation woks well only fo low-fequency lighting High-fequency shadows need Wavelets + comession to make it fast! I 97

102 Diffuse PRT Questions? 98

103 99

104 1

105 Pecomuted Radiance Tansfe: Theoy and Pactice 2

106 Geneal PRT Jaakko Lehtinen Helsinki Univesity of Technology, Remedy Entetainment Ltd. 3

107 Paameteization of Emission The object is lit by a distant lighting envionment Usually eesented as an envionment ma L env ( ω Distant means that location in the scene does not affect the diectional distibution of incident light. In othe wods, in absence of occlusion, the diect light incident uon all oints on the object is the same, i.e., L_env(,omega = L_env(omega fo all. We usually eesent the lighting envionment using an a so-called light obe image, but it is also ossible to use analytic aea lights (cones, etc.. These will be coveed in a subsequent at of the couse. 4

108 Pojected Emission Poject the envionment ma into a lowdimensional function sace to yield basis coefficients L env ( ω n i= 1 l i y i (ω l y 1 y 2 y 8 y 12 y 3 y 4 y 15 y 19 As descibed in the ealie section, the envionment ma that we want to use fo lighting the object/scene, is ojected into a function basis {y_i} with i=1,,n. This yields an aoximation to the oiginal envionment ma, and the aoximation is fully defined by the vecto l of coefficients. 5

109 Goals With time-vaying lighting, want to Shade objects that have shiny BRDFs Shininess: The aeaance of sufaces change accoding to the viewing diection diffuse case doesn t aly any moe! Aoximate the lighting aound the object Account fo both occlusion and light eflected by the object Used fo shading othe nea-by sufaces (i.e., not equied fo just a single object Limitation: Scene cannot defom Why the lighting aound the object is useful: Think of the object being a landscae scene with mountains, hills, valleys etc. If we can comute the lighting incident to oints in fee sace in the scene, we can shade moving objects (chaactes, say with lighting that is affected by the envionment, and thus get colo bleeding and soft shadowing effects onto the chaacte. The effect of the chaacte on the scene needs to be handled seaately, then. Hee we deal exclusively with scenes and objects that ae igid, i.e., they do not change thei shaes. It is ossible to otate the object (that coesonds just to an oosite otation of the incident lighting. Real-time Global Illumination fo defoming scenes is vey challenging. 6

110 Goals visually What does look like, given L env?? L ( L ( ω? env ω out Some linea oeation fo each = Tansfe matix Given some ojected envionment ma, what do oints on the object look like? That is, what values should we assign to ixels? The deendence of the aeaance of each oint is linea w..t. the distant lighting envionment, so we ae looking fo some linea oeato that takes the emission and oduces the aeaance. In the end, we will eesent this aeaance in some linea basis, so the linea maing will be a matix that mas the coefficients of incident, distant lighting envionment into coefficients fo outgoing adiance fo the oint. 7

111 Goals visually What does look like, given L env??? L ( L ( ω? env ω out Some linea oeation fo each = Tansfe matix Comute tansfe fo many oints and inteolate to get aeaance of the whole object We ll comute this maing fo many oints on the object, so that we can get a easonable aoximation fo the aeaance of the whole object by inteolating fom these samles. 8

112 Goals visually What does see, given L env? L ( L ( ω? env ω xfe What do the oints nea the object see when the object is illuminated using an envionment ma? Some light oceeds diectly fom the distant envionment to the oint. 9

113 Goals visually What does see, given L env? L ( L ( ω? env ω xfe Some gets shadowed by the object 10

114 Goals visually What does see, given Lenv? Lenv (ω Lxfe ( ω? Some linea oeation fo each = Tansfe matix? Some light gets eflected by the object towads the oint. Again, we ae looking fo the matices that encode this tansfomation of distant incident lighting L_env into the lighting incident fo each aticula oint. 11

115 Deivation of Tansfe Matices 12

116 Tansfeed Incident Radiance Define Tansfeed Incident Radiance Lxfe ( ω fo oint as the light flowing to both diectly and afte inteaction with the object = what sees Eithe on suface o in fee sace unified teatment of both goals 13

117 L Tansfeed Incident Radiance Pecomute tansfe matices off-line use them at untime to detemine out integate with BRDF * cosine to get outgoing adiance accoding to the usual eflectance equation (details late ( ω = L ( ω f (, ω ω cosθ dω out L xfe xfe Ω( n in in L xfe out in Outgoing adiance Tansfeed incident adiance BRDF * cosine The matices that ma the incident lighting into tansfeed incident adiance will be ecomuted off-line. At untime these matices will be used fo comuting the tansfeed incident adiance fom the cuent lighting envionment. Then the tansfeed incident adiance is eflected once moe fo comuting the intensity of light flowing fom towads the viewing diection omega_out. 14

118 Tansfeed Incident Radiance Why tansfeed incident adiance? Fo sufaces, useful: Can inteolate tansfeed adiance fom sase samles, add detail in final eflection [Sloan03BRT] (details late Comutation and stoage of tansfe is exensive Fo fee sace, must: No way of knowing the suface that we want to shade in advance Fo sufaces of the object, it is ossible to aameteize the outgoing adiance diectly, and many methods do that. But it is also ossible to decoule the samling ates of tansfeed incident adiance and outgoing adiance. The ationale is that tansfeed incident adiance often (but not always vaies quite slowly ove sace, wheeas the outgoing adiance fom a suface often has high satial fequencies. Thus, the tansfeed incident adiance can often be inteolated fom sase samles, and tuned into highe-fequency outgoing adiance (bi-scale adiance tansfe. If we want to lace a new object in the scene, we need to know the light incident uon the object fom all diections. Of couse, if we efom the final eflection integation at untime, we can in incile change the BRDF of the final bounce at untime. In that case, though, the global illumination solution will not be coect any moe. 15

119 Diectional Inne Poduct Ω( a(, ω b( ωdω ω = : a(, b In the est of the esentation we ll denote a diectional inne oduct using wedges like this. The aguments to this inne oduct ae always two functions: One that vaies both in sace ( and diection (omega, and anothe that vaies only in the diection domain (omega. We ll use this notation fo getting ojection coefficients fo diffeent functions. 16

120 Basic Idea Poject tansfeed incident adiance into a sheical function basis {z j }, j=1,,m in a numbe of oints in the scene (e.g., at vetices of mesh, texels Basis not necessaily the same as used fo lighting L xfe ( ω ( ω T = z l m i= 1 l i zi ( ω We ll stat ou way towads deiving the tansfe matices that ma the incident illumination into tansfeed incident adiance. It all stats fom eesenting tansfeed incident adiance fo oint in a function sace sanned by sheical basis functions z. The coefficient vecto that descibes this aoximation of tansfeed incident adiance is denoted by l^. Note that what we ll do hee coesonds to a simle discetization of the continuous endeing equation, but we ski the details of this. 17

121 The Tansfe Matix We want to aoximate lighting incident to oint in a basis {z j } by coefficients The distant lighting envionment L env is given in basis {y i } by coefficients Lighting and ojection ae linea thee is a linea elationshi between these vectos: l = T l l l The coefficients l^ ae unknown, the coefficients l we know. What we want to do is find thei elationshi; this elationshi between the two vectos is linea, and thus it can be exessed using a matix. We call this matix the tansfe matix fo oint : It mas the distant, incident illumination to an aoximation of the lighting that eaches. We ll come to what the matix looks like next 18

122 The Tansfe Matix Intuition T The i:th column of aoximates the aeaance of the scene as seen fom oint in tems of the basis {z}, when the scene is illuminated by the i:th lighting basis function alone. Since the scene is lit using a linea combination of the basis lights (weights secified by l, just take a linea combination of these basis aeaances (matix-vecto multilication! l = T l Matix vecto multilication T*l is nothing but taking a linea combination of the columns of T with weights taken fom the comonents of l. Thus, if we think of a lighting envionment vecto l that only has a single one at index i and the est of the elements ae zeo, we see that the matix-vecto multilication just icks out the i:th column of the matix. And since the esult, l^, aoximates the lighting incident to in tems of the basis z, we can conclude that the columns of the matix ae basis aeaances of the scene unde the illumination of a single lighting basis function y_i. Ou lighting envionment is secified using a linea combination of basis lights, and so this matix-vecto oduct just tells us that we can get an aoximation to what the wold looks like to by taking a linea combination of these basis solutions (=the columns of T^, weighed by the elative weights of the lighting basis functions (= the comonents of the vecto l. The simlest case is if we use a iecewise constant basis fo the lighting envionment; then all the indices i coesond to a diffeent diectional light by which the scene is illuminated, and the columns of this matix just tell us what the scene looks like fom oint, when it is lit by a unit-intensity light at the i:th diection. 19

123 Enties of the Tansfe Matix Columns ae basis aeaances The matix enty T ji is the j:th ojection coefficient of the adiance incident to that comes fom the i:th light basis function y i alone (denote by L i xfe ( ω i i T ji L ~ z j ( d : L (, ~ xfe ( ω ω ω = xfe z j = Ω Examle: Diect lighting fom i :th light basis function = y i ( ω V (, ω Let s conside the lighting cast on the scene by a single basis light y_i. Let us denote the light that comes fom the i:th light basis function y_i to oint eithe diectly o though any numbe of bounces by L^i_xfe. By ou above definition, the enty T^_ji is the lighting incident to due to lighting basis function y_i, ojected against the dual basis function z~_j. Fo examle, the diect lighting due to basis function y_i to fom diection omega is just the value of y_i in diection omega, modulated by the visibility function V(,omega fom : V = 1 if can see the envionment in diection omega, and V = 0 othewise. 20

124 Enties of the Tansfe Matix Visually y i 1 y i y i+1 L i xfe = the light fom basis function y i that eaches eithe diectly o though any numbe of bounces off the object The object ~ ( ω z j Integate L i xfe against dual basis function ove shee centeed on T ji i = xfe L j Ω ( ω ~ z ( ω dω The y and z functions ae dawn as hat functions fo illustational uoses only. Such functions can be used fo comuting tansfe, but othe basis functions, ossibly with global suots like the Sheical Hamonics, ae often used. 21

125 Enties of the Tansfe Matix T ji Ω Can evaluate suitable algoithm i = L ω ~ xfe ( z ( ω dω L i xfe ( ω using any Monte Calo ath tacing Photon Maing Pogessive adiosity like ocedue, etc. Must be able to ende with negative light (e.g., Sheical Hamonics have negative values j All you need is to be able to comute the adiance that is incident to the oint when the scene is lit using the i:th light basis function alone! NOTE that the lighting basis functions can be negative, so the endee must be able to handle negative light as well. Next we ll conside the simle case of diect illumination only; that is, we conside only the light incident to the object that has been shadowed by the object, but has not bounced on the suface befoe iminging uon. This simle case will show that this ehas abstact-looking comutation can eally be simle in some cases. 22

126 Easiest Case: Tansfe Matix fo Diect Lighting The diect lighting incident to (denote by L 0 is just the lighting envionment modulated by the visibility fom : L 0( ω = L env ( ω L ( ω V (, ω env V (, ω The fist tem, the diect lighting tem L_0, is just ou lighting envionment L_env times the visibility function V(,w that encodes whethe o not the object blocks the sightline fom towads w: If can see the envionment at diection w, the lighting is taken fom the envionment; othewise thee is no diect light fom that diection, i.e., L_0( w = 0 fo that diection. 23

127 Easiest Case: Tansfe Matix fo Diect Lighting The diect lighting to due to basis function i y i is then And thus L 0( ω = y ( ω V (, ω T 0, ji = Ω = i L 0( ω, y ( ω ~ z ( ω V (, ωdω Call the diect-lighting-only matix i j i ~ z j T 0 The matix enty T_ji is the integal of the visibility function times the i:th lighting basis function times the j:th tansfeed incident adiance coefficient! 24

128 Easiest Case: Tansfe Matix fo Diect Lighting L = 0 ( ω m z ( ω n j j= 1 i= 1 m j= 1 l i l 0, j T z 0, ji j ( ω = z( ω T T 0 l If we substitute the evious into the equation that gives the aoximated tansfeed incident adiance fom the its basis coefficients, we end u with the following. The sum in the middle can be witten in matix fom as given on the second line. The tansfe matices T0 ma the incident illumination (exessed in basis {y} by vecto l into diect, shadowed incident adiance at the oint, exessed in basis {z}. 25

129 Tansfe Matix fo Diect Lighting (examle Ω T0, ii = yi ( ω yi ( ω V (, ω dω Conside a iecewise constant basis Use the same basis fo light and tansfe: z i = y i Then the tansfe matix is eally simle Basis functions do not ovela: Diagonal matix! Faction of light fom i:th diection that eaches If we use a iecewise constant basis fo both incident lighting and tansfeed incident adiance, this is easy to gas. Fist, all non-diagonal enties of the matix will be zeo: Unde the integal thee ae basis functions y_i and y_j. Fo i!= j thei suots do not ovela, and thus the integand is identically zeo. The matix enty (i,i gives the faction of diect light incident fom i:th diection that eaches the oint. 26

130 A Pogessive Method of Simulation Accounting fo Indiect Lighting The next section deives a aticula method fo comuting the tansfe matices that account fo indiect lighting as well. It esembles a ogessive gatheing adiosity method, and beas a close esemblance to an ealy non-diffuse global illumination method of Sillion and othes fom Siggah 91. In what follows, we ll deive a ecusive fomula that comutes the tansfe matices that coesond to k+1 bounces of light, given that we know the tansfe matices that account fo k bounces fo light eveywhee in the scene. As we aleady have a method fo comuting T_0, the tansfe matix fo diect lighting, we ae able to account fo an abitay numbe of bounces this way. And as lighting is linea, the tansfe matix T^ that accounts fo all bounces is just the sum of the matices T_0^, T_1^, that each account fo a aticula numbe of bounces. 27

131 Rendeing Equation fo Tansfeed Incident Radiance Rewite the endeing equation fo tansfeed incident adiance L xfe ( ω = L L xfe Ω( ' env ( ' ω' f Woks also in fee sace, not only on sufaces ( ω V (, ω + ( ', ω' ωcosθ ' dω'? ω ω' Diect w/ shadows Reflected In ode to deive the method, we ll use a tansosed vesion of the endeing equation; one that we have witten secifically using tansfeed incident adiance as the unknown, not outgoing adiance as is usually done. These equations ae equivalent; if we know the solution to the othe, we get the othe etty easily fom it. What the equation says: The lighting incident to is the sum of diect lighting fom the lighting envionment, shadowed by the object Light eflected by the scene towads. Note that the fist tem is only non-zeo if the ay fom towads omega doesn t intesect the scene anywhee, and the second tem can only be non-zeo if the ay does intesect the scene. P is the oint that the ay fom towads omega intesects. Note that the usual eflectance integal is efomed above the oint hee, not as in the usual vesion of the endeing equation. This woks just as well fo oints not on the sufaces, but in fee sace. 28

132 Recusion by Neumann Seies The Neumann seies gives ecusion fo L k+1 fom L k : Lk + 1( ω = Lk ( ' ω' f ( ', ω' ωcosθ ' dω' Ω( ' once you know L k, you can comute L k+1 You still have to know L k eveywhee in the scene? ω' The incident vesion of the endeing equation can be solved using Neumann seies just as well as the outgoing vesion: The lighting uon is the sum of diect light, light that has taken one, two, thee, etc. bounces. Given that we know the lighting in the scene that has taken k bounces: Then the Neumann seies gives the elationshi of this known k-times eflected light and k+1 times eflected light, i.e., if we know the incident adiance fom the evious bounce, the next one is obtained fom the eflectance equation. 29

133 Pojection of Tansfeed Incident Radiance At each oint, oject the tansfeed adiance into a function sace {z j }, j=1,,m l j = Ω L L xfe xfe ( ω ( ω ~ z j m j= 1 l j z ( ω dω = : j ( ω L xfe (, ~ z j To deive tansfe matices fo all bounces, we ll stat fom the ojection equation: We want to exess L_xfe at in tems of the basis functions z. In the end this will yield the tansfe matices fo all bounces. As we saw ealie: In ode to get the j:th ojection coefficients, we have to integate the function against the dual basis function z^tilde. 30

134 Pojection of Tansfeed Incident Radiance l = L (, ~ z j xfe But by the Neumann seies L ( ω = xfe j L0 ( ω + L1 ( ω + L2 ( ω + K diect, shadowed 1bounce L env ( ω 2 bounces diect shadows As we saw ealie, the endeing equation may be solved by the Neumann seies as the sum of diect lighting, light bounced off the object once, twice, etc. That is, the adiance incident to is the sum of adiance diectly fom the lighting envionment L that is shadowed by the object 31

135 Pojection of Tansfeed Incident Radiance l = L (, ~ z j xfe But by the Neumann seies L ( ω = xfe j 1 bounce L0 ( ω + L1 ( ω + L2 ( ω + K diect, shadowed 1bounce 2 bounces L env ( ω the light that eaches though one bounce off the object 32

136 Pojection of Tansfeed Incident Radiance l = L (, ~ z j But by the Neumann seies L ( ω = xfe l j xfe = L (, L (, 1 0 j L0 ( ω + L1 ( ω + L2 ( ω + K diect, shadowed 1bounce ~ z ~ z j j + + L 2 (, 2 bounces ~ z j +K two bounces, and so on. Since the inne oduct is linea, the ojection coefficients can be comuted by ojecting each bounce of light seaately and adding the esults togethe. 33

137 The Stoy So Fa We ve aleady deived e-oint matices that ma coefficients of lighting envionment to coefficients that eesent diect lighting ma the lighting envionment in basis {y} to incident diect light in basis {z} T 0 Next: deive matices fo each oint that give bounce k+1, given the matices fo bounce k 34

138 Tansfe Matix fo Bounce k+1 Subsequent bounces: l k + 1, j = = k Ω( n' L k+ 1 (, ~ z j L ( ' ω' f ( ', ω' ωcosθ 'dω', L k ( ' ω' ω ~ z j 1 2 Difficulty: Don t know L k in all oints, only some need to inteolate We ll stat the deivation of the matices T_^{k+1} fom the ojection coefficients l^_k. The ojection of the bounce k+1 at oint is the diectional inne oduct (at of the adiance eflected towads fom the evious bounce k and the dual basis functions z_j^tilde. That is, fo a lage numbe of diections aound, we have to evaluate the k+1 bounce adiance eflected towads fom each diection. That eflected adiance is detemined by eflecting the evious bounce at the oints whee the ays fom hit by efoming the usual eflectance integation at the oints whee ou ays hit. NOTE that if a ay does not hit the object, thee is no light of bounce k+1 coming fom that diection! We will esent efficient methods fo evaluating the eflectance integal late when we talk about oducing outgoing adiance fom tansfeed incident adiance. Notice that the ays fom may hit the suface of the object anywhee, not just at the oints whee we know the evious bounce k. We know an aoximation to tansfeed incident adiance fom the evious bounce at the finite set of oints, not in all oints in the scene to get L_k fo a oint that falls between the samles, inteolate linealy fom the neaest known oints. 35

139 Inteolation of L k L Inteolate L k fom neaby, known oints 1, 2, o with weights w s : k o ( ' ω' w L ( ω' s= 1 s k s L k ( 1 ω' L k ( ' ω' 1 L k ( 2 ω' 2 Inteolate tansfeed incident adiance fom neaest known oints. The w_s ae inteolation weights (ositive, sum to one. In actice we know L_k at the vetices of ou model (o texels, if we samle tansfe using textues. This means we can easily inteolate using baycentic coodinates (fom vetices o bilinealy (fom textues. 36

140 Tansfe Matix fo Bounce k+1 o s= 1 L ( ' ω' w L ( ω' k s k s L k ( ' ω' ω 1 l = k + 1, j = L k+ 1 o Ω ( n' s= 1 (, ~ z w L ( s k j s ω' f ( ', ω' ωcosθ 'dω', ~ z j 2 = o w m s Ω ( n' s= 1 a= 1 l k, a za ( ω' f ( ', ω' ωcosθ 'dω', z s ~ L ( k s ω' m a= 1 j s lk, a za ( ω' Now we have the tools to comute the next bounce. When evaluating the adiance eflected fom the last bounce by oint towads, substitute the inteolated tansfeed incident adiance into the eflectance integal. The tansfeed incident adiance though k bounces we know; we can comute its coefficients using the matices fo k bounces T_k. 37

141 Tansfe Matix fo Bounce k+1 l = k + 1, j o ws m Ω ( n' s= 1 a= 1 i= 1 n l i T s k, ai z a ( ω' f ( ', ω' ωcosθ 'dω', ~ z j = n i= 1 l = T k + 1 i m o s w T z f ~ s k, ai a ( ω' ( ', ω' ωcosθ 'dω', z j Ω ( n' a= 1 s= : = T k + 1, ji l No deendence on l, just the evious matices T k Visually next At the oints fom which we inteolate, tansfeed incident adiance of the evious bounce is linea in the incident coefficients (induction hyothesis we ve oved this fo diect lighting, i.e., k=0, aleady. On the to ow we substitute this linea elationshi to the evious equation, Then move the sum out of the inne oduct by lineaity, And notice that what s left inside the inne oduct is just integals of basis functions, the BRDF and the enties of tansfe matices fo the evious bounce, and that thee ae again two fee indices, and that this is again just a matix-vecto multilication with the incident lighting coefficients l. We define the stuff inside the the inne oduct as T_{k+1}, the tansfe matix fo bounce k+1.!!! You should notice that the coefficients l fo the lighting envionment DO NOT affect the tansfe matix T_k+1 it is comuted uely fom suitable integals and inteolations of the tansfe matices T_k. This is just to convince you that it woks, a moe gahical exlanation is coming u next 38

142 Tansfe Matix fo Bounce k+1 visually Tansfeed incident adiance fom incident basis function y i, catued at 1 and 2 by basis functions z a, inteolated to o w s T k, ji s= 1 T s ' k, ji L k ω ( ' ω' 1 2 s T, ~ z m o k + 1, ji = ws Tk, ai za ( ω' f ( ', ω' ωcosθ 'dω' Ω ( n' a= 1 s= 1 j When evaluating the ojection of tansfeed incident adiance of bounce k+1 fom lighting basis function y_i to oint, we have to comute the adiance eflected towads fom the evious bounce. To do this, we have to evaluate a diectional inne oduct fo many diections omega aound. Fo a single diection omega, the boxed fomula is just the tansfeed incident adiance fom y_i afte k bounces at the oint whee the ay fom towads omega hits the suface of the object, inteolated in the fashion we just descibed. 39

143 Tansfe Matix fo Bounce k+1 visually Tansfeed incident adiance fom incident basis function y i, catued at 1 and 2 by basis functions z a, inteolated to, eflected towads ω' ω 1 2 s T, ~ z m o k + 1, ji = ws Tk, ai za ( ω' f ( ', ω' ωcosθ 'dω' Ω ( n' a= 1 s= 1 j The boxed fomula is now the adiance eflected by oint fom the evious bounce towads ; this is just the usual eflectance equation. 40

144 Tansfe Matix fo Bounce k+1 visually Tansfeed incident adiance fom incident basis function y i, catued at 1 and 2 by basis functions z a, inteolated to, eflected towads, ojected to z j ω' ω 1 2 s T, ~ z m o k + 1, ji = ws Tk, ai za ( ω' f ( ', ω' ωcosθ 'dω' Ω ( n' a= 1 s= 1 j And finally, as we eeat this ocess fo many diffeent omegas, we oject tansfeed incident adiance fom y_i afte k+1 bounces into the function sace sanned by the z functions. The inne oduct (=oute integal <> is just a gathe oeation as seen fom oint : Shoot a numbe of ays fom, detemine the light eflected towads fom whee the ays hit (using inteolation fom neaby samles, and oject that light into the basis z. If we use sheical hamonics fo eesenting the tansfeed incident adiance, this beas quite some esemblance to an ealy non-diffuse adiosity method fom 1991 by Sillion and othes; they used sheical hamonics fo eesenting outgoing adiance fom suface oints, and used a simila gatheing scheme. They woked with fixed lighting, though. 41

145 Comlete Tansfe Matix Each bounce esults in a new matix so that the final oeato fo oint is T k + 1 T = T0 + T1 + T 2 +K Subsuface scatteing is simle to add just simulate it with you favoite method when comuting the T k+1 s fom T k s In the beginning we gave an exlicit fomula fo T_0 and now we have a ecusion fo T_{k+1} fom T_k we can comute tansfe matices fo all bounces. The comound tansfe matix that accounts fo all bounces is obtained fom these by just summing them u. It is woth noticing that nothing in the evious deivation events inclusion of subsuface scatteing; all you need to do is simulate with you favoite method when detemining the matices T_{k+1} fom the evious bounce. 42

146 Tansfe Matix in Fee Sace In a non-scatteing medium, oints not on sufaces do not affect the aeaance of the scene comute solution with all bounces fo suface oints fist, then gathe all bounces at once fo oints in fee sace 43

147 Outgoing Radiance The Final Bounce Towads the Eye 44

148 Outgoing Radiance Once we have tansfeed incident adiance, must eflect it once moe to oduce outgoing adiance fo sufaces must know this to ende ixels Many methods [Sloan02] (oiginal PRT, [Kautz02], [Lehtinen03], [Sloan03] (bi-scale tansfe, [Ng03], [Liu04] & [Wang04] (wavelets 45

149 Outgoing Radiance L out Lxfe Ω ( ω out = ( ω f in (, ω in ω cos' θ : = max(cosθ,0 out cos' θ dω in Again a linea oeation on L xfe! l out = O T l L xfe ( ω in L out ( ωout?? We ll add anothe linea oeato O^ that mas the tansfeed incident adiance at into outgoing adiance. O may be a matix, in which case it mas the tansfeed incident adiance into full, sheical outgoing adiance exessed in a new function basis, o it can be a vecto that deends on the outgoing viewing diection, in which case it is just dotted with the tansfeed incident adiance coefficients to oduce outgoing adiance into the viewing diection. In ode to define the integation domain as the whole shee instead of the hemishee centeed at the nomal as done usually, we ll change the cosine tem cos theta = dot(nomal, omega_in to ead cos theta := max( dot(nomal, omega_in, 0 to signify claming of the cosine to zeo fom below. 46

150 Rotation to Tangent Fame L Fist, otate L xfe into the tangent fame of the suface at xfe Reason: in global fame the BRDF would need to be tabulated fo each oientation If suface mateial is constant, the BRDF looks the same fo all oints in the tangent fame 3 DOF less ( ω L R xfe ( ω Global coodinates Tangent fame In ode to simlify things a bit, we will efom the outgoing adiance integation in the canonical fame of each oint. This canonical sace is the so-called tangent sace. Note that this has nothing to do with the oientation of the object; only the oientation of the vetex tangent sace w..t. the object sace. The ationale fo this is that in the tangent sace the BRDF takes the simlest ossible fom; indeed, if the comutation would be done in global coodinates, we would need to efom diffeent comutations fo suface oints oiented diffeently. This would add a significant stoage buden. Howeve, this ste is not stictly necessay. Fo instance Ng et al. [2004] (tile oduct wavelet integals make the BRDF highe-dimensional by adding the suface oientation as anothe dimension, and comessing this high-d signal with wavelets. 47

151 Rotation to Tangent Fame Can be done using a otation matix l R, = R l = R T l (Details on SH otation matices in [Kautz02] L R xfe = z( ω ( ω = T R T l m j= 1 l R, j z( ω Since otation is a linea oeation, we can get coefficients of otated tansfeed incident adiance by alying a suitable otation matix R to the coefficients of tansfeed incident adiance. We denote this by l^{r,}. Only the Sheical Hamonics and othe steeable bases ae nice in the otation sense, since they ae closed unde otation. This means that any SH exansion can be otated an abitay amount, and still eesented exactly using the same basis functions. This means the ojected lighting envionment does not suffe fom suious wobbling o aliasing unde otation. Othe basis sets not necessaily closed unde otation, which means that the otation cannot be eesented exactly with the same basis set. This might esult in temoal wobbling and simila aliasing atifacts. 48

152 Outgoing Radiance fom Rotated L xfe L = = out ( ω Ω j= 1 j= 1 out m R, l j z j ( ωin f m l L R xfe R, j Ω (, ω z j in ( ω f in L T out ωout ( ωout (, ω (, ω ( = O l in in ω ω : = O j ( ωout out out cos' θ dω cos' θ dω in in R, View-deendent m-vectos (can be stoed in an env.ma [Kautz02] used this with SH In ode to get outgoing adiance fom tansfeed, otated incident adiance, we ll fist lug its basis exansion into the usual eflectance equation. The comonents of the vecto l^{r,} ae the basis coefficients of tansfeed, otated incident adiance. The integal in the ed box only deends on the basis index j and the outgoing angle thus, they ae view-deendent vectos of length m (the numbe of basis functions used fo eesenting tansfeed incident adiance. They can be stoed in an envionment ma, and outgoing adiance can be comuted simly by looking the aoiate vecto with the view diection and dotting it with the l^{r,}. 49

153 Outgoing Radiance by BRDF Matix In SH, can use the doubly-ojected BRDF matix of Westin et al. [1992] f (, ω ω s t in ωout cos' θ B jiyj ( ωout Yi ( in i= 1 j= 1 Plugging this in yields [Lehtinen03] L ω = Y( ω out ( out out T B R T l B mas tansfeed incident adiance, exessed in SH, into outgoing adiance, exessed in SH (all in tangent fame Note that B may have moe ows than T and R the outgoing adiance is eesented with moe basis functions as the tansfeed incident adiance. This can be useful fo highly glossy and/o anisotoic BRDFs. The chain of linea tansfomations takes the SH coefficients of the distant envionment (l fist to tansfeed incident adiance SH coefficients at by T^, then the tansfeed incident lighting is otated into the tangent fame by R^, And the matix B tuns the tansfeed, otated incident lighting into SH coefficients that eesent outgoing light. 50

154 Phong-like BRDFs [Sloan02] If BRDF*cos is ciculaly symmetic w..t. the eflected viewing diection, can use sheical convolution w/ SH coefficients [Ramamoothi01] Looks like the evious case, but will be diagonal, and looku done in eflected diection Cannot model cosine deendence not valid fo all BRDFs B This is what was used by Sloan and othes in 2002, but the method has limited alicability because it cannot suot all BRDFs. 51

155 Geneal Case Change of Basis fo Outgoing Radiance Can exess outgoing adiance in yet anothe function sace {u k }, k=1,,n L m j= 1 out l ( ω R, j z j out, u~ k ( ω f in = (, ω in ω cos' θ dω, Ω out ( C B kj (C B is a comound BRDF + basis change matix Can do this in two stes [Lehtinen03], but this will intoduce band-limiting in u~ k Now fo the geneal case: We ll oject the outgoing adiance into a new function sace sanned by the functions u_k, again by taking the inne oduct with the outgoing adiance and the dual basis functions u~. Reaanging the integation and again moving tansfeed, otated incident adiance out, we get a new matix that mas the tansfeed, otated coefficients to the coefficients fo the basis functions u. This matix is a comound eesentation of the BRDF and the basis ojection. This can be done in two stes (fist a BRDF matix followed by a basis change, but this intoduces atificial bandlimiting, since the intemediate esult is tuncated by ojecting the outgoing adiance fist into the same basis as used fo eesenting tansfeed incident adiance. 52

156 Outgoing Radiance in New Basis := M L ω = u( ω out ( out out T ( C B R T l M has dimension N x n N is the dimension of the outgoing adiance basis n is the dimension of the emission basis E.g., 25 x 25 (lage! Let s walk though the whole chain of tansfomations: Fist we have the incident lighting, that is, its basis coefficients l. My multilication by T^, the incident lighting is tuned into tansfeed incident adiance, i.e., an aoximation to what oint sees when the scene is lit using the lighting envionment secified by l. Multilied by R^, the tansfeed incident adiance is otated into the tangent fame at oint. Finally, it is tuned into outgoing adiance coefficients by multilication with the comound BRDF + basis change matix CB. To get the adiance fom to the viewing diection, the basis functions u ae evaluated in the viewing diection and modulated using the outgoing adiance coefficients. Done! M is a comound adiance tansfe oeato; it mas the distant, incident illumination into the outgoing adiance fom oint, i.e., its aeaance. Fo instance, using 4th ode (25-tem SH fo incident adiance, and the same basis fo outgoing adiance, we have a 25x25 matix e oint. 53

157 Otimal Basis: Outgoing Radiance fom Seaable BRDF BRDFs can be aoximated using the SVD as sums of 2D oducts [Kautz99]: f b in ωout fk ( ωin uk ( out k = 1 (, ω ω Each facto deends eithe on incident o outgoing diection, not both To find a vey good basis fo outgoing adiance, we ll look at the BRDF. The incident/outgoing aameteization must be used hee, contay to what is geneally done in BRDF aoximation, whee fo instance half-angle aameteizations ae commonlace. Such a aameteization will not lead to seaation of the incident and outgoing ats, as we want. 54

158 Outgoing Radiance fom Seaable BRDF Plugging this in yields [Liu04, Wang04] L ( ω = : = ( C B = out m j= 1 l = u( ω R, j out T b k = 1 out u ( C k ( ω B out R Ω z T Comound C*B*R*T is small, since C*B only has size b x m (b = #tems in BRDF j ( ω l in f k ( ω cos' θ dω in kj in Plugging the seaable aoximation into the eflectance equation yields a comound BRDF & basis change matix that has the integals of the incoming BRDF factos with the cosine and the basis functions used fo eesenting tansfeed incident adiance. This esults in a fomula that is exactly like the geneal basis change we just deived, with the imotant diffeence that now the comount oduct only has the same numbe of ows as thee ae in the BRDF aoximation. Fo nice BRDFs the SVD aoximation is usually vey good aleady with a vey small numbe of tems; only a few may be needed. This means the matix M^ = C*B*R*T is much smalle than with othe basis functions. In effect we ve adated the outgoing adiance basis to the oeties of the BRDF. 55

159 Bi-Scale Radiance Tansfe Samle at a highe ate, e-use fo many L ω = u( ω out ( out out T ( C B R T l Samle at a lowe ate, inteolate Sloan et al. [2003] used Bidiectional Textue Functions (BTF whose light-deendence was ojected into SH vey comlex aeaance with less simulation The simulation and stoage of tansfe is exensive, and often the adiance incident onto the sufaces of the scene vaies much moe slowly than the outgoing adiance. This motivates slitting the geneal equation into two ats. The tansfe at we samle sasely and inteolate, while the oeato that mas tansfeed incident adiance into outgoing adiance we samle moe densely. Since it is defined only locally, we can e-use the same C*B fo many oints on the suface. 56

160 Bi-Scale Radiance Tansfe (examle [Sloan et al. 2003] Notice the fine textue on the suface, and how the weave attens shadow and mask each othe. Simulation of simila quality this without the bi-scale factoization would esult in an excessive amount of data. Much of it would be unnecessay too; the small, scale effects usually do not affect the maco-scale tansfe (lage shadows and inteeflections too much. 57

161 Summay: Chain of Tansfomations Distant light exessed in basis {y i } by Tansfe matix mas distant light to tansfeed incident light at, exessed in basis {z j } by l Final otation and eflection mas tansfeed incident adiance to outgoing adiance, exessed in basis {u k } by M T Comound mas to diectly l l out l out l 58

162 Outgoing Radiance by Diect Simulation All the evious methods ae based on a factoed (2-stage eesentation: 1. Incident light tansfeed incident adiance 2. Tansfeed incident adiance outgoing light Useful, but not necessay The two-stage method that we ve esented hee is not the only ossible way. 59

163 Outgoing Radiance by Diect Simulation Can also oject outgoing adiance diectly Diectly estimate the comound matices M : = C B R T Good: No bandlimiting due to factoization Bad: Costly Columns of all matices M ae comletely indeendent Used e.g., by Ng et al. [2003] fo image elighting (single outgoing DOF M has one ow only In this case each column of M diectly answes the question: What does oint look like when the scene is illuminated by basis light i? Because the comutation is not slit anywhee, all the matices must be comuted seaately by fo instance Monte Calo ath tacing. Note! All image elighting methods basically do this, but with a single DOF fo the outgoing adiance (only light towads the camea in a fixed osition Because all esults ae ath taced, we can eesent abitaily comlex tansot aths egadless of the samling on sufaces. But because of the this, cannot utilize a Neumann seies and intemediate esults fo othe oints, and thus must comute all bounces fo all oints seaetely. While all this is cetainly ossible, this method is comutationally heavie than the methods we ve esented befoe. 60

164 Connections PRT: Fix the lighting L L ω = u( ω out ( out out out ωout ( ωout ( = u l = suface light fields out [Mille98] [Nishino99] [Wood00] [Chen02] [Matusik02] T M = l out Fixed aeaance vecto l We can ask: What if we fix the lighting? Then fo each, the oduct M^ l can be ecomuted; this vecto diectly encodes the aeaance of oint. This is a suface light field. Note that we cannot otate the object w..t. the lighting envionment any moe, but we can look at it fom any viewoint. 61

165 Connections PRT: Fix the view Lout, ωout ( L ω = u( ω out ( out out u l = out = u out = image elighting o diffuse PRT M [Aiey90] [Dosey91] [Nimeoff94] [Teo97] [Sloan02] [Ng03] T Tansfe vecto, not a matix l And what if we fix the view? This means, fo instance, fixing the camea so that it cannot move; we ae endeing a fixed ictue of the object. Then the basis function vecto u is fixed, and its oduct with M^ can be ecomuted as befoe. Now the lighting may vay, but the view not this is image elighting. Also, if all sufaces ae diffuse, thei aeaance is catued by a single vecto, not matix; i.e., diffuse PRT (what was descibed ealie may be seen as a secial case of this. 62

166 Tile Poducts: Anothe Way fo Diect Lighting If inteeflections ae not needed, can exand all factos in eflectance equation in an othonomal basis Incident lighting, visibility, BRDF*cos Comute outgoing adiance using tiling coefficients and light/vis/brdf coefficients Clebsch-Godan coefficients fo SH Haa tiling coefficients given by [Ng04] Hee we assume that the lighting envionment, visibility and BRDF*cos ae all ojected into the same function sace, although that is not stictly necessay, as tile oducts can be defined also fo mixed bases. 63

167 Tile Poducts L out ( ω out = n m li Yi ( ωin v j Yj ( ωin Ω i= 1 j= Lenv ( ωin V (, ω in o bk ( ωout k = f (, ω ω in Yk ( ωin dωin 4443 out cos' θ in Lout ( ωout = li v j bk ( ωout C i, j, k C ijk = tiling coefficients C ijk ijk = Yi Yj Yk The outgoing adiance fom oint (without inteeflections is a ti-linea function of the lighting envionment, visibility and BRDF*cos, i.e., the esulting value is linea seaately in each of these functions. All the basis exansion sums and coefficients can be moved out of the eflectance integal, and so we ae left with a tilinea exession with the vectos l, v, and b and the tiling coefficients C_ijk. The tiling coefficients ae actually a 3D tenso, as can be seen fom the tile sum exession. Note that the tenso has high symmety, since the indices i,j,k can be feely emuted and the tiling coefficients emain the same this is obvious fom thei definition. Note that if we used diffeent basis sets fo the thee functions, this symmety would be lost. 64

168 Tile Poducts in SH Lout ( ωout = l v b ( ωout C i, j, k In SH, if lighting envionment has n coefficients and BRDF*cos has o coefficients, visibility needs to be ojected using n + o coefficients It is elatively easy to show that C ijk = 0 always when j > i+k in SH i j k ijk 65

169 Tile Poducts in SH Can also comute diect-lighting-only tansfe matices with tile oducts Poject visibility seaately fo each, then use C ijk fo comuting the elements T ji Again, need n + o coefficients fo visibility 66

170 Poeties of Diffeent Basis Sets 67

171 Choice of Light Basis Sheical Hamonics [Sloan02] [Kautz02] Haa wavelets [Ng02] [Ng03] [Liu04] [Wang04] Diectional o comact [Hao03] Steeable [Nimeoff94] [Ashikhmin02] Custom 68

172 Light Basis SH Low-fequency Good: Can eesent lage aea souces efficiently Bad: Can t eesent small souces efficiently Good: Closed unde otation (no wobbling Bad: Global suot (all functions non-zeo ove whole shee Bad: Quite noticeable inging 69

173 Light Basis Haa Good: Efficient decoelato, all-fequency Can eesent many env.mas with only few coefficients (non-linea aoximation Bad: Still need to e-comute tansfe fo all basis functions Bad: Not closed unde otation, and no analytic otation fomulae Bad: Non-linea aoximation causes temoal flickeing Non-linea aoximation means only using those basis functions whose ojection coefficients ae sufficiently fa fom zeo. The basis functions that cay significant enegy fo an envionment ma change fo each envionment; thus, if we want to ende using abitay lighting, the tansfe must be comuted fo each basis function, even if only a small subset of them would be equied fo endeing with any given fixed envionment. This is a lot of wok if we want to suot all-fequency lighting, but if the ice can be aid, esults ae vey convincing. 70

174 Light Basis diectional o comact Good: Comact suot, easy to undestand ~ diectional lights in diffeent diections Bad: Inefficient aoximation (almost all env.mas need many coefficients Bad: Not closed unde otation (ojection wobbles All the bad sides of Haa wavelets, but not the good one (do not oject to sase vectos. 71

175 Light Basis Steeable Steeable bases ae closed unde otation Good: otation does not alias Bad: Not good decoelatos (inefficient aoximation 72

176 Light Basis Custom Can always take advantage of io knowledge of the ossible set of lighting envionments tailo a basis Examle: Few bight, small lights and the est lowfequency include small lights as diectional basis functions, use SH fo the est 73

177 Choice of BRDF & Outgoing Basis L out (, ω out = u( ωout Sheical Hamonics Eithe in thei usual fom [Sloan02] o leastsquaes otimal hemisheical [Sloan03CPC] Comact / diectional [Lehtinen03] Secialized basis fom seaable BRDF aoximation [Liu04] [Wang04] Hemisheical [Gauton04] T M l These ae not all of the ossibilities. Fo instance, wavelet bases could be emloyed hee. The simlest case, the Haa wavelet, is not well suited fo diect visualization of outgoing adiance, howeve; its exessive owe coesonds to a iecewise constant diectional basis! 74

178 BRDF & Outgoing Basis SH Bad: Global suot always need all coefficients fo any outgoing diection Good: No aliasing, smooth esults Least-squaes otimal hemisheical SH [Sloan03CPC] Good: Can use less basis functions 75

179 BRDF & Outgoing Basis Diectional / comact Good: Comact suot Need only few coefficients fo any single outgoing diection Bad: Need many functions bloats M Bad: May see banding (functions not necessaily smooth enough 76

180 BRDF & Outgoing Radiance Secialized Basis is built otimally fom BRDF by SVD Good: Need few tems only M has only few ows Bad: Hade to suot satially-vaying BRDFs Need many SVD factoizations stoage oblem 77

181 The End Questions? 78

182 1

183 Pecomuted Radiance Tansfe: Theoy and Pactice 2

184 Light Reesentation Jan Kautz MIT 3

185 Light Reesentation Many ways of ceating incident light Ray-tace olygonal models o scenes Fom HDR light obes o envionment mas Samled actual incident lighting Diectly fom analytical solutions e.g. solution fo a disk light souce, angle Numeically fom ola functions τ 4

186 Oveview Light Reesentation Fixed Lighting Analytical Solutions (SH Diectional, disc, and sheical light souces Envionment Mas (SH/Haa Vaying Lighting Rotate Envionment Mas (SH/Haa Inteolate fom ecomuted incident lighting (SH Use gadient fo bette inteolation (SH 5

187 Fixed Lighting Analytical Solutions (SH Diectional light souce Coming fom a single diection Incident illumination is basically a Diac eak Integation against basis functions boils down to evaluating basis functions in that diection: l i y (Ψ i = i Ψ The easiest way to ceate a light souce is to integate the basis functions against a diectional light souce (basically a Diac. 6

188 Fixed Lighting Analytical Solutions (SH Disc souces (cente aound z: d τ ( θ, ϕ ( τ θ 1 > 0 = 0 othewise Integate symbolically in Male: l i = 2π τ ϕ = 0θ = 0 ( θ ϕ, sinθ dθ dϕ Use SH otation matix to move light y i A disc-shaed aea souce can be analytically ojected into SH. It is efeed to cente the disc aound the z-axis. In that case, only vey few coefficients ae non-zeo. Hee ae the non-zeo coefficients fo the fist 5 bands: Y(l=0,m=0 = *cos(theta Y(l=1,m=0 = *sin(theta^2 Y(l=2,m=0 = *cos(theta^ *cos(theta Y(l=3,m=0 = *cos(theta^ *cos(theta^ Y(l=4,m=0 = *cos(theta^ *cos(theta^ *cos(theta The light souce can be otated aound using the SH otation matices (see late slides. 7

189 Fixed Lighting Analytical Solutions (SH Disc souces: Due to SH constuction any function that is otationally symmetic aound z only has non-zeo coefficients fo modes m=0 Efficient eesentation and also otation It's inteesting to note, that the lighting vecto will only have vey few coefficients in that case (only fo modes m=0, i.e. fo the fist 5 bands, only 5 out of the 25 coefficients will be non-zeo. This also makes the otation (see late slides moe efficient. 8

190 Fixed Lighting Analytical Solutions (SH Sheical light souces Sheical light souces aea as disc souces d 2D examle Distance of sheical light to cente d and shee adius detemines size of ca: sinτ = d A secial case ae sheical light souces. They coesond to disc-like souces, but thei size changes with the distance to the samle oint, whee the light is samled. This samling can be actually efomed on-the-fly. 9

191 Fixed Lighting Envionment Mas Sheical Hamonics Integate basis functions against fixed HDR ma l i 1 Monte-Calo integation = L ( s y ( s d s env 2 Pecomute mas in same sace (e.g. cube ma that contain the basis functions big dot-oduct fo each coeff. i This is a standad scenaio. A HDR envionment ma is given and ojected into basis functions. If the envionment ma is given as a cube ma, it is necessay to know the solid angle of a texel in the cube ma. It is: 4/( (X^2 + Y^2 + Z^2^(3/2, whee [X Y Z] is the vecto to the texel (not nomalized. 10

192 Fixed Lighting Envionment Mas Haa Wavelets Reesent envionment mas as cube mas Do standad wavelet tansfom on each face Comess/quantize data (select most significant coefficients Refeence 4096 coeffs. 100 coeffs. 11

193 12 Vaying Lighting Rotating Envionment Mas Vaying Lighting Rotating Envionment Mas Rotation of HDR ma Sheical Hamonics Thee is a simle linea matix that will otate SH coefficients diectly: The matix is block diagonal No sill acoss bands Fo the fist few SH bands, it can be comuted exlicitly Geneal case: ecuence fomula l R l SH = ' = O M M M M M M M M M L L L L L L L L L X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X R SH Sheical Hamonics ae a steeable basis. It means that no infomation is lost unde otation. Thee ae vaious ways of ceating this otation matix. Thee ae ecuence fomulas, but in case only a few bands ae needed, the matix can be ceated exlicitly.

194 Vaying Lighting Rotating Envionment Mas Rotation of HDR ma Haa Wavelets No convenient otation matix Best solution: Rotate actual envionment ma Flatten otated envionment into cube ma Redo wavelet tansfom of each cube face In case of Haa Wavelets, thee is no convenient otation matix. All one can do is to otate in image sace and then eoject. 13

195 Vaying Lighting Moe local effects (SH Incident light will vay within a scene Boow "Iadiance Volume" technique [Gege98] Pecomute incident adiance on a gid Coutesy of Rafal Mantiuk Sofa, we have assumed that lighting is infinitely fa away, but that doesn't go well with inteio scenes. In that case, we can samle the incident adiance at gid oints thoughout the scene (assuming the scene doesn't change and stoe that in SH. 14

196 Vaying Lighting Moe local effects (SH Incident light will vay within a scene Boow "Iadiance Volume" technique [Gege98] Pecomute incident adiance on a gid I.e., fo each samle oint k ecomute a light vecto (assumes static lighting l l k Fo new oints q, linealy inteolate light fom neaby samle oints k [Mantiuk02] E.g., this technique is used in Max Payne 2 At un-time, one just queies the closest neighbos and inteolates the lighting coefficient vectos. 15

197 Vaying Lighting Iadiance Volumes Note how bunny is illuminated diffeently Coutesy of Rafal Mantiuk This gives a nice sense of an object being "in the scene". Even just the fist 2 SH bands (4 coefficients gives a geat sense. 16

198 Vaying Lighting Gadients Can we do bette than simle inteolation? How does incident light change when we move away fom samle oint? da(x x n(x s s Aoximate with gadient! [Annen04] d Instead of just linea inteolation, one can use a gadient in addition. 17

199 Vaying Lighting Gadients Lighting is ojected into SH as follows: l = L ( s y ( s d s i Ω S We want to comute the gadient when we move d away fom And euse at un-time: l i + d = l i + ( l d i i 18

200 Vaying Lighting Gadients Rewite integal as integal ove sufaces (because of imlicit deendency on Take gadient: (analytical integand n x l i = y σ ( σ i S LS ( x da 3 σ σ n = nomal σ = x The actual deivation is slightly involved and omitted hee. It is available in the ae [Annen04]. Code is available fom the authos. 19

201 Gadients Putting it all togethe Comute incident lighting Stoe distance and nomal as well Comute coefficients l i = Pe-ixel integation with SH basis Comute gadient l i = Pe-ixel integation of simle analytical fomula Requies distance & nomal Fo endeing: Extaolate to coefficients +d l i 20

202 Gadients Results Single Samle (extaolate to evey vetex no gadient with gadient gound tuth Multile Samle (exta/inteolate to evey vetex samle locations eight samles eight gadients 21

203 Conclusion Lighting can be moe than a static cube ma Analytical model Samle on the fly Pecomute, but inteolate O even extaolate using gadients 22

204 23

205 1

206 Pecomuted Radiance Tansfe: Theoy and Pactice 2

207 Pactical PRT I Comession and Samling Issues Pete-Pike Sloan Micosoft Cooation 3

208 Rendeing Basic equation being solved L (, ω = u( ω T M l out out out Outgoing Radiance Outgoing basis in view diection Comosite Tansfe Matix Distant lighting envionment We will fist evisit the basic equation that is used to geneate outgoing adiance at a suface oint ( in diection (omega out. You evaluate the outgoing basis functions in the diection omega out (this is a ow vecto of coefficients You then multily that times the tansfe matix at the oint (. And final dot the esult with the coefficients of the envionment lighting vecto l_env 4

209 Rendeing Basic equation being solved Lout (, ωout = u( ωout T e Outgoing adiance coefficients Anothe way to look at the above equation is that we evaluate the outgoing adiance function at the oint (e_ in the diection omega_out. 5

210 Rendeing Lighting needs to be tansfomed into global fame of igid object Evaluate/oject/otate Whee should tansfe matices be stoed? Vetices o textues How should the outgoing basis be evaluated? Tabulated in textues Analytically in the ixel shade The light on the ight hand side has to be eesented in the global fame of the object. This can be achieved in seveal ways, fo examle if the light basis functions have efficient otation fomula you can just aly them, in geneal you can oject the lighting into the basis, o you can evaluate simle analytic models as Jan talked about ealie. The tansfe matices can be stoed eithe at evey vetex o at evey texel. The outgoing basis is geneally evaluated e-ixel (based on inteolated o comuted omega out. This is geneally done in a ixel shade, and can eithe be tabulated (fo examle in cube mas o evaluated analytically. 6

211 Poblems With PRT Big matices at each suface oint 25-vectos fo diffuse, x3 fo sectal 25x25-matices fo glossy at ~50,000 vetices Slows glossy endeing (4hz Fozen View/Light can incease efomance Not as GPU fiendly Limits diffuse lighting ode Only vey soft shadows Thee ae a coule of limitations of PRT that makes it difficult to use it as discussed so fa fom a actical stand oint. The matices can be vey lage, even in the diffuse case 25d vectos ae equied, 3 time as many if you want to model colo bleeding. The glossy case equies 25^2 matices, at oughly 50k vetices like most of ou examles this equies ove 100mb. Glossy endeing is also slow you can feeze the view/light to gain some efomance, but this is a seious constaint. Also glossy endeing is not as GPU fiendly as the diffuse case with much of the wok being done on the CPU. Finally, the light ode fo the diffuse case is limited because of the numbe of coefficients that can be stoed e vetex on cuent hadwae. 7

212 Comession Goals Decode efficiently As much on the GPU as ossible Rende comessed eesentation diectly Incease endeing efomance Make non-diffuse case actical Reduce memoy consumtion Not just on disk Ou chief goal fo comession is efficient decoding. As much of this wok has to be done on the GPU as ossible and ideally we would ende diectly fom the comessed eesentation. This could lead to inceased endeing efomance making the glossy case actical. Also it s imotant to educe the un time memoy equiements, not just the size of the datasets on disk. 8

213 Comession Examle Suface is cuve, signal is nomal These high dimensional signals ae difficult to visualize, so I will use a much simle oblem to descibe the comession techniques. Ou suface hee is a cuve, and the signal we ae comessing is the nomal at each oint a 2D signal. 9

214 Comession Examle Signal Sace On the ight, we visualize the signal sace. Each nomal mas onto a oint on the unit cicle; this is the gauss ma of the cuve. 10

215 VQ Cluste nomals Vecto quantization is a simle but common comession technique. It gous the signal into a small numbe of clustes of simila samles, 3 in this examle. 11

216 VQ Relace samles with cluste mean M M% = M C VQ aoximates each samle as the mean of the cluste it s a membe of. Note that the clusteing is done comletely in signal sace; the satial elationshis of the samles is ignoed. 12

217 PCA Relace samles with mean + linea combination N 0 i % = + w i= 1 M M M M i Pincile comonent analysis is anothe common comession technique. Given a signal in some high dimensional sace, PCA comutes an otimal lowe dimensional linea aoximation to the signal, in a least squaes sense. This is done by comuting the mean of the signal, and a set of basis vectos. In this case we ae ojecting the signal fom 2D down to a single line. Each samle is then elaced by its ojection coefficients in this new basis. The chief limitation is evident fom this examle. While the signal is a 1D manifold, it is clealy not globally linea. But a linea aoximation *is* aoiate locally on the cicle the tangent is a good local aoximation to the signal. Ou signal is a high dimensional vecto maed to a suface in 2D. If the signal is smooth, locally thee should be a 2D tangent lane in 625D signal sace at each oint, so a local linea aoximation should be aoiate. 13

218 CPCA Comute a linea subsace in each cluste N 0 i i % = C + w C i= 1 M M M M Ou technique, called CPCA fo clusteed incial analysis, exloits this by comuting a linea subsace in each cluste. Each suface oint needs to stoe an index into a cluste, and a set of ojection weights fo the PCA basis in the coesonding cluste. 14

219 CPCA Clustes with low dimensional affine models How should clusteing be done? Static PCA VQ, followed by one-time e-cluste PCA otimizes fo iecewise-constant econstuction Iteative PCA PCA in the inne loo, slowe to comute otimizes fo iecewise-affine econstuction So CPCA aoximates samles in each cluste using a low dimensional affine model. Thee ae a coule of ways to do the clusteing. The most staightfowad aoach is just to cluste using VQ and then comute a PCA basis in each cluste, which we call static PCA. VQ otimizes fo iecewise constant econstuction, while we use iecewise-affine econstuction, so this is clealy subotimal. Iteative PCA clustes using distance to affine sub-sace instead of distance to mean. It comutes PCA in the inne loo and so is slowe than static PCA. But it otimizes fo the econstuction we use duing endeing. 15

220 Static vs. Iteative Hee we see two images clusteed with the diffeent techniques using a single PCA vecto. When shading, using a single PCA vecto means each cluste will have a linea gadient. Static PCA ceates clustes that ae elatively isotoic, while iteative PCA shaes the clustes based on obable shadow gadient diections. 16

221 Related Wok VQ+PCA [Kambhatla94] (static VQPCA [Khambhatla97] (iteative Mixtue PC [Dony95] (iteative Indeendently used with BTF s [Muelle03] Moe sohisticated models exist [Band03], [Roweis02] Maing to GPUs is challenging Vaiable stoage e vetex Patitioning is moe difficult (o equies moe asses Woth investigating again on cuent GPU s While this is new to CG, it s been done in machine leaning. Kambhatla used the tem VQ+PCA to define the static case, and VQPCA to define the iteative case. Dony settled on the tem Mixtue of PC to define the iteative case. Muelle and collegues used simila techniques in a ae at VMV in Thee ae moe sohisticated techniques fo non-linea dimensionality eduction. Matt Band s wok is used in a ae esented at siggah in 2003 fo examle. Maing these techniques to the evious geneation of GPU s is challenging. Since samles ae classified in a vaiable numbe of clustes, vaiable stoage is equied. Patitioning the mesh into indeendent sets of data is also moe difficult. As featues like deendent textues become moe efomant, it will be woth investigating these othe algoithms as well. 17

222 Equal Rendeing Cost VQ PCA CPCA Hee we see quality obtained when a bid model is endeed at equal fame ates using 3 comession techniques. With VQ, iecewise-constant econstuction atifacts ae aaent. This signal is clealy not eesented well with global PCA thee is substantial enegy loss. CPCA does a good job of aoximating the signal, with faily low eo. 18

223 Rendeing with CPCA e ( v = y( v T M l N 0 i i M MC + wmc i= 1 N T 0 i i ( = y( MC + MC l i= 1 e v v w N T i i e( v = y ( v C + w C Me l M l C i= 1 (( 0 (( i e C Rendeing with CPCA is staight fowad. Fist, aoximate each tansfe matix with the mean fom its cluste and a linea combination of the cluste s basis vectos. In this equation the C subscit eesents the cluste fo the coesonding oint. We can lug this diectly into the endeing equation. While this exession looks moe comlex then the oiginal equation, we can distibute the dot oduce with the light vecto l and aive at the following equation. The highlighted exession ae just exit adiance column vectos. So scala exit adiance is comuted by taking a linea combination of exit adiance vectos and dotting it with the exit basis function evaluated in the view diection. 19

224 Rendeing with CPCA e v v w N T i i ( = y( ec + ec i= 1 ( 0 ( Constant e cluste ecomute on the CPU Rendeing is a dot oduct Comute linea combination of vectos Only deends on # ows of M The imotant thing to note about this equation is that highlighted tems ae constant e cluste, so they can be comuted on the CPU at each fame. The dimensionality of these vectos only deends on the numbe of exit adiance basis functions, so the math on the GPU is indeendent of the light s dimension. Fo ou glossy tansfe matices, this eesents a comutational savings if N is lowe than

225 Non-Local Viewe N 0 ( ( T i i e( v = y( v MC l + w MC l i= 1 Assume: v constant acoss object (distant viewe ( ( ( 0 N T i T i y M l y( M l e v w v = + g C g C i= 1 Rendeing indeendent of view & light odes - linea combination of colos Using a non-local viewe model is a common technique in comute gahics. The assumtion is that the view does not change much acoss the object, so a common view diection can be used fo all suface locations. This allows us to distibute the constant view diection into the ecomuted exessions, making the shade indeendent of both view and light odes. All we need to ende is a simle linea combination of N colos. 21

226 Muelle et al Using CPCA at two scales [Muelle04] Don t blow the coase scale out Ceate dot oducts between clustes/pca matices at coase scale and PCA basis vectos at fine scale Thee was anothe nice ae by Muelle and collegues whee they used datasets that had infomation at two scales (coase tansfe matices and fine scale BTF s and encoded both using local PCA. The naïve aoach to doing this would be to blow out the SH signal in between the scales, and eojet this satially vaying function at the fine scale howeve this would be extemely slow. They obvseved that you can diectly oject fom coefficients on one CPCA sace to coefficients in anothe by simly ojecting all of the basis vectos fo elighting the BTF s though the basis matices used fo coase scale tansfe. These coefficients would then be udated when the lighting change and eveything would emain efomant. 22

227 Why not cluste satially? Any featue that vaies fine than the satial cluste size will be oblematic Coelations exist that ae not adjacent and should be exloited Instead of leaning the clustes, you could just ty and atition them satially (this has been done in image e-lighting wok. If the geomety vaies quickly elative to the cluste size, it will equie a highe dimensional linea sace to aoximate the data. By clusteing egions that ae not satially adjacent, fo examle the un-occluded egion with a vetical nomal highlighted hee, you can get away with fewe clustes and lowe dimensional linea saces. 23

228 Satial Samling Issues Relationshi between satial samling densities ove objects and light fequencies Umba Penumba When looking at PRT in geneal, it is woth thinking about the elationshi between light fequency and satial samling ates. PRT geneally deals with steeable lighting models, but it is woth thinking about the finest light that can be modeled when detemining a bounds on satial samling densities. Given a simle scene, with a lage aea light, a simle blocke and a simle eceive, the intensity on the eceive is boken into 3 categoes. Regions that ae comletely lit, egions that ae comletely in shadow (the umba and a tansition egion (the enumba. Lage aea lights have slow tansitions, this means they induce lowe satial samling ates. 24

229 Satial Samling Issues Light shinks -> enumba tightens Highe samling density to move light ove object/scene Umba Penumba Howeve fo smalle light souces, the enumbas tighten, which means the tansitions ae moe aid, so the samling ates have to be highe. 25

230 Angula Samling Issues Lage lights need to be samled a lot It is also woth thinking about angula, o diectional samling issues. Lage/smooth lights subtend lage solid angles, so to get an accuate estimate of the illumination aiving at a oint, many diectional samles have to be taken. This means that taditional inteactive techniques fo shadowing (fo examle, would equie many asses to geneate an accuate model of the illumination. 26

231 Angula Samling Issues Small lights clealy less Small light souces clealy equie fewe samles, which means they ae moe feasible to wok with at un time (assuming a small numbe of small light souces. 27

232 Samling Issues Lage (low fequency lights Coase satial samling Not a lot of stoage Lage solid angles Run time integation would be exensive Small (high fequency lights Fine satial samling High stoage Small solid angles Run time integation isn t that bad So fo diect lighting, lage low fequency lights induce coase satial samling ates which means less stoage is equied and ecomuting makes moe sense. They ae difficult to handle using un time integation due to the lage solid angles. Small lights induce high satial samling densities, and ae moe amenable to un time integation. 28

233 Samling Issues: Tansot Bounced light is etty much always low fequency An illuminated wall is an aea light Do you need to use high fequency basis to model high fequency inte-eflections??? Simila to duality in [Ramamoothi2001], inteeflections ae imlicitly lage aea lights When looking at moe comlex tansot effects (beyond diect lighting it is woth noting that they etty much always behave as low fequency light souces a wall illuminated by a tight light souce geneally acts as a lage emitte. So ecomuting indiect lighting fom high fequency lighting seems like a easonable idea in geneal. But that than begs the question, is it feasible to simulate the indiect lighting fom high fequency lighting using a low fequency ojection of that lighting? This could be simila to the duality discussed by avi in his siggah 2001 ae between light and mateial fequency. 29

234 Pactical PRT II Albedo/Nomal Maing GPU Simulato DX 30

235 Albedo Factoing out the albedo almost always makes sense Often vaies at a much fine scaled comaed to incident adiance Commonly done with lightmas in games The albedo should be factoed out of the tansfe vectos/matices. The incident lighting is almost always much smoothe than eflectivity, so it makes sense to not conflate the two. 31

236 Factoing Out Albedo Simlest way is to only multily by albedo when gatheing/shooting data to model inteeflections Always geneates a esult that is coect afte multilied by local albedo Woks fo any tye of light tansot albedo is only factoed out on the bounce befoe light eaches the eye The easiest way to do this is to only multily by albedo when gatheing (o befoe shooting duing tansot simulation. So all tansfe vectos (fo any tansot ath model eveythign excet the multily by the albedo at a oint on the suface. 32

237 Nomal Maing fo PRT Why use nomal maing? Two scales tangent fames/lighting at vetices, fine scale (albedo/nomal mas e texel Don t blow out tansfe vectos to fine scale Just looking at diffuse sufaces Bi-Scale Radiance Tansfe does this in the geneal case but is much heavie weight One of the most beneficial oeties of nomal maing is that it enables modeling things at multile scales. Coase infomation, eithe e vetex o in a lowe esolution textue (like incident lighting should be stoed at the aoiate samling ates. Nomal mas model fine scale etubations of the suface nomal but ideally you wouldn t blow out tansfe vectos/matices to that fine scale. The technique we ae about to descibe is only fo diffuse mateials, the bi-scale adiance tansfe ae coves the moe geneal case, but is a much heavie technique (both in tems of comutation and data 33

238 Iadiance Envionment Mas [Ramamoothi2001], alication of the SH convolution fomula 4π h f h f h f l 2l ( * = =α m m m l l l l l h is ciculaly symmetic function h(z f is any function ove the shee Evaluate convolved SH in a diection comutes integal of h oiented in that diection against f The nomal maing technique is based on the iadiance envionment ma ae fom siggah This ae is essentially an alication of the sheical hamonic convolution fomula. Which is simle the convolution of a ciculaly symmetic kenel h (can be thought of as a function of Z only when oiented with the Z axis with an abitay function f eesented in sheical hamonics can be eesented by scaling each of the coefficients in a given band of f (l by a constant and the coefficient of m=0 in h at that band. Cicula symmetic functions of z only have one non-zeo ojection coefficient (coesonding to m=0 Evaluating a convolved function in a given diection gives you the integal of the oiginal function against the kenel oiented in that diection. The eason the kenel has to be ciculaly symmetic is because functions with cicula symmety ae aameteized by 2 DOF, so the esult of the convolution can be eesented on the unit shee a geneal kenel would geneate a function on SO3. In avi s ae he obseves that the clamed cosine kenel can be eesented in closed fom and that most of the enegy is in the coefficients though the quadatics. This is the same motivation used in image ocessing lage kenels ae moe efficiently convolved in fequency sace. 34

239 Nomal Maing Account fo GI of coase scale (geomety but not fine scale (nomal ma Could use DC of visibility to oke hole in lighting envionment, with tile oducts this amounts to scaling light coeffs by DC Mathematical justification fo ambient occlusion L xfe just needs to be quadatic [Ramamoothi2001] The technique esented will also only account fo global illumination at the scale of the geomety, how bums cast shadows/eflect light on themselves will not be modeled. If you scaled the lighting envionment by the DC tem of the SH ojection of visibility, that s mathematically equivalent to taking the oduct of the lighting and the visibility ojected into SH which is a mathematical justification fo ambient occlusion. A esult of the iadiance envionment ma ae is that tansfeed adiance only need to be quadatic if the shadowing of the bums is not being taken into account. 35

240 Nomal Maing fo PRT Intuitive descition Iadiance envionment mas whee the envionment ma vaies ove the suface Evaluate with nomal fom nomal ma [Ramamoothi2001] The easiest way to think about the technique that will be descibed is you have a smoothly vaying iadiance envionment ma, that you want to evaluate using a nomal looked u fom the nomal ma. 36

241 Nomal Maing Demo [ load simle scene ] [ go into nomal ma mode ] [ tun off light obe, tun on diectional light ] Hee is a faily coasely tesselated mesh with a nomal ma, the ecomutation knows nothing about the actual nomal ma being used. We can change the scale of the nomal ma, o switch in a diffeent nomal ma, and use the same simulation esults. This achieves the desied effect a coas PRT simulation is combined with a high fequency nomal ma. Hee I will toggle between the ideal case and an aoximation that I will descibe in a bit. This aoximation is quite good, but thee is clealy some eo nea the silhoutte which might be a esult of comession. 37

242 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l Valve showed a simila technique at GDC04, but just fo static lighting. The idea esented hee is fo a aameteized model of lighting. 38

243 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l SH Lighting Vecto The fist tem is the distant envionment ma lighting. 39

244 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l Distant lighting to quadatic SH local lighting The next tem is a tansfe matix that mas distant lighting to quadatic tansfeed incident lighting 40

245 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l Rotation fom global to local fame (tangent sace Then thee is a otation fom the global to the local fame 41

246 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l Convolution with nomalized cosine kenel And finaly a diagonal matix that convolves the signal with the coefficients fo the nomalized cosine kenel 42

247 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l SH basis functions evaluated in nomal diection You then have the SH basis functions (quadatic evaluated in the nomal diection. 43

248 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l Outgoing adiance as a function of nomal Finally this geneates outgoing adiance fo the given nomal 44

249 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation ( y( n T e n = CRM l Convolved local lighting envionment (Tansfeed Incident Radiance 45

250 Nomal Maing fo PRT Related to technique esented last yea by Valve TM fo Half-Life 2 But fo aameteized model of light Mathematical fomulation e( n = y( n T l Convolved local lighting envionment (Tansfeed Incident Radiance 46

251 Nomal Maing fo PRT Faily heavy weight 9xO 2 (16-36 matix vetex/lr textue Can be comessed just like diffuse tansfe Geneates 3*9 coefficients (iadiance ema What ae cheae aoximations? Evaluating the technique as outlined in the evious slides is still faily heavy weight. You equie 9xode^2 matices e vetex o in a low es textue (which can be comessed. Moe imotantly that geneates 27 coefficients that ae the iadiance envionment ma that have to then be evaluated. So we ae going to go ove some cheae aoximations. 47

252 Nomal Maing fo PRT Hemisheical function that fo a given nomal vecto comutes outgoing adiance Poject convolved adiance into an analytic basis besides SH HL2 basis (3 ows fo the matix Shifted Associated Legende Polynomials [Gauton2004] Looks decent in male will have to exeiment The basic oblem is that you have a hemisheical function (outgoing adiance that is a function of the nomal (not view diection as we have done befoe. One simle aoach would be to oject this hemisheical function into some othe basis. HalfLife2 uses 3 quadatic olynomials on otions of the hemishee. You could also use the basis functions fom last yeas EGSR ae, these look like than can do a easonable job aoximating convolved SH functions in male, but moe exeimentation needs to be done. 48

253 Nomal Maing fo PRT Use same ideas as BRDF factoization e( n = b( n A l b( n A Bi-linea basis functions ove hemishee (4 non-zeo Matix, ows nomal diections columns quadatic SH light A ij equals evaluating convolved light basis function j in nomal diection i Anothe aoach, that we will discuss hee is to use the same ideas that ae used fo BRDF factoization, which Jaakko talked about ealie. That is build a matix A samling the convolved lighting envionment ove a hemishee of nomals. The ows eesent the nomal diections, the columns the lighting envionment. 49

254 Nomal Maing fo PRT A M A M A L A L A i0 i8 L A M M N0 N8 This is just tying to give moe intuition to the matix A. You have some samling of the hemishee, each ow coesonds to a samle. The columns coesond to illumination fom the 9 quadatic SH basis functions. A coefficient in the matix eesents the integal of a cosine kenel in the given diection against the lighting envionment. Not that the lighting envionment should be clamed to the hemishee befoe this integal haens. Bi-linea basis functions ae used on the unit disk, and then maed to the hemishee to geneate a value at any oint on the hemishee (so thee ae at most 4 non-zeo enties fo a given nomal. 50

255 Nomal Maing fo PRT Comute SVD of A A = USV t U S t V Nx9 matix (each column is a nomal basis textue 9x9 diagonal matix (singula values 9x9 matix Then you comute the singula value decomosition of this matix and you get 3 tems. A matix U, whee each column eesents a nomal basis textue. A diagonal matix S (the singula values And a matix Vt,which is 9x9 (fo quadatic SH. 51

256 Nomal Maing fo PRT Old equation ( y( n T e n = CRM l New equation T ( ( n e n = U SV CRM l This geneates a new equation that simly elaces the evaluation of the quadatic SH in the nomal diection. 52

257 Nomal Maing fo PRT Use fist M singula values T ( ( n e n = U SV CRM l MxO 2 matix M channel nomal diection textue Then instead of using the full matix, just use the fist M singula values. 53

258 Nomal Maing fo PRT Use fist M singula values M = 4, SE 93.9/96.4 M = 6, SE 98.7/99.3 Then you only need a MxO 2 matix Shade is simle dot inteolated scalas with nomal deendent textue Hee you see the accuacy with unifom samling ove the hemishee, and cosine weighted samling (nomals alligned with Z in tangent sace ae efeed. 4 tems is faily accuate. 54

259 Nomal Maing Shades Vetex Shade conventional PRT, but outut 3 4 channel values (instead of 1 RGB colo Pixel shade This is not much moe comlex than a conventional PRT shade. 55

260 Pixel Shade StandadSVDPS( VS_OUT In, out float3 gb : COLOR { float2 Nomal = tex2d(nomalsamle, In.TexCood; float2 vtex = Nomal*0.5 + float2(0.5,0.5; float4 vu = tex2d(usamle,vtex; gb. = dot(in.cr,vu; gb.g = dot(in.cg,vu; gb.b = dot(in.cb,vu; } gb *= tex2d(albedosamle, In.TexCood; 56

261 Nomal Maing fo PRT Even cheae aoximation Comute shadowed value e vetex (conventional PRT, luminance only Comute unshadowed value (using iadiance emas on luminance only Take the atio shadowed/unshadowed Inteolate ove mesh and modulate with nomal ma evaluation A much moe aoximate solution would be to just use a coase PRT esult to modulate a e-ixel evaluation. 57

262 Nomal Maing fo PRT Pojecting into an analytic basis o SVD shadow blue light and can inte-eflect ed light off wall 58

263 Nomal Maing fo PRT Most aoximate technique effectively Just dims the light 59

264 Nomal Maing fo PRT If nomal oints towads blue light Outgoing Radiance is blue, not bounced ed 60

265 Nomal Maing fo PRT Relatively chea, allows fo sase PRT that esonds to high fequency changes in suface nomal Could imove by moving to a YUV colo sace Moe ecision in luminance, less in choma Like video/image codecs 61

266 GPU Simulato Simulation on the CPU is faily exensive It would be nice to get a quick esult, even if it is of lowe quality Pesented technique addesses diect lighting only fo diffuse objects See GPUGems2 fo indiect lighting Simila to ambient occlusion technique in GPUGems Tile oducts can be used to geneate tansfe matices 62

267 How to Comute? Use MC integation to estimate integal Foeach P on suface T = 0 Foeach D on shee fcostem = dot(d,n if (fcostem <= 0 continue if (GeomIntesect(P,D continue T += EvalBF(D*CosTem T *= NomFac 63

268 Comute on GPU Doesn t ma well to GPU Ray tacing inefficient Sum eduction inefficient Use of hemicubes fo NI would also not ieline as well Revese ode of loos, accumulate contibutions fo a given diection to all samles in aallel 64

269 GPU Setu Comute 2 textues G geomety textue eithe ack vetices into textue (any ode but should be coheent o use aameteization N nomal textue 1 to 1 coesondence with G Ceate 3 MRT textue stacks of same esolution (high ecision this will be T 65

270 GPU Comutation Initialize G,N and T (T is zeo Foeach D on shee Rende deth into textue DT Set D and EvalBF(D as constants Set tansfom as constants (3x4 mat Foeach P in G [asteize tiangle] fcostem = dot(d,n if (fcostem <= 0 continue P = xfom P if (P.z > DT(P.xy continue T += EvalBF(D*CosTem*NomFac 66

271 GPU Comutation Looku in G Looku in N Poject into shadow Accumulate scaled by cos 67

272 GPU Issues No blending with high ecision fomats Use ing-ong buffe tick always ende into buffe A, have shade add buffe A + B[FameNum%2] into B[(FameNum+1%2] No contol flow Use conditional instuctions to multily esult Otimizations Do multile diections in a ass Multily textue coodinates by zeo if cos tem < 0 68

273 PRT and DiectX CPU simulato Pe vetex o e texel Diect fo SH + any numbe of indiect fo anything Subsuface scatteing Tansfe matices comuted at any oint in sace o on mesh (diect o bounced Albedo factoed out is default Pe vetex o e texel albedo 69

274 Adative Simulato Refines mesh based on oeatos change ove the suface Can esamle into textues (moe efficient than a e-texel simulation in geneal RobustMeshRefinement Kind of like discontinuity meshing fo PRT Initial efinement so featues aen t skied 70

275 Adative Simulato 71

276 Comession in DX VQ/PCA Local PCA Low Quality VQ+PCA High Quality VQPCA (much faste than in ae Can comess any data (not just PRT 72

277 Miscellaneous Stuff UVAtlas Imlementation of iso-chat algoithm fo geneating aameteizations GPU Simulato fo diect lighting Pe vetex o e texel GutteHele Poagates data fom inteio of aameteization to gutte 73

278 Samles PRTDemo Pe-vetex simulato, SS, adative, etc. Iadiance envionment ma and LDPRT also PRTCmdLine Command line a that uns the simulato LDPRTDemo bat, just un time fo LDPRT 74

279 75

280 1

281 Pecomuted Radiance Tansfe: Theoy and Pactice 2

282 Final Thoughts Pete-Pike Sloan 3

283 Distant Lighting Basis SH only models low fequencies Can t cast had shadows Lowe satial samling densities Wavelets/comact basis Can handle high/all fequency lights Lage tansfe vectos/matices Highe satial samling densities 4

284 BRDF/BTF s Secialized (factoed foms Fewest numbe of ows Only eally woks with homogenous BRDF s Pojection into othe basis Moe ows Can vay the BRDF BTF s Comess at two scales 5

285 Scenaios fo Games Sky lighting (not sun SH fo both diect/indiect Sun Diect with shadow zbuffe/volume Indiect with steeable/all-fequency/sh Glossy Might still just use taditional techniques 6

286 Comession No eason not to Minimally do PCA CPCA does much bette, aticulaly on tansfe matices Reduces data and comutation significantly 7

287 No Single Technique Slit on light fequencies Slit on tansot aths Taditional techniques fo diect lighting fom high fequency lights SH/PRT fo all tansot aths fo low fequency lights PRT fo indiect lighting only fom high fequency lights Maybe just ojecting into SH is good enough 8

288 What s Not Handled Inte-Object effects [Kontkanen05],[Zhou05] fo diect lighting Defomable Objects [Kautz04],[James03] ae good stats though 9

289 Acknowledgements Coauthos Timo Aila, Thomas Annen, Fedo Duand, Jesse Hall, John Hat, Xinguo Liu, Ben Luna, Hans-Pete Seidel, John Snyde, Hay Shum Figues Ren Ng, Robin Geen, Rafal Mantiuk At/Light Pobes/Samles Paul Debevec, Shanon Done, Jason Sandlin, John Steed 10

290 11

291 1