Spherical Harmonics. Volker Schönefeld. 1st July Introduction 2. 2 Overview 2

Transcription

1 Spherica Harmonics Voker Schönefed 1st Juy 25 Contents 1 Introduction 2 2 Overview 2 3 Orthogona functions Associated Legendre Poynomias Sine and Cosine Spherica functions Spherica poar coordinates Definition Spherica Harmonics Definition Properties Harmonic expansion Convoution Rotationa invariance Fast Integration Transfer Matrix Rotation Spherica Harmonics in Computer Graphics Irradiance Environment Maps Spherica Harmonic Lighting Introduction The new agorithm Transfer functions Light functions Concusion 23

2 1 Introduction Just as the Fourier basis represents an important too for evauation of convoutions in a one- or two dimensiona space, the spherica harmonic basis is a simiar too but defined on the surface of a sphere. Spherica harmonics have aready been used in the fied of computer graphics, especiay to mode BRDF and incident radiance as we as BRDF inference [1, 2, 3, 8, 9]. But spherica harmonics have just recenty become feasibe to be used in rea time computer graphics, especiay in enhancing the dynamic ighting of scenes in rea time as wi be shown ater in this work. The motivation of this pro-seminar is to demystify spherica harmonics in a simiar way as Robin Green did [12] but with more mathematica background on the functions themseves and ess focus on the actua appications, which in Green s case was a technique caed spherica harmonic ighting. Athough the spherica harmonics are not the easiest mathematica functions this is an attempt at expaining and iustrating them as pausibe as possibe - without eaving out the critica mathematica reationships. Spherica harmonics are sometimes caed the swiss army knife of mathematica physics and this metaphor is extensibe to computer graphics to a certain degree, as the attentive reader wi hopefuy understand at the end of this work. 2 Overview First the required mathematica fundamentas wi be reviewed. This incudes a short evauation of orthogona functions, where the associated Legendre poynomias wi be introduced and the sine and cosine functions wi be examined regarding some intriguing properties. Thereafter spherica functions and spherica poar coordinates wi be reviewed shorty. Once the fundamentas are in pace they are foowed by a definition of the spherica harmonic basis whie evauating its most important properties. Finay the focus wi move on exampes for the usage of spherica harmonics to sove the common ighting function in a rather new and partiay precomputed way. Aso, the use of spherica harmonics to quicky reight objects using pre-fitered irradiance environment maps wi be discussed. 3 Orthogona functions Orthogona functions their domain [a, b]: and [6, 7] are casses of functions {p n (x)} that obey an orthogonaity reation over b a b a w(x)p n (x)p m (x)dx c n δ nm c m δ nm (1) w(x)p n (x)p m (x)dx c n δ nm c m δ nm (2) for rea- and compex-vaued functions respectivey. δ k denotes the Kronecker deta [6], defined by { 1 k δ k k. w(x) is an arbitrary weighting function independent of n as we as m and φ R(φ) ii(φ) denotes the compex conjugate of the compex number φ. With this property p n (x) is then caed a basis function. If c n 1 for a n the cass is even orthonorma, which is a stronger reation and provides some additiona properties. Amongst other things orthogona and orthonorma basis functions aow the expression of any piecewise continuous function over [a, b] as a inear combination of an infinite series of ineary independent basis functions. In other words: The basis functions p n are sma pieces of information. Scaing and combining them produces either exacty the origina function f (if an infinite series of basis functions is used or the function is band-imited) or a band-imited approximation f of the source function (if ony a finite number of basis 2

3 PSfrag repacements R R R R k.3 k 1.2 k k 1 k 2 PSfrag repacements k 2.4 k 3 k 3.7 f (x) (a) Projection of the red poynomia into the first few Legendre coefficients k n. (b) Reconstruction of the origina function by summing the scaed basis functions. Figure 1: An exampe of an expansion and reconstruction of f (x) 1.75x 3 +.6x x.6 using the first four Legendre poynomias 1, x, 1 2 (3x2 1) and 1 2 (5x3 3x). functions is used whie the source function aso consists of signas of higher frequency than the threshod). Band-imiting means that frequencies higher than a certain threshod are removed and this is simiar to a ow-pass fiter appied on the function before the harmonic expansion. For the approximation the maximum error is in genera proportiona to the number of basis functions used. max f (x) f (x) (3) x Projection and Reconstruction: So everything needed to approximate a given function f arbitrariy accurate is to compute the coefficients k n describing how much each basis function p n is ike f. This is done by integrating the product f (x)p n (x)dx k n (4) over the fu domain of f. The aforementioned process is caed projection or expansion. An exampe is shown in figure 1(a). Its inverse process is defined as the inear combination of a basis functions scaed by their associated coefficients f (x) n k n p n or f (x) This is caed reconstruction and it is demonstrated in figure 1(b). N n k n p n (5) Integration of orthonorma series: In addition to the orthogona set of properties an important property of the orthonorma functions is the foowing: Consider the integra of a product of any two arbitrary piecewise continuous functions a(x) and b(x). a(x)b(x)dx? (6) 3

4 Expanding these functions into band-imited functions ã and b with coefficients a n and b n respectivey transforms the integra into a simpe dot product of the projection coefficients. ã(x) b(x)dx N i a n b n. (7) This effectivey reduces a symboic integration of the product of two functions into a series of mutipy-adds which are a ot easier to compute. Numerica Integration: When evauating integras for harmonic expansion (e.g. an integra of a function that describes the incident ight intensity for an given point in a compex scene), instead of performing symboic integration it is often necessary to use other soutions - especiay in the fied of numerica computer simuation. One such soution is caed the Monte Caro Integration. It basicay consists of taking a number of sampes of the function (probabiistic gathering) and using them to approximate the correct integra resut. The more sampes are taken the ess error is introduced in this integration. One keyword that is used ater on when computing spherica harmonic coefficients is known as the Monte Caro Estimator and it is defined as f (x) 1 N N f (x i )w(x i ) (8) i1 where f denotes the function we want to integrate, N the number of sampes, f (x i ) represents one sampe and w(x i ) represents a weighting function for each sampe, which is its reciproca probabiity. More information on the topic can be found here [6, 1, 11, 13]. Exampes: A prominent exampe that makes use of the orthogonaity reationship is the Fourier series [6], which provides a convenient method of expanding periodic functions into an infinite sum of sines and cosines. There wi be more detais on the sine and cosine orthogonaity ater in this section since they are one of the fundamentas spherica harmonics are buit upon. Additionay a cass of orthogona poynomias wi be anayzed, named the associated Legendre poynomias. They are often interconnected with and wi be used to define as we as expore the spherica harmonics. 3.1 Associated Legendre Poynomias The first cass of orthogona functions is named after Adrien-Marie Legendre ( ), a french mathematician. In genera represented by the symbo P m the associated Legendre poynomias are rea-vaued and defined over the range [ 1, 1]. An expicit definition is P m (1 ( 1)m 2 x! 2 ) m d+m dx +m (x2 1) (9) athough it is rarey used for computationa purposes, because the evauation is tricky and numericay unstabe. The function takes two integer arguments and m which are constrained by N and m [,]. is used as the band index to divide the cass into bands of functions resuting in a tota of ( + 1) poynomias for a -th band series. With respect to and w(x) 1 the associated Legendre poynomias obey the orthogonaity reationship 1 P m (x)p m (x)dx δ 2( + m)! (2 + 1)( m)!. (1) 1 However, for different m on the same band, the poynomias are orthogona with respect to a different constant and another weighting function. If neither m m nor the poynomias are not orthogona at a. When used in spherica harmonics, this orthogonaity needs to be estabished by another orthogona poynomia. For a better understanding the first few functions are shown in figure 3.1 and used for the harmonic expansion in figures 1(a), 1(b) with..3 and m. If m, as it can be found in the aforementioned 4

5 m P m (x) 1 1 x P3 2(x) P2 2(x) x (3x 2 1) 3x 1 x 2 P 3 (x).5 P 1 3 (x) 1 P 1 (x) P (x) (1 x 2 ) 1 2 (5x 3 3x) 3 2 (1 5x2 ) 1 x 2 15x(1 x 2 ) P2 (x) 1 P1 1(x) P2 1(x) P 3 3 (x) (1 x 2 ) 3 Figure 2: First four bands,..., 3 of the associated Legendre poynomias exampe, they degenerate to the unassociated Legendre poynomias, which wi however not be discussed in this work. The associated Legendre poynomias can aso be defined using a set of recurrence reations P m m (x) ( 1) m (2m 1)!!(1 x 2 ) m/2 (11) Pm+1 m (x) x(2m + 1)Pm m (x) (12) ( m)p m (x) x(2 1)P 1 m (x) ( + m 1)Pm 2 (x) (13) which wi come in handy when impementing the function in a computer appication, especiay since they are easier to compute and ess susceptibe to numerica errors compared to other methods [6, 14]. To evauate a given function vaue P m (x) primariy equation (11) is used to generate the highest P m m possibe. Thereafter for m the correct vaue has been computed. Otherwise a that is eft to do is to raise the band, so (12) is used once to get to the next band, and then (13) can be iterated (because it depends on 1 and 2 resuts the second rue needs to be appied once) unti the correct answer is found. 3.2 Sine and Cosine The other important set of orthogona functions is the sine and cosine set. Despite being the key functions when taking of spherica or circuar systems, they aso obey the orthogonaity reation over [ π, π]. Foowing key integra identities can be defined, and wi be of good use when defining spherica harmonics: π π π π π π π π sin(mx)dx sin(mx)sin(nx)dx πδ mn (14) cos(mx)cos(nx)dx πδ mn sin(mx) cos(nx)dx π π cos(mx)dx whereas m, n are caed phases. As noted above, these integra identities are one of the key properties the Fourier Series is buit upon. 5

6 Another important property of the sine and cosine functions is reated to the compex numbers. It is caed the Euer identity and defined as For more information, see [6, 5]. e iφ cos(φ) + isin(φ). (15) 4 Spherica functions Now the key fundamentas for the main topic are defined and expained. But before this work jumps into the coorfu word of spherica harmonics, there are some sma conventions that need to be reviewed in order to ensure a proper understanding. 4.1 Spherica poar coordinates When taking about spherica functions it is convenient to use spherica poar coordinates instead of the Cartesian ones. The spherica coordinate system is PSfrag repacements defined by two anges θ and φ, whereas φ < 2π describes the azimutha ange in the xy-pane originating at the x-axis and θ < π denotes the poar ange from the z-axis. Additionay a radius r is used to represent the distance from the origin of the coordinate system, but it can be omitted for normaized coordinates, which a ie on a unit sphere and therefore have a uniform distance from the origin of r 1. See figure 3. Care has to be taken when comparing spherica functions from different sources, since there is no generay accepted convention about the semantics of φ and θ, thus they may be swapped or defined differenty. x φ Figure 3: Poar coordinate system The reations between a point in the Cartesian and spherica poar coordinate systems for this work are defined as and anaogousy the inverse reations 4.2 Definition r x 2 + y 2 + z 2 (16) ( y φ cot (17) x) ( ) x θ sin y 2 ( cos 1 z ) (18) r r x r cosφsinθ (19) y r sinφsinθ (2) z r cosθ. (21) A spherica function is a mapping of spherica coordinates (θ,φ) to a scaar vaue. In this work spherica functions are assumed to be rea-vaued, athough compex-vaued functions can be harmonicay expanded by the compex spherica harmonics series anaogousy. Spherica functions can easiy be visuaized by either dispaying a textured sphere, where the intensity of a point on the surface represents the vaue of the function at that point (Figure 4(b)), or by dispacing the points on the surface of the sphere aong their corresponding norma vector based on the vaue of the function (Figure 4(a)). An integration can be thought of as summing infinitesima patches of area. The parameterization of a sphere using poar coordinates causes the patches on the equator to be bigger and thus shoud have more infuence on the soution of the integra compared to the patches at the poes. To encode this effect an integration of a spherica function is pre-mutipied by sin(θ) which is 1 at the equator, vanishes at the poes and is directy proportiona to the area of the patches (which ony depends on θ). To enhance readabiity, z θ r y 6

7 (a) Dispaced unit sphere with random coorization. (b) Textured unit sphere, using the spectrum shown on the right side to visuaize high and ow function vaues. Figure 4: An exempary spherica function f (θ,φ) 1 4 (cos(6φ)3 +sin(θ) 4 +1) potted with both presented techniques. such an integration of a spherica function f (θ,φ) over the surface of a unit sphere S wi be expressed either in the expicit spherica coordinate form or as an integra over the impicit surface S. 2π π f (θ,φ)sin(θ)dθdφ f (ω)dω (22) S Anaogousy, an integra over the surface of an hemisphere Ω( n) in direction n is denoted as 2π π f (θ,φ)sin(θ)h( n,θ,φ)dθdφ f (ω)dω (23) Ω( n) whereas h( n,θ,φ) { 1 n v θφ otherwise with v θφ being the vector from the center of the unit-sphere to the point described by θ and φ. Additionay ater on x ω wi be used to describe any point in direction ω. 5 Spherica Harmonics Now we finay arrived at the actua topic of this work and the spherica harmonics are very cose. With a the required fundamentas defined now a forma write up and expanation of the definition of the spherica harmonics wi foow. After that detais of the important properties that resut from this definition wi be discussed. Finay rotation of a spherica harmonic function are described briefy. 5.1 Definition In section 3 it is shown that the associated Legendre poynomias can be used to express any piecewise continuous function over the interva [ 1, 1] either as an infinite series of poynomias, a finite series of poynomias for a band-imited approximation or a finite series of poynomias in case the function itsef does not have frequencies higher than a certain threshod. When ooking at the definition of spherica 7

8 coordinates in section 4 it may become obvious that we can express any circuary symmetric function (ike 2,m in figure 6), which has no dependence on φ, in terms of the associated Legendre poynomia by mapping θ into the [ 1,1] domain using cosθ. But we need some mechanism to provide orthogonaity in case of non-circuar symmetric functions. This can be reaized by combining the associated Legendre poynomias for the θ dependence with the sine and cosine functions for the φ dependent part. Now with the basic idea expained a forma description of the spherica harmonics wi foow and finay they wi be investigated. A compete forma definition N and m is given by of the compex-vaued spherica harmonic series with two arguments Y m (θ,φ) N m P m (cosθ)e imφ (24) where N m is a normaization coefficient. As for the Legendre poynomias denotes the band index. Using Euer s formua the equation can be rewritten as Y m (θ,φ) N m P m (cosθ)(cos(mφ) + isin(mφ)) (25) and it becomes evident that, as described above, the spherica harmonics are based on the associated Legendre poynomias for the θ and sine and cosine functions for the φ dependence. See figure 5. X PSfrag repacements y 2 4 (θ,φ) sin(2φ) P 2 4 (cosθ) 25 2 (7cos2 θ 1)sin 2 θ Figure 5: A demonstration of the functiona dependencies of both coordinate axes. The eft image shows the φ dependence with the corresponding sine-wave (phase m 2) whie the center image dispays the θ dependence aong with its associated Legendre poynomia P4 2. Combining both yieds the right picture of the spherica harmonic basis function y 2 4. The normaization factor can then be derived from S Y m (ω)y m (ω)sin(θ)dω δ mm δ (26) which concurrenty proofs the orthogonaity of the spherica harmonics. The sin(θ) weights the function vaues by the distance from the equator. This is due to the aforementioned fact that integrating spherica coordinates can be seen as integrating sma patches on the sphere. 8

9 1 This figure iustrates the first 5 spherica harmonic bands...4. Green parts indicate positive extends whie red shows negative ones. The itte sphere on the top right shows the distribution on the unit sphere. 9 Figure 6: SH basis functions 2 3 PSfrag repacements 4 m 4 m 3 m 2 m 1 m m1 m2 m3 m4

10 Soving the equation (26) by expanding Y m yieds: 2π π Y m (θ,φ)y m 2π 1 Y m (θ,φ)y m 1 2π 1 N m N m 1 P m N m N m 1 P m 1 (θ,φ)sin(θ)dθdφ (θ,φ)d(cosθ)dφ (cosθ)p m (cosθ)e imφ e im φ d(cosθ)dφ (cosθ)p m (cosθ)d(cosθ) }{{} θ-dependent part 2π e imφ e im φ dφ }{{} φ-dependent part (27) Soving the φ dependent integra, which is based on the sine and cosine integra identities, resuts in 2π 2π 2π e imφ e im φ dφ (cos(mφ) + isin(mφ))(cos(m φ) isin(m φ))dφ cos(mφ)cos(m φ) + sin(mφ)sin(m φ) + icos(m φ)sin(mφ) icos(mφ)sin(m φ)dφ cos(mφ)cos(m φ)dφ+ sin(mφ)sin(m φ)dφ+i( cos(m φ)sin(mφ)dφ cos(mφ)sin(m φ)dφ) } {{ } } {{ } } {{ } } {{ } πδ mm πδ mm 2πδ mm by appying the Euer formua (15) and the aforementioned integra identities 1 (14). Simiary for the θ dependent integra, which on the other hand is based on the associated Legendre poynomias: 1 1 P m (cosθ)p m (cosθ)d(cosθ) (28) ( + m)! ( m)! δ (29) whie assuming 2 m m. Therefore, by pugging the two derived identities (28) and (29) into (27), which yieds N m 4π ( + m)! N m ( m)! δ δ mm δ δ mm, (3) it becomes obvious that N m ( m)! 4π ( + m)!. (31) Rea Spherica Harmonics: Considering that most appications of spherica harmonics require ony reavaued spherica functions, ike BRDF cacuation and approximation, irradiance estimation and spherica harmonic ighting, it is convenient to define the rea-vaued spherica harmonics function as 2R(Y m ) 2N y m m cos(mφ)p m (cosθ) if m > Y N P (cosθ) if m 2I(Y m ) (32) 2N m sin( m φ)p m (cosθ) if m < Whie the compex spherica harmonic basis incudes a pair of sines, the separated imaginary and rea parts of the rea spherica harmonics ony have one sine, and thus the normaization needs to be adjusted by a factor of 2 for those cases. In the foowing paragraphs, ony the rea spherica harmonic part is covered, for the aforementioned reasons. 1 Note that R π π sin(x)dx R π+a π+a sin(x)dx because of the periodica appearance of sine and cosine. 2 There is no need for the m m case because it is aready handed by (28) 1

11 Instead of using two parameters it is sometimes usefu to fatten the spherica harmonic functions in a specific order into a one dimensiona vector, so that they can easiy be enumerated. Thus the function y i (θ,φ) y m (θ,φ) with i ( + 1) + m is used whenever suited. To get a better understanding of the spherica harmonics and what they ook ike, the first few functions can be seen in figure 6. A three casses of spherica harmonics are iustrated: The zona harmonics are a spherica harmonics with m which means that they are circuar symmetric as described in the exampe at the beginning of this section. They are termed zona since the visua curves that appear on the unit sphere are paraes of the equator (atitude) and they divide the unit sphere into zones. It may aso be reevant to mention that since m the harmonics reduce to associated Legendre poynomias. Sectora harmonics are those spherica harmonics of the form Y m m. Tessera harmonics are a other spherica harmonics. The reated unit sphere is usuay divided into severa bocks in ongitude and atitude. 5.2 Properties The foowing section wi cover the most important properties of the spherica harmonics Harmonic expansion Since it has aready been iustrated that spherica harmonics form an orthonorma basis, projecting a spherica function into spherica harmonic coefficients is simpe and a straight forward appication of definition (4). Repacing the arbitrary poynomia basis p n by the rea spherica harmonic basis function y i transforms the equation into k i f (s)y i (s)ds. (33) S Source function n n2 n4 n6 n8 n1 n15 Source function n n4 n6 n1 n15 n2 n25 Figure 7: An exampe of a band-imited spherica harmonic expansions. n denotes the order. By combining this with the aforementioned Monte Caro estimator a numerica soution for the coefficients k i can be defined. In the Monte Caro estimator a weighting function is used for every sampe. If the sampes are carefuy chosen, by making them independent of the parameterization of the sphere using a technique ike stratified samping as described by [12], the weighting function turns into a constant term because a sampes have an equa probabiity. This probabiity is 4π 1 for an equa distribution of sampes 11

12 on the surface of a unit sphere and therefore the resuting weighting function is w(x i ) 4π. The numerica soution for the integration probem can then be rewritten as k i 1 n n f (x j )y i (x j )4π 4π j1 n n f (x j )y i (x j ) (34) j1 with n. Some exampes for such expansions can be seen in figure 7. What can aso be seen in the figure is that using a finite series of coefficients wi ony reconstruct an approximation of the origina function except the source function does not have higher frequencies than the ones that can be captured by the highest order spherica harmonic band used. (n + 1) 2 coefficients are used for such a band-imited approximation of the n-th order. This quadratic growth makes it crucia to find the right trade-off between quaity of the approximation and memory consumed by the coefficients. Ramamoorthi and Hanrahan [3] as we as Basri [4] have shown that for ambertian refectance and irradiance approximation a second order approximation (using a 9 coefficients, a 9D subspace as they ca it) is aready sufficient. More detais about this wi foow ater when providing some exampes of spherica harmonics in computer graphics Convoution Imagine a convoution of a spherica function f with a kerne function k. For exampe it can be a owpass fiter in order to remove high frequencies from the spherica function, maybe to prevent artifacts when computing a band-imited expansion of a high frequency function. This kerne function has to be circuar symmetric, which impies that it does not have any φ dependence, because the resut of a non-symmetric convoution woud not be defined over the sphere, but rather in the corresponding specia orthogona group SO 3. Appying the Funk-Hecke-Theorem [4] yieds 4π (k f ) m k f m α k f m (35) which means that harmonic expansion of the convoution is equa to expanding both functions separatey, scaing them by α. Therefore the spherica harmonic reconstruction of a convoution can be written as Rotationa invariance ḟ k f m (α k f m )Y m. (36) Let g be a copy of f rotated by an arbitrary rotation R over the unit sphere. The foowing reationship can be defined: g(s) f (R(s)) (37) In other words it does not matter if the function or the input is rotated - the resut is the same. This is a very critica property caed rotationa invariance (readers famiiar with the one-dimensiona Fourier transformation may see the anaogy to the shift-invariance property). In practice this means that no aiasing artifacts wi occur when sampes from f are coected at a rotated set of sampe points. A more specific exampe woud be that rotating a ight function wi not cause any ight ampitude fuctuation Fast Integration The fast integration of the product of any two spherica functions a and b has aready been expained in the section about orthonorma poynomias. But to underine the importance of this property consider having some sort of ight transfer function that transforms any given incident ight into a certain amount of exiting radiance. If both the incident ight function and the transfer function are expressed in terms of spherica harmonics, the evauation of the exiting radiance can be reduced to a dot product of the coefficients. This can even be quicky cacuated in a fragment or vertex shader on the newer generations of graphics hardware at 12

13 an incredibe fast rate, whie the evauation of an arbitrary integra over the upper hemisphere in the shader woud just not work at a. The Spherica Harmonic Lighting technique (section 6.2) wi make good use of this property Transfer Matrix Now consider expanding the mutipication of two spherica functions c(ω) a(ω)b(ω) into spherica harmonic coefficients, whereas a(ω) can be evauated at projection time, whie it is not possibe for b(ω). Think of b(ω) as a visibiity (or maybe ighting) function and a(ω) as some kind of weighting function, representing occuders that imit the visibiity function (or the ighting) or maybe even extend it when the function vaue is 1 - which is a bit hard to imagine though, but coud possiby be expained by some kind of optica magnification effect (binocuars). Soan et a. [16] described this as a inear transformation of b j s coefficients by a matrix â which can be obtained by factoring out b in the spherica harmonic expansion of c: and thus c i S c(ω)y i (ω)dω a(ω)b(s)y i (ω)dω S S k ( j j (a k y k (ω)) (b j y j (ω))y i (ω)dω j S k (a k y k (ω))y j (ω)y i (ω)dω)b j (a k y k (ω)y j (ω)y i (ω)dω)b j k S â i j b j (38) j â i j a k y k (ω)y j (ω)y i (ω)dω. (39) k S In other words transforming any given vector of spherica harmonic coefficients b by the transfer matrix wi yied a new vector of coefficients c, which is equa to the harmonic expansion of a(ω)b(ω). The described matrix is mosty sparse and symmetric which provides room for programmatic optimizations. What this technique makes possibe is to integrate over a tripe product of functions, with two distinct unknown parameters (whie using the aforementioned fast integration ony aows the dependence on one unknown parameter). An exempary usage of this transfer matrix can be found in Soan, Snyder and Kautz s paper [16]. They are using it to mode gossy radiance transfer, whereas both the view-dependent refection direction (BRDF) and the incident ight function (incident radiance) are unknown at pre-computation time. 5.3 Rotation It has aready been expained that when rotating a spherica harmonic expanded function its extents wi be sustained exacty. But cacuating such a rotation is not as easy as rotating an ordinary vector space. Robin Green [12] has described this probem in depth and even caed it a roya pain-in-the-ass when using a naïve approach. I wi ony cover a sma presentation of the probem and the soution that has been proposed by Kautz et a. [17]. As we wi see this ony works we for the first coupe of spherica harmonic bands, but that is enough for the cause of this work. The interested reader can find more information on the probem in various sources [18, 19, 2, 21]. Here is how a ow-order rotation may be computed: From the orthogonaity property it can be derived that when transforming coefficients of a particuar band in the resuting function ony coefficients of that same band are affected. What does this mean? Firsty it means that a rotation can be expressed as a inear transformation, a n n matrix to be exact. Secondy this matrix wi be bock diagona sparse, which means that it wi contain severa "transformation bocks" on the principa diagona each spanning the rectanguar region of a transformation of a singe band. So the key to a rotation woud be to find a efficient set of recurrence reations that depending on the n-th 13

14 band rotation returns the n + 1-th band matrix. Such reations can be found in the aforementioned papers but wi not be described here. Let us get back to the matrix. Each entry in the matrix M i j describes how much a source coefficient j is ike the rotated coefficient i. Recaing the method to determine how much a orthogona basis is ike another this can be formuated as M i j y i ( ˆRω)y j (ω)dω (4) S whereas ˆR denotes the rotation. For exampe, an arbitrary rotation about the z-axis ( φ) by α is M z i j (α) S y i (θ,φ + α)y j (ω)dω. (41) The rotated coefficients c j can be computed from the source coefficients c j using the inear transformation c j i M i j c j. But this method computed for an arbitrary ange can quicky become very expensive. Instead the trick is to convert the rotation matrix into its Y-Euer ange decomposition. A Y-Euer rotation works by first rotating around the z-axis, then the y-axis and finay the z-axis again. Computing the z-axis rotation is easy [12], but the y-axis is more compex. However the y-axis rotation can be decomposed: Firsty, rotate about the x-axis by 9 o and secondy do a genera rotation about the z-axis. Finay rotate back about the x-axis by 9 o. The two x-axis matrices are transposed counterparts and can be pre-cacuated, since they are fixed. This can even be optimized further as described by Robin Green [12] but since this work is not about impementationa specifics those optimizations are not covered here. 6 Spherica Harmonics in Computer Graphics In the foowing section I wi give some exampes of what spherica harmonics can be used for in the fieds of computer graphics. The exampes focus on enhancing rea-time rendering by sourcing out or approximating parts of the ighting equation using spherica harmonics. Usage of spherica harmonics is not imited to these appications though, since they can aso be utiized for object recognition and image based reighting [3, 4]. 6.1 Irradiance Environment Maps The first technique has been deveoped by Ravi Ramamoorthi and Pat Hanrahan at the Stanford University. They propose an efficient representation for irradiance environment maps using spherica harmonics [3]. Environment maps: An environment map is used to store distant ighting distribution denoted by L. Since it is assumed to be very far away, there is no noticeabe variance in the ighting on an object, a points on the surface are it equay. Based on the norma vector n the incident ight at an arbitrary point can be ooked up. Such a ookup represents the resut of an integration over the upper hemisphere Ω( n) at that point, whie not taking into account near iumination or visibiity information (e.g. no shadows). The ight mode is assumed to be ambertian and thus such an integra can be described as the convoution Ω( n) L(ω)( n ω)dω L A( n) E( n) (42) whereas E( n) represents the irradiance of the surface and describes the grand tota of incident ight. So an environment map represents a way to map a norma n to its corresponding irradiance E( n) (which mutipied by the surface abedo corresponds to the image intensity). It is possibe to pre-compute this convoution for every norma direction, since the environment map is static. Such a pre-computation is caed pre-fitering and the resut is a very burry environment map. 14

15 Figure 8: Image courtesy of Ramamoorthi and Hanrahan. It shows the resut of their Efficient Representation of Irradiance Environment Maps [3] rendered in rea-time. The new approach: What Ramamoorthi et a. noticed is that this convoution can be expressed in terms of spherica harmonics and soved very efficienty in frequency space. Expanding the ight function L in terms of spherica harmonics yieds L(θ,φ) L m Y m (θ,φ) (43) and defining max( n ω, ) max(cos θ, ) as the circuar symmetric kerne function, aso in its spherica harmonic expansion with m, it turns out (36) that the convoution (42) can be written as A( n) max(cosθ,) A Y ( n) (44) E( n) α A L m Y m ( n). (45) Very intriguing about this soution is that because α vanishes very fast the irradiance is we approximated by ony 9 coefficients. The maximum error for any pixe is 9% [3]. Especiay for rendering this is a very important property since when writing down the first 9 spherica harmonics in Cartesian coordinates Y (x,y,z) {Y1 1 ;Y1 ;Y 1 1 }(x,y,z).48863{x;z;y} Y2 (x,y,z) (3z2 1) Y 2 2 (x,y,z) (x2 y 2 ) {Y 2 2 ;Y 1 2 ;Y 1 2 }(x,y,z) {xz;yz;xy} (46) 15

16 Grace Cathedra Lightprobe Eucayptus Grove Lightprobe Figure 9: Image courtesy of Ramamoorthi and Hanrahan. It shows a source environment map and its pre-fitered counterparts using the standard image-space method (eft) and the spherica harmonic approach (right). it becomes evident that this computation can easiy be performed on the fy on modern fragment or vertex shader hardware by soving the quadratic poynomia (45). E(x,y,z) 2 m α A L m Y m (x,y,z) c 1 L 2 2(x 2 y 2 ) + c 3 L 2 z2 c 5 L 2 + c 4L + 2c 1(L 2 2 xy + L1 2xz + L 1 2 yz) (47) + 2c 2 (L 1 1 x + L 1 1 y + L 1 z) c c c c c Aternativey the spherica harmonic coefficients for every point on the surface can be precomputed once, which reduces the rea-time cacuation to ony a scaar product of the vertex norma s coefficients and the irradiance coefficients. It shoud be denoted that each coor channe needs a separate set of coefficients because the ighting function ony represents ampitude of ight, no coorization. For rendering, the resut of the aforementioned equation need ony to be mutipied by the surface abedo, which is usuay taken from texture map, to yied the fina pixe output. Pre-fitering resuts: Traditionay an environment map is represented by either a cube map (which is constructed using six two-dimensiona textures representing the sides of a cube) or a two dimensiona texture - using a specia coordinate mapping, caed sphere mapping, to warp the 2D image onto the surface of a sphere. Figure 9 shows such a two-dimensiona mapping of two ight-probes [15]. As asserted before the quaity does not differ much. The images on the right side represent the pre-fitered irradiance maps. With a traditiona approach pre-fitering takes O(S T) time, where S is the size of the source texture (ightprobe in this case) and T the size of the resuting irradiance texture (usuay something around 64x64496 texes or even arger). The new approach by Ramamoorthi et a. ony needs O(9 T ), because ony 9 coefficients need to be computed. It is obvious that whie saving a ot of processing time this aso saves a ot of memory. 16

17 6.2 Spherica Harmonic Lighting Introduction This may be the most impressive exampe for the usage of spherica harmonics in computer graphics. Spherica harmonic ighting is a technique that can effectivey deiver rea-time dynamic goba iumination at a very high performance. There are some imitations, but most of them can be circumvented. The Rendering Equation: Before starting with the spherica harmonic soution of the ighting equation, one of the hoy grais of rea-time computer graphics wi be introduced: the accurate computation of what Kajiya has referred to as the rendering equation [22] - competey in rea-time. The rendering equation expresses the ight intensity transferred from a point x in direction ω v and in differentia ange form it is defined as L(x,ω v ) L e (x,ω v ) + R(x, ω,ω v )L(x ω, ω)g(x,x ω )V (x,x ω )dω. (48) S L e (x,ω v ) describes the ight that is emitted by the point x in direction ω v independent of any incident ight (e.g. an emissive surface ike phosphor). Modeing this is the easiest part. R(x,ω,ω v ) is the scaarvaued bi-directiona refectance distribution function (BRDF) and it scaes the ight refected at point x into direction ω v depending on the incident direction ω. The geometric reationship G(x,x ) describes how the two parameter points x and x are reated to each other whereas x is some other point of the scene in direction ω from x. One of the most prominent exampes for such a reationship is the ambertian cosine term. Finay V(x,x ) describes the visibiity reationship between the two points, which is either 1 or. It becomes true if a ray cast from x to x is not occuded by any other geometry and is fase otherwise. The most difficut probem of this equation is to sove the recursion of L(x,ω v ) aso termed interrefection. Review of previous soutions: Here is a quick review of previous rea-time soutions for this equation: Id Software s Quake (1996) pioneered with a soution caed rea-time ight-mapping. This technique uses a radiosity agorithm [24] to pre-compute the diffuse ighting (which was very time consuming and took hours to days) of the entire scene, storing it into textures. One impication of this approach was that no dynamic ighting coud be taken into account when computing the ightmaps. Additionay the BRDF had to be static since the ambertian refection mode uses an equa distribution of ight into a directions (R(x,ω,ω v ) is constant). Dynamic ighting was faked by additive bending of unshadowed diffuse point ight sources with V(x,x ) 1 and no interrefections. This technique can sti be found in a ot of modern games, with the most prominent exampe being Haf-Life 2 by Vave Software. They are using a modified version of this technique where the ightmap is computed from three different anges. Thereby they circumvent the BRDF imitation and are abe to approximate gossy refections [23]. A more modern approach is to use stenci voume shadows or shadowmaps to sove the equation. Whie this technique aows competey dynamic ighting, there are no interrefections. Additionay point ights are assumed. Since those have no area and ony consist of one point - hard shadow edges are impied. There are workarounds that simuate spherica area ights using points ights foowed by a smoothing of the shadow edge. So in the past you aways had to choose between quaity (in terms of interrefection and area ights) and fexibiity (dynamic ight sources). Using spherica harmonics it is possibe to have both - rea-time dynamic ightsources with interrefections and area ight sources. Athough there are imitations it sti is a step towards the goa The new agorithm Now the soution deveoped by Soan et a. [16] be described in detai (a programmatic expanation has been written by Robin Green [12]). The basic idea is to divide the rendering equation into a ight source function L I as we as a (pre-computabe) transfer function T : L(x,ω v ) L e (x,ω v ) + L I (x, ω)t (x, ω,ω v )dω (49) S 17

18 Figure 1: Exampe of a scene it by pre-computed spherica harmonic ighting. There is ony one ight source, the sun, but notice the interrefections that cause the ceiing, the top of the pedesta and the wings of the statue to be sighty iuminated. The ight source function L I (x,ω) describes exacty the fraction of the ight incident at point x from direction ω that has been emitted directy from a ightsource - without taking into account any interrefections (indirect ighting). There are two possibe settings for L I. Firsty, given the idea setting of ow variation of the ight source function for the entire scene ony one copy of the ight source function is required and shared by a points. For instance a setting where a distant ight ike the sun ighting a sma scene (ike a chessboard) woud fufi this property. This works because the ight is so far away that given any two points x and x ω on the surface of the scene the difference between L I (x) and L I (x ω ) is negectibe. Secondy, if the environment consists of oca ighting (eg. ight sources that are cose to the object) L I wi vary at different points on the object. In this case it is possibe to take sampes of the ight function and interpoate between them when rendering, effectivey creating a fied of incident ighting. This does, however, break some assumptions about interrefections that we need to make, so this can not be used The transfer function T (x,ω,ω v ) describes how ight arriving from L I is transformed into irradiance in direction ω v. Note that not ony ight arriving at the point x is transformed. Since there possiby are interrefections arriving at x, T might transform ight that does not directy arrive from the ightsource into irradiance. Depending on the ight mode used, T can be expressed as a spherica function which optionay depends on the view-direction. For view-dependent ight modes the transfer functions need to be approximated by a spherica harmonic transfer matrix (Chapter 5.2.5) to transform incident ight into a new set of coefficients that can ater be convoved with the BRDF. On the other hand the transfer functions for viewindependent ight modes can be approximated by ony using a vector of spherica harmonic coefficients. The transfer functions for both cases are generated in a pre-computing step - that is ikey to take a ong time for a detaied approximation. When rendering a point on the surface its intensity is the soution of equation (49). But since both functions are expressed in terms of spherica harmonics the equation reduces to L(x,ω v ) L e (x,ω v ) + L I i T i (5) i 18

19 for diffuse and L(x,ω v ) L e (x,ω v ) + i α i R i ( T i j L I j )yi (ω v ) (51) j for view-dependent ight modes whereas j T i j L I j is the inear transformation of the ight coefficients by the transfer function and the sum over i describes the convoution of the BRDF kerne R with the transfered ight function. Evauating y m returns the vaue of the convoved function at ω v (which is the view-direction). From now on the ambertian refection mode wi be assumed and therefore no gossy refections wi be described, athough an in depth discussion of this refection type can be found in [16] Transfer functions After this rather quick and dirty overview it is time to go into more detai about the transfer function. Since in the ambertian mode ight is refected equay in a directions, the aforementioned rendering equation can be simpified by removing a view-dependent parts: L am (x) L e (x) + ρ(x) π S L(x ω, ω)g(x,x ω )V(x,x ω )dω (52) The BRDF in this case reduces to constant term ρ(x) which is caed the surface abedo at point x. It describes the ratio of radiosity (which corresponds to image intensity) versus irradiance. Now a that remains in the equation is the amount of incident ight L, the geometric reationship, and the visibiity functions. Recaing the geometric reationship in a diffuse environment being defined as the non-negative max( n x v,) (has aready been used in the section on environment maps), whereas n x is the norma vector at the point x and v the direction of the incident ight, the equation can be expanded to L am (x) L e (x) + ρ(x) π Ω( n x ) L(x ω, ω)max( n x v ω,)v(x,x ω )dω. (53) The integra domain reduces to the upper hemisphere since for the ower hemisphere the cosine term woud be. Three different types of transfer functions can be derived from this equation. Each one adds more detai to the ighting, but aso takes more time to compute. I wi ony give a brief idea on how the foowing transfer functions can be evauated using ray-casting, for detais consut Green s paper [12]. Diffuse Unshadowed Transfer Function: The first transfer function T DU does not take into account the visibiity term V(x,x ) or interrefections. Therefore the rendering equation reduces to L DU (x) L e (x) + ρ(x) L I (x, ω)max( n x v ω,)dω. (54) π Ω( n x ) Comparing (54) to (49) it quicky becomes cear that the transfer function has to be T DU (x,ω) max( n x v ω,). (55) The resut of a scene ighted with this function ooks very simiar to norma dot-product ighting, except that instead of point or directiona ight sources any area ight source can be used. Exampe usage of diffuse unshadowed transfer functions can be found in figures 11(a) and 11(b). Diffuse Shadowed Transfer Function: The second transfer function T DS is very simiar to the aforementioned one. In addition to T DU it does, however, incude the visibiity term. Therefore, L DS (x) L e (x) + ρ(x) L I (x, ω)max( n x v ω,)v(x,ω)dω. (56) π Ω( n x ) whereas V(x,ω) is true as soon as the ray with origin at x in direction ω is intersecting parts of the scene, opposed to V(x,x ) where the ray check is ony performed in between x and x. Foowing (49) yieds T DS (x,ω) max( n x v ω,)v(x,x ω ). (57) 19

20 Transfer function (a) Schematic diagram of a diffuse unshadowed transfer function. Green arrows indicate unoccuded rays, whie yeow rays describe occuded rays, that are nevertheess accounted into the transfer function. (b) Exampe of a scene rendered with diffuse unshadowed ight transfer. Note that there are no shadows except for the cosine term. Transfer function (c) Schematic diagram of a diffuse shadowed transfer function. Green arrows indicate unoccuded rays, whie red rays describe occuded rays that are now not taken into account (cmp. figure 11(a)). (d) Exampe of a scene rendered with diffuse shadowed ight transfer. Note that there now are shadows. Figure 11: Shadowed and unshadowed ight transfer functions. 2

21 This means that to evauate this function, a ray needs to be cast in every direction to determine if there is an occuder bocking incident ight for that direction. With T DS the resuts are comparabe to dynamic singe-pass radiosity. Exampes can be found in figures 11(c) and 11(d). Diffuse Interrefected Transfer Function: The diffuse interrefected transfer function is the most compex of the three. It soves equation (53) competey, athough it needs to be divided into two parts - one of them we have aready seen: L IR (x) L e (x) + ρ(x) L I (x, ω)max( n x v ω,)v(x,ω)dω (58) π Ω( n x ) + ρ(x) L R (x, ω)max( n x v ω,)(1 V(x,ω))dω. π Ω( n x ) L R (x,ω) depicts the interrefected part of the incident ight arriving at x in direction ω. So the second integra describes a ight which is arriving at x that was refected by another point of the scene, using an inverse visibiity check to mask out direct ighting (Note: this is not needed here, because L R does ony cover the interrefected parts anyway. But it wi be required in a moment.). By using L DS the equation can be simpified, yieding L IR (x) L DS + ρ(x) L R (x, ω)max( n x v ω,)(1 V(x,ω))dω. (59) π Ω( n x ) Now a probem arises: What part of equation L IR is the ight source function, and where can we find the transfer function? The ight source function (obviousy) is the same, but the transfer function is more compex now. Finding an expicit expression wi not be easy, because the two integras somehow need to be joined together. But it is possibe to define an infinite series of transfer functions Tn IR that for n converge towards T IR. It can be defined as T IR (x,ω) T DS (x,ω) (6) Tn+1 IR (x,ω) T DS (x,ω) + ρ(x) max( n x v ωi,)v(x,x ωi )Tn IR π (x,ω i )dω i (61) Ω( n x ) What does this construct do? In the first pass ony the shadowed direct ighting is taken into account yieding the same resuts as T DS. But in the second pass the shadowed direct ighting is reintroduced and an integration over the upper hemisphere is performed. This integration gathers a interrefected ighting from the previous pass. The above aso mathematicay describes the impementation suggested by Green and Soan et a. The first pass is performed just ike the diffused shadowed cacuation. A foowing passes are simiar, but instead of rejecting ight if an intersection occurs the transfer function of the intersecting point is determined and taken into account. Continuing this for severa iterations wi quicky approximate the interrefections. One important imitation that arises from the interrefected transfer function is the "baking" of materia properties (ρ(x)) into the transfer function. Whie this enabes effects such as coor beeding, it means that materia properties can not change at runtime - which is possibe with the diffuse shadowed and unshadowed transfer functions. Again, exampes can be found in figures 12(a) and 12(b) Light functions A ight function maps a direction ω to a ight intensity. It is usuay described using the three RGB coor channes. Light function are very simiar to the environment maps described before and those can actuay be used as ight functions. But computing and pre-fitering environment maps at rea-time is probematic at the moment and therefore approximations are often used. For exampe, in a demo a dynamicay tinted environment map is used for skybox rendering whie the ight function is just a quick, dirty and scripted approximation of sunight that was heaviy tweaked unti it fitted we enough. It uses a coored directiona 21

22 + PSfrag repacements Fina transfer function (a) Schematic diagram of a diffuse interrefected transfer function. The eft part is the shadowed transfer function of x, whie the center image represents another shadowed transfer function that sends out interrefections. These are scaed by the cosine term and added to the eft function to yied the fina interrefected transfer function (cmp. figures 11(a) and 11(c)). Note that this is ony a schematic exampe. In a correct impementation a red rays woud gather interrefections. Interrefected Shadowed (b) Exampe of a scene rendered with diffuse interrefected ight transfer. An in depth ook at the ceiing above the opposite wa and at the top of the pedesta show interrefections that are missing in figure 11(d). Five iterations have been used to gather the interrefections by shooting rays per vertex. Computation time was about 4 hours per pass resuting in a tota of 2 hours. Figure 12: Diffuse interrefected ight transfer. 22

23 ight projected into spherica harmonic coefficients for the sun and a sighty bueish ambient term that is distributed equay into a directions (to fake the ight refractions by the atmosphere and the nighty iumination by the moon). The ambient term can be reaized with spherica harmonics by simpy scaing the zeroth spherica harmonic order basis function (since it represents equa distribution into a directions). To simuate the movement of the sun the ight function is brought into the proper rotation using the aforementioned Y-Euer operation described above. The coor channes are aso modified to simuate dusk and dawn. More information on approximating sunight can be found in Green s paper. 7 Concusion This work has presented some orthogona basis functions, and how they can be combined to form the spherica harmonics. Aso, some properties of the spherica harmonics have been discussed and exampes of how they can be used have been given. Whie the intention was to expain the spherica harmonic as simpe and iuminating as possibe, it sti is not trivia - both to write and expain as we as to understand. Hopefuy it has become cear what spherica harmonics can be used for in computer graphics, athough ony two of the pentifu number of appications coud be iustrated here. It has yet to be determined what ese those itte adaptive spherica shaped friends can be used for - not ony in computer graphics, but aso computer vision (as Basri and Jacobs have shown [4]) and maybe other graphics reated areas ike coision detection, artificia inteigence, and more. References [1] B. Cabra, N. Max and R. Springmeyer. Bidirectiona Refection Functions from Surface Bump Maps SIGGRAPH , 1987 [2] M. D mura. Shading Ambiguity: Refection and Iumination. In Computationa Modes of Visua Processing Landy and Movshon, eds., MIT Press, Cambridge, , 1991 [3] Ravi Ramamoorthi, Pat Hanrahan. An Efficient Representation for Irradiance Environment Maps SIG- GRAPH 497-5, 21 [4] Ronen Basri, David W. Jacobs. Lambertian Refectance and Linear Subspaces IEEE Transactions on Pattern Anaysis and Machine Inteigence Vo [5] Richard P. Feynman. The Feynman Lectures on Physics, vo. I - part 1. Inter European Editions, Amsterdam 1975 [6] Eric W. Weisstein. MathWord [7] W. Magnus, F. Oberhettinger, R.P. Soni. Formuas and Theorems for the Specia Functions of Mathematica Physics Chapter V, pp 24, New York 1966 [8] F. Siion, J. Arvo, S. Westin and D. Greenberg. A Goba Iumination Soution for Genera Refectance Distributions SIGGRAPH [9] S. Westin, J. Arvo, K. and Torrance. Predicting Refectance Functions from Compex Surfaces SIG- GRAPH , 1992 [1] State of the Art in Monte Caro Ray Tracing SIGGRAPH Course 29,21 [11] Peter Shirey. Reaistic Ray Tracing A. K. Peters 21 [12] Robin Green. Spherica Harmonic Lighting: The Gritty Detais SCEA Research and Deveopment, 23 [13] Matt Pharr. Physicay Based Rendering: From Theory to Impementation Morgan Kaufman, Juy 24 23

24 [14] Numerica Methods in C: The Art of Scientific Computing Cambridge University Press, pp , 1992 [15] Pau Devebec. Light Probe Image Gaery [16] Peter-Pike Soan, Jan Kautz, John Snyder. Precomputed Radiance Transfer for Rea-Time Rendering in Dynamic, Low-Frequency Lighting Environments Microsoft Research and SIGGRAPH, Juy 22 [17] Jan Kautz, Peter-Pike Soan and John Snyder. Fast Arbitrary BRDF Shading for Low-Frequency Lighting Using Spherica Harmonics 13th Eurographics Workshop on Rendering, 22 [18] J. Ivanic and K. Ruedenberg. Rotation Matrices for Rea Spherica Harmonics, Direct Determination by Recursion Journa of Physica Chemistry A Vo. 1, pp , 1996 [19] J. Ivanic and K. Ruedenberg. Additions and Corrections: Rotation Matrices for Rea Spherica Harmonics Journa of Physica Chemistry A Vo. 12, No. 45, pp , 1998 [2] Cheo Ho Choi et a. Rapid and stabe determination of rotation matrices between spherica harmonics by direct recursion Journa of Chemica Physics Vo 111, No. 19, pp , 1999 [21] Migue A. Banco, M. Forenz, M. Bermejo. Evauation of the rotation matrices in the basis of rea spherica harmonics Journa of Moecuar Structure (Theochem), 419, pp 19-27, 1997 [22] J.T. Kajiya. The Rendering Equation SIGGRAPH, pp , 1986 [23] Garry McTaggart. Haf-Life 2 / Source Shading Game Deveopers Conference, 25 [24] Cohen and Waace. Radiosity and Reaistic Image Synthesis Academic Press,

25 Figure 13: Pictures of a demo renderer featuring the presented spherica harmonic ighting technique combined with a high dynamic range pipeine. The ower images show day and night transitions whie the others show arbitrary scenes. 25