Vrtual Noctluca: Interacton between Lght and Water usng Real-tme Flud Smulaton and 3D Moton Measurement Kyouhe Ada and Norko Nagata Kwanse Gakun Unversty, School of Scence and Technology 2-1 Gakuen, Sanda, 669-1337 Japan { ada, nagata} @kwanse.ac.jp http://st.ksc.kwanse.ac.jp/ nagata/ Abstract. In recent years, wth the rapd mprovement of the performance of computers, new possbltes for real-tme smulaton technologes are emergng. In ths study, we smulated the behavor of water n real tme and combned ths smulaton wth a measurement of the user s 3D moton to smulate the nteracton between water and lght, as observed n Noctluca. Noctluca s oceanc plankton that produces lght when physcally stmulated. Strrng the surface of water contanng Noctluca n a completely dark place causes the surface to glow, and an observer can gan the mystcal and fantastc experence of watchng the glow becomng dm wth the flow of the water. Key words: Interactve art, GPU, stereo vson, Smoothed Partcle Hydrodynamcs, anmaton, llumnaton 1 Introducton Smulatons based on physcal nteractons have been wdely used n computer graphcs to produce anmatons. In partcular, many methods have been developed to vsualze fluds realstcally smulatons; however, the computatonal cost of these methods have always been hgh [1][2]. Wth the recent developments of technques for obtanng faster smulatons usng hgh performance-computers, t has now become possble to calculate flud smulatons n real tme. Therefore, there s a possblty that nteractve applcatons usng real-tme flud smulatons wll be developed soon. In ths study, we smulated the behavor of water n real tme and combned ths smulaton wth a measurement of a user s 3D moton; n ths manner, we smulated the nteracton between water and lght, as observed n Noctluca. Noctluca s oceanc plankton that produces lght when physcally stmulated. Strrng the surface of water contanng Noctluca n a completely dark place causes the surface to glow, and an observer can gan a fantastc and almost mystcal experence of watchng the glow becomng dm wth the flow of the water. We use Smoothed Partcle Hydrodynamcs (SPH), whch s a method of
partcle-based flud smulaton, to smulate realstc water flow. Furthermore, we ncreased the smulaton speed usng graphcs processng unts (GPUs) so that we can receve the response quckly. We also used a stereo camera to measure the 3D motons of nputs such as a user s hand. By usng 3D motons as the nput data, we can experence vrtual Noctluca. 2 Related Work Flud smulaton can be categorzed nto grd methods and partcle methods. In grd methods, many grds are needed for exact computatons, so t s not suted for real-tme applcatons. In partcle methods, Movng Partcle Semmplct (MPS) [7] and Smoothed Partcle Hydrodynamcs (SPH) [8] are typcal methods. SPH methods are sutable for real-tme smulatons because of ther low computatonal cost [6]. For real-tme smulatons, the growth n computatonal power of GPUs has materally contrbuted to ther effcency. GPUs were orgnally desgned for 3D graphcs tasks and have also been to speed non-graphc tasks such as cellular automata smulaton, partcle smulaton, solvng of lnear equatons, and so on. Regardng flud smulaton, several studes have been done on ncreasng speed [9, 10], and Harada et al. recently proposed a method for SPH on GPUs [11]. On the other hand, several studes have been conducted on the use of physcally based smulatons n nteractve applcatons. Cassnell et al. developed an applcaton n whch a user can send portons of a projected mage forward or backward n tme by actually touchng and deformng the projecton screen, usng a CCD camera [12]. Arga et al. presented a system to show wnd flow, computed by flud smulaton, projected on the screen, and then obstructed by the shadows of observers. Matsuo developed an applcaton that controls a group of brllant butterfles by operatng a glowng ball [13]. No prevous studes, however, have examned the same nteractons between water and lght as we propose here. In ths study, ntegraton of real-tme flud smulaton and stereo vson has been realzed. 3 Smulaton Of Water In Smoothed Partcle Hydrodynamcs (SPH) 3.1 Governng Equatons The velocty of a flud s nfluenced by the forces assocated wth t, such as pressure, vscosty, and external forces. In ncompressble fluds, the densty remans constant even wth the applcaton of pressure. Vscosty s a force that makes the velocty constant. And the only external force s gravtatonal force. Ths ndcates that the changes n the velocty of ncompressble fluds s dependent on the sum of the forces assocated wth the flud. The governng equatons for
ncompressble flow are expressed by the mass conservaton equaton and the momentum conservaton equaton as follows Dρ Dt = 0 (1) ρ Dv Dt = p + µ 2 v + f (2) where ρ, v, p, µ, and f are densty, velocty, pressure, dynamc vscosty coeffcent of the flud, and external force, respectvely. The left hand sde of Lagrange s dfferental equaton, ndcates the changes n the physcal values of the flow. It s expressed by the followng equaton Dϕ Dt = ϕ t + v ϕ (3) The second term on the rght hand sde of Eq.(2) ndcates advecton phase. When the fxed grd method s used, we must calculate the parameters assocated wth the advecton phase. Ths method requres computatons by lnear nterpolaton because the grd ponts have physcal values. Ths leads to serous problems n that the velocty of the flud decreases to zero and the mass on the surface of the flud decreases. Smulaton wth small tme steps mnmzes the problems, but ths s not suffcent these smulatons would fal wth real-tme applcatons. Lagrange s dfferental equaton ndcates the dfferental calculus on the movng partcles. A partcle method such as SPH, that problem does not occur because we can calculate the advecton phase only by movement of partcles. 3.2 Dscretzaton SPH s a partcle method n whch the flud s regarded as a group of partcle. In SPH, a physcal value at a poston x s calculated as a weghted sum of the physcal values ϕ j of neghborng partcles j as follows: ϕ(x) = j m j ϕ j ρ j (x x j ) (4) where m j, ρ j, and x j represent the mass, densty, and poston of partcle j, respectvely, and W s a weght functon. The densty of the flud s calculated usng Eq.(4) as ρ(x) = m j W den (x x j ) (5) j Snce W s dstant from the poston x, the value of W becomes approxmately 0. Therefore, W s the sum of the physcal values of neghborng partcles. In ths secton, we explan a method to dscretze Eq.(2) wth Eq.(4). In order to calculate acceleraton, we separately compute pressure, vscosty, and external
forces. The acceleraton s then calculated as a sum of these forces. The pressure on the flud s calculated usng the consttutve equaton p = p 0 + k(ρ ρ 0 ) (6) where, p 0 and ρ 0 are the pressure at rest and densty, respectvely. To compute the momentum conservaton equaton, gradent and laplacan operators, whch are used to solve for the pressure and vscosty forces on partcles, have to be modeled. The pressure force f press and the vscosty force f vs are computed as follows: f press = j m j p + p j 2ρ j W press (x x j ) (7) f vs = µ j m j v + v j 2ρ j W vs (x x j ) (8) where x and x j are the postons of partcles and j, respectvely. The weght functons used by M uller et al. are also used n ths study [6]. The weght functons for the pressure, vscosty, and other terms are desgned as follows. W press (r) = 45 πre 6 (r e r ) 2 r r (9) W press (r) = 45 πre 6 (r e r ) (10) W press (r) = 315 64πre 9 (re 2 r 2 ) 3 (11) In these functons, the value s 0 outsde the effectve radus r e. 3.3 Neghbor Search In SPH, each partcle must be used to search for neghborng partcles n order to calculate the nteracton among the partcles. The computatonal cost of searchng for neghborng partcles s hgh when a large number of partcles are used. To reduce ths computatonal cost, a 3D grd coverng the computatonal regon, called a bucket, s ntroduced, as descrbed by Mshra et al. [14]. Each voxel encoded as a pxel n the texture s assgned a 3D computatonal space. Then, for each partcle, we compute a voxel to whch the partcle belongs and store the partcle ndex n the voxel. If ths bucket can be obtaned, we do not have to search for the neghborng partcles of a partcle because the neghborng partcles are present n the voxels surroundng the voxel to whch the partcle belongs.
Fg. 1. Neghbor search usng a bucket 3.4 SPH on Graphcs Processng Unts (GPUs) To compute SPH on GPUs, physcal values are stored as textures n vdeo memores. The textures of poston, velocty, densty, and bucket are prepared. A texel of a texture corresponds to a partcle, and the physcal values of a partcle are stored n the texel, as shown n Fg. 2 (a). Although a bucket s a 2D array, the current GPUs cannot wrte to a 3D buffer drectly. Therefore, we employed a flat 3D texture n whch a 3D array s dvded nto set of 2D arrays. We dvded the 3D textures nto 2D arrays, as shown n Fg. 2 (b). The ndces of the Fg. 2. (a) All the partcles postons are stored at each texel of the poston texture. (b) The 3D array s dvded nto a set of 2D arrays. neghborng partcles of a partcle can be found usng the generated bucket texture. Usng the ndex of the partcle, the poston can be obtaned from the poston texture. The densty of a partcle s then calculated by the weghted sum of mass of the neghborng partcles, whch s then wrtten nto the densty texture. The pressure can be calculated from the densty and poston textures. Further, the vscosty force can be calculated from the poston, velocty, and densty textures. These forces are computed usng Eq.(7) and (8). Assumng that the external force s only gravtaton, we can update the velocty, and by usng ths updated velocty texture, the poston s calculated usng an explct
Euler ntegraton as follows: v t+ t = v t t (f press + f vs + f g ρ ) (12) x t+ t = x t + v t+ t t (13) where x t and vt are the prevous poston and velocty of partcle, respectvely. Although there are hgher order schemes, they were not ntroduced because we dd not encounter any stablty problems. Fg. 3. Neghbor search usng bucket texture. (a)dsposton of partcles n the regon. (b) poston texture (c) bucket texture comprsng stored partcles. 4 Manpulaton of a Stream of Water We used a stereo camera (Bumblebee2, Pont Grey Research Inc.) to measure the 3D moton of the user s hands and the parts moved by the user n real tme; the 3D coordnates of the pont sequence of the object surface are entered nto the system. Table 1 shows the specfcatons of the stereo camera. The stereo camera measures the 3D ponts by employng the prncple of stereo vson. The stereo camera cannot adjust the angle of vew because the poston of the two lens s fxed. However, the stereo camera can measure wthout calbraton snce t has defned parameters of the camera. 4.1 Collson Detecton In ths secton, we explan our method for detectng collsons between water partcles, whch are smulated by usng SPH, and the 3D coordnates of the pont sequence on the object surface. The computatonal cost of calculatng the collson detecton among partcles s hgh, and hence, we compute t on GPUs as well as SPH. As descrbed n chapter 3, four types of textures are prepared n SPH on GPUs. Further, the texture of the object s poston, called object texture, s prepared to compute the collson detecton. The postons of the ponts measured
Table 1. Specfcatons of the stereo camera Sony 1/3 progressve scan CCD Imagng Sensor ICX204 (1032 776 max pxels) 4.65 m square pxels Baselne 12 cm A/D Converter 12-bt analog-to-dgtal converter Frame Rates 20 FPS Camera Specfcaton IlDC 1394-based Dgtal Camera Specfcaton v1.31 by usng the stereo camera are stored n each texel, and the empty texels have a value of 0. By usng the object texture, we calculate the postons of the object s ponts and mapped them nto a bucket texture. The collson detecton between partcles s generally computed n cases where the dstance between the partcles s 0 or below an optonal constant value. Our method of collson detecton s decded from each voxel wthout computng dstance. We regard the water partcles that are stored n the texel of the mappng object s partcles as collded partcles. In such a manner, we reduce the computatonal cost, and the method retans precson because each voxel s suffcently small. Fg. 4. Collson detecton on GPUs. (a) 3D coordnates of the pont sequence on the object surface. (b) Object texture. (c) Bucket texture. 4.2 Takng nto Consderaton User s Moton We also need to consder nterference from users on the behavor of water; hence the average of all object postons s calculated for each frame. Ths s expressed by the followng equaton v t = 1 x t 1 x t 1 j, ( = 0,, N; j = 0,, M) (14) N M j
where v t s velocty of the water partcle, N and M are the number of ponts measured by stereo camera, and x s the poston of the ponts. The rate of change n the value s consdered to be the speed of the collded water partcles. 5 Result The system was mplemented usng a system equpped wth a Core(TM)2 CPU and a GeForce 7950GTX. The programs were wrtten n C++ and DrectX, and the shader programs were wrtten n HLSL. The system comprse a PC, whch smulates the behavor of water and regulates the overall system, and a stereo camera, whch measures the user s 3D moton. Fg. 5 shows the composton of the system. Collsons between the surface pont sequence and water partcles Fg. 5. Overvew of the system were detected by the camera, and the water partcles that were nvolved n the collson were made to produce lght. In addton, n order to smplfy the regulaton of parameters for lght producton, varous patterns of lght producton were developed by usng ntutve expressons such as beautful, mystcal, and fantastc. Fg. 6 llustrates a demonstraton. Our system was used represent a stream of water by flud smulaton. Furthermore, we adopted sound effects Fg. 6. The motons of a user s hand and the change n the llumnatons.
Fg. 7. Varous examples of llumnatons that change accordng to the number of water partcles as well as ther speed, usng a musc vsualzaton system [15]. Ths system produced synergstc effects wth vsual sensatons and engaged users. Even wth such a complex pattern of lght producton, we realzed a frame rate of 10 fps. The system allows users to experence the natural lumnescence of Noctluca. 6 Concluson In ths study, we have smulated the nteractons of Noctluca wth water by usng real-tme flow smulatons and 3D moton measurement. Although computatonal cost of flud smulaton s hgh, we can receve the response more quckly usng GPUs effectvely. Furthermore we can operate our applcaton ntutonally measurng user s 3D moton wth stereo camera. We developed the nterestng applcaton, whch s not only the sense of sght, but also hearng, by adoptng sound effects that change accordng to the user s nput. Usng ths applcaton, mysterous and fantastc sensatons that cannot be experenced n daly lfe can be felt. References 1. Geoffrey Irvng, Eran Guendelman, Frank Losasso, Ronald Fedkw, Effcent Smulaton of Large Bodes of Water by Couplng Two and Three Dmensonal Technques,
SIGGRAPH2006, ACM TOG 25, pp.805-811, (2006). 2. Losasso, Tamar Shnar, Andrew Selle, Ronald Fedkw, Multple Interactng Lquds, SIGGRAPH2006, ACM TOG 25, pp.812-819, (2006). 3. Nck Foster and Dmtrs Metaxas. Modelng the moton of a hot, turbulent gas. In SIGGRAPH 97: Proceedngs of the 24th annual conference on Computer graphcs and nteractve technques, pp. 181-188. ACM Press/Addson- Wesley Publshng Co., (1997). 4. Jos Stam. Stable fluds. In Proceedngs of the 26th annual conference on Computer graphcs and nteractve technques, pp. 121-128. ACM Press/Addson-Wesley Publshng Co., (1999). 5. Mark Carlson, Peter J. Mucha, and Greg Turk. Rgd flud: anmatng the nterplay between rgd bodes and flud. ACM Trans. Graph., Vol. 23, No. 3, pp. 377-384, (2004). 6. M. M uller, D. Charypar, and M. Gross. Partcle-based flud smulaton for nteractve applcatons. Proc. of Sggraph Symposum on Computer Anmaton, pages 154-159, (2003). 7. S. Koshzuka and Y. Oka. Movng-partcle sem-mplct method for fragmentaton of ncompressble flud. Nucl.Sc.Eng., 123:421-434, (1996). 8. J.J. Monaghan. Smoothed partcle hydrodynamcs. Annu.Rev.Astrophys, 30:543-574, (1992). 9. P.Kpfer, M.Segal, and R.Westermann. Uberflow: A gpu-based partcle engne. Proceedngs of the ACM SIGGRAPH/EUROGRAPHICS 6 Conference on Graphcs Hardware, pages 115-122, (2004). 10. A.Kolb, L.Latta, and C.Rezk-Salama. hardware based smulaton and collson detecton for large partcle systems. Proceedngs of the ACM SIG- GRAPH/EUROGRAPHICS Conference on Graphcs Hardware, pages 123-131, (2004). 11. Takahro Harada, Sech Koshzuka, Yochro Kawaguch. Smoothed Partcle Hydrodynamcs on GPUs. Computer Graphcs Internatonal, (2007) 12. Alvaro Cassnell, Masatosh Ishkawa. Khronos projector, ACM SIGGRAPH 2005 Emergng technologes, Artcle No. 10, (2005) 13. Takahro Matsuo. Phantasm, ACM SIGGRAPH 2008 art gallery, Pages 101-101, (2008) 14. B.K. Mshra. A revew of computer smulaton of tumblng mlls by the dscrete element method: Part-contact mechancs. Internatonal Journal of Mneral Processng, 71(1):73-93, (2003). 15. Fujsawa T. X., Tan, M., Nagata, N., and Katayose, H. Musc Mood Vsualzaton based on Quanttatve Model of Chord Percepton. J. Informaton Processng Socety of Japan, 50(3), 1133-1138, (2009).