Reasoning About Liquids via Closed-Loop Simulation

Transcription

1 Reasoning Abou Liquids via Closed-Loop Simulaion Connor Schenck and Dieer Fox Absrac Simulaors are powerful ools for reasoning abou a robo s ineracions wih is environmen. However, when simulaions diverge from realiy, ha reasoning becomes less useful. In his paper, we show how o close he loop beween liquid simulaion and real-ime percepion. We use observaions of liquids o correc errors when racking he liquid s sae in a simulaor. Our resuls show ha closed-loop simulaion is an effecive way o preven large divergence beween he simulaed and real liquid saes. As a direc consequence of his, our mehod can enable reasoning abou liquids ha would oherwise be infeasible due o large divergences, such as reasoning abou occluded liquid. (a) Color image (b) Ground Truh (c) Open-loop simulaion (d) Closed-loop simulaion I. I NTRODUCTION Liquids are ubiquious in human environmens, appearing in many common household asks. Recen work in roboics has begun o invesigae ways in which robos can reason abou and manipulae liquids. While some research eams have successfully solved liquid pouring asks using relaively weak models of he physics underlying liquid flow [33, 25, 22], oher work has shown ha physics-based models have he poenial o enable far richer undersanding of acions involving liquids [11]. Physics-based models are very general ools for enabling robos o reason abou heir environmens. Work on rigidbody acions using physics-based models has enabled robos o perform a wide variey of asks [3, 2, 5]. However, o use such models requires racking heir sae using realime percepion. For rigid-body models and deformable objecs such as clohing and owels, here has been a lo of work on racking he modeled sae using sensory feedback [16, 27, 28]. For liquids, hough, here has no ye been any work connecing physics simulaion wih real-ime percepion for roboic asks. Unlike modeling rigid or deformable bodies, modeling liquids is much higher dimensional and lacks he same kind of inheren srucure, and hus small perurbaions can quickly lead o large deviaions. As an example, Figure 1 shows a comparison beween real liquid (Figure 1b) and he resul of performing a carefully uned liquid simulaion wih he same seup (Figure 1c). I is clear ha wihou any feedback, he liquid simulaor and he real liquid have significan differences. In his paper, we invesigae ways o incorporae sensory feedback ino physics-based liquid simulaion. By closing he loop beween simulaion and real-ime observaions, a robo can rack liquids wih much higher accuracy, as illusraed in Figure 1d. Ulimaely, he abiliy o accuraely rack he sae of a liquid will enable a robo o reason abou liquids in a wide variey of conexs, addressing quesions such as How much waer is in his conainer?, Where did his liquid come Acknowledgemen This work was funded in par by he Naional Science Foundaion under conrac numbers NSF-NRI and NSF-NRI Fig. 1: A comparison beween open-loop and closed-loop liquid modeling. The upper-lef shows he color image of he scene for reference and he upper-righ shows he same image wih he acual liquid pixels labeled. The lower wo images show he color image, bu wih he liquid from he simulaor shown. from?, Wha is he viscosiy of his liquid?, or How can I move a specific amoun of his liquid wihou spilling?. Toward his goal, our work only assumes ha he robo can rack 3D mesh models of he objecs in is environmen and can differeniae beween liquid and everyhing else in is camera observaions, boh asks ha have been addressed in prior work [26, 7, 24]. We demonsrae ha our closedloop liquid simulaion enables a robo o reason abou liquids in ways ha were infeasible before, such as esimaing he amoun of waer in an opaque conainer during a pouring ask, or deecing parial obsrucion in waer pipes. In his paper we firs discuss relaed work, followed by a deailed descripion of he liquid simulaor we use as he base for our closed-loop physics-based model. Nex we describe wo differen mehods for using he observaions of real liquid o correc errors in he base liquid simulaor. Afer ha we describe hree experimens we performed using his mehodology and heir resuls. We end he paper wih a discussion of he implicaions of our mehod and fuure work. II. R ELATED W ORK Liquid simulaion and fluid mechanics are well researched in he lieraure [1]. They are commonly used o model fluid flow in areas such as mechanical and aerospace engineering [9], and o model liquid surfaces in compuer graphics [2, 6, 17]. Work by Ladicky e al. [12] combined hese mehods wih regression foress o learn he updae rules for paricles in a paricle-based liquid simulaor. There has also been some work combining real world observaions wih deformable objec simulaion. Schulman e al. [28], by applying forces in he simulaor in he direcion of he gradien of he error beween

2 deph pixels and simulaion, were able o rack cloh based on real observaions. Our warp field mehod, described in secion IV-C, applies a similar concep o liquids. Finally, he only example in he lieraure of combining real observaions wih liquid simulaion is work by Wang e al. [32], which used sereo cameras and colored waer o reconsruc fluid surfaces, and hen used fluid mechanics o make he resuling surface meshes more realisic, alhough hey were limied o making realisic appearing liquid flows raher han using hem o solve roboic asks. In roboics, here has been work using simulaors o reason abou liquids, alhough only in consrained seings, e.g., pouring asks. Kunze and Beez [1, 11] employed a simulaor o reason abou a robo s acions as i aemped o make pancakes, which involved reasoning abou he liquid baer. Yamaguchi and Akeson [35, 34] used a simulaor o reason abou pouring differen kinds of liquids. However, hese works use raher crude liquid simulaions for predicion asks ha do no require accurae feedback. Schenck and Fox [24] used a finie elemen mehod liquid simulaor o rain a deep nework on he asks of deecing and racking liquids. They did no use he simulaor o reason abou perceived liquid, hough. Yamaguchi and Akeson followed up heir simulaed work wih pouring on a real robo [36]. Several ohers have also performed he pouring ask using a real robo [25, 13, 18, 29, 3, 22]. However, mos of hese simply dump he enire conens from he source conainer ino he arge, bypassing he need o reason in any deail abou he liquid dynamics. Only [25, 22] acually aemped o pour specific amouns of liquid, requiring a leas a parial undersanding of liquids on he robo s par. There has been some limied work on perceiving liquids in real daa. Yamaguchi and Akeson [33] used opical flow combined wih sereo vision o perceive liquid flows in 3D. Work by Griffih e al. [8] used liquids o assis a robo in undersanding conainers from sensory daa. In boh [24, 25], hey use deep neworks o boh perceive liquids in color images and o reason abou heir behavior. However, heir deep neworks are limied o he specific seing hey are rained on, and do no have he broad applicabiliy of general liquid simulaors. III. OPEN-LOOP LIQUID SIMULATOR Our physics-based model is based on a liquid simulaor. The sae of he simulaor racks he liquid over ime, simulaing i forward while observaions preven i from deviaing from he real liquid dynamics. In his secion, we describe how he liquid simulaor compues he dynamics of he liquid, and in he following secion we describe how he observaion modifies he liquid sae. To simulae he rajecory of liquid in a scene, he liquid is represened as a se of paricles and he Navier-Sokes equaions [1] are applied o compue he forces on each paricle. The Navier-Sokes equaions require cerain physical properies of liquid (e.g., pressure, densiy) o be defined for all poins in R 3. This is implemened using Smoohed Paricle Hydrodynamics (SPH) [31], which compues he value of a propery a a specific locaion in space as he weighed average of he neighboring paricles. This is in conras o finie elemen liquid simulaions [21], which divide he scene ino a voxel grid and sore he values of he given propery a each locaion in he grid. One major disadvanage of he finie elemen simulaions is ha as he size of he environmen grows, he requiremens of he voxel grid in boh memory and run ime grows as O(n 3 ), making hem inefficien for large environmens wih sparsely locaed liquids. This is he case for he simulaions in his paper, and so we chose o use SPH, which is beer suied o his ype of ask. The implemenaion used in his paper is based off he implemenaion from he paricle simulaion library Fluidix [15]. The res of his secion briefly describes ha implemenaion. Smoohed Paricle Hydrodynamics is essenially a mehod for represening a coninuous vecor field of a physical propery in space via a discree se of paricles. I is based around he following equaion for evaluaing ha field a any arbirary poin in space, where A is he physical propery in quesion: A(r) = A j m j W ( r r j, h) ρ j j where m j is he mass of paricle j, A j is he value sored in paricle j, is he densiy of paricle j, W is a kernel funcion ha weighs he conribuion of each paricle by is disance, and h is he cuoff disance for W. In SPH, he mass m j of each paricle is consan, however he densiy is no, and mus be compued via he SPH equaion above. Tha is, he physical value we wan o compue A is se o be he densiy ρ, which resuls in ρ appearing on he righ side of he equaion wice. The issue of recurrence (requiring he densiy o be known in order o compue he densiy) is handled by he densiy in he denominaor canceling ou: ρ(r) = j m j W ( r r j, h) = j m j W ( r r j, h). To implemen a liquid simulaion using SPH, each paricle mus sore 6 physical properies: 3D posiion (wihou orienaion however since paricles are infiniesimally small poins), velociy, force, mass, densiy, and pressure. As saed above, he mass for each paricle is consan. A each imesep, he force is used o updae he velociy as follows: v +1 i = v i + f i ρ i T where T is he amoun of simulaion ime one imesep corresponds o. The posiion a each imesep is hen updaed by he velociy in a similar manner r +1 i = ri + vi T. The densiy of each paricle a each imesep is compued using he equaion in he previous paragraph. The pressure is compued as p i = c 2 i (ρ i ρ ) where c 2 i is he speed of sound and ρ is he reference densiy of he liquid. The force is compued by summing he conribuions from pressure, viscosiy, graviy, and surface ension. The pressure

3 force a paricle i is defined as: ( p i ρ 2 i f pressure i = m j ρ j j The force due o viscosiy is f viscosiy i µ m j ( v i ρ 2 i + p j ρ 2 j + v j ρ 2 j ) ) W (r i r j ). = 2 W (r i r j ) j where µ is he viscosiy consan of he liquid (recall ha v i is he velociy of paricle i). To compue he surface ension acing on each paricle, we mus firs compue he normal of each paricle: m j W (r i r j ). n i = j Inuiively, he normal n i for any paricle in he cener away from he surface of he liquid will have approximaely equal conribuions from all direcions, resuling in he magniude of n i being small. Conversely, for paricles near he surface, n i will have a large conribuion from paricles in he direcion of he inerior of he liquid and very lile conribuion in he direcion of he surface, resuling in an n i wih a large magniude in he direcion away from he surface. The force due o surface ension is compued as f ension i = σ n i n i j m j 2 W (r i r j ) where σ is he liquid s ension consan. To preven numerical insabiliy when n i is small, we only compue he ension force when he normal magniude is greaer han a hreshold, i.e., he paricle is near he surface. To simulae he flow of liquid in a scene during an ineracion, we assume he simulaor is given 3D models of he objecs ha inerac wih he liquid as well as heir 6D poses over he course of he ineracion (obained for example from an objec racking sysem such as [26]). We iniialize he liquid paricles in he scene (deails on his in secion VI) and simulae he paricles forward a each imesep as he simulaor racks he objecs poses. Our liquid simulaor is implemened using he paricle simulaion library Fluidix [15], which efficienly compues paricle ineracions on he GPU. We performed a bes-firs grid search over he space of parameers (e.g., he viscosiy consan) o find he se of values ha bes mach he real liquid dynamics. For each se of parameers in he grid, we used he evaluaion crieria described in secion V-D o score hem wih respec o he daa we colleced (described in secion V-B), and seleced he parameers ha bes fi he real daa. In doing so, we aemped o make our open-loop simulaion as close as possible o he real liquid dynamics. For efficiency reasons, we use beween 2, and 8, paricles in our experimens. For a deailed derivaion of Smoohed Paricle Hydrodynamics, please refer o [31]. IV. CLOSED-LOOP LIQUID SIMULATORS While liquid simulaors model fluid dynamics based on physical properies, hey ofen don model every possible force ha could affec he liquid; and even he bes simulaors sill have some error relaive o real liquids. Over ime, even small errors can lead o a large divergence beween real and simulaed liquid behavior. While his may no be a problem in some cases (e.g., in 3D animaion i may only be necessary for a liquid o appear realisic), if we wish o use liquid simulaion as a robo s inernal model of is environmen, i mus mach he real liquid behavior as closely as possible. One poenial mehod for alleviaing his issue is o improve he fideliy of he simulaor. However, his mehod has many pifalls. I requires knowledge of every possible force ha could affec he rajecory of he liquid, no only he sandard forces such as pressure and viscosiy, bu also forces for example due o vacuum sucion (as in he case of a plunger), which may require modeling addiional elemens of he environmen. I can also be very brile, as every propery of every objec in he environmen mus be known ahead of ime (e.g., he fricion consans over he enire surface of every objec). Finally, and mos imporanly, even if he simulaor is almos perfecly accurae, he iniial sae of he simulaor migh no be known (e.g., unknown amoun of waer in a cup), and i will sill deviae slighly from realiy and hus accumulae drif, which a purely open-loop sysem has no way o esimae or correc for. We propose wo mehods for dealing wih noise when racking real liquid dynamics using a simulaor. Boh mehods involve closing he loop, ha is, uilizing observaions of real liquid dynamics in order o beer approximae hem in he simulaion. The firs, inspired by sandard Bayes filers in roboics, is a MAP filer, which uses he observaion o correc simulaion errors relaive o he observaion. The second, based on modeling physical forces in he simulaor, applies a warp field ha pulls paricles oward observed liquid. We describe hese wo mehods in he following secions. A. Bridging he Observaion and he Sae Before describing our wo closed-loop mehods, we briefly describe how we map he full 3D sae of he liquid simulaor ino he robo s percepion space. In his work, we assume ha he robo s camera only provides 2D images labeled wih pixel deecions, based on he observaion ha mos liquids, especially waer, are no deeced by deph cameras. A any imesep, he robo s percepion is hus a binary image I, wih pixels labeled as liquid or no-liquid. In order o direcly compare he paricles represening he 3D liquid sae wih he 2D image, he pose of he paricles mus be projeced ino he image. This is done using he following equaion: ([ ] ) 1 x i 1 x i = Ax i where x i is he pose of paricle i a ime, x i is ha pose projeced ono he 2D image plane, and A is he camera inrinsics marix: [ F Lx P P x F L y P P y ] where F L is he focal lengh and P P is he principle poin of he camera. When projecing paricles ino he image plane, we can ake ino accoun occlusions by casing a ray from he paricle s 3D pose ino he camera s 3D pose and checking if i collides wih any of he rigid objecs in he scene. Any paricle whose ray collides wih an objec is no included when updaing he dynamics of he simulaor as here is no way o

4 direcly observe ha paricle. For he paricles ha are no occluded, we can compue he disance in 2D space beween pixels in he image and liquid paricles, which can hen be used o inform he dynamics of he liquid simulaor. Addiionally, we can use his projecion o compue he likelihood of an image, ha is, how well he overall se of liquid paricles explains each of he observed pixels. We define he funcion Φ o be he coverage funcion ha maps a pixel locaion o he number of paricles ha cover ha pixel. To compue his, we place a small, fixed radius sphere a each liquid paricle locaion, hen projec hose spheres back ino he camera, ignoring occluded spheres. The value of Φ a a given pixel locaion is hen simply he number of hese spheres ha projeced ono ha pixel. We use his funcion in boh our closed-loop mehods. B. MAP Filer Simulaor We use a maximum a poseri (MAP) filer as one of our closed-loop simulaion mehods. We model each paricle as is own filer, wih is own se of hypoheses, and use he MAP hypohesis a each ime sep o compue he dynamics. Le P be a se of liquid paricles in a scene a ime, O be he objecs and heir corresponding 6D poses, and I be he observaion. We define S (P 1, O ) = P o be he funcion as described in secion III ha compues he sae of he liquid paricles a imesep given he previous sae of he liquid paricles. A he beginning of each imesep, all he liquid paricles are propagaed forward in ime by one sep via S using he objecs and heir poses O. Since S is deerminisic, we perform he dynamics sampling sep in he filer separaely. Given a liquid paricle x i, we sample one hypohesis paricle for each locaion in a grid cenered a ha liquid paricle s posiion. The grid has dimension and he size of each grid cell is se a a small, fixed consan (we use 5mm in his paper). This resuls in 27 hypoheses sampled for each liquid paricle. x i,n Nex we mus compue P ( x i,n I, P ), he probabiliy of each hypohesis paricle given he observaion and he se of liquid paricles. Here, we mus condiion on all paricles in order o ake ino accoun ha hese paricles may already explain a cerain liquid pixel. We firs apply Bayes rule P ( x i,n I, P ) P (I x i,n, P )P ( x i,n P ). For simpliciy, we use a uniform prior P ( x i,n P ) over all hypohesis paricles ha are feasible, eliminaing hose ha violae physical consrains, such as moving hrough a 3D objec mesh. Thus, for all feasible hypohesis paricles, P ( x i,n I, P ) P (I x i,n, P ). When compuing P (I x i,n, P ), wha we really wan o know, since his is a MAP filer, is which x i,n maximizes his probabiliy. However, he ineracion beween I, x i,n, and P is highly complex and difficul o compue analyically. Insead, we approximae his value wih an acivaion funcion Ψ which we define o be Ψ(I, x i,n, P ) = j liquid(i ) W ( x i,n j, h) Φ(j, P ) + 1 where liquid(i ) is he se of all liquid pixels in I, W is a kernel funcion, x i,n is x i,n projeced ono he image plane (as described in he previous secion), h is he limiing radius for W, and Φ reurns he coverage of j by P (also described in he previous secion). Inuiively, his funcion sums he number of liquid pixels around x i,n, weighed by heir disance o x i,n divided by heir coverage, i.e., how well explained ha pixel is by P. Thus, he more liquid pixels around a hypohesis paricle, he higher is Ψ value, and he less he pixels are covered by he liquid paricles, he higher he Ψ value. For W we use a squared exponenial kernel wih a 1 lengh scale of 33, and we se he limiing radius o 1. 2 Inuiively, his means ha he uni lengh under his kernel is 33 pixels wih a limiing radius of 1 pixels. Finally, we se x i from he MAP hypohesis paricles as follows: x i = argmax x i,n Ψ(I, x i,n, P ). Noe ha we also adjus he velociy of x i o mach he change in posiion from x i 1 so as o preserve he correc momenum. C. Warp Field Simulaor The second mehod we use for closing he loop in he simulaor is a warp field, somewha similar o he approach applied in [28]. Here, he observaion applies a force in he simulaor ha aemps o make he liquid paricles beer mach he observed liquid. Each observaion poin is essenially a magne in he scene, pulling nearby paricles owards i. However, if all observaion poins pulled wih he same amoun of force, hen paricles would end o clump around a subse of he observaion poins, leaving oher observaion poins wih no nearby paricles as he forces from he former cancel ou hose from he laer. Thus, he amoun of force an observaion poin applies o nearby paricles mus vary wih he number of nearby paricles. When aken ogeher, all he observaion poins creae a field of forces ha warp he paricles o beer mach he real liquid observaions. Once again le P be a se of liquid paricles in a scene a ime, O be he objecs and heir corresponding 6D poses, I be he observaion, and S be he funcion ha compues he dynamics of he paricles for a single imesep. The force due o he observaion warp field is compued as f i,obs = λ j liquid(i ) u ij Φ(j, P ) + 1 W ( xi j, h) where λ is he warp consan, liquid(i ) is he se of all liquid pixels in I, u ij is a uni vecor poining from paricle x i (projeced ono he image plane as described in secion IV-A) o liquid pixel j, Φ(j, P ) is he coverage of pixel j (described in secion IV-A) and W is he same kernel funcion used in he MAP simulaor (wih same parameers). The warp consan λ adjuss he srengh of he warp force, wih higher values resuling in a higher warp force and lower values in a lower force. Again, he coverage of a pixel Φ(j, P ) is a measure of how many liquid paricles cover i, ha is, how many liquid paricles are nearby. The force applied o each paricle by each liquid pixel is divided by ha pixel s coverage, hus as more

5 (a) Cup (b) Bole (c) Pipe Juncion (a) Unblocked (b) Parial (c) Blocked Fig. 3: The 3 ypes of blockages placed in he pipe juncion. (lef o righ) Pipe juncion wih no blockage; lef leg is parially blocked; and lef leg is fully blocked. (d) Pan (e) Bowl (f) Frui Bowl Fig. 2: Objecs used during he experimens. The op row shows he wo conainers he robo poured from as well as he pipe juncion. The lefmos bowl in he boom row was used in he pouring and he righ wo were used during he pipe juncion experimens. paricles cover an observed liquid pixel, i pulls paricles o i wih less force. Conversely, pixels ha have lower coverage pull paricles o hem wih more force, hus encouraging he simulaor o move paricles so as o fill he conour of he observed liquid. i,obs f The force is hen projeced back ino 3D space. This is done by applying he inverse of he projecion described in secion IV-A. Because his is 2D o 3D, he projecion has an unspecified degree of freedom. To compensae for his, we assume ha he force vecor is in a plane parallel o he image plane in 3D space. Finally, we apply he SPH equaion o smooh he forces across he paricles f i,obs f j,obs m j W ( r i r j, h). = j i,obs The resuling force f is hen added o he oher forces described in secion III and S is compued as normal. A. Robo & Sensors V. EXPERIMENTAL SETUP The robo used in he experimens in his paper was an upper-orso robo wih wo 7-DOF arms, each wih an elecric parallel gripper. A able was fixed in fron of he robo. To sense is environmen, he robo used is Asus Xion Pro RGBD camera, which recorded boh color and deph images a resoluion a 3 Hz during each ineracion, and is Infrared Cameras Inc. 864P Thermal Imaging camera, which recorded hermographic images a resoluion a 3 Hz during each ineracion. The hermal camera was used in combinaion wih heaed waer o acquire he ground ruh pixel labelings. The cameras were locked in fixed relaive posiions and placed jus below he robo s head a approximaely ches heigh. B. Daa Collecion 1) Pouring: We colleced 16 pouring ineracions. We varied he source conainer (cup, Figure 2a, or bole, Figure 2b) and is iniial fill amoun (empy, 3%, 6%, or 9% full). Before each pouring ineracion, a bowl (he pan, Figure 2d) was placed on he able in fron of he robo. Nex he source was placed in he robo s gripper, filled wih waer, and he gripper moved over he bowl. The robo hen proceeded o roae i s wris along a fixed rajecory such ha he opening of he conainer iled down owards he bowl and waer poured ou. During each pouring ineracion, he robo recorded from is RGBD and hermal cameras as well is join poses. We colleced wo rials for each combinaion of source conainer and fill amoun. 2) Pipe Juncion: We colleced 5 pipe juncions ineracions. Before each of he pipe juncion ineracions, wo bowls (bowl, Figure 2e, and frui bowl, Figure 2f) were placed sideby-side on he able in fron of he robo. Nex, he robo held he ends of he pipe juncion (Figure 2c) wih is grippers over he bowls and recorded from is RGBD and hermal cameras while 1 lier of waer was poured in he op opening. Each leg of he pipe juncion could be fully blocked or parially blocked, i.e., he flow going o ha leg could be parially resriced or enirely sopped. A diagram of he pipe juncion and how he blockages affeced flow is shown in Figure 3. The blockage can be placed in eiher leg, for a oal of 5 possible configuraions. C. Daa Processing Before we can use our simulaors o rack he flow of liquid in he ineracions described in he previous secion, we mus firs perform some pos-processing on he daa. Firs, boh he open-loop and closed-loop simulaors require he objec poses o be known over he course of he ineracion. We uilize an objec racking mehod based on poin cloud daa o do his. Second, boh closed-loop simulaors require an image wih pixels labels for he liquid. We use a hermal camera o acquire his labeling. In his paper we perform hese seps offline, however boh are capable of operaing in real-ime in online siuaions. 1) Objec Tracking: We use he sofware program DART [26] (Dense Ariculaed Real-Time Tracking) o rack he objecs in each ineracion. DART uses deph images o rack objecs over ime. We iniialize he pose of he bowls by using he Poin Cloud Library s [23] buil-in ableop segmenaion algorihm o find he poin cluser on he able, and hen se heir iniial pose o he cenroid. We iniialize he conainers by compuing he robo s forward kinemaics o find he gripper pose. Once iniialized, DART reurns a pose for each objec a each poin in ime over he ineracion. 2) Liquid Labeling: For each pouring and pipe juncion ineracion, he waer was heaed o a emperaure significanly above he surrounding environmen bu below is boiling poin. The ineracions were recorded wih a hermal camera, and he hermal image was simply hresholded o locae he liquid pixels. Figure 4b shows an example hermal image recorded

6 (a) RGB (b) Thermal (c) Threshold (d) Overlay Fig. 4: Acquiring liquid labels from he hermal camera. The upperlef is a color image of he scene, he upper-righ shows he corresponding hermal image ransformed o he color image s space. The lower-lef image shows he liquid labels acquired via hresholding he hermal image, and he lower-righ shows he labels overlayed on he color image. during a pipe juncion ineracion, and Figure 4c shows is corresponding hresholded values. In addiion o generaing labels from he hermal image, i mus also be calibraed o he deph image (he objec poses generaed by DART, and hus he enire simulaor, operae in he deph camera frame of reference). Tha is, for each pixel in he hermal camera, we mus deermine which pixel in he deph camera i corresponds o. This is no as simple as i may appear. Waer is no visible in he deph image as he projeced infrared ligh does no reflec properly off he surface. However, our deph camera also collecs color images and calibraes i o he deph frame auomaically. We can use he color image hen o calibrae he hermal camera. While here exis mehods for doing a full regisraion beween color and hermal images [19], hese end o be noisy and unreliable. In his paper, because he waer remains a a fixed disance from he camera, we use a simpler soluion. Firs we ake a checkerboard paern prined on a wooden board and place i under a high-inensiy halogen lamp. The ligh and dark paern on he board absorbs ligh from he lamp a differen raes, causing he dark squares o hea faser han he ligh squares. We hen hold his board in fron of boh he hermal and color cameras a he same disance as he waer. The differenial heaing causes he checkerboard paern o be visible in boh cameras, allowing us o find correspondence poins beween he wo images. We hen use hese poins o compue an affine ransformaion beween he images, and use i o ransform he hermal image ono he color image. Figures 4a and 4b show an example color image and is corresponding hermal image ransformed ono he color space (he hermal camera has a narrower field of view han he color camera, which is why here are no hermal values around he edge of Figure 4b). Figure 4d shows he hresholded hermal image overlayed ono he color image. D. Evaluaion Crieria We use wo crieria for evaluaing our mehodology. The firs is inersecion over union (IOU). In his case, he sae of he liquid simulaion is projeced ino he camera by placing small spheres a each paricle locaion and projecing hose ino he camera, aking ino accoun occlusions by objecs. We hen compare he se of pixels labeled as liquid by his projecion o he se of pixels labeled as liquid by he hermal image. The IOU is simply he inersecion of hese wo ses divided by he union. When comparing he probabiliy of muliple simulaions for he purposes of esimaing hidden sae, we use P (I π Iπ ) MAP Filer 73.38% 77.12% 65.22% 79.85% 8.69% 76.3% Warp Field 75.94% 79.41% 67.1% 82.8% 83.22% 78.41%.8 Open Loop MAP Filer Warp Field.6 IOU Cup Bole 3% 6% 9% Overall Open Loop 6.17% 67.25% 35.56% 77.62% 77.94% 65.66% Timesep Fig. 5: The able shows he IOU for each mehod. The graph shows he IOU a every imesep across one of he pouring experimens (bole filled o 3%). where I π is a se of prediced images for ineracion π, and Iπ is he se of ground ruh images. To compue his, we firs apply Bayes rule P (I π Iπ ) P (Iπ I π )P (I π ). For our experimens, we assume he prior P (I π ) is uniform. To compue P (Iπ I π ), we assume each pixel is independen and simply muliply heir individual probabiliies ogeher T Y Y P (Iπ I π ) = P (j j ) =1 j where we se P (j j ) equal o δ if j and j are equal (boh liquid or boh no-liquid), and o 1 δ if hey are no. Due he he large number of pixels across all images and imeseps, we se δ =.51 o preven underflow1. Afer compuing he probabiliies, we hen normalize hem so hey sum o 1. VI. E XPERIMENTS & R ESULTS We ran hree experimens o evaluae our simulaors a racking he sae of real-world liquids. The firs uilized he pouring ineracions and focused on quaniaively evaluaing he open and closed loop simulaors. The second and hird experimens es our simulaion mehods a esimaing he sae of an unknown variable in he environmen. This is an imporan abiliy for a robo, as ofen liquids are occluded by conainers or oher objecs, forcing robos o reason abou he hidden sae of he liquids based on oucomes during an ineracion, somehing ha is no always necessary during rigid objec ineracions. Our second wo experimens examine wo differen cases of hidden sae esimaion using liquids. A. Comparing Open and Closed Loop Simulaion Mehods To compare each of he hree simulaion mehods (open loop, MAP filer, and warp field), we simulaed hem on he daa colleced for each pouring ineracion. A he sar of each ineracion, we fill he 3D model of he conainer wih he same amoun of liquid as was filled in he real conainer. To do his, we perform binary search on he iniial number of paricles, running he simulaion forward, holding he objec poses consan, unil each has seled and hen compuing he level of he liquid in he conainer. We hen simulae he liquid forward in ime, updaing he objec poses based on he racked poses acquired using DART. We evaluae each mehod by comparing heir IOUs, compued as described in secion V-D2. 1 Even in log-space, values would sill periodically underflow wih higher values for δ due o he large quaniy of pixels. 2 The 4 pouring ineracions where he conainer was lef empy were no included in his analysis because he union par of he IOU would be, resuling in a division by.

7 Open Loop MAP Filer Warp Field *Empy 3% 6% 9% Empy 3% *6% 9% Empy *3% 6% 9% Empy 3% 6% *9% Fig. 6: Probabiliy disribuion over he esimaed iniial fill amouns. They are aggregaed by he rue fill amouns. From op o boom hey are empy, 3% full, 6% full, and 9% full (indicaed by he *). The blue bars show resuls from he open loop mehod, cyan for he MAP filer, and red for he warp field. The IOU for he hree simulaion mehods is shown in he able in Figure 5. The upper wo rows show he IOU for he mehods condiioned on he wo ypes of conainers used; The middle rows show he IOU condiioned on he iniial percen full of he conainer; and he las row shows he overall IOU for each mehod. This able reveals some ineresing phenomena. I is no immediaely clear why all he simulaors seem o perform slighly beer on ineracions where he robo poured from he bole raher han he cup. However, he middle of he able shows ha all of he mehods end o perform beer when more liquid is involved. We noice ha he bole, while having a similar diameer as he cup, is aller, meaning if hey are filled o he same raio full (e.g., 3%), hen he bole will have more overall liquid. This explains he sligh bump in performance from one conainer o he oher. The mos imporan revelaion, however, is ha boh closedloop simulaion mehods ouperform he open-loop simulaion by a significan margin. This is illusraed graphically by he graph on he righ in Figure 5, which shows he IOU a every imesep over one sequence, and clearly shows ha he closedloop mehods are beer able o mach he locaion of he real liquid han he open-loop mehod. Addiionally, boh he able and he graph show ha he warp field mehod ouperforms he MAP filer mehod. This clearly shows ha closing he loop in liquid simulaions can make he rajecory of he liquid beer mach real world liquid dynamics. B. Esimaing he Iniial Amoun of Liquid We evaluaed all hree simulaion mehods on he same hidden sae ask. For each pouring ineracion, he iniial amoun of liquid in he conainer was no given o he robo. Insead, he ask of he robo was o esimae his amoun based on he observaions and is own liquid simulaions. To do his, he robo needs o run muliple simulaions for each ineracion, one for each possible fill amoun, and compare he predicions of each simulaion o he observaion. For each pouring ineracion, he robo ran 4 simulaions: one where he conainer was lef empy, one where he conainer was filled o 3% full, one where he conainer was filled o 6% full, and one where i was filled o 9% full. For each simulaion, he liquid paricles are simulaed forward in ime as he objec poses are updaed via heir racked poses. We compue he probabiliy of each simulaion by evaluaing he probabiliy of heir prediced images as described in secion V-D. Figure 6 shows he resuls of performing his for each of he pouring ineracions, aggregaed by he ground ruh fill amoun (indicaed by he * in he x-axis of each graph). The blue bars show he probabiliy disribuions for he open-loop mehod, he cyan bars show he disribuion for he MAP filer mehod, and he red bars show he disribuion for he warp field mehod. All mehods are easily able o correcly place he highes probabiliy on he empy simulaion when here is in fac no liquid in he ineracion, which follows inuiion as here are no observed liquid paricles. Addiionally, even hough here is slighly more confusion, all of he mehods place he highes probabiliy on he 9% simulaion when he conainers sar ou 9% full. Again, his aligns wih inuiion as i is easy o disinguish a lo of liquid from almos no liquid. The mos confusion occurs when rying o disinguish a lile (3%) from some (6%). The open loop mehod is almos compleely unable o disinguish beween he wo, boh disribuions being very similar. The MAP filer mehod is slighly beer, bu sill ges confused when he rue amoun in he conainer is 6%. Only he warp field mehod is able o correcly esimae he iniial amoun of liquid, placing over 7% probabiliy on he correc simulaion in every case. C. Solving he Pipe Juncion Task The final experimen we performed was he pipe juncion ask. Here he ask is for he robo o find he blockage in a pair of conneced pipes simply by observing he liquid as i exis he pipes, a siuaion he robo may find iself in if, say, rying o diagnose a broken sink. We assume ha he robo knows a priori he defaul, unblocked flow rae of liquid hrough he pipes, and hus mus use he change in flow o find he blockage. To es his, a pipe T-juncion was held invered over wo bowls such ha he legs of he T empied ino differen bowls, boh visible o he robo. However, he ask is o find he blockage based only on he oupu of he pipes, so he T-juncion was held high enough so ha he robo could only see he openings on he boom and no he op opening. To simulae a consan flow ino he pipes, a conainer wih exacly 1 lier of waer was iled a a consan angular velociy so ha he liquid flowed ino he op opening of he juncion. The ype of blockage used (if any), unblocked, parially blocked, or blocked, was placed inside he pipe, no visible o he robo. We used he daa colleced during he pipe juncion ineracions o evaluae he robo on his ask. To solve his ask, like in he previous experimen, he robo needs o run muliple simulaions wih differen values for he hidden sae (he pipe blockages) and compare heir oucomes. For each ineracion, he robo ran 5 simulaions: one for boh legs unblocked, one for he righ leg parially blocked, one for he righ leg fully blocked, one for he lef leg parially blocked, and one for he lef leg fully blocked. The probabiliy of each simulaion is compued using he mehod described in secion V-D. Figure 7 shows he probabiliy for each of he simulaed blockages over ime for one of he ineracions using he bes closed-loop mehod (warp field). The robo ran one simulaion for each blockage ype, and he diagrams across he op of he figure indicae where he blockage in ha simulaion was

8 Likelihood Waer Firs Becomes Visible Timesep Fig. 7: Probabiliy disribuion over he blockage locaion over ime for a single ineracion. The 5 diagrams across he op correspond o he five differen simulaions he robo ran, each color-coded o he corresponding line in he plo. The rue blockage was placed in he lef leg and only parially blocked he leg (in he keys in he op row, second from righ). Bes viewed in color. placed. The color bordering each diagram corresponds o he color of he line indicaing ha simulaions probabiliy over ime. Afer only a shor ime window, he robo is able o place 1% probabiliy on he correc blockage (parial-lef). Indeed, we ran his on all 5 pipe juncion ineracions, and by he end of each, he robo had placed 1% on he correc blockage in every case. We also evaluaed he 5 ineracions using he open-loop mehod. I was able o correcly esimae wih 1% probabiliy in he simpler cases (no blockage or fully blocked) as would be expeced. However, for he more difficul ineracions (parial blockage), i only picked he correc blockage ype and locaion in one case (when he rue blockage was parial-lef) and in he oher case incorrecly placed 1% probabiliy on here being no blockage. While he poin of his experimen was o show he possible ype of reasoning ha can be done wih full physics-based liquid models, even here he closed-loop mehods ouperform he open-loop mehods, if only in 1 ou of 5 cases. Regardless, by using he closed-loop liquid simulaion mehods developed here, he robo is clearly able o robusly solve his ask. VII. DISCUSSION Reasoning abou Liquids: So far, reasoning abou liquids applied o real robos has been limied o resriced asks such as pouring [25, 22, 3]. Wih our physics-based model, reasoning abou liquids can be done on a much wider variey of asks. The las wo experimens in his paper boh involve compleely differen asks, one reasoning abou pouring, he oher abou blockages in pipes, ye he same algorihm is able o solve boh asks, wihou any special knowledge aside from generic 3D models. Anoher advanage of our mehod over mehods such as a deep learning approach [24] or even a nonphysics model-based approach [33] is ha he persisence of a liquid is rivially inferred. For example, a robo using his model could observe a pouring ineracion, and i would be immediaely obvious ha he new liquid in he arge conainer originaed in he source conainer, and ha he overall liquid is he same a he end of he pour as i was a he beginning. Generalizing o Oher Liquids: Anoher advanage of a physics-based model is ha i can generalize o differen ypes of liquid. Yamaguchi and Akeson [33] developed a modelbased deecor ha could deermine he locaion of liquids in a scene, and hey showed ha i could generalize o a wide array of liquid ypes. This is unlike learning-based models, which canno generalize o liquids oo differen from heir raining se. Wih he aleraion of a few physical parameers, a physicsbased model can generalize o liquids as diverse as waer, oil, honey, and even dough. I is currenly an open challenge as o how o infer hese parameers efficienly from observaion. Predicing Liquid Behavior: While ohers have used physics-based models for liquids [11], none have ye combined hem wih real percepion. As a resul, due o he quick divergence of open-loop models wih realiy, here has been lile prior work exploring he possible acion spaces around liquids. Closed-loop liquid simulaions enable robos o use he same model o inerac wih liquids in a wide variey of seings, such as carrying a conainer across a room wihou spilling is conained liquid, scooping liquid wih a spoon, and ejecing liquid from a syringe in a conrolled manner. Wihou closed-loop liquid simulaions, each of hese asks would require developing a separae model. Using an algorihm such as model predicive conrol [4], he robo could plan for a shor ime horizon ino he fuure using he open-loop simulaion, bu rack he curren sae using he closed-loop simulaion, hus prevening a faal divergence from realiy. VIII. CONCLUSION In his paper, we proposed wo mehods for racking he sae of liquid wih a closed-loop simulaor. The firs, inspired by Bayes filer echniques in roboics, used a MAP filer o correc errors in he simulaor. The second, inspired by he physical forces underlying he simulaor, applied a warp field o he paricles o correc he error. The resuls clearly show ha boh our closed-loop mehods are beer a racking he liquid han he open-loop mehod. We also showed how hese closed-loop simulaions can be used o reason abou and infer he hidden variables of an ineracion involving liquids. To our knowledge, his is he firs ime real liquid observaions have been combined wih liquid simulaions for roboics asks. In he immediae fuure, we plan o coninue his work along muliple avenues of invesigaion. In his paper, we uilized a hermal camera o acquire liquid deecions o focus he evaluaion on our experimenal mehodology. In he fuure, we plan o combine our mehodology wih deep learning mehods like he ones in [24, 14] o perceive liquids, bypassing he need for a hermal camera. Deep learning can also be applied o perform sysem idenificaion, i.e., o learn he correc physics models and updae hem in real-ime based on percepion. This migh addiionally enable more efficien simulaion, allowing he use of more paricles. Our curren sysem requires running a separae simulaor for each hidden sae, making i hard o scale o more complex scenarios. One ineresing quesion is how o bes incorporae independencies beween muliple conainers of liquid in order o improve scaling. Addiionally, we also plan o apply our mehodology o solving closedloop conrols asks wih real liquids, somehing which was difficul or impossible before. Finally, we plan o make our daa publicly available o oher researchers.

9 REFERENCES [1] David J Acheson. Elemenary fluid dynamics. Oxford Universiy Press, 199. [2] Rober Bridson. Fluid simulaion for compuer graphics. CRC Press, 215. [3] Maya Cakmak and Andrea L Thomaz. Designing robo learners ha ask good quesions. In ACM/IEEE Inernaional Conference on Human-Robo Ineracion (HRI), pages 17 24, 212. [4] Eduardo F Camacho and Carlos Bordons Alba. Model predicive conrol. Springer Science & Business Media, 213. [5] Nilanjan Chakrabory, Sephen Berard, Srinivas Akella, and Jeffrey C Trinkle. A geomerically implici imesepping mehod for mulibody sysems wih inermien conac. The Inernaional Journal of Roboics Research, 33(3): , 214. [6] Simon Clave, Philippe Beaudoin, and Pierre Poulin. Paricle-based viscoelasic fluid simulaion. In Proceedings of he 25 ACM SIGGRAPH/Eurographics symposium on Compuer animaion, pages ACM, 25. [7] Crisina Garcia Cifuenes, Jan Issac, Manuel Wührich, Sefan Schaal, and Jeannee Bohg. Probabilisic ariculaed real-ime racking for robo manipulaion. IEEE Roboics and Auomaion Leers (RA-L), 216. [8] Shane Griffih, Vladimir Sukhoy, Todd Weger, and Alexander Soychev. Objec caegorizaion in he sink: Learning behavior grounded objec caegories wih waer. In Proceedings of he 212 ICRA Workshop on Semanic Percepion, Mapping and Exploraion. Cieseer, 212. [9] Philip G Hill and Carl R Peerson. Mechanics and hermodynamics of propulsion. Reading, MA, Addison- Wesley Publishing Co., 1992, 764 p., 1, [1] Lars Kunze. Naïve Physics and Commonsense Reasoning for Everyday Robo Manipulaion. PhD hesis, Technische Universiä München, 214. [11] Lars Kunze and Michael Beez. Envisioning he qualiaive effecs of robo manipulaion acions using simulaion-based projecions. Arificial Inelligence, 215. [12] L ubor Ladický, SoHyeon Jeong, Barbara Solenhaler, Marc Pollefeys, and Markus Gross. Daa-driven fluid simulaions using regression foress. ACM Transacions on Graphics (TOG), 34(6):199:1 199:9, 215. [13] Joshua D Langsfeld, Krishnanand N Kaipa, Rodolphe J Genili, James A Reggia, and Sayandra K Gupa. Incorporaing failure-o-success ransiions in imiaion learning for a dynamic pouring ask. In IEEE Inernaional Conference on Inelligen Robos and Sysems (IROS) Workshop on Complian Manipulaion, 214. [14] Jonahan Long, Evan Shelhamer, and Trevor Darrell. Fully convoluional neworks for semanic segmenaion. In IEEE Inernaional Conference on Compuer Vision and Paern Recogniion (CVPR), pages , 215. [15] Adam Macdonald. Fluidix. OneZero Sofware, Canada, 217. URL hp:// [16] Igor Mordach, Kendall Lowrey, and Emanuel Todorov. Ensemble-cio: Full-body dynamic moion planning ha ransfers o physical humanoids. In Inelligen Robos and Sysems (IROS), 215 IEEE/RSJ Inernaional Conference on, pages IEEE, 215. [17] Mahias Müller, David Charypar, and Markus Gross. Paricle-based fluid simulaion for ineracive applicaions. In Proceedings of he 23 ACM SIG- GRAPH/Eurographics symposium on Compuer animaion, pages Eurographics Associaion, 23. [18] Kei Okada, Misuharu Kojima, Yuichi Sagawa, Toshiyuki Ichino, Kenji Sao, and Masayuki Inaba. Vision based behavior verificaion sysem of humanoid robo for daily environmen asks. In IEEE-RAS Inernaional Conference on Humanoid Roboics (Humanoids), pages 7 12, 26. [19] Peer Pinggera, Toby Breckon, and Hors Bischof. On cross-specral sereo maching using dense gradien feaures. In IEEE Conference on Compuer Vision and Paern Recogniion (CVPR), 212. [2] Michael Posa, Cecilia Canu, and Russ Tedrake. A direc mehod for rajecory opimizaion of rigid bodies hrough conac. The Inernaional Journal of Roboics Research, 33(1):69 81, 214. [21] Junuhula Narasimha Reddy. An Inroducion o Nonlinear Finie Elemen Analysis: wih applicaions o hea ransfer, fluid mechanics, and solid mechanics. OUP Oxford, 214. [22] Leonel Rozo, Pedro Jimenez, and Carme Torras. Forcebased robo learning of pouring skills using parameric hidden markov models. In IEEE-RAS Inernaional Workshop on Robo Moion and Conrol (RoMoCo), pages , 213. [23] Radu Bogdan Rusu and Seve Cousins. 3d is here: Poin cloud library (pcl). In ICRA, 211. [24] Connor Schenck and Dieer Fox. Towards learning o perceive and reason abou liquids. In Proceedings of he Inernaional Symposium on Experimenal Roboics (ISER), 216. [25] Connor Schenck and Dieer Fox. Visual closed-loop conrol for pouring liquids. In Proceedings of he Inernaional Conference on Experimenal Roboics (ICRA), 217. [26] Tanner Schmid, Richard A Newcombe, and Dieer Fox. Dar: Dense ariculaed real-ime racking. In Roboics: Science and Sysems, 214. [27] Tanner Schmid, Kaharina Herkorn, Richard Newcombe, Zolan Maron, Michael Suppa, and Dieer Fox. Deph-based racking wih physical consrains for robo manipulaion. In Roboics and Auomaion (ICRA), 215 IEEE Inernaional Conference on, pages IEEE, 215. [28] John Schulman, Alex Lee, Jonahan Ho, and Pieer Abbeel. Tracking deformable objecs wih poin clouds. In Roboics and Auomaion (ICRA), 213 IEEE Iner-

10 naional Conference on, pages IEEE, 213. [29] Minija Tamosiunaie, Bojan Nemec, Aleš Ude, and Florenin Wörgöer. Learning o pour wih a robo arm combining goal and shape learning for dynamic movemen primiives. Roboics and Auonomous Sysems, 59 (11):91 922, 211. [3] Yuval Tassa, Tom Erez, and Emanuel Todorov. Synhesis and sabilizaion of complex behaviors hrough online rajecory opimizaion. In Inelligen Robos and Sysems (IROS), 212 IEEE/RSJ Inernaional Conference on, pages IEEE, 212. [31] Damien Violeau. Fluid Mechanics and he SPH mehod: heory and applicaions. Oxford Universiy Press, 212. [32] Huamin Wang, Miao Liao, Qing Zhang, Ruigang Yang, and Greg Turk. Physically guided liquid surface modeling from videos. In ACM Transacions on Graphics (TOG), volume 28, page 9. ACM, 29. [33] Akihiko Yamaguchi and Chrisopher Akeson. Sereo vision of liquid and paricle flow for robo pouring. In Proceedings of he Inernaional Conference on Humanoid Roboics (Humanoids), 216. [34] Akihiko Yamaguchi and Chrisopher Akeson. Differenial dynamic programming for graph-srucured dynamical sysems: Generalizaion of pouring behavior wih differen skills. In Proceedings of he Inernaional Conference on Humanoid Roboics (Humanoids), 216. [35] Akihiko Yamaguchi and Chrisopher G Akeson. Differenial dynamic programming wih emporally decomposed dynamics. In IEEE-RAS Inernaional Conference on Humanoid Roboics (Humanoids), pages , 215. [36] Akihiko Yamaguchi and Chrisopher G Akeson. Neural neworks and differenial dynamic programming for reinforcemen learning problems. In IEEE Inernaional Conference on Roboics and Auomaion (ICRA), pages , 216.