Featue Aticle Digital Route Panoamas Route panoama is a new image medium fo digitally achiving and visualizing scenes along a oute. It s suitable fo egistation, tansmission, and visualization of oute scenes. This aticle exploes oute panoama pojection, esolution, and shape distotion. It also discusses how to impove quality, achieve eal-time tansmission, and display long oute panoamas on the Intenet. Jiang Yu Zheng Indiana Univesity Pudue Univesity, Indianapolis In ancient westen at, compositions showed the pespective of a scene s pojection towad a 2D view plane. In contast, an ancient oiental painting technique exposed scenes of inteest at diffeent locations in a lage field of view, which let viewes exploe diffeent segments by moving thei focus. A typical and famous example is a long painting scoll named A Cathay City (see Figue 1), painted 900 yeas ago. It ecoded the pospeity of the capital city of ancient China in the Song Dynasty by dawing scenes and events along a oute fom a subub to the inne city. The invention of pape in ancient China allowed most paintings to be ceated on foldable and scollable papes instead of canvases. As opposed to a single-focus pespective pojection at one end of a steet o a bid s eye view, one benefit of this scollable style is that a pathoiented pojection can display moe detailed visual infomation in an extended visual field than a single-focus pespective pojection at one end of a steet o a bid s-eye view fom the top. Today we can ealize a composition appoach simila to A Cathay City by ceating a new image medium Route Panoama, a pogam that contains an entie scene sequence along a oute. We geneate long oute panoamas by using digital image-pocessing techniques and ende oute views continuously on the Intenet. Now we can captue, egiste, and display oute panoamas fo many miles taken fom vehicles, tains, o ships. We can use this appoach fo many pactical puposes, including pofiles of cities fo visitos o fo intoductoy indexes of local hometowns. In an extended manne, we could pobably even use it as pat of something like an enhanced vesion of Mapquest fo people navigating though lage cities like Los Angeles o Beijing. Figue 2 (next page) shows an example of a oute panoama fom a one-hou video in Venice, a place whee people often get lost. Route scenes on the Intenet Thee ae many things to conside when ceating a quality oute panoama. To begin with, we ceate a oute panoama by scanning scenes continuously with a vitual slit in the camea fame to fom image memoies. We call it a vitual slit because it isn t an actual lens cove with a slit in it; athe, fo each camea image in the video sequence, we e copying a vetical pixel line at a fixed position. We paste these consecutive slit views (o image memoies) togethe to fom a long, seamless 2D image belt. We can then tansmit the 2D Figue 1. A Cathay City painted on an 11 m scoll in the Song Dynasty 900 yeas ago. (Fom the collection of the National Palace Museum, Taipei, Taiwan, China.) 1070-986X/03/$17.00 2003 IEEE Published by the IEEE Compute Society 57
Figue 2. Segment of oute panoama ecoding canal scenes in Venice. IEEE MultiMedia image belt via the Intenet, enabling end uses to easily scoll back and foth along a oute. The pocess of captuing a oute panoama is as simple as ecoding a video on a moving vehicle. We can ceate the oute panoama in eal time with a potable PC inside the vehicle o by eplaying a ecoded video taken duing the vehicle s movement and inputting it into a compute late. Nevetheless, the geneated image belt with its consecutive slit views pieced togethe has much less data than a continuous video sequence. My colleagues and I fist invented a oute panoama 10 yeas ago fo mobile obot navigation. 1-3 We called it a genealized panoamic view because we discoveed it while ceating the fist digital panoamic view. In this ealy study, we mounted a camea on a moving vehicle and it constantly captued slit views othogonal to the moving diection. The oute panoama is a special case of a moe geneal epesentation called a dynamic pojection image, 4 which foms the 2D image using a nonfixed, tempoal pojection. Figue 3 (next page) illustates how a vetical slit can scan the steet scenes displayed in Figue 2 when the camea is positioned sideways along a smooth path. This viewing scheme is an othogonal-pespective pojection of scenes othogonal towad the camea path and pespective along the vetical slit. Geneally speaking, common modes of tanspotation such as a fouwheeled vehicle, ship, tain, o aiplane can povide a smooth path fo the camea. Compaed with existing appoaches to model a oute using gaphics models, 5,6 ou oute panoama has an advantage in captuing scenes. It doesn t equie taking discete images by manual opeation o textue mapping onto geomety models. A oute panoama can be eady afte diving a vehicle in town fo a while. It yields a continuous image scoll that othe image stitching o mosaic appoaches, 7-9 in pinciple, ae impossible to ealize. A mosaicing appoach woks well fo stitching images taken by a camea otating at a static position. Fo a tanslating camea, howeve, scenes with diffeent depths have diffeent dispaities (o optical flow vectos) in consecutive images. Ovelapping scenes at one depth will add layes and ceate dissonant scenes at othe depths, much like ovelapping steeo images. A oute panoama equies only a small faction of data compaed to a video sequence and has a continuous fomat when accessed. If we pile a sequence of video images togethe along the time axis, we obtain a 3D data volume called spatial tempoal volume that s full of pixels (see Figue 4, p. 60). The oute panoama compises pixel lines in consecutive image fames, which coespond to a 2D data sheet in the spatial tempoal volume. Ideally, if the image fame has a width w (in pixels), a oute panoama only has 1/w of the data size of the entie video sequence (w is 200~300), since we only extact one pixel line fom each fame when viewing though the slit. The oute panoama neglects edundant 58
scenes in the consecutive video fames, which we can obseve fom the taces of pattens in the epipola plane images (Figue 4 shows one EPI). This shows a pomising popety of the oute panoama as an Intenet medium, which can delive lage amounts of infomation with small data. The missing scenes ae objects unde occlusion when exposed to the slit. Also, as nomal 2D images captuing dynamic objects, a oute panoama only feezes instantaneous slit views, athe than captuing object movements as video does. Figue 5 (see p. 60) is anothe example of a oute panoama. It displays a segment of the Fist Main Steet of China. We used a small digital video camea on a double-decke bus to ecod the 5-km oute panoama. No image device othe than an image gabbe is equied in pocessing. (a) Buildings Route panoama S Camea Slit Camea tace Viewing plane (plane of sight) Pojecting scenes Compaed with existing media such as photo, video, and panoamic views, oute panoama has its own pojection popeties. Fo instance, much can depend on the smoothness of the ide while the camea is ecoding. In an ideal situation oads ae flat and the vehicle has solid suspension, a elatively long wheelbase, and tavels at a slow speed. Unde these conditions, the camea has less up-and-down tanslation and can tavese a smooth path along a steet. In this aticle, we assume the camea axis is set hoizontally fo simplicity. We define a pathoiented coodinate system SYD fo the pojection of scenes towad a smooth camea tace S on the hoizontal plane. We denote a point in such a coodinate system as P(S, Y, D), whee S is the camea-passed length fom a local stating point (see Figue 3b), Y is the height of the point fom the plane containing the camea path, and D is the depth of the point fom the camea focus. Assume t and y ae the hoizontal and vetical axes of the oute panoama, espectively. The pojection of P in the oute panoama denoted by p(t, y) is t = S/ y = Yf/D (1) whee (m/fame) is the slit sampling inteval on the camea tace, which is the division of the vehicle s speed V (m/second) by the camea s fame ate pe second (fps). The camea s focal length f (in pixels) is known in advance afte calibation. Because the oute panoama employs an othogonal-pespective pojection, the aspect atio of an object depends on its distance fom y (b) Camea motion Vitual slit (pixel line) Camea path Route panoama in long memoy Image fame the path. Figue 6 displays a compaison of views between an odinay pespective image and a oute panoama. The futhe the object, the lowe the object height is in the oute panoama. Object widths in the oute panoama ae popotional to thei eal widths facing the steet. In this S Y D Automatic copy and paste t P Y D Figue 3. Route scene scanning by constantly cutting a vetical pixel line in the image fame and pasting it into anothe continuous image memoy when the camea moves along a smooth path on the hoizontal plane. S 59
Figue 4. Data size of a oute panoama is one sheet out of a volume of video images. y Spatial-tempoal volume Route panoama t (image numbe) Oigin x Image fame Figue 5. A segment of oute panoama geneated fom a video taken on a bus (befoe emoving shaking components). Figue 6. 2D pojections of 3D objects in a oute panoama compaed with a pespective pojection image. W stands fo the object width, H is object height, and D is object depth. (a) Odinay pespective pojection. (b) Othogonal-pespective pojection. (c) A typical object in pespective pojection and (d) in an othogonal-pespective pojection images. Hf /D 1 Hf /D 1 L1 L1 L1 Hf /D 2 Wf /D 1 (a) W/ (b) Wf /D 2 Image fame L4 W/ L1 Hf /D 2 Vanishing point L2 Hoizon Vanishing point (c) Pespective pojection L4 Asymptote L2 Hoizon Asymptote (d) Route panoama 60
sense, distant objects ae extended hoizontally in the oute panoama. Thus, oute panoamas aen t likely to miss lage achitectues. Small objects such as tees, poles, and signboads ae usually naowe than buildings along the t axis and might disappea afte squeezing in the t diection, while buildings won t disappea. In a nomal pespective pojection image, a small tee close to the camea may occasionally occlude a lage building behind it. We examine seveal sets of stuctue lines typically appeaing on 3D achitectues (see Figues 6c and 6d) and find thei shapes in the oute panoama fom a linea path. Assuming a linea vecto V = (a, b, c) in the global coodinate system with its X axis paallel to the camea path, the line sets ae L1 {V c = 0}: lines on vetical planes paallel to the camea path. These lines may appea on the font walls of buildings. L2 {V a = c = 0} L1: vetical lines in the 3D space. These lines ae vetical ims on achitectues. L3 {V c 0}: lines stetching in depth fom the camea path. L4 {V c 0, b = 0} L3: hoizontal 3D lines nonpaallel to the camea path. Obviously, lines in L2 ae pojected as vetical lines in the oute panoama though the vetical slit. Denoting two points P 1 (S 1,Y 1,D 1 ) and P 2 (S 2,Y 2,D 2 ) on the line, whee P 1 P 2 =τv, thei pojections in the oute panoama ae p 1 and p 2. The pojection of the line is then S v = p2 p1 = accoding to the pojection model in Equation 1. Fo a line in L1 whee D = D 2 D 1 = 0 and Y/ S is constant b/a, its pojection becomes v = S S f Y = D aτ 2 1 2 2 fy fy 1 D D fbτ = a fb τ D D 1 1 1 which is linea in the oute panoama. Theefoe, a line in L1 is still pojected as a line in the oute panoama fo a linea path. 1 We can obtain easonably good visual indexes of oute scenes as long as we allow fo smooth cuves in the oute panoama. The most significant diffeence fom pespective pojection is a cuving effect on line set L3 (Figue 6d). Fo a line in L3, its pojection in the oute panoama is a cuve, because point P 2 in the oute panoama is S2 fy2 S S1 f( Y1 Y) ( t2, y2)= + + = D2 D1 + D aτ + S1 f( Y1 + bτ ) = D + cτ which is a hypebolic function of τ. This cuve appoaches a hoizontal asymptotic line y = fb/c fom p 1 when τ. Paticulaly fo lines in L4, thei pojections ae cuves appoaching towad the pojection of hoizon (y = 0) in the oute panoama. The path cuvatue also defines the lines cuving effect if the camea moves on a cuved tace. Because of space, we omit the analysis hee. Nevetheless, we can obtain easonably good visual indexes of oute scenes as long as we allow fo smooth cuves in the oute panoama. Anothe inteesting popety of the oute panoama is the common asymptote fo a set of paallel lines stetching in depth (paallel lines in L3). Unde pespective pojection, we poject paallel lines with a depth change in the 3D space onto the image plane as nonpaallel lines and thei extensions on the image plane coss at a common point called the vanishing point (accoding to the pinciple in compute vision). In the oute panoama obtained fom a linea camea path, howeve, a set of 3D paallel lines stetching in depth has a common asymptotic line in the oute panoama. This is because paallel lines in L3 have the same diection (a, b, c), and thei pojections in the oute panoama all appoach the same hoizontal asymptotic line y = fb/c when τ. 1 July Septembe 2003 61
Suface 1 Suface 2 Suface 3 Slit Camea path Figue 7. Ovesampling ange, just-sampling depth, and undesampling ange of the oute panoama. IEEE MultiMedia P J S 1 S 2 Slit view coveed egions Ovesampling ange Just-sampling depth Ove sampled Sampled suface Slit view Undesampling ange Focal length of camea If we fix the camea axis so that the plane of sight though the slit isn t pependicula to the camea path, we obtain a paallel-pespective pojection along a linea path because all the pespective planes of sight ae paallel. We can futhe extend this to a bended-paallel-pespective pojection when the camea moves along a cuved path. We can extend most popeties of the othogonal-pespective pojection similaly. Stationay image blu and close object filteing When we e actually ecoding a oute panoama, we obtain slit views by cutting a pixel line in the image fame of a video camea. Evey slit view itself then is a naow-pespective pojection. The sampling ate of the slit has a limit lowe than the video ate. If the vehicle speed isn t slow enough, it s eflected in the oute panoama, because the panoama is actually the connection of naow pespective pojections at discete positions along the oute (as Figue 7 depicts). Scenes contained in a naow wedge ae pojected onto the onepixel line at each time instance. We examine sufaces that can appea in thee depth anges fom the camea path. Fist, at the depth whee suface 2 in Figue 7 is located, each pat of the suface is taken into consecutive slit views without ovelapping, just as a nomal pespective pojection does. We call this the just-sampling ange (depth). Second, fo a suface close than the justsampling ange, the consecutive slit views don t cove evey fine pat of the suface (suface 3 in Figue 7). Sufaces in this ange ae undesampled in the oute panoama. If the spatial fequency of intensity distibution is low on the sufaces that is, the suface has elatively homogeneous intensity we can ecove the oiginal intensity distibution fom the sampled slit views (the oute panoama), accoding to the Nyquist theoem in digital signal pocessing. Othewise, we may lose some details within that ange. Theefoe, oute panoama has a function of filteing out close objects such as tees, poles, people, and so foth. By educing the camea s sampling ate, this filteing effect becomes cleae and moe distinct. This is helpful when we e mainly inteested in achitectues along a steet. Thid, if a suface is fathe than the justsampling ange, the camea could ovesample the suface points. Because a slit view accumulates intensities in the naow pespective wedge, a point on suface 1 may be counted in the ovelapped wedges of the consecutive slit views. Theefoe, a distant object point etaining at a position in the pespective pojection may cause a blu hoizontally in the oute panoama. We call this phenomenon stationay blu, since it s convese to the motion blu effect in a dynamic image whee a fast tanslating point wipes acoss seveal pixels duing the image exposue. We can give the just-sampling depth s numeical computation in a moe geneal fom to include bended paallel-pespective pojection. If we set a global coodinate system O-XYZ, we can descibe the smooth camea path by S[X(t), Z(t)]. If the vehicle is moving on a staight lane without obvious tuns, the camea path almost has zeo cuvatue (κ 0). Selecting a camea obseving side, we can divide a path oughly as linea, concave, o convex segments, depending on the sign of cuvatue. Fo simplicity, we assume the adius of cuvatue R(t) of a cuved camea path is constant between two consecutive sampling positions S 1 and S 2, whee R(t) > 0 fo a concave path, R(t) < 0 fo a convex path, and R(t) = fo a linea path. The cuve length between S 1 and S 2 is. The wedge s angle is 2θ whee f tanθ = 1/2, because we cut one pixel as the slit width (see Figue 8). Fo bended paallel-pespective pojection, the plane of sight which is the wedge s cental plane has an angle α fom the camea tansla- 62
Camea axis P j D j θ =S 2 P j 1 pixel α Radius of cuvatue Motion vecto Camea path S 1 S 2 R(t) Camea path (a) (b) Figue 8. Just-sampling ange fo a cuved camea path. (a) Cuved camea path and a physical sampling wedge. (b) Two consecutive slit views and just-sampling depth. tion diection that is the cuve s tangent vecto. The two consecutive wedges of thin pespective pojection meet at a vetical line though point P j in Figue 8. We have a vecto elation on the hoizontal plane as S 1 P j = S 1 S 1 + S 1 P j in tiangle S 1 S 2 P j. Then we obtain S 1 P j and S 2 P j as SP 1 S P 2 j j 2Rt ()sin + + 2Rt () sin α θ 2Rt () = sin 2θ + Rt () 2Rt ()sin 2Rt () sin α θ 2Rt () = sin 2θ + Rt () by using sine theoem. It isn t difficult to futhe calculate the just-sampling depth D j in the plane of sight to the just-sampled suface. Figue 9 shows the equation fo this. It s impotant to note that the just-sampling ange not only elies on the camea s sampling ate, vehicle speed, image esolution, and camea s focal length, but also depends on the camea path s cuvatue. The just-sampling ange tends to be close when the camea moves on a concave path and fa on a linea path. When the camea SP 1 j SP 2 j cosθ D j = SP 1 j + SP 2 j Rt + + Rt Rt 2 ()sin 2 () sin α θ 2 () sin α θ 2Rt () cos θ = + Rt + + sin 2θ sin α θ + sin α θ () 2Rt () 2Rt ( ) Figue 9. Calculating the just-sampling depth D j in the plane of sight to the just-sampled suface. path changes fom linea to convex (R(t) vaies fom to 0 ), D j extends to the infinity (D j + ) and then stats to yield negative values (P j flips to the path s othe side). This means that the consecutive pespective wedges won t intesect when the convex path eaches a high cuvatue and the entie depth ange towad infinity is undesampled. Fo a simple othogonal-pespective pojection towad a linea path, we can simplify the equation in Figue 9 to D j = 2 tanθ by setting α=π/2 and R(t). Oveall, thee ae diffeently sampled egions in a oute panoama depending on the subjects depths. We can select a sampling ate so that the just-sampling ange is July Septembe 2003 63
IEEE MultiMedia Vehicle tun (Camea pan R y ) Wheelbase T y Camea Pitch R x Roll R z Figue 10. Vehicle and camea model in taking a oute panoama. Note that (T x, T y, T z, R x, R y, and R z ) = (fowad tanslation, up-and-down tanslation, tanslation sideways, pitch, pan, and oll), espectively. Also, tanslation sideways doesn t occu fo a vehicle movement. appoximately at the font sufaces of the buildings of inteest. The collection of slit views is a pocess of smoothing spatial intensity distibution: output value at a point by aveaging intensities ove a space aound it. We can estimate the degee of stationay blu as follows. At depth D, the width of the pespective wedge is W = D tanθ. We can aveage the colo distibution at depth D ove W to poduce a pixel value fo the slit view, which is the convolution between the intensity distibution and a ectangula pulse function with width W and height 1/W. If we set a standad test patten at depth D that s a step edge with unit contast o shapness, we can easily veify that the convoluted esult is a tiangula wave with the contast educed to 1/(D tanθ). Theefoe, an edge s shapness is invesely popotional to its depth. This is impotant when estimating objects depth in a oute panoama. If a segment of oute panoama mainly contains objects fa away, we can squeeze it along the t axis to educe the stationay bluing effect and, at the same time, educe shape distotion. This scaling may visually impove the objects appeaances in the oute panoama if thee s no equiement to keep the exact scale o esolution of the oute panoama hoizontally. Dealing with camea shaking Impoving image quality is a cucial step towad the eal application of oute panoamas in multimedia and the Intenet. In Figue 5, we T x T z pushed a small video camea fimly on a window fame of the bus to avoid uncontollable accidental camea movement. We still can obseve sevee zigzags on hoizontal stuctual lines in the oute panoama. This is because the camea shook when the vehicle moved ove an uneven oad. To cope with the camea shaking, some have tied to compensate by using a gyoscope. Howeve, adding special devices might incease the difficulty of speading this technology. Ou appoach was to develop an algoithm to ectify distotion accoding to some constaints fom scenes and motion. We can descibe the camea movement at any instance by thee degees of tanslations and thee degees of otations (as displayed in Figue 10). Among them, the camea pitch caused by left-and-ight sway and the up-and-down tanslation caused by the vehicle shaking on uneven oads ae the most significant components affecting the image quality. The latte only yields small up-and-down optical flow in the image if the vehicle bumping is less than seveal inches. Oveall, camea pitch influences image quality the most and luckily we can compensate fo it with an algoithm that educes camea shaking (many algoithms along these lines exist). 10,11 Most of these algoithms wok by detecting a dominant motion component between consecutive fames in the image sequence. Fo a oute panoama, howeve, we only need to deal with shaking components between consecutive slit lines. The idea is to filte the hoizontal zigzagged lines in the oute panoama to make them smooth, which involves spatial pocessing. We do this by using the following citeia: smooth stuctual lines in the 3D space should also be smooth in a oute panoama, and vetical camea shaking (in pitch) joggles the slit view instantly to the opposite diection. As Figue 11 illustates, we estimate the camea motion fom the joggled cuves in the oute panoama and then align vetical pixel lines accodingly to ecove the oiginal staight stuctual lines. The way to find an instant camea shaking is to check if all the staight lines joggle simultaneously at that position. We tack line segments hoizontally in the oute panoama afte edge detection and calculate thei consecutive vetical deviations along the t axis. At 64
each t position, we use a median filte to obtain the common vetical deviation of all lines to yield the camea s shaking component. The median filte pevents taking an oiginal cuve in the scene as a camea-joggled line and esulting in a wong pitch value. Afte obtaining the sequence of camea paametes along the t axis, we pepae a window shifting along the hoizontal axis and apply anothe median filte to the camea sequence in the window, which eliminates distubances fom abupt vehicle shaking. Suppose the oiginal stuctue lines in the scenes ae hoizontal in an ideal oute panoama (Figue 11a). The camea, howeve, shakes vetically ove time (Figue 11b), which joggles the stuctue lines invesely in the captued oute panoama (Figue 11c). By shifting all the vetical pixel lines accoding to the estimated camea motion sequence, we can align cuved lines popely to make hoizontal stuctue lines smooth in the oute panoama (Figue 11d). (a) (b) (c) (d) Figue 11. Recoveing staight stuctue lines fom cameajoggled cuves in a oute panoama. Figue 12. Recoveing smooth stuctue lines in the oute panoama by emoving the camea-shaking components. Figue 12 shows the oute panoama of Figue 5 afte emoving the camea shakes in the pitch. This algoithm is good at emoving small zigzags on the stuctue lines to poduce smooth cuves. Once we apply the algoithm, it s easy to modify the oute panoama to make majo lines smooth and staight. Figue 13. Real-time steaming data tansmission ove the Intenet to show oute scenes. Real-time tansmission and display Ou next step is to tansmit a long oute panoama on the Intenet and to seamlessly scoll it back and foth. Displaying and steaming oute panoamas gives uses the feedom to easily maneuve along the oute. We developed thee kinds of oute panoama displays to augment a vitual tou. The fist type is a long, opened fom of oute panoama (see Figue 2). The second type is a side-window view (see Figue 13) that continuously eveals a section of the oute panoama back and foth. We call it a oute image scoll. You can contol the diection of movement and the scolling speed with a mouse. The thid type is a fowad view of the vehicle fo steet tavesing; we call it a tavesing window (see Figue 14, next page). We can combine these diffeent displays in vaious ways. In endeing a tavesing window, we map both side-oute panoamas onto two sidewalls July Septembe 2003 65
Figue 14. The tavesing window dynamically displays two sides of oute panoamas in an open, cylindical panoamic view fo vitual navigation along a steet. East <- The 5th Steet The 4th Steet -> West Move fowad Speed up Pause IEEE MultiMedia along the steet and then poject to a local panoamic view (a cylindical image fame). We then display the opened 2D fom of this panoamic view so that uses can obseve the steet stetching fowad as well as achitectues passing by, while tavesing the oute. We ende the tavesing window continuously accoding to the moving speed specified by the mouse. Although the tavesing window isn t a tue 3D display, majo potions in the tavesing window have an optical flow that esembles eal 3D scenes. As anothe fom of use, it s even possible to display these pseudo-3d outes within a ca s navigation system. As a oute panoama extends to seveal miles, it s unwise to download the whole image and then display it. We developed a steaming data tansmission function in Java that can display oute image scolls and tavesing windows duing download. Because of the oute panoamas small data sizes, we achieved much faste tansmission of steet views than video. The image belt povides a visual pofile of a long steet fo its compactness. By adding clicking egions in the image, the oute panoama becomes a flexible visual index of outes fo Web page linking. On the othe hand, we can automatically scoll a oute panoama in a small window to give viewes the feeling that they e viewing achitectues and shops fom a sightseeing bus. With two cameas diecting left and ight sides of the vehicle, we can establish two side views of a oute by synchonizing the oute panoamas. If we dive vehicles passing evey steet in a town fo all the oute panoamas, we can geneate a visual map of the town fo vitual touism on the Intenet. With the tools discussed hee, we can egiste and visualize an uban aea using panoamic views, oute panoamas, oute image scolls, and maps. All these images have much less data compaed to video and look moe ealistic than 3D compute-aided design models. We can link aeas within a city map on the Web to coesponding aeas in the oute panoamas so that clicking a spot on the map can a update oute image scoll accodingly (and vice vesa). Eventually, we can use these tools to visualize lage-scale spaces such as a facility, distict, town, o even city. Conclusion Steets have existed fo thousands of yeas and thee ae millions of steets in the wold now. These steets have a temendous amount of infomation including ich visual contexts that ae closely elated to ou lifestyles and eflect human civilization and histoy. Registeing and visualizing steets in an effective way is impotant to ou cultue and commecial life. With the Route Panoama softwae, when you click a map to follow a cetain oute, the oute panoama will also be scolled accodingly, showing eal scenes along the oute. This will geatly enhance the visualization of Geogaphic Infomation Systems (GIS). Because of the 2D chaacteistics of captued oute panoamas, we can use them in a vaiety of ways. Fo instance, we can display and scoll them on wieless phone sceens o handheld teminals fo navigation in cities o facilities. By connecting a oute panoama database with the Global Positioning System, we can locate ou position in the city and display the coesponding segment of a oute panoama on a liquid cystal display. Displaying oute panoamas and panoamic images ae basically aste copy of sections of images. Hence, the poposed techniques ae even applicable fo 2D animation and game applications, potentially poviding iche, moe ealistic content than befoe. Cuently, we e woking on seveal innovations fo ou Route Panoama softwae. This includes constucting 3D steets fom oute panoamas, shapening distance scenes due to the stationay blu, establishing oute panoamas with a flexible camea setting, captuing high ises into oute panoamas, combining oute panoamas with local panoamic views in cityscape visualization, and linking oute panoamas to existing GIS databases. MM 66
Refeences 1. J.Y. Zheng, S. Tsuji, and M. Asada, Colo-Based Panoamic Repesentation of Outdoo Envionment fo a Mobile Robot, Poc. 9th Int l Conf. Patten Recognition, IEEE CS Pess, vol. 2, 1988, pp.801-803. 2. J.Y. Zheng and S. Tsuji, Panoamic Repesentation of Scenes fo Route Undestanding, Poc. 10th Int l Conf. Patten Recognition, IEEE CS Pess, vol. 1, 1990, pp.161-167. 3. J.Y. Zheng and S. Tsuji, Panoamic Repesentation fo Route Recognition by a Mobile Robot, Int l J. Compute Vision, Kluwe, vol. 9, no.1, 1992, pp. 55-76. 4. J.Y. Zheng and S. Tsuji, Geneating Dynamic Pojection Images fo Scene Repesentation and Recognition, Compute Vision and Image Undestanding, vol. 72, no. 3, Dec.1998, pp. 237-256. 5. T. Ishida, Digital City Kyoto: Social Infomation Infastuctue fo Eveyday Life, Comm. ACM, vol. 45, no. 7, July 2002, pp. 76-81. 6. G. Ennis and M. Lindsay, VRML Possibilities: The Evolution of the Glasgow Model, IEEE MultiMedia, vol. 7, no. 2, Ap. June 2000, pp. 48-51. 7. Z. Zhu, E.M. Riseman, and A.R. Hanson, Paallel- Pespective Steeo Mosaics, Poc. Eighth IEEE Int l Conf. Compute Vision, IEEE CS Pess, vol. I, 2001, pp. 345-352. 8. H.S. Sawhney, S. Aye, and M. Gokani, Model- Based 2D and 3D Motion Estimation fo Mosaicing and Video Repesentation, Poc. 5th Int l Conf. Compute Vision, IEEE CS Pess, 1995, pp. 583-590. 9. S.E. Chen and L. Williams, Quicktime VR: An Image-Based Appoach to Vitual Envionment Navigation, Poc. Siggaph 95, ACM Pess, 1995, pp. 29-38. 10. Z. Zhu et al., Camea Stabilization Based on 2.5D Motion Estimation and Inetial Motion Filteing, Poc. IEEE Int l Conf. Intelligent Vehicles, IEEE CS Pess, vol. 2, 1998, pp. 329-334. 11. Y.S. Yao and R. Chellappa, Selective Stabilization of Images Acquied by Unmanned Gound Vehicles, IEEE Tans. Robotics and Automation, vol. RA-13, 1997, pp. 693-708. Jiang Yu Zheng is an associate pofesso at the Depatment of Compute and Infomation Science at Indiana Univesity Pudue Univesity, Indianapolis. His cuent eseach inteests include 3D modeling, dynamic image pocessing, scene epesentation, digital museums, and combining vision, gaphics, and human intefaces. Zheng eceived a BS degee fom Fudan Univesity, China, and MS and PhD degees fom Osaka Univesity, Japan. He eceived the 1991 Best Pape Awad fom the Infomation Pocessing Society of Japan fo geneating the fist digital panoamic image. Reades may contact Jiang Yu Zheng at jzheng@ cs.iupui.edu. Januay Mach Contibutions to Computing Lean how yesteday s contibutions have shaped today s computing wold. Featued aticles cove topics such as the fist assemblyline poduction of a digital compute, a manage s account of woking at Compute Sciences Copoation in the mid-1960s, and seaching fo tactable ways of easoning about pogams. Apil June Evolution of Digital Computes Digital computes have evolved into poweful wokstations, but that wasn t always the case. Read about one man s foay into digital computing as well as the making of the MCM/70 micocompute. This issue also featues aticles on the gloy days of Datamation and the histoy of the secto, an analog calculating instument. http://compute.og/annals 2003 Editoial Calenda July Septembe Histoical Reconstuctions This special issue is devoted to ecoding and ecounting effots to peseve computing pactice though physical econstuction, estoation, and simulation. In addition to poviding accounts of such pojects though substantive aticles, the issue will seve as a digest of pojects and initiatives as well as societies and oganizations active in these aeas. Octobe Decembe Women in Computing Since the days of Ada Lovelace, women have played an impotant ole in the histoy of computing, and this ole has eceived inceasing attention in ecent yeas. Scholaship in this aea has begun to move beyond simply demonstating women's pesence in the histoy of computing to consideing how computing and gende constucts have shaped one anothe ove time. 67 July Septembe 2003