ERAD Optical flow in radar images. Proceedings of ERAD (2004): c Copernicus GmbH 2004

Transcription

1 Proceedings of ERAD (2004): c Copernicus GmbH 2004 ERAD 2004 Optical flow in radar images M. Peura and H. Hohti Finnish Meteorological Institute, P.O. Box 503, FIN Helsinki, Finland Abstract. Optical flow is one of the standard computer vision methods for computing motion vectors. The key idea is to derive motion vectors from intensity gradients and differences of subsequent images. This model is analogous to advection equations in meteorology which motivates testing the method for weather radar data. Also the computational simplicity of optical flow makes it an interesting alternative to autocorrelation based methods. Our final target is to apply motion vectors in nowcasting precipitation for 0 3 h. (One should keep in mind that motion of a precipitation area generally differs from Doppler measurable motion of droplets.) Related products typically exceed pixels in size and the maximal allowed processing time is below a minute for deriving the motion vectors. In this paper, we briefly review the optical flow model and suggest techniques for fast, quality weighted computing in extraction of motion vectors. We emphasize weather radar data specific issues such as the discontinuity and multi-scale nature of precipitation. Nevertheless, our first experiments suggest that the proposed techniques are applicable also in a wider context of motion extraction problems. 1 Introduction Nowcasting of precipitation is the most central application in operational radar meteorology. As to automated nowcasting of the next couple of hours, direct radar data extrapolation techniques are typically superior to physical models applied in numerical weather prediction (NWP). Principally, generalpurpose image extrapolation schemes can be considered as far as attention is paid on application specific constraints. In processing weather radar data, we would like to stress the following three issues. Correspondence to: M. Peura (Markus.Peura@fmi.fi) First, accepted processing time for radar image products is typically from one to five minutes. Hence, the applied algorithms should be computationally light. Second, the objects appearing in radar data are distinct, separated by large clear areas. This is due to discontinuous nature of precipitation and also due to sensitivity limits of radar. However, motion fields are continuous; motion vectors should be assigned also to echo free areas. Third, precipitation objects have large differences in size, ranging from convective cells in half-kilometre scale to precipitation areas hundreds of kilometres wide. A motion detection algorithm should however work consistently regardless of target size. This far, for extracting motion in image sequences, a standard technique has been to apply autocorrelation due to its conceptual clarity and robustness (Holmlund, 1998). However, an alternative approach, the optical flow technique (Sonka et al., 1993) seems to be simpler and hence faster than autocorrelation. While autocorrelation is essentially a matching technique, optical flow applies differential computing that requires fewer computation loops. The problem remains to find out whether this speedup compromises the quality of results. An excellent discussion on different optical flow methods is presented by J. L. Barron and Fleet (1994). We apply a general optimization problem outlined in that paper and adopt respective notations. In this paper, we focus on computational speed and utilization of quality information. We start by reviewing the optical flow model (Sect. 2) and outline our modified scheme for weather radar data (Sect. 3). In Sect. 3.1, we show how radar data can be treated as a continuous flow by smoothing input images. Both pre-smoothing and actual optical flow computation can be accelerated by sliding window techniques discussed in Sec The proposed computation scheme supports generating and using quality information in both extraction and application of motion vectors; two quality descriptors are suggested in Sect. 3.3.

2 M. Peura and H. Hohti: Optical flow in radar images 455 Fig. 1. The proposed overall scheme. 2 Optical flow f 1 (5th July 2004, 14:30) f (5th July 2004, 14:45) Fig. 2. Sample images from FMI Anjalankoski radar. In optical flow, the basic idea is to derive a continuous motion field a flow from temporal and spatial derivatives of two subsequent images Sonka et al. (1993). Formally, flow of quantity f = f (x, y, t) can be modelled as df dt = f t + f x u + f y v = f t + f v. (1) where df/dt is the change observed in the coordinate system flowing with data and the partial derivatives indicate the changes in the rigid image frame coordinates. Motion vector v = [u v] T is the quantity to be derived. This model is analogous to meteorological advection equations, which suggests using this approach in related problems as well. Practically, one assumes no changes in the data, df/dt = 0, and hence f v + f t = 0. This single equation contains two unknowns, u and v, which suggest solving a larger set of respective equations within some neighborhood of each (x, y). This overdetermined set leads to minimizing a squared sum of type w ( f v + f t ) 2 (2) where w is a weighting function defined in image coordinates and/or neighborhood coordinates. The role and definition of w is discussed further in Sect After some matrix manipulation, we obtain [ ] 1 Gyy G v = xt G xy G yt G 2 xy G (3) xxg yy G xy G xt + G xx G yt where G xx = wf x f x, G xy = wf y f x, G yy = wfy f y, G xt = wf x f t, and G yt = wf y f t (within ).

3 456 M. Peura and H. Hohti: Optical flow in radar images Fig. 3. Image f obtained through multi-scale-smoothing of f (Fig. 2, bottom). Fig. 4. Gradient quality image w(x, y) of f. Light areas contain pixels of high quality. 3 Proposed modifications The proposed scheme is shown in Fig. 1, illustrating the separate roles of motion vector extraction and application. The details discussed in this section refer to this image. Likewise, we shall focus on two sample images shown in Fig Pre-smoothing Direct application of optical flow solution (3) is possible for data that are flow-like: smoothly continuous thus differentiable. Optical flow approach needs gradient information to work. However, weather radar data contains objects with distinct edges, and the objects are separated from each other by echo-free areas. This suggests that information of the edges should be somehow spread into echo-free areas. This can be achieved Fig. 5. Motion vector quality q (top), motion vector component u (center) and its q-weighted-averaged version u (bottom). for example by smoothing input images with an averaging convolution mask. As a result, not only the effective scope of the objects will increase but also discrete edges will become more differentiable. Specifically, we suggest multiscale smoothing, which means blurring the original image f

4 M. Peura and H. Hohti: Optical flow in radar images 457 f +3 f +12 f +6 f +15 Fig. 6. Frames from an image series extrapolated from f 1 and f. f +9 f +18

5 458 M. Peura and H. Hohti: Optical flow in radar images with an averaging window operator A and mixing the result in the original image: f = cf + (1 c)a{f } with some c ]0, 1[. (One can apply this recursively.) As shown in Fig. 3, the resulting image contains both original details and smoothed regions. As a rule of thumb, the dimensions of the averaging window should be comparable to those of the object displacements between subsequent image frames. 3.2 Accelerating computation by algorithm design Image processing operations engaging computation of cumulants can be often accelerated. For example, performing the above-mentioned smoothing for an N N image with an n n window requires a computational effort proportional to N 2 n 2 using a brute force algorithm (with four nesting loops). However, the same result can be obtained by two subsequent averaging operations of window dimensions n 1 and 1 n, with effort proportional to N 2 n. Moreover, using a sliding window technique, that is, by incrementally updating the intensity sum in a window, the computational complexity is of order N 2 +2n N 2 which is remarkable in the case of large n. Also the actual optical flow computation can be accelerated. The crucial point here is that if w = w(x, y, t) in image coordinates, also the cumulants in (3) can be updated using a sliding-window technique, yielding complexity of order N 2 n instead of N 2 n 2. (If w = w(i, j) in coordinates, sliding is impossible.) 3.3 Weighting computation with quality descriptors If available, quality information should be taken into account in computations, especially in compositing multi-source data but also in spatial or temporal interpolations of single-source data. Sometimes quality information is available for input data, sometimes a computation scheme provides it as a byproduct, with negligible extra effort, and sometimes it is a central component of a system (Holmlund, 1998; Peura, 2002). In this context, we consider quality information that shares the same format with the data, i.e. an image array. Then, one can apply weighted averaging or weighted median filtering for the original image. As a result, data with high quality smoothly overrides (spatially or temporally adjacent) data of lower quality. In applying optical flow, there seems to be at least two stages where to use data quality. First, weighting function w mentioned in (2) and (3) should describe the quality that is, usability or confidence of each image location (x, y) for the optical flow algorithm. We suggest that this function is a gradient stability measure of the form w(x, y) = w o ( f (x, y) f 1 (x, y) ) where w o is an increasing function. A related image is shown in Fig. 4. Zero or small gradient change tells that that we are probably still on the same slope while changed gradient indicates that the slope has gone past (x, y). Second, there is a simple way to associate quality information to motion vectors. If neighborhood window is small compared to the size of details in image data, the gradient values inside are nearly same, and the calculation of a motion vector is ill-determined. This is the so called aperture problem. The nominator in (3) is actually an inverted and negated determinant (Peura and Hohti, 2004); the value of the determinant is zero if the motion vector is ambiguous. Hence, we propose a monotonous mapping of type q(x, y) = q o (G xx G yy G 2 xy ) where q o is an increasing function. A related image is shown in Fig Extrapolation As we mentioned in the introduction, we aim at nowcasting products based on direct extrapolation of weather radar data. Samples from an image series extrapolated from the two images of Fig. 2 are shown in Fig Conclusions We presented how optical flow approach can be applied in extracting motion vectors in radar images. Pre-smoothing is recommended, because radar images contain discontinuities at the edges of precipitation areas. The basic optical flow method is simple as such, but with minor efforts in algorithm design it can made even faster. Hence, in time critical applications, optical flow should be considered as an alternative to autocorrelation based methods. As to quality computation, optical flow algorithm both applies and produces quality information fluently. The results obtained this far encourage continuing this research. In the future, we will study parallel processing for convective and widespread rain as well as detection of divergence and converge. References Holmlund, K.: The Utilization of Statistical Properties of Satellite- Derived Atmospheric Motion Vectors to Derive Quality Indicators, Weather and Forecasting, pp , Barron, S. S. B. and Fleet, D. J.: On Optical Flow, in 6th Int. Conf. on Artificial Intelligence and Information-Control Systems of Robots (AIICSR), edited by I. Plander, pp. 3 14, World Scientific, Bratislava, Slovakia, citeseer.nj.nec.com/ article/barron94optical.html, sept , 1994, Smolenice Castle, Slovakia, Peura, M.: Computer vision methods for anomaly removal, in Second European Conference on Radar Meteorology (ERAD02), pp , Copernicus Gesellschaft, Peura, M. and Hohti, H.: Motion vectors in Weather Radar Images, in Seventh International Winds Workshop, Eumetsat, to appear, Sonka, M., Hlavac, V., and Boyle, R.: Image Processing, Analysis and Computer Vision, Chapman & Hall Computing, 1993.