2 The MiníMax Principle First consider a simple problem. This problem will address the tradeos involved in a two-objective optimiation problem, where

Size: px

Start display at page:

Download "2 The MiníMax Principle First consider a simple problem. This problem will address the tradeos involved in a two-objective optimiation problem, where"

Miles Watkins
5 years ago
Views:

1 Determining the Optimal Weights in Multiple Objective Function Optimiation Michael A. Gennert Alan L. Yuille Department of Computer Science Division of Applied Sciences Worcester Polytechnic Institute Harvard University Worcester, MA Cambridge, MA September 21, 1998 Abstract An important problem in computer vision is the determination of weights for multiple objective function optimiation. This problem arises naturally in many reconstruction problems, where one wishes to reconstruct a function belonging to a constrained class of signals based upon noisy observed data. A common approach is to combine the objective functions into a single total cost function. The problem then is to determine appropriate weights for the objective functions. In this paper we propose techniques for automatically determining the weights, and discuss their properties. The MiníMax Principle, which avoids the problems of extremely low or high weights, is introduced. Expressions are derived relating the optimal weights, objective function values, and total cost. 1 Introduction An important problem in computer vision is the determination of weights for multiple objective function optimiation. This problem arises naturally in many reconstruction problems, where one wishes to reconstruct a function belonging to a constrained class of signals based upon noisy observed data. There is usually a tradeo between reconstructing a function that is true to the data, and one that is true to the constraints. Instances of this tradeo can be found in shape from shading ë3ë, optical ow ë4ë, surface interpolation ë2ë, edge detection ë7ë, visible surface reconstruction ë9ë, and brightness-based stereo matching ë1ë. The recently popularied regulariation method for solving ill-posed problems ë6, 8ë always requires the tradeo of conicting requirements. The basic framework denes a cost or error functional which reects the ëbadness" of a proposed solution to the reconstruction problem. Mathematical techniques, such as the calculus of variations, are used to nd the best solution to the reconstruction problem. The contribution of each constraint to the cost funtional is weighted, and the weights may be adjusted to achieve a desired tradeo. In some cases, a priori knowledge may be used to determine a ëbest" set of weights. Typically, one must know some property of the observed data, such as the signal-to-noise ratio, before proceeding. This is not always possible. It would be better to have a method for determining weights that did not depend on a priori knowledge. In this paper we propose techniques for automatically determining weights, and discuss properties of these techniques. Let us assume that there are several objectives to be achieved, and let there be a non-negative objective function for each objective which measures distance from that objective. We would like to combine the objective functions into a single cost function. In the following sections we will examine ways to construct the desired single objective function. 1

2 2 The MiníMax Principle First consider a simple problem. This problem will address the tradeos involved in a two-objective optimiation problem, where a cost function is to be minimied over a single variable. Our goal is to shed some insight into the more complex general problem, where the cost function incorporates arbitrarily many objectives to be minimied over a multiple-variable eld. 2.1 Linear Weight Constraint Let y 1 èè and y 2 èè be non-negative functions of a single state variable. It may be assumed, without loss of generality, that y 1 èè = 0 and y 2 èè = 0 for some ènot necessarily the sameè. These are the two objective functions. Let the overall cost function be a linear combination of the objectives, dened by eèç; è =çy 1 èè+è1, çèy 2 èè; 0 ç ç ç 1: For a given value of ç, there will exist a value of which minimies eèç; è. Let that minimum value be e èçè = min eèç; è =eèç; èçèè: Given ç, one generally tries to nd èçè, the value with the minimum total cost. We can plot e èçè as a function of ç, as shown in gure 1. Since there exists some such that y 1 èè = 0, and the y i 's are non-negative, e è1è = min y 1 èè =0. Likewise, e è0è=0. e èçè 6 e 0 0 ç 1 - ç Figure 1: Total cost e èçè vs. ç. e èçè is convex with e è0è = e è1è = 0. 2

3 The total cost function must be convex, irrespective of the convexity of each objective function. Let e 1 = e èç 1 è and e 2 = e èç 2 è. Then for any between 0 and 1, e èç 1 +è1, èç 2 è ç e 1 +è1, èe 2. The proof follows: e èç 1 +è1, èç 2 è = min eèç 1 +è1, èç 2 ;è = min èç 1 y 1 èè+è1, ç 1 èy 2 èèè +è1, èèç 2 y 1 èè+è1, ç 2 èy 2 èèè ç min ç 1 y 1 èè+è1, ç 1 èy 2 èè +è1, è min = e 1 +è1, èe 2 ç 2 y 1 èè+è1, ç 2 èy 2 èè Note that the proof does not depend on the convexity of either y i. The main diculty inchoosing ç is that if it is either too low or too high, one of the objective functions will be inadequately represented in the total cost, and the total cost will be too low. One way to ensure that the total cost will not be too low istopick the maximum cost solution. That is, nd ç such that e = e èç è is maximied. This may seem at rst to be far from optimal, but if one recalls that e has already been minimied over, it will be seen that the maximiation over ç does make sense, while avoiding the problems of ç excessively low or high. This is the MiníMax Principle. Note that if there exists a value of ç not equal to ero or one for which the total cost is ero, then the optimal cost found using the MiníMax Principle will also be ero. This follows from the convexity ofe èçè. Let us see what the extremiation of the total cost implies. If ç is the weight which maximies e èçè, then denote the value of the corresponding state variable by = èç è. Since e èçè is maximied at ç, conclude that 0 = d dç e èçè ç=ç = d dç eèç; èçèè eèç; è eèç ;è d dç èçè ç=ç But the last term, eèç ;è, equals ero, because minimies e. è, y 2 è eèç; è = y 1 è ç=ç Each objective function assumes the same cost value, although the weights ç and 1, ç are not necessarily identical, and the weighted contribution of each objective function will not in general be identical. 2.2 Non-linear Weight Constraint In this section we consider a variation on the previous solution technique in which the total cost function is not a linear combination of the weights. Instead, let the weights of the objectives, when squared, sum to a constant. The constant can be set to 1 without loss of generality. That is, eèç; è =çy 1 èè+ p 1, ç 2 y 2 èè; 0 ç ç ç 1: As before, the optimal value of given ç is èçè, where e èçè = min eèç; è =eèç; èçèè. A plot of e èçè would look much the same as in the linear combination case, gure 1. It can be shown that e èçè is convex. 3

4 The MiníMax Principle requires that ç be found which maximies the total cost, avoiding the problems of ç excessively low or high. As before, e = max ç e èçè =e èç è. Using the chain rule for dierentation we obtain: After rearrangement, eèç; è = y 1 è ç=ç 1 y 1 è 1 è= p 1, ç 2 y 2è è: ç ç p 1, ç 2 y 2è è: Each objective function, when divided by its weight, is equal. The weights ç and p 1, ç 2 will not in general be equal, therefore, the objective functions will not be equal. Also, the objective function with the greatest value will have the largest weight, so that its contribution to the total cost function is further increased. This contrasts with the linear weight case discussed above. 3 General Case In this section we consider the generaliation to any number of objective functions and any dimensionality state space. Replace the scalar variable by avector eld èxè dened over domain x. We use a vector eld for because there may be more than one quantity ofinterest. For example, in optical ow would be the horiontal and vertical components of optical ow. In stereo would be the horiontal and vertical disparities. If the image model has other parameters, they may be incorporated into. The two objective functions are replaced by n objective functionals Z yèè = Lèèxèè dx; where L is a set of possibly non-linear operators on the vector eld. The weights are given by vector ç, so that eèç; è = ç T yèè. The least cost solution for a given set of weights is èçè, yielding a total cost of e èçè = min eèç; è =eèç; èçèè. The cost functional is convex in ç. The proof follows: Let e èç 1 è=e 1 and e èç 2 è=e 2. e èçè is convex if, for any between 0 and 1, e èç 1 +è1, èç 2 è ç e 1 +è1, èe 2. e èç 1 +è1, èç 2 è = min eèç 1 +è1, èç 2 ; è = minèç 1 +è1, èç 2 è T yèè ç min ç T 1 yèè + min è1, yèè èçt 2 = e 1 +è1, èe 2 The MiníMax Principle requires that we nd ç to maximie e èçè; the maximum is obtained at ç. Note that if there exists a ero-cost solution with all positive weights, it will be found by the MiníMax proceduire. This follows from the convexity of e èçè. Thus, in cases where there is an obvious optimal solution, the proposed method will discover it. This property is independent of any constraint on the weights, apart from the positivity assumption. Usually, a constraint is needed for the weight vector ç. Otherwise, as ç increases without limit, so does the cost e èçè. Two methods of constraining ç have been considered earlier. When generalied, these correspond to 1 T ç =1 and jçj =1; where 1 T =è1; 1;:::èisavector of 1's. In both cases we require that ç ç 0, that is, every component of ç must be non-negative. The rst constraint forces the weights to lie on the hyperplane which is tangent to1 èand which passes through point 1= j1jè. The second constraint forces the weights to lie on the unit sphere. 4

5 3.1 Linear Weight Constraint We now investigate the behavior of the solution to the rst problem. The weights are constrained to lie on the hyperplane 1 T ç =1. Use the method of Lagrange multipliers, adjoining the total cost functional e = ç T y to the constraint to form L = ç T y + lè1 T ç, 1è; where l is the Lagrange multiplier. To extremie e, it must be true = y T + l1 T =0 T =1 T ç, 1=0: where 0 T = è0; 0;:::è. This shows that y =,l1 = 1e =n, which is constant for all components of y. Therefore, all objective functionals achieve the same value when the total cost functional is minimied. 3.2 Non-linear Weight Constraint When the weights are constrained to lie on the unit sphere, the method of Lagrange multipliers gives Setting the partials to ero, we have and Therefore, and L = ç T y + lèç T ç, = yt +2lç T =0 = çt ç, 1=0: e = ç T y =,2l y = e ç: The objective functionals are parallel to the weights, and every objective functional value, when divided by the corresponding weight, is equal. 4 Conclusions In this paper we have proposed methods for determining the weights in multiple-objective optimiation problems. The MiníMax Principle, which avoids the problems of extremely low or high weights, was introduced. Expressions were derived relating the optimal weights, objective functional values, and total cost. The optimal weights are not described in closed form; they must be found by a search technique. The Fibonacci search technique ë5ë is a good choice because the total cost is a unimodal function. For each set of weights considered, the entire problem of determining the state èçè must be solved. This may be computationally intensive. However, if is determined iteratively for each value of ç, then the state èç k èmay be used as the initial guess for the next set of weights ç k+1. This should greatly reduce the computation required. 5

6 Acknowledgements The support of the Worcester Polytechnic Institute Research Development Council is gratefully acknowledged. References ë1ë M.A. Gennert, ëbrightness-based Stereo Matching," These proceedings. ë2ë W.E.L. Grimson, ëa Computational Theory of Visual Surface Interpolation," Phil. Trans. Royal Soc. of London B, Vol. 298, pp. 395í427, ë3ë B.K.P. Horn, ëimage Intensity Understanding," Articial Intelligence,Vol. 8, No. 2, pp. 201í231, April ë4ë B.K.P. Horn & B.G. Schunck, ëdetermining Optical Flow," Articial Intelligence, Vol. 17, Nos. 1í3, pp. 185í203, August ë5ë D.A. Pierre, Optimiation Theory with Applications, John Wiley & Sons, New York, ë6ë T. Poggio & V. Torre, ëill-posed Problems and Regulariation Analysis in Early Vision," MIT AI Laboratory Memo 773, April ë7ë T. Poggio, H.L. Voorhees, & A. Yuille, ëa Regularied Solution to Edge Detection," MIT AI Laboratory Memo 833, May ë8ë D. Teropoulos, ëregulariation of Inverse Problems Involving Discontinuities," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 8, No. 4, pp. 413í424, July ë9ë D. Teropoulos, ëthe Computation of Visible Surface Represenations," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 10, No. 4, pp. 417í438, July

REDUCTION OF CODING ARTIFACTS IN LOW-BIT-RATE VIDEO CODING. Robert L. Stevenson. usually degrade edge information in the original image.

REDUCTION OF CODING ARTIFACTS IN LOW-BIT-RATE VIDEO CODING Robert L. Stevenson Laboratory for Image and Signal Processing Department of Electrical Engineering University of Notre Dame Notre Dame, IN 46556