A multicriteria framework with voxel-dependent parameters for radiotherapy treatment plan optimization a)

Transcription

1 A multicriteria framework with voxel-dependent parameters for radiotherapy treatment plan optimization a) Masoud Zarepisheh, Andres F. Uribe-Sanchez, Nan Li, Xun Jia, and Steve B. Jiang b) Center for Advanced Radiotherapy Technologies and Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, California (Received 11 July 2013; revised 10 December 2013; accepted for publication 11 February 2014; published 19 March 2014) Purpose: To establish a new mathematical framework for radiotherapy treatment optimization with voxel-dependent optimization parameters. Methods: In the treatment plan optimization problem for radiotherapy, a clinically acceptable plan is usually generated by an optimization process with weighting factors or reference doses adjusted for a set of the objective functions associated to the organs. Recent discoveries indicate that adjusting parameters associated with each voxel may lead to better plan quality. However, it is still unclear regarding the mathematical reasons behind it. Furthermore, questions about the objective function selection and parameter adjustment to assure Pareto optimality as well as the relationship between the optimal solutions obtained from the organ-based and voxel-based models remain unanswered. To answer these questions, the authors establish in this work a new mathematical framework equipped with two theorems. Results: The new framework clarifies the different consequences of adjusting organ-dependent and voxel-dependent parameters for the treatment plan optimization of radiation therapy, as well as the impact of using different objective functions on plan qualities and Pareto surfaces. The main discoveries are threefold: (1) While in the organ-based model the selection of the objective function has an impact on the quality of the optimized plans, this is no longer an issue for the voxel-based model since the Pareto surface is independent of the objective function selection and the entire Pareto surface could be generated as long as the objective function satisfies certain mathematical conditions; (2) All Pareto solutions generated by the organ-based model with different objective functions are parts of a unique Pareto surface generated by the voxel-based model with any appropriate objective function; (3) A much larger Pareto surface is explored by adjusting voxel-dependent parameters than by adjusting organ-dependent parameters, possibly allowing for the generation of plans with better trade-offs among different clinical objectives. Conclusions: The authors have developed a mathematical framework for radiotherapy treatment optimization using voxel-based parameters. The authors can improve the plan quality by adjusting voxel-based weighting factors and exploring the unique and large Pareto surface which include all the Pareto surfaces that can be generated by organ-based model using different objective functions American Association of Physicists in Medicine. [ Key words: treatment planning, intensity modulation, optimization, Pareto surface, voxel-dependent parameters 1. INTRODUCTION Treatment planning in cancer radiation therapy can be treated as a decision-making problem. The planner seeks for a treatment plan to deliver a certain amount of prescription dose to a cancerous target, while sparing dose to nearby critical structures and organs at risk. The apparent conflicts between those objectives make a multicriteria decision-making technique an appropriate tool to solve this problem. The main idea of the multicriteria technique is to reduce the size of the candidate set of plans by considering only those plans, called Pareto plans, for which it is impossible to improve some objectives without worsening others. The next step is to look for a clinically acceptable plan under the trade-offs between different criteria among the set of Pareto plans, referred to as Pareto surface. There exist several methods to handle this complicated procedure, such as fine-tuning optimization parameters in a trial-and-error fashion or by some heuristic approaches, 1, 2 precomputing a well discrete representative of Pareto surface and navigating among them, 3 15 and prioritizing the evaluation criteria to avoid sacrificing those goals of more importance while improving those less important ones Among them, fine-tuning optimization parameters is arguably the most common paradigm. In the optimization parameter-tuning regime, a commonly used approach is to adjust organ-dependent parameters, such as the weighting factors and the prescription doses for the PTV (planning target volume) and the thresholds for the critical organs. The planner tries to achieve clinical goals, like dose-volume constraints, by manipulating the organ-dependent parameters. Many biological-based and dose-based evaluation criteria have also been introduced Med. Phys. 41 (4), April /2014/41(4)/041705/10/$ Am. Assoc. Phys. Med

2 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization by researchers into the radiotherapy optimization, e.g., minimum and maximize dose, mean dose, equivalent uniform dose (EUD), generalized equivalent uniform dose (geud), tumor control probability (TCP), and normal tissue complication probability (NTCP) (see Romeijn et al. 23 and references therein). Clinical experiments have demonstrated a great impact of using different evaluation criteria in the optimization process on the optimized plan quality From the mathematical viewpoint, different criteria result in different Pareto surfaces. Romeijn et al. 23 and Hoffmann et al. 27 showed that some organ criteria generate the same Pareto surface; however, there are still a wide variety of criteria leading to different Pareto surfaces. So far there is no general agreement on the choice of the objective function and it still remains unanswered as for the question of which evaluation criterion leads to a better treatment plan? In this paper, we will show that this issue can be withdrawn by exploiting a voxel-based optimization model. In our model, a large Pareto surface exists and different parts of this surface are explored by applying different sets of organ evaluation criteria. Moreover, the entire Pareto surface is explored by a voxel-based optimization model, as long as the voxel penalty terms and their derivatives are increasing functions. The second motivation of this work is from the recent research works about voxel-based optimization models, in which one tunes parameters associated to each voxel in the objective function, as opposed to the organ-based model in which all voxels within a specific organ are tied together and treated equally in the objective function. It has been demonstrated that the voxel-based optimization tends to result in more qualified plans compared to the conventional organ-based approaches for clinical cases. Moreover, voxelbased optimization planning also facilitates interactive treatment planning by providing specific access to certain parts of the dose-volume histogram (DVH) curves and isodose layouts of interest. If a specific part of the DVH or isodose curves does not comply with the clinician s requirements, the planner can pick out the involved voxels and adapt their contributions in the objective function. 28, 29, 34 Up to now, there are two different schemes for adjusting voxel penalty terms in the objective function: voxel weighting factor adjustment 28 33, 35 and voxel reference dose adjustment (It has been referred to as the prescription dose in literature.). 30, 34, 36 Despite the success achieved by the voxel-based model in clinical studies, those models are mainly proposed heuristically and the fundamental reasons regarding the efficacy of them is unclear. In this paper, we aim at building a mathematical framework for this voxel-based optimization approach, which enables us to answer the following questions naturally raised regarding this model: (1) Why do voxel-based models lead to better plan quality than organ-based models? (2) How much improvement in DVH curves can be expected by utilizing the voxel-based approach compared to the conventional organ-based approach? In particular, can we expect to improve some parts of the DVH curves without worsening other parts? (3) What is the appropriate voxel-based objective function? (4) How do we assure the Pareto optimality of the solutions while the weights and references are adjusted? This paper unfolds as follows. The new framework is presented in Sec. 2. Section 3 elaborates on the results of the new framework and clarifies the differences between the variants of parameter adjustment. Finally, Sec. 4 is devoted to the conclusions and future research areas. 2. A NEW FRAMEWORK FOR TREATMENT OPTIMIZATION 2.A. Three different Pareto surfaces Pareto optimality is an inevitable part of inverse treatment planning when approached from the direction of multicriteria optimization. It helps us to get rid of the so-called non-pareto plans that are not worth consideration. Generally, the Pareto optimal solutions are those feasible ones for which it is impossible to improve some criteria without deteriorating others. Equivalently speaking, improvement at no cost is possible for non-pareto solutions, while this is impossible for Pareto ones. Pareto optimality is defined with respect to specific criteria used in evaluating the plan quality. In intensity modulated radiation therapy (IMRT) literature, Pareto optimality has been conventionally defined based on the evaluation criteria (objective functions) associated to the organs. At least one evaluation criterion is given to each organ and a plan is called Pareto optimal if there does not exist another plan that is better in terms of at least one criterion and not worse with respect to every other criterion. Since the traditional definition of Pareto optimality in IMRT is based on the given evaluation criteria, the set of the Pareto solutions (Pareto surface) would depend on the specific evaluation criteria used in the studies, and it is not clear in advance which Pareto surface includes a plan with clinically better trade-offs. For example, a TCP/NTCP-based Pareto surface may contain a better plan for one patient, while a geud-based Pareto surface may include a better plan for another one. To overcome this issue, we define Pareto optimality based on the DVH and dose distribution concepts as they are the most commonly used to evaluate the plan quality. For the sake of simplicity, we treat the PTV over-dosing and under-dosing equally; however, the results could be generalized easily. 1. (X OEC ): A treatment plan is called organ evaluation criteria Pareto (OEC Pareto), if improvement in some organ evaluation criteria is only possible at the cost of another organ evaluation criterion. For instance, the organ evaluation criteria could be the maximum dose to OAR (organ at risk) and the minimum dose to the PTV. In this case, the set of all Pareto treatment plans (Pareto surface) is denoted by X OEC and is called OEC Pareto surface. 2. (X DD ): A treatment plan is called dose distribution Pareto (DD Pareto), if it is impossible to improve the radiation doses in some voxels without worsening those in other voxels (improvement of doses in PTV voxels means getting closer to the prescribed dose, while that in voxels in OARs means delivering less

3 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization radiation). In this case, the Pareto surface is denoted by X DD and is called DD Pareto surface. 3. (X DVH ): A treatment plan is called DVH Pareto, if we cannot improve a certain part of the DVH curve of an organ without deteriorating either other part of that DVH or the DVH curves of other organs. Similarly, the Pareto surface in this case is denoted by X DVH and is called DVH Pareto surface. Let us illustrate these three definitions by a simple example. Suppose that we have a patient with a PTV and an OAR. Let D and x denote the dose deposition matrix and fluence map, respectively. Then, D ={Dx x 0} shows the set of all feasible plans for this patient. To see whether a specific feasible plan d D is OEC Pareto or not, we should define the OEC first. For example, if we consider the maximum dose as the OAR OEC and the minimum dose as the PTV OEC, apland D is OEC Pareto if there does not exist a plan d D with either (1) the lower maximum dose in OAR and the same or higher minimum dose in PTV or (2) the higher minimum dose in PTV and the same or lower maximum dose in OAR. To check if d is also DD Pareto, we need to compare the dose distribution of d with the dose distributions of all feasible plans. If there does not exist a plan d with better dose distribution [better in every voxel, meaning lower dose in an OAR voxel and closer to the prescription dose in a PTV voxel (For a PTV voxel, closer to the prescription dose is not necessarily better from the clinical point of view. This problem can be easily handled by differentiating between PTV overdose and underdose in our definitions and later on in our optimization models.)], then plan d is DD Pareto. We can have a similar argument for DVH Pareto optimality. Plan d is DVH Pareto if there does not exist a feasible plan d with a part of the DVH curve improved for an OAR/PTV without compromising other parts of the DVH curve for this OAR/PTV or other DVH curves for other OARs/PTVs. Among these Pareto surfaces, the OEC Pareto surface is the commonly used one in current IMRT treatment planning systems and can be explored by minimizing the weighted sum of the OEC. Since this surface is defined based on the given OEC, it would be changed as the OEC change. In contrary, the other two Pareto surfaces are unique since they are defined based on the dose distribution and DVH concepts. In Subsections 2.B and 2.C, we will show how we can explore these surfaces and we will also prove their relationships. In particular, we will prove that the DD Pareto surface can be explored by employing the voxel-based optimization model using any appropriate objective function. 2.B. Exploring Pareto surfaces In commonly used organ-based treatment planning, at least one evaluation criterion (e.g., maximum, mean, or minimum organ dose) is associated with each organ, and a desired plan is obtained by finding an appropriate tradeoff between these criteria. All voxels within a specific organ are weighted equally in each organ criterion. Mathematically speaking, each criterion is a function of the corresponding voxel doses in this organ that is invariant under permutation of voxel indices. As opposed to the organ-based model, a voxel-based model allocates nonuniform penalty to the voxels. Let us consider a typical IMRT inverse planning problem where the fluence map x is the decision variable. Here we just consider IMRT optimization problem for the sake of simplicity, and the proposed framework can be easily applied to any other treatment plan optimization problems (e.g., volumetric modulated arc therapy). Problems (1) and (2) demonstrate a typical voxel-based and an organ-based model, respectively: x(w) = arg min x 0 w j F j (D j x), (1) σ S j v σ x(w) = arg min x 0 w σ G σ (D σ x), (2) σ S where S = T C is the set of structures with T accounting for tumors and C for critical structures, v σ denotes the set of voxels belonging to the structure σ. w j and w σ are the weights corresponding to the voxel j in the voxel-based model and the structure σ in the organ-based model. D denotes the dose deposition matrix and its entry D jk specifies the dose received by the voxel j from a beamlet k at its unit intensity. D j is the jth row of matrix D that corresponds to voxel j, and D σ is the set of rows corresponding to the organ σ. F j is a voxel penalty function, and G σ is an organ penalty function. In Problem (1), each voxel has its own penalty contribution in the objective function with its specific parameter such as w j defining its importance. For target voxels, it is preferred to have dose close to the prescription value r σ, while low radiation dose is desirable for the voxels belong to OARs. Therefore, the penalty functions for voxels in OARs should be the increasing functions of dose, and penalty functions for tumor s voxels should be increasing functions of deviation to the prescribed value. The following theorem reveals that we can explore the OEC and DD Pareto surfaces by employing organ-based and voxel-based models, respectively. The proof of this theorem is given in the Appendix. Theorem 1: (a) The optimal solutions of Problem (2) are OEC Pareto with respect to the criteria G, and almost all parts of OEC Pareto surface can be explored by Problem (2); i.e., x(w) X OEC and x(w) = X OEC w>0 w>0 (b) Let F j be increasing functions for each σ C, j v σ and increasing functions of Dj σ x rσ for each σ T,j v σ, then x(w) = X DD w>0 x(w) X DD and w>0 (c) Let F j and its derivative be increasing functions for each σ C, j v σ and increasing functions of

4 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization Dj σ x rσ for each σ T,j v σ, and the derivative of F be positive on its domain, then x(w) = X DD w>0 The first part of the above theorem shows that almost all parts of the OEC Pareto surface can be explored by organ-based model. It says almost because we are able to generate the Pareto points which are called properly Pareto solutions and we might miss the others referred to as nonproperly Pareto solutions. 37, 38 The second part of the theorem reveals that if the voxel penalty functions are appropriate increasing functions, then the whole DD Pareto surface, except for nonproperly Pareto points, could be explored by the voxel-based model. The third part of the theorem states that the whole Pareto surface could be explored provided that the voxel penalty functions and their derivatives are appropriate increasing functions and the derivatives are positive on their domains. Moreover, it is easy to show that having the increasing derivatives results in convexity and assures the global optimality. The OEC Pareto surface is defined based on the given OEC and different OEC result in different Pareto surfaces. Since we do not know in advance which OEC Pareto surface may contain the desirable treatment plan, choosing a set of appropriate OEC is critical for the organ-based model. On the other hand, the DD Pareto surface is a unique surface defined regardless the objective function selection and based on the dose distribution concept. The above theorem reveals that almost the entire DD Pareto surface could be explored by using the voxel-based optimization model, as long as the voxel penalty functions are appropriate increasing functions, and the whole Pareto surface could be generated if the derivatives are increasing and positive as well. In particular, this means generating the X DD is irrespective of the specific forms of the penalty functions. For instance, by employing the following popular power penalty function, we are able to explore almost the entire DD Pareto surface by changing the voxel-based weights regardless the value of the exponent q σ providing that q σ > 1: min x 0 w j (D j x) qσ w j D j x r σ q σ. (3) σ C j v σ σ T j v σ Since the derivative of the power function is not positive everywhere in Eq. (3), we might miss nonproperly Pareto points. In order to catch the entire Pareto surface we can use, for example, the following exponential penalty function: min x 0 w j exp(d j x)+ w j exp( D j x r σ ). σ C j v σ σ T j v σ (4) 2.C. Relationship between Pareto surfaces The following theorem clarifies the relationship between three different Pareto surfaces that have been introduced in Sec. 2.A. The proof of this theorem is also given in the Appendix. Theorem 2: If G σ is an increasing function for each σ C, and is an increasing function of D σ x r σ for each σ T, then X OEC X DVH X DD. (5) Relation (5) contains three important messages. The first message comes from the relation X OEC X DD. Given that X DD is a unique Pareto surface and X OEC is an objective function dependent, the relation X OEC X DD implies that all the Pareto surfaces generated by the organ-based model by using different objective functions belong to X DD, which can be generated by any appropriate objective function using the voxel-based model. In fact, the Pareto optimal solutions generated by the organ-based model are part of the optimal solutions generated by the voxel-based model even if Problems (1) and (2) have different objective functions. The second message also comes from the relation X OEC X DD. It means that a larger Pareto surface is explored by adjusting voxel-based weighting factors than by adapting organ-based weighting factors, possibly leading to a plan with better trade-offs among different clinical criteria. In fact, we are exploring different parts of the large Pareto surface X DD by using different set of the organ evaluation criteria in the organ-based model. That is, the organ-based problem picks certain plans out of the Pareto surface X DD based on the given OEC, and leaves out others as the non-pareto plans. However, the part of the X DD that is not navigated by this approach may contain some more desirable plans, because there is generally no scientific or fundamental theory behind choosing OEC. The third message can be drawn from the relation X OEC X DVH, which states that the plans generated by the organ-based model are already DVH Pareto, where a uniform weighting factor is assigned to all voxels within an organ. Therefore, it is impossible to improve some parts of the DVH curves without worsening others by allocating the nonuniform weights to voxels. In particular, by adjusting the voxel-based weights after the organ-based weight adjustment, it is impossible to improve treatment plans in terms of DVH criteria at no cost. However, this statement does not exclude the possibility of improving the plan at a small 28 31, 33 cost. 3. TREATMENT PLAN OPTIMIZATION BY USING QUADRATIC PENALTY FUNCTIONS In this section, by considering a specific problem with a popular quadratic penalty function, we will show that how the results of the above two theorems can be useful in developing optimization models and also in adjusting the parameters. The quadratic dose function has been a very popular choice in IMRT treatment plan optimization. Using a quadratic function makes, it possible to get rid of the absolute function in Eq. (3), and we can also employ the existing efficient algorithms developed for the quadratic optimization problems. 39 In the rest of this section, first we will show that how we can assure the DD Pareto optimality in organ-based model, and then we will compare the organ-based and voxelbased weighting factor adjustment, and finally we will have a discussion about the reference dose adjustment. Three case

5 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization studies are also provided. To facilitate reading, hereafter we refer to DD Pareto optimality as Pareto optimality, and DD Pareto surface as Pareto surface. 3.A. Organ-dependent parameter adjustment The following problem illustrates a typical organ-based quadratic optimization model for IMRT: min x 0 σ Cw σ (D j x r σ ) w σ (D j x r σ ) 2, j v σ σ T j v σ where (D j x r σ ) + denotes the vector (D j x r σ ) with all negative elements replaced by zeros. The weights w σ (For σ S) are the parameters that we need to fine-tune to get a plan with desirable trade-offs. This can be done by a trial-and-error based approach or by using a heuristic update scheme, e.g., developed by Xing et al. 1, 2 For tumor voxels, reference doses (r σ, σ T) are the prescribed doses given by the clinicians, but for OARs they are parameters, which are adjusted to reshape the DVH curves. In fact, the objective function focuses on the particular tail parts of the DVH curves for OARs by considering penalty only for doses which are higher than the reference doses. For an OAR, the penalty function in Problem (6) is not an increasing function, because it does not differentiate the doses that are lower than the reference dose. Hence, the conditions of Theorem 1 are not satisfied and Pareto optimality might be lost. The conditions are satisfied only if the reference doses are equal to zero for all OAR voxels. Problem (7) overcomes this issue by penalizing the doses lower than the reference dose with a linear penalty function and doses higher than the reference dose with an extra quadratic dose function. The conditions of Theorem 1 are then fulfilled for this problem, and hence, the Pareto optimality is guaranteed. In fact, the main idea for reshaping the DVH curves is to focus more on certain parts of the DVH by considering the penalty only for overdoses (Doses those are higher than the reference doses.). Instead of penalizing only for over-doses, Problem (7) reshapes the DVH by penalizing over-doses more than the under-doses: min x 0 w σ (D j x + (D j x r σ ) 2 + ) j v σ σ C (6) + σ T w σ j v σ (D j x r σ ) 2. (7) 3.B. Voxel-dependent weight adjustment Cotrutz and Xing, 28, 29 and subsequently Yang and Xing 31, 33 and Shou et al., 33 extended their previous work on organ-based quadratic objective function to the voxel-based quadratic model. Wu et al. 30 proposed their systematic approach to adjust the weighting factors per voxel to modify the initial plan and guide it to clinically acceptable one. Breedveld et al. 32 introduced their voxel-based automatic treatment planning in order to generate a plan that complies with all or at least the most important given DVH and maximum-dose constraints. We recently introduced our approach to adjust the voxel weights automatically to generate a plan with DVH close to the given one for application in automatic treatment planning and adaptive radiotherapy replanning. 35 The following voxel-based quadratic model is used in all of these works: min x 0 wj σ (D j x) 2 + wj σ (D j x r σ ) 2. (8) σ C j v σ σ T j v σ Since the conditions of Theorems 1 and 2 are satisfied in above problem, we can take advantage of these theorem s results. According to Theorem 1, all plans obtained by weighting factor adoption are Pareto, and moreover, almost the entire Pareto surface could be characterized by changing the weighting factors. Regarding Theorem 2, if the plan is achieved by adjusting the organ-based weights, we cannot expect to improve some parts of the DVH at no cost by employing the voxel-based model; however, we can expect to improve some parts at a reasonable cost since we are able to explore some previously missed parts of the Pareto surface. These facts will be demonstrated in the following case studies. 3.C. Organ-based model versus voxel-based model: Case studies In this section, we would like to confirm some results of Theorems 1 and 2 using three case studies, and particularly the fact that the voxel-based model is able to improve the plan quality in terms of the DVH at the reasonable price by exploring the parts of the Pareto surface missing in the organ-based models. The way that the weighting factors should be adjusted is beyond the scope of this paper, and here we just borrow the approach that we have used in our previous work 35 in which the organ and voxel weighting factors are adjusted automatically and iteratively to project the reference plan back onto the DD Pareto surface by employing the approximation of the Pareto surface. We studied two gynecologic and a head-andneck cancer cases that have been planned previously using the Eclipse treatment planning system (Varian Medical System, Palo Alto, CA); however, the results can also be confirmed using the case studies provided in other voxel-dependent research works , 33 The plans generated with Eclipse and used for treatment are used as reference plans. Two new plans, one using the organ-based model and the other using the voxel-based model, are then generated for each case by automatically adjusting the organ or voxel weighting factors to match the DVHs of the reference plan, using our in-house GPU-based dose and optimization engines. The quadratic objective function (8) has been chosen for both models due to its computational efficiency and meeting the desirable conditions of Theorems 1 and 2. According to part (b) of Theorem 1, the much larger DD Pareto surface can be explored by adjusting voxel weighting factors regardless how they are adjusted, while due to Theorem 2 and part (a) of Theorem 1, just a part of the DD Pareto surface can be explored through organ weight adjustment. The results of gynecologic and head-and-neck cases are illustrated in Fig. 1, where the dashed lines represent DVHs

6 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization FIG. 1. Comparison of the treatment plans optimized with a voxel-based model and an organ-based model for the two gynecologic and the head-and-neck cases. Dashed line: voxel-based model; solid line: organ-based model. The vertical lines indicate the PTVs prescription doses. for the plan generated by the voxel-based model, and the solid lines are for the plan generated by the organ-based model. Comparing these two plans for the first gynecologic case (the first picture in Fig. 1), it is clear that by utilizing nonuniform weights to different voxels in an organ, the right and left femoral heads and pelvic bone marrow (PBM) are significantly improved at a reasonable cost of rectum and bladder. It can be also found in Table I(a) where two plans are compared based on some clinically important metrics. Since the higher value is preferred for the PTV under-dose and the lower value is preferred for the PTV over-dose, the better value for each criterion is shown in bold to make the visual comparison easier. The last column represents the absolute difference of two metrics. According to Table I(a), bowel maximum dose, bone marrow V(10Gy) and V(20Gy), and left and right femoral head maximum doses are improved respectively by 1%, 12%, 7%, 5%, and 13% at the 1% cost of rectum maximum dose. The second gynecologic case also demonstrates the significant improvement in plan quality by using the voxel-based model. According to Table I(b), bone marrow

7 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization TABLE I. Comparison of the plans in Fig. 1 using some clinically relevant metrics. Parts (a) and (b) represent the first and the second gynecologic cases and Part (c) represent the head-and-neck case. For each criterion, the better score is bolded. Organ Specific interest Organ-based plan Voxel-based plan Abs. diff. (a) PTV V (99%) 96% 96% 0 V (110%) OARs Bowel V (45Gy) Max dose 100% 99% 1% Rectum Max dose 98% 99% 1% Bone marrow V (10Gy) 72% 60% 12% V (20Gy) 45% 38% 7% Bladder Max dose 99% 99% 0 L-femoral head Max dose 49% 44% 5% R-femoral head Max dose 37% 24% 13% (b) PTV V(99%) 95% 95% 0 V(110%) OARs Bowel V (45Gy) 1% 3% 2% Max dose 102% 106% 4% Rectum Max dose 95% 95% 0 Bone marrow V (10Gy) 80% 67% 13% V (20Gy) 45% 35% 10% Bladder Max dose 98% 97% 1% L-femoral head Max dose 49% 33% 16% R-femoral head Max dose 41% 32% 9% (c) PTVs PTV-70 D (5%) 109% 107% 2% V (107%) 17% 8% 9% V (95%) 99% 99% 0 D (99%) 97% 98% 1% PTV-60 D (5%) 100% 98% 2% V (107%) 46% 30% 16% V (95%) 99% 99% 0 D (99%) 84% 83% 1% PTV-54 D (5%) 90% 87% 3% V (107%) 43% 25% 18% V (95%) 99% 99% 0 D (99%) 77% 76% 1% OARs Cord D (0.03%) 40% 30% 10% Max 41% 30% 11% Brainstem D (1%) 27% 19% 8% Max 41% 30% 11% L-parotid D (50%) 22% 15% 7% Mean 29% 21% 8% R-parotid D (50%) 30% 19% 11% Mean 33% 23% 10% V(10Gy) and V(20Gy), bladder, right and left femoral head maximum doses are respectively improved by 13%, 10%, 1%, 16%, and 9% at the 2% and 4% costs of bowel V(45Gy) and bowel maximum dose. By using the voxel-based model, the head-and-neck case demonstrates the significant improvement in all OARs and PTVs over-doses at the reasonable cost of the PTV-54 and PTV-60 under-doses. Part (c) of Table I compare the organ-based and the voxel-based plans for PTVs and OARs. The significant improvement in all OARs and PTVs overdoses can be observed at the 1% costs of PTV- 60 V (99%) and PTV-54 V(99%). Here we would like to point out that the whole DD Pareto surface does not need to be explored using our voxel-based model in order to get a desirable plan. A well-designed Pareto surface navigation (i.e., voxel weight adjusting) algorithm can reach a desirable solution (e.g., a Pareto surface point close to the reference plan) along an efficient path on the Pareto surface. Therefore, the computational efficiency of the voxel-based plan optimization is not necessarily worse than that of the organ-based optimization. In our three case studies, the organ- and voxel-based weight adjustment processes took 17 and 19 s for the first gynecologic case, 22 and 18

8 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization s for the second gynecologic case, and they took 15 and 17 s for the head-and-neck case, respectively. The main reason why the organ-based model took longer for the second gynecologic case is that more iterations in the organ-based model are required to get results comparable with the voxel-based model. In general, the amount of time that we have to spend in each model depends on the patients, the objective functions used in the models, and more importantly, the algorithm used to explore the Pareto surface. In the algorithms that we used in this work, the organ and voxel weighting factors are adjusted iteratively based on the deviation of the current plan to the reference one, and then the optimization problem is solved with the new weighting factors at each iteration. Adjusting voxel weights is slightly more time consuming than adjusting the organ weights, but the most time consuming part is the solving of the optimization problem which is common in both models. So, for the same number of iterations, both algorithms take about the same computational time. 3.D. Voxel-dependent reference dose adjustment In addition to changing voxel weighting factors, voxel reference doses can also be adjusted as optimization parameters 30, 34, 36 to produce a desirable plan as in the following model: min x 0 w σ (D j x r j ) 2. (9) j v σ σ S As opposed to the weighting factor adjustment, the Pareto optimality is not always guaranteed by adjusting reference doses in the above problem. In fact, the penalty functions for the OAR and PTV voxels in the above problem are not necessarily the increasing functions of D j x and D j x r σ which makes the conditions of Theorem 1 unsatisfied. In our recent work, 34 we introduced two techniques to cope with this issue. The first technique is to make sure that the reference is always outside the feasible region and the second technique is to modify the objective function to satisfy the conditions of Theorem 1. In that work, we proposed an algorithm to automatically adjust the reference values and weighting factors to generate a plan with DVH and dose distribution close to the physician s desired ones for application in interactive treatment planning. 4. DISCUSSION AND CONCLUSIONS In this paper, we introduced a new mathematical framework for treatment plan optimization and presented two theorems behind the organ- and the voxel-based optimization models. The new framework defines the desirable properties of the objective functions for the organ- and the voxel-based models in order to generate the Pareto optimal solutions. It also clarifies the differences between the organ and the voxel weight adjustment in terms of the Pareto surface navigation and plan quality. According to the new framework, in the organ-based model different objective functions may define different Pareto surfaces, while in the voxel-based model all objective functions with appropriate properties correspond to the same Pareto surface defined based on the dose distribution concept. In fact, the organ-based model capture some parts of the much larger DD Pareto surface, depending on the given organ evaluation criteria. Therefore, we are more likely to get a plan with more desirable trade-offs by exploring the entire DD Pareto surface via the voxel-based model. Up to now, the quadratic objective function is the most popular one in the voxel-based model due to its computational efficiency. Regarding Theorem 1, this function is able to generate the whole DD Pareto surface excluding the nonproperly Pareto points. There are still two remaining questions here. First, how important is the missing nonproperly Pareto points in exploration of the Pareto surface? This question is especially important because these points can be included by other objective functions such as one used in Problem (4). Second, since the objective function has no impact on the Pareto surface in voxel-based model, what is the best objective function, in terms of the computational efficiency, that satisfies the conditions of Theorem 1? The conditions provided in Theorem 1 can also be useful to establish the new optimization models and algorithms. Problems (6) (9) are good examples where we took advantage of these conditions. We showed that in organ-based model, when we are changing the organ reference doses in order to reshape their DVH curves, it is better to use a linear penalty term for doses lower than the reference dose to assure the Pareto optimality. One practical difficulty in the voxel-based optimization models is the dramatically increased number of parameters that cannot be manually adjusted in a trial-and-error fashion. In our recent works, 34, 35 we introduced new algorithms to automatically adjust the voxel parameters in order to efficiently navigate through the Pareto surface to reach a clinically desirable plan for application in adaptive radiotherapy replanning, automatic treatment planning, and interactive plan tuning. In each of these applications, there is a reference plan that can be used as a guidance to facilitate the weighting factor and reference value adjustment. In adaptive radiotherapy replanning, the initial plan which contains the clinically approved trade-offs among the organs for the patient s initial geometry can be used as the reference plan to develop a new plan for patient s new geometry by adjusting the voxel parameters. For automatic treatment planning, the reference plan can be extracted from a library of previously treated patients of similar medical conditions 40 and used to guide the plan optimization for a new patient by adjusting the voxel parameters. For interactive plan tuning, we allow the physicians to interactively fine-tune the DVH and isodose lines of an initially optimized plan, and the algorithm will adjust the voxel parameters to guide the plan to match the tuned DVHs and isodose lines. ACKNOWLEDGMENTS This work is supported by the University of California Lab Fees Research Program and a Master Research Agreement from Varian Medical Systems, Inc. The authors would like

9 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization to thank Dr. Edwin Romeijn for reading this paper and for providing constructive comments. APPENDIX: PROOF Throughout the proofs, we need some fundamental results from multicriteria optimization which can be found in both Ehrgott 37 and Miettinen Proof of Theorem 1 (a) For an arbitrary positive weight vector, an optimal solution of Problem (2) is Pareto with respect to the objective function G. It implies that there is no possibility to decrease some G σ without increasing others, which means that the optimal solution is Pareto in terms of the evaluation criteria. Moreover, each properly Pareto point can be generated with some positive weights. (b) Let x be an optimal solution of Problem (1) for an arbitrary positive weight vector. Then, x is a Pareto solution of min x 0 F(Dx). Since function F j is increasing for each σ C, j v σ, and increasing of D j x r σ for each σ T,j v σ, then x is a Pareto solution of Problem (A1) as well. Now, by the definition of DD Pareto optimality, it is obvious that each Pareto solution of Problem (A1) is Pareto in terms of the dose distribution, i.e., x X DD : min x 0 {{D j x} σ C,j vσ, { D j x r σ } σ T, j vσ }. (A1) (c) Let x X DD be an arbitrary Pareto solution in terms of the dose distribution. We need to prove that x is an optimal solution of Problem (1) for some positive weights. According to the definition of DD Pareto optimality, x is a Pareto solution of Problem (A1) which can be converted to the following equivalent linear multicriteria optimization problem by change of variables: 41 min{{d j x} σ C,j vσ, {z + j + z j } σ T,j v σ } s.t. D j x r σ = z + j z j, σ T,j v σ x,z +,z 0. (A2) If we define z + j ( x) = Max{0,D j x r σ },z j ( x) = min{0,d j x r σ }, then it can be readily shown that ( x,z + ( x),z ( x)) is a Pareto solution of Problem (A2). Due to the linearity of Eq. (A2), there is a positive weight vector w for which ( x,z + ( x),z ( x)) solves the corresponding weighted version of the above problem. Now, it is not difficult to show that x solves the weighted version of Problem (A1) corresponding to w It implies that x is a so-called properly Pareto solution of this problem. Since F and its derivative are increasing functions with respect to the objective functions of Problem (A1) and the derivative of F is positive on its domain, x is a properly Pareto solution of min x 0 F(Dx) as well. 42 So, there are some positive weights for which x is an optimal solution of Problem (1) due to the strict convexity of F. 2. Proof of Theorem 2 First of all, we need to provide a mathematical definition for improvement in DVH curves. Let π( ) denote the permutation that sorts every given vector in an ascending order. Then, for a specific structure σ C(σ T) improvement in DVH curve is equivalent to the component wise decrease in vector. π(d σ x)[π( D σ x r σ )]. Now, we prove the relations X OEC X DVH and X DVH X DD separately. (X OEC X DVH ): If we prove that for each structure σ,the corresponding objective function G σ decreases as the DVH curve of that structure improves, and G σ does not change as the DVH curve remains unchanged, then the relation X OEC X DVH can be proved easily by contradiction. We prove this property for two different cases σ C and σ T individually. (σ C): Let x, ˆx 0 be two feasible solutions of Eq. (2). At first, consider the case for which these two plans produce the same DVH curves for structure σ For this case we need to show that G σ (D σ x) = G σ (D σ ˆx). The relation π(d σ x) = π(d σ ˆx), and subsequently G σ (π(d σ x)) = G σ (π(d σ ˆx)), can be deduced from the fact that these two plans generate the same DVH curves for structure σ. Since the objective function G σ is symmetric and indifferent to the permutation, we have G σ (D σ x) = G σ (π(d σ x)), G σ (D σ ˆx) = G σ (π(d σ x)), and hence, G σ (D σ x) = G σ (D σ ˆx). Now, consider the case for which plan x generates the better DVH curve than plan ˆx for structure σ. In this case, we need to prove that G σ (D σ x) <G σ (D σ ˆx). Since x generates the better DVH curve than ˆx, we have π(d σ x) π(d σ ˆx) (Forvectors a and b, a b a b and a b). Moreover, the objective function G σ is an increasing function concluding that G σ (π(d σ x)) <G σ (π(d σ ˆx)). Now, the relation G σ (D σ x) <G σ (D σ ˆx) can be obtained by taking into account the symmetric property of G σ. (σ T): The main idea to prove this part is same as the previous part. If x and ˆx generate the same DVH curves for structure σ then π( D σ x r σ ) = π( D σ ˆx r σ ) and so G σ (π( D σ x r σ )) = G σ (π( D σ ˆx r σ )). Since the objective function G σ is symmetric with respect to the D σ x r σ,wehaveg σ ( D σ x r σ ) = G σ ( D σ ˆx r σ ). Now, if the DVH curve of plan x for structure σ is better than of plan ˆx then π( D σ x r σ ) π( D σ ˆx r σ ). The relation G σ ( D σ x r σ ) <G σ ( D σ ˆx r σ ) can be achieved by taking into account the increasing and symmetric properties of G σ with respect to D σ x r σ. (X DVH X DD ): We argue by contradiction to prove this part. Suppose that there exists ˆx X DVH which does not belong to X DD. ˆx / X DD means that there exists another plan like x with better amount of radiation in some voxels. Therefore, for each σ C(σ T), we have D σ x D σ ˆx( D σ x r σ D σ ˆx r σ ) and relation holds at least for one structure. Relations D σ x D σ ˆx and D σ x r σ D σ ˆx r σ result in π(d σ x) π(d σ ˆx) and

10 Zarepisheh et al.: A voxel model for radiotherapy treatment plan optimization π( D σ x r σ ) π( D σ ˆx r σ ), respectively (with strict inequality for at least one voxel). It shows that some DVH curves of plan ˆx could be improved without deteriorating the others, meaning ˆx / X DVH, which is a contradiction. a) This work was originally presented at the 54th Annual Meeting of the AAPM, Charlotte, NC, b) Author to whom correspondence should be addressed. Electronic mail: Steve.Jiang@UTSouthwestern.edu 1 L. Xing, J. G. Li, S. Donaldson, Q. T. Le, and A. L. Boyer, Optimization of importance factors in inverse planning, Phys. Med. Biol. 44, (1999). 2 L. Xing, J. G. Li, A. Pugachev, Q. T. Le, and A. L. Boyer, Estimation theory and model parameter selection for therapeutic treatment plan optimization, Med. Phys. 26, (1999). 3 K.-H. Küfer, H. W. Hamacher, T. R. Bortfeld, A multicriteria optimization approach for inverse radiotherapy planning, Presented at the 13th ICCR, Heidelberg (ICCR, Berlin: Springer, 2000), pp C. Cotrutz, M. Lahanas, C. Kappas, and D. Baltas, A multiobjective gradient-based dose optimization algorithm for external beam conformal radiotherapy, Phys. Med. Biol. 46, (2001). 5 H. W. Hamacher and K. H. Küfer, Inverse radiation therapy planning: A multiple objective optimization approach, Discrete Appl. Math. 118, (2002). 6 M. Lahanas, E. Schreibmann, and D. Baltas, Multiobjective inverse planning for intensity modulated radiotherapy with constraint-free gradient-based optimization algorithms, Phys. Med. Biol. 48, (2003). 7 K.-H. Küfer, A. Scherrer, M. Monz, F. Alonso, H. Trinkaus, T. Bortfeld, and C. Thieke, Intensity-modulated radiotherapy: A large scale multicriteria programming problem, OR Spectrum 25, (2003). 8 D. L. Craft, T. F. Halabi, H. A. Shih, and T. R. Bortfeld, Approximating convex Pareto surfaces in multiobjective radiotherapy planning, Med. Phys. 33, (2006). 9 C. Thieke, K.-H. Küfer, M. Monz, A. Scherrer, F. Alonso, U. Oelfke, P. E. Huber, J. Debus, and T. Bortfeld, A new concept for interactive radiotherapy planning with multicriteria optimization: First clinical evaluation, Radiother. Oncol. 85, (2007). 10 L. Shao, and M. Ehrgott, Approximately solving multiobjective linear programmes in objective space and an application in radiotherapy treatment planning, Math. Methods Oper. Res. 68, (2008). 11 D. Craft and T. Bortfeld, How many plans are needed in an IMRT multiobjective plan database?, Phys. Med. Biol. 53, (2008). 12 M. Monz, K. H. Küfer, T. R. Bortfeld, and C. Thieke, Pareto navigation: Algorithmic foundation of interactive multi-criteria IMRT planning, Phys. Med. Biol. 53, (2008). 13 T. S. Hong, D. L. Craft, F. Carlsson, and T. R. Bortfeld, Multicriteria optimization in intensity-modulated radiation therapy treatment planning for locally advanced cancer of the pancreatic head, Int. J. Radiat. Oncol., Biol., Phys. 72, (2008). 14 D. Craft, and M. Monz, Simultaneous navigation of multiple Pareto surfaces, with an application to multicriteria IMRT planning with multiple beam angle configurations, Med. Phys. 37, (2010). 15 D. L. Craft, T. S. Hong, H. A. Shih, and T. R. Bortfeld, Improved planning time and plan quality through multicriteria optimization for intensitymodulated radiotherapy, Int. J. Radiat. Oncol., Biol., Phys. 82, e83 e90 (2012). 16 M. Langer, E. K. Lee, J. O. Deasy, R. L. Rardin, and J. A. Deye, Operations research applied to radiotherapy: An NCI-NSF-sponsored workshop, February 7 9, 2002, Int. J. Radiat. Oncol., Biol., Phys. 57, (2003). 17 K.-W. Jee, D. L. McShan, and B. A. Fraass, Lexicographic ordering: Intuitive multicriteria optimization for IMRT, Phys. Med. Biol. 52, (2007). 18 J. J. Wilkens, J. R. Alaly, K. Zakarian, W. L. Thorstad, and J. O. Deasy, IMRT treatment planning based on prioritizing prescription goals, Phys. Med. Biol. 52, (2007). 19 J. O. Deasy, J. R. Alaly, and K. Zakaryan, Obstacles and advances in intensity-modulated radiation therapy treatment planning, in Frontiers of Radiation Therapy and Oncology, edited by J. L. Meyer, B. D. Kavanagh, J. A. Purdy, and R. Timmerman (Karger, Basel, 2007), pp V. H. Clark, Y. Chen, J. Wilkens, J. R. Alaly, K. Zakaryan, and J. O. Deasy, IMRT treatment planning for prostate cancer using prioritized prescription optimization and mean-tail-dose functions, Linear Algebra Appl. 428, (2008). 21 T. Long, M. Matuszak, M. Feng, B. A. Fraass, R. K. T. Haken, and H. E. Romeijn, Sensitivity analysis for lexicographic ordering in radiation therapy treatment planning, Med. Phys. 39, (2012). 22 M. Falkinger, S. Schell, J. Müller, and J. J. Wilkens, Prioritized optimization in intensity modulated proton therapy, Z. Med. Phys. 22, (2012). 23 H. E. Romeijn, J. F. Dempsey, and J. G. Li, A unifying framework for multi-criteria fluence map optimization models, Phys. Med. Biol. 49, (2004). 24 D. Craft, T. Halabi, and T. Bortfeld, Exploration of tradeoffs in intensitymodulated radiotherapy, Phys. Med. Biol. 50, (2005). 25 M. L. Kessler, D. L. McShan, M. A. Epelman, K. A. Vineberg, A. Eisbruch, T. S. Lawrence, and B. A. Fraass, Costlets: A generalized approach to cost functions for automated optimization of IMRT treatment plans, Optim. Eng. 6, (2005). 26 P. Xia, N. Yu, L. Xing, X. Sun, and L. J. Verhey, Investigation of using a power function as a cost function in inverse planning optimization, Med. Phys. 32, (2005). 27 A. L. Hoffmann, D. den Hertog, A. Y. D. Siem, J. H. A. M. Kaanders, and H. Huizenga, Convex reformulation of biologically-based multi-criteria intensity-modulated radiation therapy optimization including fractionation effects, Phys. Med. Biol. 53, (2008). 28 C. Cotrutz and L. Xing, Using voxel-dependent importance factors for interactive DVH-based dose optimization, Phys. Med. Biol. 47, (2002). 29 C. Cotrutz and L. Xing, IMRT dose shaping with regionally variable penalty scheme, Med. Phys. 30, (2003). 30 C. Wu, G. H. Olivera, R. Jeraj, H. Keller, and T. R. Mackie, Treatment plan modification using voxel-based weighting factors/dose prescription, Phys. Med. Biol. 48, (2003). 31 Y. Yang, L. Xing, Inverse treatment planning with adaptively evolving voxel-dependent penalty scheme, Med. Phys. 31, (2004). 32 S. Breedveld, P. R. M. Storchi, M. Keijzer, A. W. Heemink, and B. J. M. Heijmen, A novel approach to multi-criteria inverse planning for IMRT, Phys. Med. Biol. 52, (2007). 33 Z. Shou, Y. Yang, C. Cotrutz, D. Levy, and L. Xing, Quantitation of the a priori dosimetric capabilities of spatial points in inverse planning and its significant implication in defining IMRT solution space, Phys. Med. Biol. 50, (2005). 34 M. Zarepisheh, N. Li, L. Cervino, K. Moore, X. Jia, and S. Jiang, A novel IMRT plan optimization algorithm for physician-driven plan tuning, Med. Phys. 40, 340 (2013). 35 M. Zarepisheh, T. Long, N. Li, E. Romeijn, X. Jia, and S. Jiang, A novel prior-knowledge-based optimization algorithm for automatic treatment planning and adaptive radiotherapy re-planning, Med. Phys. 40, 530 (2013). 36 P. Lougovski, J. LeNoach, L. Zhu, Y. Ma, Y. Censor, and L. Xing, Toward truly optimal IMRT dose distribution: Inverse planning with voxel-specific penalty, Technol. Cancer Res. Treat. 9, (2010). 37 M. Ehrgott, Multicriteria Optimization (Springer, Berlin, 2005). 38 K. Miettinen, Nonlinear Multiobjective Optimization (Kluwer Academic Publishers, Dordrecht, 1999). 39 S. Breedveld, P. R. M. Storchi, M. Keijzer, and B. J. M. Heijmen, Fast, multiple optimizations of quadratic dose objective functions in IMRT, Phys. Med. Biol. 51, (2006). 40 M. Karimi, N. Li, M. Zarepisheh, L. Cervino, X. Jia, K. Moore, and S. Jiang, Selecting reference patients for automatic treatment planning using multiple geometrical features, Med. Phys. 40, 379 (2013). 41 K. G. Murty, Linear Programming (John Wiley & Sons, New York, 1983). 42 M. Zarepisheh, Transformation of multiobjective optimization problems with natural and lexicographical ordering, Ph.D. thesis, Amirkabir University of Technology, Tehran, Iran, 2011.