Stepwise selection of variables in data envelopment analysis: Procedures and managerial perspectives

Transcription

1 European Journal of Operational Research 180 (2007) Continuous Optimization Stepwise selection of variables in data envelopment analysis: Procedures and managerial perspectives Janet M. Wagner *, Daniel G. Shimshak Department of Management Science and Information Systems, University of Massachusetts Boston, 100 Morrissey Blvd., Boston, MA 02125, United States Received 23 December 2004; accepted 7 February 2006 Available online 30 June Abstract One of the most important steps in the application of modeling using data envelopment analysis (DEA) is the choice of input and output variables. In this paper, we develop a formal procedure for a stepwise approach to variable selection that involves sequentially maximizing (or minimizing) the average change in the efficiencies as variables are added or dropped from the analysis. After developing the stepwise procedure, applications from classic DEA studies are presented and the new managerial insights gained from the stepwise procedure are discussed. We discuss how this easy to understand and intuitively sound method yields useful managerial results and assists in identifying DEA models that include variables with the largest impact on the DEA results. Ó 2006 Elsevier B.V. All rights reserved. Keywords: Data envelopment analysis; Efficiency measurements; Data reduction 1. Introduction Data envelopment analysis (DEA), a mathematical technique based on linear programming first introduced by Charnes et al. (1978), is a way of determining the efficiency for a group of decisionmaking units (DMUs) when measured over a set of multiple input and output variables. For a given set of input and output variables, DEA produces a single comprehensive measure of performance (efficiency score) for each DMU. This is done by * Corresponding author. Tel.: ; fax: addresses: janet.wagner@umb.edu (J.M. Wagner), daniel.shimshak@umb.edu (D.G. Shimshak). constructing an empirically based best-practice or efficient frontier as a result of identifying a set of efficient DMUs (on the efficient frontier) and inefficient DMUs (not on the efficient frontier). The major advantage of DEA over other methods that determine efficiency, such as cost benefit analysis or regression, is that the relative weights of the variables do not need to be known, a priori. Multiple variations of this technique exist, differing in how the efficient frontier is determined and in how the distance to the frontier for inefficient DMUs is measured. DEA results rely heavily on the set of input and output variables that are used in the analysis. However, in the literature relatively little attention has been paid to how, in a real-world situation, these /$ - see front matter Ó 2006 Elsevier B.V. All rights reserved. doi: /j.ejor

2 58 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) input and output variables should be chosen. Many of the existing papers on DEA treat the input and output variables used in their studies as simply givens and then go on to deal with the DEA methodology. As noted by Golany and Roll (1989), few studies give an overall view of DEA as an application procedure that must focus on the choice of data variables in addition to the methodology of DEA. The attention to variable selection is particularly crucial since the greater the number of input and output variables, the less constrained are the model weights assigned to the inputs and outputs, and the less discerning are the DEA results (Jenkins and Anderson, 2003). While it is advantageous to limit the number of variables, there is no consensus on how best to do this. Several methods have been proposed that involve the analysis of correlation among the variables, with the goal of choosing a set of variables that are not highly correlated with one another. Unfortunately, studies have shown that these approaches yield results which are often inconsistent in the sense that removing variables that are highly correlated with others can still have a large effect on the DEA results (Nunamaker, 1985). Other approaches look at the change in the efficiencies themselves as variables are added and removed from the DEA models, often with a focus on determining when the changes in the efficiencies can be considered statistically significant. As part of these approaches, procedures for the selection of variables to be included in the model have been developed by sequentially applying statistical techniques. In this paper, we develop an algorithm for the progressive or stepwise selection process, and examine the managerial insights gained from using this method. Our formal stepwise method of variable selection measures the effect or influence of variables directly on the efficiencies by considering their average change as variables are added or dropped from the analysis. This method is intended to produce DEA models that include only those variables with the largest impact on the DEA results. For managers and decision-makers, this method is easy to understand, intuitively sound, and does not require extensive additional calculations. This paper is organized as follows. Section 2 includes a literature review regarding the selection of variables in DEA modeling, including approaches for reducing the total number of DEA variables. In Section 3 we present a methodology for stepwise DEA modeling for selecting variables through the use of both a backwards approach and a forwards approach. We illustrate these methods using sample datasets from the literature and discuss the new managerial insights resulting from the stepwise results. In Section 4, some alternatives to the stepwise method are discussed. Our conclusions are presented in Section Selection of DEA variables The initial list of potential variables to be considered for DEA is often very large. Any resource used by a DMU should be treated as an input variable. The output variables come from the performance and activity measures that result when a DMU converts resources to produce products or services. In addition, environmental variables which may affect the production process need to be included in the list. An environmental variable is an external variable which could nevertheless influence the availability or requirement of resources (Klimberg and Puddicombe, 1995). Environmental variables which add resources are treated as inputs whereas those that require resources are treated as outputs (Boussofiane et al., 1991). A number of guidelines have been proposed in the literature suggesting limiting the number of variables relative to the number of DMUs. In general, the total number of input and output variables in the DEA model should be no more than one-third the number of DMUs in the analysis (Boussofiane et al., 1991; Friedman and Sinuany-Stern, 1998). The challenge in DEA is to find a parsimonious model, using as many input and output variables as needed but as few as possible. The greater the number of input and output variables in a DEA, the higher is the dimensionality of the linear programming solution space, and the less discerning is the analysis (Jenkins and Anderson, 2003). Golany and Roll (1989) explain that a large number of variables in the analysis will result in explaining away a larger portion of the differences among the DMUs. This will tend to shift the compared units towards the efficient frontier, resulting in a relatively large number of units with high efficiency scores. Nunamaker (1985) proves that not only will adding variables to a DEA result in higher efficiency scores, but will also expand the set of efficient DMUs. Prior DEA studies have suggested several approaches for identification of those variables considered most relevant to be included in the DEA model. One approach involves judgmental screening

3 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) of the list of variables by expert decision makers (Golany and Roll, 1989). The screening is meant to determine which variables contribute to the objectives of the DEA application and convey pertinent information not included in other variables. Possible procedures that may be used to obtain these judgments include the Delphi-method or variations of the analytic hierarchy process. Another commonly used approach for reducing the list of variables for inclusion in the DEA model is to apply regression and correlation analysis (Lewin et al., 1982). This approach purports that variables which are highly correlated with existing model variables are merely redundant and should be omitted from further analysis. Therefore, a parsimonious model typically shows generally low correlations among the input and output variables, respectively (Chilingerian, EJOR, 1995; Salinas- Jimenez and Smith, 1996). Norman and Stoker (1991) proposed a method of adding variables to the DEA model one at a time. They started with a simple model involving one single output and one single input. Efficiencies for all the DMUs were then calculated. All variables not in the model were correlated with the measure of efficiency. They claimed that high statistical correlation was an indicator that a particular variable influenced performance. A new variable was then added to the DEA model based on the correlation values and incorporated into the measure of efficiency. The process was repeated until no further influential variables remained. The authors did note that the observation of high statistical correlation alone was not sufficient. A logical causal relationship to explain why the variable influenced performance was necessary. Another application of variable selection based on correlating the efficiency scores can be found in Sigala et al. (2004). In his analysis of DEA modeling, Nunamaker (1985) found that for selected DMUs, the addition of a highly correlated variable may substantially alter the DEA efficiency scores. He concluded that because a variable was redundant within a regression model did not mean that it was redundant within a DEA model. The existence of high correlation among variables did not necessarily mean that one of the variables could be excluded without changing the subsequent DEA results. Therefore, it would be unwise to rely strictly on regression and correlation analysis as a means of reducing the number of variables. At best, these quantitative techniques could assist in variable reduction. In a similar vein, Golany and Roll (1989) claim that one-at-a-time regression tests on the inputs and outputs should not be regarded as reliable rules for eliminating variables but rather as indicators for a need to examine some of the variables more closely. In a recent study by Jenkins and Anderson (2003), regression and correlation analysis was used to identify which variables were to be omitted from the DEA model on the basis of the least loss of information. Information was related to the variance of an input or output variable about its mean value. Their statistical approach using partial correlation analysis resulted in a measure of information contained in each variable. Similar to some of the prior studies, the authors found that the DEA results could vary greatly according to which highly correlated variables were included or omitted from the DEA model. In contrast to correlation based methods, which look at the input and output variables before the application of DEA in an attempt to determine the likely effect on the efficiency scores after the application of DEA, other approaches look directly at the effect on efficiency scores as input and output DEA variables are changed. Statistical tests developed by Banker (1993, 1996) have been used to evaluate the marginal impact on the efficiencies of a adding or subtracting a given variable. While focusing on evaluating the statistical significance of the changes in the efficiencies, Kittelson (1993) presented an iterative technique for building a DEA model involving statistical procedures. He began by selecting an initial model involving a portion of the total number of input and output variables. He did not present guidelines for this first step but rather suggested using prior theoretical or empirical reasons for including variables in the initial model. Next the efficiency estimates for the initial model were compared to those for a new model in which one additional variable was added. Four separate statistical tests were performed to determine whether the addition of the new variable would significantly increase the efficiency estimates. This procedure was continued until no new variables could be added to the model. A similar methodology for variable selection involving statistical procedures was described by Pastor et al. (2002). They explored nested DEA models whose data sets differed in one single input or output variable. Their method evaluated a reduced DEA model, which did not include one

4 60 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) particular variable, and an extended model, which did include that one variable. Efficiencies were calculated for each DMU under both the reduced and extended forms of the model. A statistical test was presented to determine the significance of the efficiency contribution of the particular variable being evaluated. This paper described how applying this methodology sequentially could lead to decisions about incorporating or deleting variables into or from the DEA model. An application of this methodology can be found in Lovell and Pastor (1997). Our paper advances the work on variable reduction methods in DEA by formalizing a stepwise method to DEA modeling and by focusing on the managerial use and insights gained by using this approach. Our method suggests some simple rules for removing variables (backwards approach) or for adding variables (forwards approach) in the DEA model, one at a time. In the backwards approach, the goal of the method is to remove those variables that have the least influence on the set of efficient DMUs which define the reference set. Likewise, the forward approach attempts to add variable that have the most influence. The decision to remove or to add a variable is based on a simple evaluation of the efficiency scores of the DMUs, thus making our method both easy to explain within a managerial setting and easy to implement. 3. Stepwise modeling for selecting DEA variables We begin by describing the basic stepwise procedure using the backwards approach for the modeling of DEA variables. The backwards approach starts by considering all possible input and output variables in the DEA model. At each step, one variable is dropped from the model by analyzing the efficiency scores of the DMUs. Theoretically, the method can continue until only one input and one output variable remain in the model. From a practical viewpoint, stopping rules can be incorporated using the decision criterion to create a parsimonious DEA model Backwards approach Assume that you have a set of j =1,...,J input variables and k = 1,..., K output variables. Start: Run a single DEA analysis that includes the full set of J input variables and K output variables. Record the efficiency scores for each DMU for this run (set E * ). Step 1: Run a set of i =1,...,J + K DEA analyses, dropping one input variable and then one output variable at a time in each run. For each analysis: Record the efficiency scores for each DMU (set E 1,i ) for all i runs. Calculate, for each DMU, the differences in the respective DMU efficiency scores (E * E 1,i ). Calculate the average difference in efficiency (over the set of i differences). Choose the single input or output to be dropped by selecting the variable with the minimum average difference in efficiency scores from above. At least one input and at least one output variable must be kept in the analysis. If the model has only one input or output variable remaining, then this one variable cannot be dropped and another variable must be considered based on the selection procedures above. For the variable selected to be dropped, label the DEA results E 1. E 1 is based on the efficiency scores of the DMUs for the remaining input and output variables. Step n + 1: Repeat each step by running a set of i = 1,...,J + K n DEA analyses. With the remaining J + K n input and output variables, compare results E n+1,i and E n (the efficiency scores from the previous step) and chose the variable to be dropped based on the minimum average difference in efficiency scores. Stop: The method is concluded when the model has been reduced to one input variable and one output variable. It would be possible to incorporate rules that will allow the procedure to stop at earlier stages, for example, when the magnitude of the change in efficiency scores reaches some predetermined level. Various stopping rules are discussed in Section 4, which presents alternative implementations of the stepwise approach.

5 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) It should also be noted that the stepwise DEA procedure does not rely on the particular form of the DEA model. This procedure can be used with either constant or variable returns to scale, or with either an input or an output orientation, as long as the same model is used consistently in all steps Sample problem: Hotel chain The backwards method for the stepwise modeling of DEA variables can easily be demonstrated by using an example. We will consider the dataset presented in a textbook by Ragsdale (2004), which describes a hypothetical example of a hotel chain comparing itself against a group of 7 other similar chains. For this group of 8 hotels, labeled A through H, the chain has gathered information regarding 6 input variables and 2 output variables. In this problem, the inputs are the following customer rankings of satisfaction with the chain s: I1 service I2 climate control I3 price I4 convenience I5 room comfort I6 food quality The outputs represent the following customer satisfaction levels: O1 overall satisfaction O2 value The results of applying the backwards approach to stepwise DEA modeling, for this input-oriented, constant returns to scale DEA model, are detailed in Table 1 and described below. Start: Running a DEA analysis with all 6 inputs and 2 outputs, we obtain the set of efficiencies called E *. With the full set of input and output variables, there are a total of 4 efficient hotels. The 4 inefficient hotels have efficiencies ranging from to Step 1: We now run a series of 8 DEA analyses, systematically dropping each input and then each output, one at a time. For each of the 8 analyses, we record the resulting efficiency score for each DMU, calculate the difference between the efficiency scores for each DMU in the new analysis (minus one input or output variable) and calculate the average difference in efficiency scores compared to the set with all 8 variables (E * ). A very interesting result occurs in this example. For three of the six input variables service (I1), climate control (I2) and food quality (I6) the average difference in the efficiency scores between E 1,i and E * is 0. In other words, these three input variables can be removed from the model without affecting a single efficiency score since they have no effect at all on the efficiency scores. In this example then, we can skip directly to a model with only 3 inputs (price, convenience, and room comfort) and the 2 outputs. This model yields the same efficiencies as in the starting model. Step 2: Starting with the reduced set of 3 input and 2 output variables, a new series of DEA analyses are run dropping each remaining variable in turn while the values of E 2,i are calculated. The minimum average difference in efficiency scores (0.001) occurs by dropping the output variable value (O2) from the DEA model. The same four hotels remain efficient. Step 3: There are now three remaining inputs (price, convenience, and room comfort) and one remaining output (overall satisfaction). Since the model must have at least one output variable, the one output, overall satisfaction, is no longer considered in the analysis for removal. Dropping each of the input variables separately results in changes in average efficiency scores that seem fairly substantial. Specifically, the efficiency scores for DMUs C and F decreases by 10% or more and the number of efficient DMUs goes from 4 to 2. Thus, it might make managerial sense to stop now and let the DEA model consist of 3 input variables and 1 output variable. However to illustrate the method we will keep going. The minimum average difference in efficiency scores (0.034) occurs by dropping the input variable room comfort (I5) from the DEA model. There are now only two efficient hotels. Step 4: There are now two remaining inputs, price and convenience, and one remaining output, value. The minimum average difference in efficiency scores (0.062) occurs by dropping the input variable convenience (I4) from the DEA model. At this point, only one hotel is efficient. Stop: With only one input and one output remaining, the final DEA model has price as the input variable and value as the output variable. This example highlights the managerial insights gained from stepwise DEA modeling. With the traditional approach of using all available variables in the model, a hotel manager using DEA analysis would continue to track these hotel chains on three variables (service, climate control, and food quality) that have no actual effect of their efficiencies. The

6 62 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) Table 1 Ragsdale hotel example benefit of recognizing this alone probably makes the extra time spent running a few additional DEA models worth the effort. The analysis also provides some additional insights, in that the final model can be thought of as a kind of core service model plainly a value model where the primary issue is for this hotel chain to continue to provide what is perceived as a good value for the price. Looking backwards in the analysis, we can then see that after price, convenience and room comfort are also important distinguishing characteristics for this hotel chain. Next, looking at overall satisfaction in addition to value may help to distinguish this chain from others. Again, we have observed, that at least for the set of hotel chains in this comparison group, service, climate control, and food quality are not important ways to distinguish this chain from the others. We also note that, Jenkins and Anderson (2003) used the same data to demonstrate their variable reduction method based on analyzing correlation coefficients and minimizing the loss of information. They also determined that input variables service and climate control could be omitted with a minimum loss of information and no change in DEA efficiency scores. However, their analysis did not detect that food quality also had no effect although it was removed later in their analysis. In summary, the cost of running a few extra DEA models is low, while the managerial insights

7 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) gained, particularly about issues that simply do not seem to be of much importance in establishing this brand, are high Sample problem: Tokyo libraries In the hotel chain example in the previous section the stepwise method was shown to produce interesting and useful results, but it was, after all, only a textbook problem. Table 2 summarizes each step for the stepwise DEA analysis of a real data set involving 23 public libraries in Tokyo (Cooper et al., 2000). This example has 4 input variables (floor area, books, staff, and population served) and 2 output variables (library registrations and borrowing rates). An input-oriented, variable returns to scale analysis was used for this example. At the Start, DEA analysis of the model containing all 4 inputs and 2 outputs yields nine efficient library branches. In Step 1, removing the input variable floor area gives the smallest average change in efficiency scores (0.0039) and is dropped from the model. The same nine library branches remain efficient. In Step 2, with 3 input and 2 output variables, the output registration is selected to be dropped with an average change in efficiency score of resulting in seven efficient library branches. In Step 3, the input variable staff with an average change in efficiency score of is dropped, followed by the input variable population served in Step 4. In this final step, a fairly large average Table 2 Tokyo library example Variables in analysis Start Step 1 Step 2 Step 3 End Remaining inputs Floor area Books Books Books Books Books Staff Staff Staff Population Population Population Population Remaining outputs Registration Registration Borrowings Borrowings Borrowings Borrowings Borrowings Variable dropped Floor area Registration Staff Population Branch Efficiency scores Chiyoda Chuo Taito Arakawa Minato Bunkyo Sumida Shibuya Meguro Toshima Shinjuku Nakano Shinagawa Kita Koto Katushika Itabashi Edogawa Suginami Nerima Adachi Ota Setagaya # Efficient DMUs Average change in efficiencies

8 64 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) change in efficiency score of occurs. The efficiency scores for some DMUs are reduced by as much as 60%. Applying a stopping rule to the stepwise approach at this point would certainly have made sense. Continuing, however, the backwards approach stops with a DEA model that contains books as the input variable and borrowing rates as the output variable. This final model has only three efficient library branches, the Chiyoda, Bunkyo, and Setagaya branches. When the stepwise method is taken to its conclusion, there will always be one input and one output identified as the most efficient for this final analysis. For this Tokyo library example the core model that has been identified for these libraries requires that they focus their efforts on increasing borrowing rates for each branch, the single remaining output variable. At the same time, they should concentrate on reducing inputs or, in this case, minimizing books. Managerially, we interpret this result as indicating that the core strategy for this library system is to focus their book acquisition process so that patrons needs are met (and hence borrowing is high) while keeping the investment in books to the minimum. Examining the order in which the input and output variables drop out, and the effect on the previously efficient DMUs as they do, provides valuable managerial information. We can also see from Step 1 that the input variable floor area has very little effect on the analysis. All of the originally efficient branches remain efficient when the input variable floor area is dropped. It is true that the efficiency scores for two of the (inefficient) library branches, Sumida and Shinagawa, decreased as a result of dropping this input variable. Thus, floor area is has an effect for those two branches. However, in most applications this modest change in efficiencies is outweighed by the gains that result in developing a more parsimonious model. When the input variable registration is dropped, the Minato branch falls from efficient to 88% efficiency and the Itabashi branch falls from efficient to 97% efficiency, thus implying a focus on maintaining library registrations for these branches. In the next step, dropping the output variable staff results in the Chuo and Suginami branches falling from efficient to 92% and 91% efficiency, respectively. Lastly, dropping the input variable population served results in previously efficient branches Meguro and Koto falling to 83% and 49% efficiency, respectively Sample problem: Academic departments (comparison with Jenkins and Anderson) Jenkins and Anderson (2003) performed an analysis on a widely studied dataset of Sinuany-Stern et al. (1994) involving the efficiency of 21 academic departments. The method proposed by Jenkins and Anderson drops variables based on losing the least amount of information in the output variables as measured by calculations of the partial covariances. We use the same academic department data and apply an input-oriented, constant returns to scale DEA model. Table 3 summarizes the results of our stepwise method, which drops both input and output variables based on minimizing the total difference in efficiency scores. This example includes 2 inputs (operating cost and salaries) and 4 outputs (grants, publications, graduate students and contact hours). Our stepwise procedure shows when the output graduate students is dropped from the analysis there is very little effect (a modest change in average efficiency scores , and no reduction in the number of efficient DMUs). The only department that has an appreciable change in its efficiency score is History, a department that is not fully efficient. So, while the number of graduate students is important to this department, it has little impact on any of the other departments. Interestingly, although the results were close, Jenkins and Anderson, who use a method for dropping variables that is only indirectly related to the resulting changes in efficiencies, found that dropping graduate students omits the most information measured by their covariance calculation. As a result, they would have identified this variable as the last output to be dropped. Continuing with our stepwise analysis, after dropping graduate students, grants, operating cost, and contact hours are dropped, in turn. Again, managerially, employing a stopping rule might be reasonable in this example since some steps result in changes in the efficiency scores of individual departments of more than 50%. However, for illustration s sake, we continue. When the stepwise approach reaches the last step, the final DEA model includes the single input variable salaries and the single output variable publications. We also note that in their original paper, Sinuany-Stern et al. (1994) performed sensitivity analysis on these data by dropping the output variable number of publications because none of the departments were relatively inefficient in this output. A systematic check

9 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) Table 3 Academic department example Variables in analysis Start Step 1 Step 2 Step 3 Step 4 Remaining inputs OperCost OperCost OperCost Salary Salary Salary Salary Salary Remaining outputs Grants Grants Public Public Public Public Public GradStd ConHrs ConHrs ConHrs ConHrs Variable dropped GradStd Grants OperCost ConHrs Department Efficiency scores Geography History Education Economics Hebrew Lit Behavioral Sci Bible Foreign Lit Social Work Philosophy Biology Geology Chemistry Math Physics Nuclear Eng Chemical Eng Material Eng Electrical Eng Mechanical Eng Ind. Eng. and Manage # Efficient DMUs Average change in efficiencies of dropping each input and output variable through a stepwise approach identifies several variables with less impact on efficiency scores and on the number of efficient DMUs than number of publications. Finally, the basis for determining variable selection using our stepwise approach is quite different than the method proposed by Jenkins and Anderson, thus when the two methods are applied to the same dataset of academic departments, the results are also different Sample problem: Baseball teams (forwards approach) If a core model of one input and one output can be determined as a starting point, then the stepwise method can also be adapted to add variables to the DEA model instead of dropping them. In the forwards stepwise approach, the goal would be to identify variables which cause the largest difference in total efficiency scores. An example of the forwards stepwise approach is presented here using another data set from Cooper et al. (2000) involving 12 Japanese baseball teams. The two input variables are total salaries paid to managers and total salaries paid to players and the two output variables are a measure of team power and attendance at the games. We again used an input-oriented, constant returns to scale DEA model for this example. In this example we selected the starting point to be represented by a core model including the input players salaries and the output team power. From among the set of variables, this pair made intuitive sense. The results of the forwards stepwise analysis are summarized in Table 4. At the Start, with only one input and one output, only the team BlueWave is identified as efficient. In Step

10 66 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) Table 4 Japanese baseball team example Variables in analysis Start Step 1 Step 2 Existing inputs Existing outputs Variable added 1, adding the input managers salaries has a larger effect on average efficiency scores than adding the output attendance. At this point another team, the Dragons, becomes efficient. In Step 2 the final remaining variable, attendance, is added to the model. Now four teams are identified as being efficient, including the BayStars and the Hawks. The stepwise approach has reached the last step now that all input and output variables have been included in the DEA model. In the forwards approach, a stopping rule could represent a minimum change in average efficiency scores required to bring about an addition of another input or output variable into the model. 4. Alternative methods Manager Sal Manager Sal Player Sal Player Sal Player Sal Team Power Team Power Team Power Attendance Manager Sal Attendance Teams Efficiency scores Swallows Dragons Giants Tigers BayStars Carp Lions Fighters BlueWave Buffalos Marines Hawks # Efficient DMUs Average change in efficiencies The stepwise approach to variable selection in DEA, as developed in this paper, provides a framework for several alternative implementations. As previously mentioned, as long as a consistent model is used in each step, the stepwise methods can be used with a variety of DEA models. Variable or constant returns to scale can be used, as well in input or output oriented models. The method to determine which variable to delete at the end of each step can also be implemented in several ways. In the procedure that we described in Section 3, the variable to be deleted is chosen as the one with the least influence (i.e., for the backwards approach, the variable giving the smallest average change in efficiency scores). Alternatively, one could delete the variable with the most influence in order to identify important variables and to determine their impact on the analysis. It would also be possible to delete the variable that retained the most number of fully efficient DMUs, using the minimum average change in efficiency scores as a tie-breaker, if necessary. In the examples in this paper, for the sake of illustration, we continued the backwards stepwise procedures until there was only one input and one output variable remaining (hence stopping when there were no variables left to be removed). However alternative stopping rules can be developed, and in most cases would be desirable. Some possible stopping rules include: (1) when the average difference in efficiency scores exceeds some maximum level, (2) when the change in any one efficiency score exceeds some maximum level, or (3) when the number of efficient DMUs falls below some minimum number. 5. Conclusion In this paper, a stepwise procedure was developed to assist in determining a useful but parsimonious set of input and output variables for a DEA analysis. Our method acts directly upon information regarding the change in efficiency scores. It is objective, requires minimal additional calculation or data storage, and is easy to do and explain. As well as developing this stepwise method, we also emphasize how this method provides valuable managerial information to the decision maker that is not available from traditional DEA analysis alone. Previous DEA studies have stressed the need to go beyond the initial results of DEA modeling, which is identification of the set of fully efficient DMUs. Stepwise DEA analysis, as developed in this paper, yields important information for each efficient DMU as to which input or output variable has the most influence in maintaining that efficiency. Also, information about the impact on inefficient DMUs by removing or adding input or output variables provides additional guidance as to the importance of factors in maintaining efficiency at the current levels. Further, stepwise DEA identifies a core model

11 J.M. Wagner, D.G. Shimshak / European Journal of Operational Research 180 (2007) that relates a single input and single output variable that are the primary determinants of efficiency. The examples in this paper illustrate the applicability and value of stepwise DEA. In the hotel chain example, backwards stepwise DEA modeling identified three input variables that simply had no effect on the efficiencies a result not recognized by existing methods aimed at reducing the number of variables in DEA studies (Jenkins and Anderson, 2003). The Tokyo library and the academic department examples showed that stepwise DEA modeling could be used on larger, realistic problems, and that additional information could be derived to evaluate the efficient units. One of these example was an input-oriented, variable returns to scale model and the other a constant returns to scale model. The Japanese baseball team example demonstrated the possibility of conducting forwards stepwise DEA modeling and the circumstances under which this method might be applicable. In summary, the managerial information gained by performing a set of simple calculations and a few extra runs using stepwise DEA modeling seems well worth the cost of this additional effort. As is true with stepwise methods for variable selection in multiple regression statistical techniques, we are aware of the danger in relying on a purely mechanistic approach. While our stepwise procedure can inform analysts and decision makers of the effect of adding and removing variables in a DEA study, the determination of the best model to represent any given situation must rely on managerial judgment and knowledge of the operations of the actual situation being represented. References Banker, R.D., Maximum likelihood, consistency and data envelopment analysis: A statistical foundation. Management Science 39 (10), Banker, R.D., Hypothesis tests using data envelopment analysis. Journal of Productivity Analysis 7, Boussofiane, A., Dyson, R.G., Thanassoulis, E., Applied data envelopment analysis. European Journal of Operational Research 52, Charnes, A., Cooper, W.W., Rhodes, E., Measuring efficiency of decision making units. European Journal of Operational Research 2, Chilingerian, J.A., Evaluating physician efficiency in hospitals: A multivariate analysis of best practices. European Journal of Operational Research 80, Cooper, W.W., Seiford, L.M., Tone, K., Data Envelopment Analysis. Kluwer Academic Publishers., Norwell, MA, pp Friedman, L., Sinuany-Stern, Z., Combining ranking scales and selecting variables in the DEA context: The case of industrial branches. Computers and Operations Research 25 (9), Golany, B., Roll, Y., An application procedure for DEA. OMEGA 17 (3), Jenkins, L., Anderson, M., A multivariate statistical approach to reducing the number of variables in data envelopment analysis. European Journal of Operational Research 147, Kittelson, S.A.C., Stepwise DEA: Choosing variables for measuring technical efficiency in Norwegian electricity distribution, Memorandum No. 06/93, Department of Economics, University of Oslo, Norway. Klimberg, R., Puddicombe, M., A multiple objective approach to data envelopment analysis, Working paper 95-05, School of Management, Boston University, MA. Lewin, A.Y., Morey, R.C., Cook, T.J., Evaluating the administrative efficiency of courts. OMEGA 10 (4), Lovell, C.A.K., Pastor, J.T., Target setting: An application to a bank branch network. European Journal of Operational Research 98, Norman, M., Stoker, B., Data Envelopment Analysis: The assessment of performance. John Wiley and Sons, Chichester, England. Nunamaker, T.R., Using data envelopment analysis to measure the efficiency of non-profit organizations: A critical evaluation. Managerial and Decision Economics 6 (1), Pastor, J.T., Ruiz, J.L., Sirvent, I., A statistical test for nested radial DEA models. Operations Research 50 (4), Ragsdale, C.T., Spreadsheet Modeling and Decision Analysis, fourth ed., South-Western, Mason OH, pp Salinas-Jimenez, J., Smith, P., Data envelopment analysis applied to quality in primary health care. Annals of Operations Research 67, Sigala, M., Airey, D., Jones, P., Lockwood, A., ICT paradox lost? A stepwise DEA methodology to evaluate technology investments in tourism settings. Journal of Travel Research 43, Sinuany-Stern, Z., Mehrez, A., Barboy, A., Academic departments efficiency via DEA. Computers and Operations Research 21 (5),