EVALUATING PORT EFFICIENCY IN ASIA PACIFIC REGION WITH RECURSIVE DATA ENVELOPMENT ANALYSIS

Transcription

1 EVALUATING PORT EFFICIENCY IN ASIA PACIFIC REGION WITH RECURSIVE DATA ENVELOPMENT ANALYSIS Hsuan-Shih Lee Professor Department of Shipping and Transportation Management National Taiwan Ocean University Keelung 0, Taiwan. Fax: Ming-Tao Chou Assistant Professor Department of Aviation and Maritime Management Chang Jung Christian University Taiwan 7, Taiwan. Fax: Sen-Guei Kuo Master Student Department of Shipping and Transportation Management National Taiwan Ocean University Keelung 0, Taiwan. Fax: Abstract: Maritime transportation for Taiwan plays an important role in trade exchange with other countries, especially in the container cargoes. The performance and efficiency of container ports is a critical factor for handling containers on the international market in maritime. However, how to raise operation efficiency and reduce operation cost is a very important issue. Evaluating operation efficiency of ports can reflect theirs status quo and reveal their advantages and disadvantages in the competitive environment. In this study, we are going to apply a new procedure based on DEA (Data Envelopment Analysis) called RDEA (Recursive Data Envelopment Analysis) to rank selected container ports in Asia Pacific region. DEA is a mathematical programming skill for measuring the relative performance of decision making units (DMUs). Basically, DEA is a nonparametric approach that does not require any assumption about the function from of the production function. However, it is controversy to rank DMUs based on their relative efficiencies, especially when their efficiencies are less than one. In this study, we will compare the ranks of ports obtained by DEA and RDEA respectively and drill down the result to unveil tactical moves needed to improve their efficiencies. Key Words: DEA (Data Envelopment Analysis), RDEA (Recursive Data Envelopment Analysis), Decision making units (DMUs), Container port. INTRODUCTION As nations are becoming more global and their industries more exposed to the pressures of international competition, there is a growing realization that services supplied to their industries must be provided on an internationally competitive basis. Thus, there is a push amongst port authorities to improve their port performance and efficiency due to increasing competition between ports and growing pressure from shippers for lower port and shipping charges. Ports from a vital link in the overall trading-chain, their level of efficiency and performance determine to a large extent a nation s international competitivenes. However, in

2 order to achieve and maintain a competitiveness edge in the international markets, port authorities need to understand the underlying factors of port competitiveness, and continually asses its performance relative to the rest of the world so that appropriate business strategies can be established. Data Envelopment Analysis (DEA) is a technique for measuring the relative efficiencies of decision-making units (DMUs) that use similar inputs to produce similar outputs where the multiple inputs and outputs are incommensurate in nature. DEA has been one of the fastest growing areas of Operations Research and Management Science in the past two decades. DEA has been applied to a wide variety of managerial and economic situations in both the public and private sectors. However, problems have also arisen as the applications of DEA advance. Two inter-related problems have long been recognized. The first is the weak discriminating power where all efficient DMUs have efficiency scores of one, receive the same ranking, and thereby make them incomparable with each other, owing to the constraint in the model that efficient scores of DMUs cannot be greater than one. The second is that it is questionable to rank those inefficient scores according to their efficient coefficients. In this article, we are going to propose a new procedure, based on DEA, called RDEA (Recursive DEA), to rank for all DMUs. We apply DEA as a basic tool to distinguish DMUs into two dichotomy groups, namely, the efficient and inefficient. Then, we apply DEA recursively to the inefficient DMUs. A hierarchy of dichotomy of DMUs can thus be established. A total ranking based on the hierarchy can be obtained. The remainder of this paper is structured as follows. Section, presents the Literature Review of DEA. Section, our method RDEA identifies those efficient units on the frontier and ranks them by the super-efficiency methodology. The RDEA of our method is also evaluated in section. Traditional DEA model result and our method are presented, which show that our method out performs traditional DEA method. Conclusions are given in section.. LITERATURE REVIEW It has been recognized that business performance is a multidimensional construct, since it covers diverse purposes and types of organizations / levels (Lewin and Minton, 8). Because a company s performance is a complex phenomenon requiring more than asingle criterion to characterize it, what a consequence is a number of studies has been argued that a multi-factor performance measurement model may be used (Bagozzi and Philips, 8; Chakravarthy, 8). The development of a multi-factor performance measure which

3 reconciles diverse financial performance measures is important because it may let policy makers to know how far a particular industry or company can be expected to increase its multiple outputs and decrease its inputs levels by simply improving its efficiency, without absorbing or wasting further resources (Zhu, ). DEA, which was developed by Charnes et al. (78) and expanded by Banker et al. (8), measures the performance of DMUs (DMU), and is recognized as a valuable decision support tool for managerial control and organizational diagnosis. To date, it has been applied successfully in various managerial contexts. The mathematical programming of DEA is particularly adept at estimating multiple inputs and multiple outputs production correspondences. It is a linear programming based method designed to assess the relative efficiencies of organizational, or more commonly, DMUs. Typically, these DMUs use the same multiple inputs in varying quantities to secure varying quantities of the same multiple outputs (Thanassoulis et al., ). DEA is a mathematical programming technique for measuring the relative performance of DMUs on the basis of the observed operating practice in a sample of comparable DMUs. DEA has typically been applied to analyze the relative productive efficiency of DMUs in a setting of multiple incommensurate input variables and multiple incommensurate output variables. A key element of the DEA methodology is constructing hypothetical composite DMUs, i.e., combinations of fractions of one or more observed units, to act as comparators for observed DMUs. The construction method employed implicitly reflects certain assumptions regarding the technical production relationships underlying the production process. In the standard DEA model formulation (Charnes et al., 78), all non-negative efficiency score values are permitted, and the performance levels of the composite units are assumed efficiency score weighed averages of the performance levels of the constituent observed DMUs. This construction method implicitly assumes continuous, locally linear input substitutability and output transformability, and constant returns-to-scale (RTS) properties. Although the standard model implicitly makes such simple production assumptions, the DEA methodology offers considerable flexibility to incorporate alternative or additional production information. DEA was initiated in 78 when Charnes, Cooper and Rhodes (CCR) demonstrated how to change a fractional linear measure of efficiency into a linear programming (LP) format. As a result, DMUs (DMUs) could be assessed on the basis of multiple inputs and outputs, even if the production function was unknown. This non-parametric approach solves an LP formulation per

4 DMU and the weights assigned to each linear aggregation are the results of the corresponding LP. Various theoretical extensions have been developed, based on the original CCR model: Banker et al. (8) developed a variable RTS variation; the multiplicative model was developed by Charnes et al. (8) in which the data is transformed using a logarithmic structure; Charnes et al. (8b) developed the additive variation, in which the objective function contains slack variables alone. Seiford and Thrall (0) provide a useful discussion and comparison of all the basic models available to date in DEA. Many additional theoretical papers in the field have adapted the models to deal with problems that have occurred in practice. One adaptation has been in the field of ranking DMUs. The basic DEA results group the DMUs into two sets, those that are efficient, and define the Pareto frontier and those that are inefficient. In order to rank all the DMUs, another approach or modification was required. Often decision-makers are interested in a complete ranking, beyond the dichotomized classification, in order to refine the evaluation of the units. One problem that has been discussed frequently in the literature has been the lack of discrimination in DEA applications, in particular when there are insufficient DMUs or the number of inputs and outputs is too high relative to the number of units. This is an additional reason for the growing interest in complete ranking techniques. Furthermore, fully ranking units is an established approach in the social sciences, in general (Young and Hamer, 87), and in multiple-criteria decision-making (MCDM), in particular. Many papers on DEA and ranking (over 0) have been published over the last decade within the DEA context. Since most decision-makers are interested in a complete ranking, the use of such techniques aids in marketing the DEA approach. It should be noted that the methods discussed here could be considered post analyses since they do not replace the standard DEA models but rather provide added value. This review describes the ranking methods developed in the literature and since many articles have been published in this field, and next section we are going to present a multi-scenario approach for the performance ranking of DMUs. With this approach, each DMU can be evaluated more objectively in a multi-facet manner.. A NEW METHOD FOR THE RDEA RANKING METHOD The results obtained using the traditional DEA model groups the DMUs into two sets: efficient and inefficient. Since most decision-makers are interested in a comprehensive ranking beyond this simple dichotomized classification, we are going to propose a method called Recursive DEA (RDEA) to completely rank all the DMUs to meet the needs for decision-makers. 7

5 The RDEA, the new integrated model we suggest here for rank scaling organizational units, is composed of two main stages. In the first stage, the DEA is run for each unit separately. For those efficient DMUs, the super-efficiency method (Andersen and Petersen, ) is applied to rank the efficient units. In the second stage, those inefficient DMUs are evaluated recursively with stage. The total ranking of DMUs can obtain by combining the results of stage and stage. Conceptually, our method RDEA identifies those efficient units on the frontier and ranks them by the super-efficiency methodology, then removes them from the reference set. The frontier of the new reference set is then identified and removed. This process is repeated until the reference set empty.. The DEA Model The basic DEA model, detailed below, is also called the CCR version (Charnes et al., 78). Suppose we have n DMUs, where each DMU j (j=,,n) produces s output y rj (r=,,s) by utilizing m inputs x ij (i=,,m). DEA use the following measure of performance for DMU j : h j s u y r r m v x rj i ij i where v i (i=,m) and u r (r=,s) are the input and the output weights for the combined m inputs and s outputs for DMU j, respectively. The weights in Equation are determined for each DMU j by the following mathematical programming problem (Charnes et al., 78) : where * h0 max h urvi v, u 0 i r o s.t. h j,,, n j h o s r m u y r v x ro i i io represents the ratio of aggregated outputs to aggregated inputs for one of the n DMUs, denoted as DMU i, (i=,,n) and y ro are respectively the ith input and rth output of DMU 0. By varying i over (,,n), we obtain all the DEA scores, h * j, with n sets of optimal weights. It is clear that the bigger the h * j, the better the performance of DMU j, since DMU j produces more aggregated output or uses less aggregated inputs. However, the highest possible value of * h is one, because of the constraints in model (). If h, then DMU j is relatively efficient. j*, j 8

6 Model () can lead to a large number of DMUs having DEA scores of one, which makes comparisons impossible. To avoid this possibility, they employ a DMU ranking procedure developed by Andersen and Petersen (). The method we called the super-efficiency methodology. According to Zhu (00), possible infeasibility of super-efficiency DEA models is only associated with extreme-efficient DMUs and indicates efficiency stability to data perturbations in all DMUs. Another type of DEA analysis is based on super-efficiency has been proposed (Seiford and Zhu (), Charnes et al. (), Rousseau and Semple () and Chernes et al. ()). That is, by excluding the constraint for the DMU 0, h0, in model (), and we obtain max vi, ur o s. t. h j h j 0 and v, u 0 For the sake of computation, we may use the following linear programming problem which is equivalent to model () by duality =min s. t. x x i,, m, r,, s,, ( j 0) 0. j i r j ij 0 i0 j, j o j xij yr 0 j, j o * Then the optimal values in model () (or model ()), 0 can be either less than, equal to, or greater than one. Now we are able to rank the DMUs according to their aggregated output to aggregated input ratios given by The RDEA Model We will also present procedure to compute RDEA method in below section. List RDEA_SORT (List L) { List EL, NEL; If L=Null then Return Else {

7 Rank L descending by calculating efficiency scores of DMUs in L with model (); Break L into EL and NEL two parts, such that EL is the list of DMUs with while NEL is the list of DMUs with efficiency less than ; Rank EL descending by calculating efficiency score of DMUs with model (), such that EL is the list with efficiency equal or greater than ; Return EL+RDEA_SORT (NEL); } } According to above procedure for fully ranking DMUs, a procedure analysis can tell us not only the appropriate form for the RDEA model, but may also provide us with good estimates of the numerical value of its ranks. These values can be checked by analysis of the CCR model and super-efficiency model. Figure summarizes the iterative approach to RDEA model building for procedure and control, which is employed in this article. Step. Rank L descending by calculating efficiency scores of DMUs in L with model () and the efficiency score of port data run by the traditional DEA. Step. Break L into EL and NEL two groups, such that EL is the list of DMUs with while NEL is the list of DMUs with efficiency less than. We apply DEA as a basic tool to distinguish DMUs into two dichotomy groups, namely, the efficient and inefficient. Step. Rank EL descending by calculating efficiency score of DMUs with model (), such that EL is the list with efficiency equal or greater than. According to the first efficient Group of DMUs data and ranking of DMUs by model (). Postulate General Class of CCR model Identify efficiency scores of DMUs Diagnostic Checking (is the efficiency scores greater than one?) No Yes The Super-Efficiency method Ranking Figure. Stages in The Iterative Approach to Model Building 0

8 . EFFICIENCY MEASUREMENT OF SELECTED INTERNATIONAL PORTS WITH OUR METHOD. Measuring International Port Efficiency with DEA The DEA technique is a useful measurement of port efficiency because the calculations are non-parametric, they can handle more than one output and they do not require an explicit a priori determination of relationships between output and inputs, as is required for conventional estimations of efficiency using production functions. Jose Tongzon () used the DEA method to measure the eficiency of ports. In Tongzon s study, selected international ports were measured. Six container ports were assessed as being inefficient and ten as being efficient. However, as noted earlier, a study such as this does not provide adequate information for managers and other decision makers. They cannot determine the relative efficiency of the 0 ports found to be efficient. Therefore, to determine the relative efficiency of the 0 ports deemed efficient, we have run the RDEA model, as outlined in section, using the data from Jose Tongzon s study, as demonstrated in the following pages.. Port Input and Output Measures In the DEA, more than one output measure can be specified. A number of different measures of port output are available, depending on which features of the port s operations are being evaluated. This article uses two outputs and six inputs to measure of port performance for the year the year for which the data on port throughputs are available. The output measures utilized are cargo throughput and ship working rate. The first output measure (cargo throughput) is the total number of containers loaded and unloaded in 0-foot equivalent units (TEUs). This output relates to the need for cargo-related facilities and services. Secondly, since ships are major port users, the second output is the ship-working rate. Ship working rate measures the number of containers moved per working hour per ship and thus is an indicator of the speed with which ships are worked. This measure can represent the level and quality of port service. Since the container handling aspect of port operation is the largest component of total ship turnaround time, the speed of moving cargoes off and onto ships at berth has considerable implications for the port users. Moreover, improving efficiency in this area is consistent with port authority intentions of maximizing berth utilization, a factor that will influence both port charges imposed on ship-owners and the actual throughput handled. To produce the above outputs and to facilitate port operations, varieties of inputs are required.

9 Based on the production framework, port inputs can be generalized as land, labor and capital. The major capital input in port operations are the number of berths, cranes and tugs. The most fundamental labor input is the number of stevedoring labor. However, due to a lack of information on this particular variable, a proxy variable is used represented by the number of port authority employees for the respective ports. This proxy variable is less difficult to obtain because it is usually published in the annual reports of some ports. The land input variable is defined as the terminal area of the port. In addition, is the amount of delay time which is the difference between total berth time plus time waiting to berth and the time between the start and finish of ship working, is an indicator of how well working time is being used. These delays could be due to labor disputes, work practices such as meal breaks, equipment breakdown, port congestion, perceived ship problems or bad weather. The values of these inputs and outputs are defined as follows and detailed in Table : Y= TEUs handled, Y= Ship rate, X= No. of Cranes, X= No. of Container berths, X= No. of tugs, X= Terminal area (m ), X= Delay time (h) and X= Labor (units). Table. Port Data of International Ports DMU Port TEUs handled Shiprate Delay Delay Terminal Labor No. of No. of No. of no Time /time Area Units Berths Cranes Tugs Y Y Y X X X X X Melbourne 0, ,8,00 8 Hong Kong,0, ,8, Hamburg,0, ,00,000,8 Rotterdam,, ,8, Felixstowe,0,. 0..7,,000,8 Yokohama,, ,8, Singapore,, ,7, Keelung,0, ,000 0 Sydney,.8. 0.,,00 0 Fremantle 0, , Brisbane, , Tilbury, , Zeebrugge, ,,00 La Spezia 87, , Tanjung Priok,, ,000, 0 Osaka 87, ,,000,070 0

10 . Results by RDEA Model Each DMU in Table was calculated by the DEA and the RDEA method respectively as the same procedure we have done in section three. Moreover, the results were generated as shown in the following Tables. Step. Rank L descending by calculating efficiency scores of DMUs in L with model () and the efficiency score of port data run by the classical DEA in Table. Table. The Efficiency Score of Port Data Run by The Classical DEA (Model ()) DMU No. Ports Efficiency Sore Melbourne Hong Kong Hamburg Rotterdam Felixstowe Yokohama Singapore Keelung Sydney Fremantle Brisbane Tilbury Zeebrugge La Spezia Tanjung Priok Osaka Step. Break L into EL and NEL two parts, such that EL is the list of DMUs with while NEL is the list of DMUs with efficiency less than. We use DEA as a basic tool to distinguish DMUs into two dichotomy groups, namely, the efficient and inefficient. In Table, Group and Ranking of DMUs by using DEA method. Step. Rank EL descending by calculating efficiency score of DMUs with model (), such that EL is the list with efficiency equal or greater than. According to the first efficient Group of port data and ranking of DMUs by model (). Table shows Efficiency score and Ranking of DMUs by model (). Step. Return. According to the second efficient Group of port data and ranking of DMUs by model (). In Table and Table shows Group and Ranking of DMUs by using DEA

11 method. Step. According Table to Table, we can combine Efficiency Scores of each group and their ranking by the RDEA Method. Table 7 Efficiency Scores of each group and their ranking by the RDEA Method. Table. Group and Ranking of DMUs by Using DEA Method (Port Data) DMU No. Ports Efficiency Score Rank Group Hong Kong Hamburg Rotterdam Felixstowe Efficiency 7 8 Singapore Keelung Brisbane Tilbury Zeebrugge La Spezia 0 Tanjung Priok Yokohama Fremantle Sydney Melbourne Osaka Inefficiency Table. Efficiency Score and Ranking of DMUs by Model () DMU No. Ports Efficiency Score Rank Group Zeebrugge. Felixstowe.7 Hamburg.7 7 Singapore.7 Efficiency La Spezia.0 Tilbury.0 Hong Kong Keelung.0 8 Brisbane.8 Rotterdam.0 0

12 Table. Efficiency Score of DMUs by The DEA Method DMU No. Ports Efficiency Score Group 0 Yokohama Sydney Fremantle Efficiency Tanjung Priok Osaka Melbourne 0. Inefficiency Table. Efficiency Score and Ranking of DMUs by Model () DMU No. Ports Efficiency Score Rank Yokohama 7.80 Osaka.7 Sydney. Tanjung Priok.8 0 Fremantle.8 Table 7. Efficiency Scores of Each Group and Their Ranking by The RDEA Method DMU No. Port The st group The nd group The rd group RDEA Rank Zeebrugge.7 Felixstowe.7 Hamburg.7 7 Singapore.7 La Spezia.0 Tilbury.0 Hong Kong Keelung.0 8 Brisbane.8 Rotterdam.0 0 Yokohama 7.80 Osaka.7 Sydney. Tanjung Priok.8 0 Fremantle.88 Melbourne 0.

13 Table 8 displays the rank obtained for each DMU by the DEA and the RDEA method. The last column, change, notes the difference in ranking for each DMU based on the results of the RDEA method. As we can see, those DMUs found to be efficient in the Tongzon article can now be compared using the results obtained from the RDEA method, whereas by the DEA results they cannot, because they all receive the same score and rank. Moreover, the inefficient DMUs can now be ranked more accurately using the results from the RDEA method. In Tongzon s paper, 0 ports had a ranking of, and therefore are incomparable. In our study, those 0 ports can now be compared. In addition, the inefficient ports had changes in ranking, demonstrating that more than just going past the dichotomized ranking, the RDEA method obtains more accurate scores resulting in changes in rank. For example, the port of Osaka is ranked last in the Tongzon study, but in our study it s rank has changed by plus, ranking number. Thus, the RDEA model we have utilized has a number of advantages over the DEA method. It retains those advantages of the DEA method outlined earlier, while improving on the method by providing a more complete ranking with more accurate scores. Table 8. Comparison of The DEA and The RDEA Method in Ranking DMU No. Ports DEA Rank (Original) RDEA Rank Change Zeebrugge - Felixstowe - Hamburg - 7 Singapore - La Spezia - Tilbury - Hong Kong 7-8 Keelung 8-7 Brisbane -8 Rotterdam 0 - Yokohama + Osaka + Sydney + Tanjung Priok - 0 Fremantle - Melbourne -

14 . CONCLUSIONS The current article, we have proposed a procedure to rank DMUs by applying DEA recursively. With DEA, DMUs can be classified into to two categories, namely, efficient DMUs and inefficient DMUs. Efficient DMUs can be differentiated by relaxing the constraint that the efficient coefficient of target DMU cannot be greater than. However, a paradox ranks the inefficient DMUs with their efficient coefficients. For example, if both DMU A and B are classified as inefficient and their efficient coefficients are 0.8 and 0.8 respectively, we may not conclude that A is more efficient than B. To differentiate inefficient DMUs, we apply DEA recursively to inefficient DMUs. Those inefficient DMUs can then be classified into two categories under the new context. To justify our procedure, we have applied our method to selected international ports. Sixteen international ports are ranked with three passes. It has been shown that the results are convincing. DEA is a nonparametric approach that does not require any assumption about the function from of the production function. However, it is controversy to rank DMUs based on their relative efficiencies, especially when their efficiencies are less than one. This article has demonstrated that our multi-scenario ranking method based on RDEA provides a viable method of evaluating relative port efficiency and port performance ranking. DEA is appealing in measuring port efficiency for its ability to handle multiple inputs and multiple outputs, which depend on which features of port operation are being evaluated. Though only CCR is demonstrated in this study, our ranking method can be extended to other DEA models with input and output multipliers. To justify our procedure, we have applied our method to selected international ports. Sixteen international ports are ranked with three passes. It has been shown that the results are convincing. In shorts, there arises two common problems in applying DEA. The first is that efficient DMUs must be little. The second is that inefficient DMUs are not differentiable. These two problems are resolved by RDEA method. Further studies related to this topic should include the RDEA method of the effects under relate completive condition, and RDEA method analysis can well be applied to inefficient DMUs if we are interested in efficiency and inefficiency DMUs ranking. ACKNOWLEDGEMENTS This research work was supported by the National Science Council of the Republic of China under grant No. NSC --H

15 REFERENCES Andersen, P., Petersen, N.C. () A procedure for ranking efficient units in data envelopment analysis, Management Science, Vol. No.0,. Bagozzi, R.P. and Phillips, L.W. (8) Representing and Testing Organizational Theories: A Holistic Construal, Administrative Science Quarterly, Vol. 7, -8. Banker, R.D., Charnes, A. and Cooper, W.W. (8) Some models for estimating technical and scale inefficiencies in data envelopment analysis, Management Science, Vol.0 No., Chakravarthy, B.S. (8) Measuring Strategic Performance, Strategic Management Science, Vol.7, 7-8. Charnes, A., Cooper, W.W. and Rhodes, E. (78) Measuring the efficiency of decision-making units, European Journal of Operational Research, Vol.,. Charnes, A., Cooper, W.W., Seiford, L. and Stutz, J. (8) A multiplicative model for efficiency analysis, Socio-Economic Planning Sciences, Vol.,. Charnes, A., Haag, S., Jaska, P. and Semple, J. () Sensitivity of efficiency classification in the additive model of data envelopment analysis, International Journal of System Science, Vol., Charnes, A., Cooper, W.W., Golany, B., Seiford, L. and Stutz, J. () Foundations of data envelopment analysis for Pareto Koopmans efficient empirical production functions, Journal of Econometrics, Vol.0, 07. Charnes, A., Rousseau, J. and Semple, J. () Sensitivity and stability of efficiency classification in data envelopment analysis, Journal of Production Analysis, Vol.7, 8. Tongzon, J. () Determinants of Port Performance and Efficiency, Transportation Research Part A: Policy and Practice, Vol., No., -. Lewin, A.Y. and Minton, J.W. (8) Determining Organizational Effectiveness: Another Look, and an Agenda for Research, Management Science, Vol., -. Rousseau, J. and Semple, J. () Radii of classification preservation in data envelopment analysis, Journal of Operation Research, Vol., 7. Seiford, L.M. and Zhu, J. () Infeasibility of Super Efficiency data envelopment Analysis, INFOR., Vol.7, Seiford, L.M. and Thrall, R.M. (0) Recent developments in data envelopment analysis: The mathematical programming approach to frontier analysis, Journal of Econometrics, Vol., 7 8. Thanassoulis, E., A. Boussofinae, and Dyson, R. G.. () Exploring Output Quality Theories: A Holistic Construal, Administrative Science Quarterly, Vol.7, -8. 8

16 Young, F.W. and Hamer, R. M. (87) Multidimensional Scaling, History, Theory and Applications. Lawrence Erlbaum. London. Zhu, J. () Robustness of the efficient decision-making units in data envelopment analysis, European Journal of Operational Research, Vol.0, 0. Zhu, J. () Data envelopment analysis with preference structure, Journal of the Operational Research Society, Vol.7, 0. Zhu, J. (00) Super-efficiency and DEA sensitivity analysis, Journal of the Operational Research Society, Vol.,.