Rank reversal phenomenon in cross-efficiency evaluation of data envelopment analysis

Transcription

1 Rank reversal phenomenon n cross-effcency evaluaton of data envelopment analyss Yong B. Shn Francs Maron Unversty, South Carolna, USA Keywords Multple Crtera Decson Makng (MCDM), Rank reversal, and Least Common Multple (LCM) Abstract Ths paper presents that the rank reversal occurs n other popular MCDM approach as well, cross-effcency evaluaton of data envelopment analyss (DEA), whch has been alternatve method for rankng decson makng unts (DMU) n the data envelopment analyss (DEA). Ths paper also attempts to llustrate that the proposed least common multple (LCM) approach successfully addresses these rank reversal problems n decson support systems area. 1. Introducton Analytc herarchy process (AHP) (Saaty, 2009) has become a very popular multple crtera decson makng (MCDM) technque. AHP has been appled to dverse felds of study such as software evaluaton, manufacturng systems, organzatonal performance evaluaton, customer requrement ratng, and fnancal ndustres. However, for nearly the same duraton, AHP has also been crtczed for rank reversals when a decson alternatve s added or dropped, frst noted by Belton & Gear (1983). In order to avod rank reversal phenomenon n the AHP where such reversals should not take place, many other dfferent mathematcal approaches (Dyer, 1985; Schoner & Wedley & Choo, 1993; Barzla & Golany, 1994; Lootsma, 1999) have been proposed. It s notceable that none of these methods have resolved ths problematc phenomenon and there are stll on-gong debates on how to avod rank reversals. Rank reversal s also found n other popular MCDM approach as well, such as the crosseffcency evaluaton of data envelopment analyss (DEA). Ths paper llustrates that the rank reversals occur n other MCDM method and presents that the proposed method successfully addresses these rank reversal problems n decson support systems area. 2. Rank reversal n the cross-effcency evaluaton of data envelopment analyss (DEA) There are a wde range of MCDM problem soluton technques, varyng n complexty and possble solutons. Each method has ts own strength, weaknesses and possbltes to be appled. For example, Borda-Kendall (BK) method s the most wdely used tool n determnng a consensus rankng because of ts computatonal smplcty. It uses a weghted ordnal rankng model n whch each of a set of n alternatves was gven an ordnal rank on a set of crtera. Smple Addtve Weghtng (SAW) s smple and the most frequently used multple attrbute decson makng (MADM) tool. However, t s obvous that these popular MCDM approaches also suffer from rank reversal. In ths secton we llustrate that the rank reversals occur n crosseffcency evaluaton of data envelopment analyss (DEA) MCDM approach. Cross-effcency evaluaton (Doyle & Green, 1994; Sexton and Slkman, 1986) has been alternatve method for rankng decson makng unts (DMU) n the data envelopment analyss (DEA) (Chrnes & Cooper, 1978). It uses self or peer evaluatons for performance assessment of DMUs. Consder n DMUs that are to be evaluated n terms of m nputs and s outputs. Let x j ( = 1,, m) and y j (r = 1,, s) be the nput and output values of DMU j (j = 1,, n). Then, the effcences of the n DMUs can be wrtten as, j = 1, n. where v ( = 7th Internatonal Trade & Academc Research Conference (ITARC), 7-8 November 2016, London, UK 60

2 1,, m) and u r (r = 1,, s) are nput and output weghts. For a specfc DMU k, effcency reltve to the other DMUs can be measured by the followng CCR model [3]: Maxmze (4) Subject to where j = 1,, n,, and -, r = 1,, s, and Let and be an optmal soluton to the above model (4). When =1 (where ), DMU k s referred to as the CCR-effcency or smple effcency of DMU k, whch s the best relatve effcency. f, ts 1, t s referred to as non-dea effcent. And s referred to as cross-effcency of DMU j, whch reflects the peer-evaluaton of DMU k to DMU j (j = 1,, n; j k).all DEA effcent unts determne an effcent fronter. CCR model [6] s computed for each DMU, ndvdually. As a result, there are n sets of nput and output weghts for the n DMUs. Each DMU has (n-1) cross-effcences plus one CCReffcency (Table 1). Snce ths model may have multple optmal solutons, ths non-unqueness could potentally hamper the use of cross-effcency. To resolve ths problem, Sexton & Slkman (1986) suggested the aggressve formulaton for cross-effcency evaluaton, whch mnmzes the cross-effcences of the other DMUs to avod the arbtrarness of cross-effcency. Mnmze (5) Subject to -, -,, and where j = 1,, n; j k, r=1,, s, and, s the CCR-effcency of DMU k. DMU Target DMU Average 1 2 n Cross-effcency 1 2 : : : : : n Table 1: Cross-effcency matrx of n DMUs (DMU) Outputs Inputs CCReffcency y1 y2 y3 x1 x2 x Table 2: Data matrx for seven departments n a unversty 7th Internatonal Trade & Academc Research Conference (ITARC), 7-8 November 2016, London, UK 61

3 However, t s found that the cross-effcency evaluaton also suffers from the rank reversal phenomenon when a non-dea effcent unt s added or removed. Consder the example nvestgated by Wong & Beasley (2012). There are seven departments (DMUs) n a unversty to be evaluated n terms of three nputs and three outputs (Table 2), whch are defned as follows: x 1 : Number of academc staff x 2 : Academc staff salares n thousands of pounds x 3 : Support staff salares n thousands of pounds y 1 : Number of undergraduate students y 2 : Number of postgraduate students y 3 : Number of research papers. It s seen from the CCR-effcences n Table 2 that DMU 4 s the only department that s rated as non-dea effcent and all the other sx departments determne an effcent fronter. Table 3 shows the aggressve cross-effcences of the seven departments whch are obtaned by solvng model (5), whch ams to mnmze the cross-effcences of the other DMUs. It s seen that DMU 4 s evaluated as the least effcent department and DMU 6 s the most effcent department. It s reasonably expected that removal of DMU 4 has no mpact on the effcences of the other sx departments because ths DMU 4 s a non-dea effcent department and not on the effcent fronter. Now DMU 4 s removed from the set of DMUs. However, ths removal s found to have a sgnfcant mpact on the cross-effcences of the other sx departments. Table 4 shows the aggressve cross-effcences of the other sx departments after the removal of DMU 4. The rankng between DMU 1 and DMU 6 s reversed wth DMU 1 becomng the best department after DMU 4 s removed from the set of DMUs. However, t also can be observed that the proposed LCM method preserves the orgnal rankngs wth DMU 6 as the most effcent department. (DMU) Target DMU Average Ran k crosseffcenc y LC M Ran k Table 3: Aggressve cross-effcences of the seven departments 7th Internatonal Trade & Academc Research Conference (ITARC), 7-8 November 2016, London, UK 62

4 Perez & Jneno (2006) ponted out that the rank reversal n the AHP could also be caused by the addton or deleton of ndfferent crtera. Ths s also true to the cross-effcency evaluaton. For convenence, we consder an nput or output as unmportant f t makes no contrbuton to CCR-effcency. When an unmportant nput or output s added or removed, the cross-effcency evaluaton may also suffer from the rank reversal phenomenon. (DMU) Target DMU Average Rank LCM Rank crosseffcency Table 4: Aggressve cross-effcences of the sx departments wthout DMU 4 As s known for most multple comparson decson makng problems, n order to get rd of the dmensons of dfferent decson attrbutes, normalzaton s necessary. Examples of the rank reversal seem to depct that the rank reversal s presumably caused by procedural flaws of the normalzaton method. The alternatve approach to yeld the most relable ntal rankng and to preserve the rankng s proposed n next secton. 3. A proposed Least Common Multple (LCM) approach Shn et al. [13] propose an alternatve approach that converts all measurement values of alternatves to the commensurate values by multplyng a least common multple (LCM) of all column sums of crtera n the decson matrx. Before the composte weghts of all alternatves are computed, a matrx, A j s multpled by L, a least common multple of all column sums of crtera, where Now the weght vector of crtera (C j ) s gven by C j = [ c 1 c 2 c c j ] T. Then, multplyng the crtera weght vector C j by the revsed value matrx A j yelds the followng data matrx, X. a c L a c L a c L a c L 1 j j a1 a2 a3 aj x a21 c1l a22 c2l a23 c3l a2 j c jl (2) x2... X j x a a a a x a 1 c1l a 2 c2l a3 c3l aj c jl... a1 a2 a3 aj Fnally, the normalzed composte weghts of alternatves are obtaned from the followng equaton, 7th Internatonal Trade & Academc Research Conference (ITARC), 7-8 November 2016, London, UK 63

5 T x x x x X '... (3) x x x x Because of the converted matrx of the unfed commensurate unt, rank reversal problems n the AHP can be prevented wthout adjustng the weghts of crtera or wonderng about structural or functonal dependency and ndependency. To verfy the valdty of our proposed approach, the next paragraphs present the results of LCM mode by re-examnng the decson matrces used n the cross-effcency evaluaton examples n the prevous secton. Consder the same numercal example n Table 2. It s seen that output 3 has no contrbuton to the CCR-effcences of the seven departments. For an unmportant nput or output, t can be removed from the set of nput or output ndces wthout any mpact on the CCR-effcences. However, Table 5 shows that the rankng between DMU 1 and DMU 6 s reversed after output3 s removed from the set of outputs. These rank reversal phenomena gve rse to a queston. That s whether an unmportant nput or output should be nvolved n the cross-effcency evaluaton. However, the proposed LCM method provdes a consstent rankng of an orgnal set of alternatves and preserves the orgnal rankngs where ether the least effcent DMU or an unmportant nput or output s dropped out from the decson makng data matrx. Addtonally, regardng the rankng of the orgnal set of DMUs, the proposed method yelds DMU 2 as the second best effcent department, whch s dfferent from that computed by the Cross-effcency method n Table 3. (DMU) Target DMU Average Rank LCM Rank cross-effcency Table 5: Aggressve cross-effcences of the seven departments wthout output3 4. Concluson It s assumed that the AHP s a powerful mult-crtera decson makng method and wll contnue to be useful for many future cases as t has been n the past. Despte ths wdespread usage, the AHP stll suffers from some theoretcal dsputes. Rank reversals are also found n many other well-known MCDM methods. Many studes argue that the rank reversal phenomenon s unpreventable when any MCDM method s appled. As seen n the rankngs n Table 3, another prmary crtcsm of MCDM methods s that due to the dfferences among dfferent technques, nconsstent results are obtaned when appled to the same decson problem. It s mportant that a good MCDM method must not yeld the rankng reversals when an alternatve s added or removed. Even though the proposed method does not suffer from those problems, t s more mportant that addtonal research n decson analyss s necessary to produce the relable rankngs one may trust. 7th Internatonal Trade & Academc Research Conference (ITARC), 7-8 November 2016, London, UK 64

6 References Barzla, J. & Golany, B., AHP rank reversal, normalzaton and aggregaton rules, INFOR, 32(2), Belton, V. & Gear, T., On a shortcomng of Saaty s method of analytc herarches. Omega, 11 (3), Charnes, A., Cooper, W. & Rhodes, E., Measurng the effcency of decson makng unts. European Journal of Operatonal Research, 2, Doyle, J. & Green, R., Effcency and cross-effcency n DEA. Journal of the Operatonal Research Socety, 45, Dyer, J. & Wendell, R., A crtque of the Analytc Herarchy Process. Techncal Report, 84/85, of Management. The Unversty of Texas at Austn. Lootsma, F., Mult-crtera decson analyss va rato and dfference judgment. Kluwer Academc Publshers: Dordrecht. Perez, J., Jmeno, J. & Mokotoff, E., Another potental shortcomng of AHP. TOP, 14(1), Saaty, T.L. & Sagr, M., An essay on rank preservaton and reversal. Mathematcal and Computer Modellng, 49, Schoner, B., Wedley, W.C., & Choo, E.U., A unfed approach to AHP wth lnkng pns. European Journal of Operatonal Research, 64, Sexton, T. Slkman, R. & Hogan, A., Data envelopment analyss: Crtque and extensons. Jossey-Bass, San Francsco, CA. Shn, Y.B. & Lee, S.H., Note on an approach to preventng rank reversals wth addton or deleton of an Analytc Herarchy Process. Journal of Educaton Revew, 3(1), Wong, Y. & Beasley, J., Restrctng weght flexblty n data envelopment analyss. Journal of the Operatonal Research Socety, 41(9), th Internatonal Trade & Academc Research Conference (ITARC), 7-8 November 2016, London, UK 65