Electricity Generation Efficiency Measures: Fixed Proportion Technology Indicators

Transcription

1 Electricity Generation Efficiency Measures: Fixed Proportion Technology Indicators Darold T Barnum (corresponding author) Department of Managerial Studies Department of Information & Decision Sciences University of Illinois at Chicago (MC 243) 601 South Morgan Street Chicago, IL USA (v) (f) dbarnum@uic.edu John M Gleason Department of Information Systems and Technology College of Business Administration Creighton University Omaha, NE USA (v) gleason@creighton.edu Darold T Barnum is Professor of Managerial Studies and Professor of Information and Decision Sciences, University of Illinois at Chicago. He earned his Ph.D. in Business and Applied Economics at the University of Pennsylvania. His recent research focuses on DEA theory and application. He has published in ournals such as Ecological Economics, Socio-Economic Planning Sciences, Management Science, Industrial and Labor Relations Review, Applied Economics, Interfaces, International Transactions in Operational Research, IEEE Transactions, and Journal of Transportation Engineering. His forthcoming article in Ecological Economics is a oint environmental and cost efficiency analysis of electricity generation. John M Gleason is Professor Emeritus of Decision Sciences, College of Business Administration, Creighton University, Omaha, NE. He earned his D.B.A. in with double maors in Quantitative Business Analysis and Logistics from the Indiana University Graduate School of Business. His recent research focuses on DEA theory and application. He has published in a variety of ournals, including Socio- Economic Planning Sciences, Management Science, International Transactions in Operational Research, IEEE Transactions on Engineering Management, OMEGA, Industrial and Labor Relations Review, Decision Sciences, Risk Analysis, International Journal of Industrial Engineering, Applied Economics, Journal of Transportation Engineering, Interfaces, and Government Information Quarterly. He has received honors and awards for research productivity from governmental, professional, and university organizations. 1

2 Electricity Generation Efficiency Measures: Fixed Proportion Technology Indicators Abstract Purpose Labor, capital and energy inputs are not substitutable for each other in the generation of electricity. This conflicts with the Data Envelopment Analysis (DEA) requirement of input substitutability. This paper seeks to demonstrate the conflict s impact on DEA efficiency estimates, and to develop efficiency indicators appropriate for fixed proportion technologies. Design/methodology/approach The paper illustrates the effects of nonsubstitutable inputs on efficiency measurement, develops efficiency measures appropriate for fixed proportion technologies, and, using a sample of 70 electricity generation plants, compares the new measures estimates with those of conventional DEA measures. Findings When compared to efficiency measures designed for fixed proportion technologies, DEA badly overestimates the relative efficiency of some plants and badly underestimates the relative efficiency of others. Research limitations/implications The new measures may have undiscovered shortcomings. Hopefully, other researchers will seek to improve them, as well as developing additional new indicators. Practical implications The paper s findings suggest that conventional DEA models should no longer be used to estimate the efficiency of electricity generation plants. Only efficiency indicators accounting for fixed proportion technologies should be used to estimate plant efficiency. Originality/value Because of the energy and environmental crises we are facing, it is critical that power plant efficiency be validly measured. Thus, governmental policy makers, utility managers and other decision makers will base their actions to improve efficiency on correct information. Replacing widely-used DEA models with more valid efficiency measures will help achieve this goal. Keywords: Fixed factor proportion technology, Data envelopment analysis, Efficiency measurement, Electricity generation plants, Energy production efficiency Classification: Research paper 2

3 1. Introduction Electricity Generation Efficiency Measures: Fixed Proportion Technology Indicators In their survey of energy and environmental efficiency studies, Zhou, Ang and Poh (2008) report that a growing number of publications use Data Envelopment Analysis (DEA). They conclude that Considering the importance of energy efficiency study and the ability of DEA in combining multiple factors, it is reasonable to believe that DEA [will] play a more important role in energy efficiency studies in [the] future (Zhou et al., 2008, p. 11). Further, they find that the largest number of these DEA efficiency studies concern electric utilities. The large and growing body of DEA studies of electric utilities is commendable, given the critical importance of energy efficiency and pollution control, and the role played by electricity generation in both (Welch and Barnum, 2009). However, as described later in this introduction, DEA theory requires that resource inputs be substitutable for each other, but electricity generation inputs are not substitutable. Therefore, DEA electricity generation applications conflict with DEA theory. Whether DEA estimates of electricity generation efficiency are materially affected by nonsubstitutable inputs has not been studied. The first essential step for successfully improving electricity generation efficiency is to validly measure efficiency. If efficiency is not measured correctly, then, based on the invalid information, governmental policy makers, utility managers and other decision makers may respond in ineffective or even harmful ways. In short, it is essential that efficiency be correctly measured before effective solutions for its improvement can be implemented. Therefore, it is worthwhile to reconsider the DEA efficiency measures being used, and, if they prove to be significantly biased, then to suggest alternative efficiency measures for this industry. 1.1 DEA theory DEA theory assumes substitutability among inputs. To elucidate, recall that an isoquant identifies the various combinations of two substitutable inputs that yield a constant output when employed by efficient producers. An isoquant can exist only if inputs are substitutable, and, further, the various possible input 3

4 combinations must form a convex set (Petersen, 1990). Farrell (1957, p. 256) states (italics added), The isoquant SS' represents the various combinations of the two factors that a perfectly efficient firm might use to produce unit output.... The curve SS', then, will be taken as the estimate of the efficient isoquant. Charnes, Cooper and Rhodes (1978, p. 437) emphasize (italics added) that their analysis has employed two assumptions which we shall refer to as the 'isoquant' and 'ray assumptions', respectively.... The former, i.e., the isoquant assumption representation, is critical and it may not be relaxed.... Banker, Charnes and Cooper (1984) identify the set of axioms required by DEA, including the convexity postulate (p. 1081) which can only be true if the inputs (and outputs) are substitutable: If ( X, Y ) T, = 1,..., n, and λ 0 are nonnegative scalars such that n n ( λ, 1 X λ 1 Y = = T ) n i= 1 λ = 1, then,. Färe, Grosskopf and Lovell (1994, pp ) state We are now prepared to introduce three important subsets of the boundaries of more sets and less sets. These subsets form the reference [isoquant] sets relative to which efficiency is measured. 1.2 DEA applications to electricity generation Inputs in DEA studies of electricity production most frequently include installed electricity generation capacity as a measure of capital, average number of employees as a measure of labor, and BTUs or physical units (such as tons of coal) as a measure of fuel. We identified 24 DEA papers published between 1983 and 2008 that studied electricity generating plants. Of the 24, 18 used installed generator capacity as a proxy for capital input, 20 used number of employees as proxy for labor input, and, as a proxy for fuel input, 20 used either some measure of heat production such as BTUs (13) or physical units (7). Thus, by far, the most common configuration of inputs in past studies is installed generator capacity, number of employees, and BTUs. As we demonstrate empirically later in this paper, these three inputs cannot be substituted for each other in the production of a fixed amount of electricity output. Employees cannot be substituted for BTUs, BTUs cannot be substituted for generator capacity, and generator capacity cannot be substituted 4

5 for employees. That is, these inputs cannot replace each other in the production of a given amount of output, because there is no factor substitutability. Such production technologies often are referred to as Fixed Factor Proportion Technologies (Beattie and Taylor, 1985). 1.3 Purpose and organization of this paper The purpose of this paper is to demonstrate the impacts of DEA theory s substitutability requirement on DEA application s efficiency estimates for electricity generation plants. First, we provide a detailed illustration of the effects of nonsubstitutable inputs on the measurement of efficiency. Next, we statistically analyze a sample of 70 electricity generating plants with data for the five years , to test whether or not capital, labor and fuel are substitutable for each other in the production of electricity. Then, we develop an additive efficiency measure, the Fixed Proportion Additive (FPA) measure, which deals with fixed proportion technologies. After doing so, we use the preceding electricity generation plant data to compare the FPA measure s efficiency estimates to those of the conventional DEA Additive efficiency measure. Next, we develop a ratio measure, the Fixed Proportion Ratio (FPR), for fixed proportion technologies. Again using the preceding data, we compare the FPR efficiency estimates to those of the Charnes-Cooper-Rhodes (CCR) measure and the Enhanced Russell Measure (ERM). Finally we present our conclusions. 2. Illustration of the effects of nonsubstitutability Consider a hypothetical case of one output and two inputs (a) when the inputs are substitutable, and (b) when the inputs are not substitutable. Suppose one unit of output is produced by the DMUs with various combinations of the inputs (Figure 1). If the two inputs shown in Figure 1 are substitutable, then the input best-practice frontier is represented by the piecewise isoquant A-B-C-D-E (ignoring composite DMU F for the time being). Conventional DEA models also would report that these five DMUs are efficient. If the two inputs are not substitutable, then at most one of these five DMUs could be efficient. For example, if it were true that, to produce one unit of output, a minimum of 1.28 units of input 1 and 0.18 units of input 2 were needed, then the DMU at point C would be efficient because it combines the required amounts and proportion (1.28/0.18 = 7.11) of the inputs needed to efficiently produce one unit of 5

6 output. It would not be possible for a DMU to produce one unit of output with the inputs available at point D (1.48, 0.14). Indeed, at point D, the required ratio of inputs would result in 0.14 * 7.11 = 1.00 unit of input 1 and 0.14 units of input 2 being used (because only 0.14 units of input two are available), thereby wasting 0.48 units of input 1 (which has 1.48 units of input available). Furthermore, the amount of output produced would be only 0.14/0.18 = 0.78 units. The preceding example is merely illustrative, of course, because at present we don t know whether the inputs are nonsubstitutable or, even if they are, what the correct ratio of inputs would be. Indeed, because we know that DMUs A, B, C, D and E all produced one unit of output, it might appear that the empirical evidence suggests that these two inputs are substitutable for each other. But, substitutability is not necessarily present for two reasons. The first is the presence of noise in the data. Because of the ubiquitous presence of noise, it is inevitable that several DMUs will be on an illusory piecewise isoquant frontier, even if the inputs are not substitutable. The second reason is that a DMU is unlikely to be equally efficient in its use of both inputs. In Figure 1, for example, if inputs are truly nonsubstitutable, the supposed piecewise isoquant frontier may be the result of A being the most efficient of all DMUs in the use of input 1 but less efficient in the use of input 2, and E being the most efficient of all DMUs in the use of input 2 but less efficient in the use of input 1. Substitutability, or the lack thereof, can be identified by logic and/or by statistical testing, but not by a deterministic empirical examination of an alleged best-practice frontier. In fact, the two inputs in Figure 1 are nonsubstitutable. One is installed generator capacity in megawatts (mw) and the other is number of full-time-equivalent employees for a set of electricity generating plants, with each plant s two inputs divided by its megawatt-hours (mwh) of output so they represent the inputs used to produce one megawatt-hour of output. It would not be possible to maintain constant output by decreasing generating capacity while increasing employment to compensate, or by decreasing the number of employees while increasing generating capacity to compensate. The number of employees needed is directly related to plant capacity (Woodruff et al., 2005). Further, simple regression 6

7 of one of the inputs on the other (with output equal to 1 for all DMUs) shows their relationship to be positive and statistically significant [t = 3.69; P(t > 3.69) < ]. Since output is equal for all DMUs, the relationship would have to be negative if the inputs were truly substitutes. Therefore, the illusory piecewise isoquant in Figure 1 is due to noise in the data and/or differences in the relative efficiency with which DMUs employ each input. (We offer more sophisticated statistical analyses later in the paper, simultaneously considering all three inputs and using statistical panel data analysis on five years of data, and reach the same conclusion.) 2.1 Estimating the Point Frontier When inputs are truly nonsubstitutable and there is only one output, as is the case in Figure 1, the best-practice frontier consists of a single point that we call the point frontier. This frontier is estimated by the composite DMU at point F in Figure 1. Although not shown on the graph, the frontiers of the reference set increase vertically and horizontally from point F, forming a right-angle or L-shaped reference set frontier. However, the only Pareto-Koopmans efficient subset of the reference set frontier is the point frontier, because only at that point can none of the inputs or outputs be improved without worsening other inputs or outputs. For substitutable inputs, the minimum level of each input is conditioned on the level of the other input. However, for nonsubstitutable inputs, the minimum level of each input needed to produce a given amount of output is not influenced by the other input. So, if all DMUs outputs are equal, when inputs are nonsubstitutable it is only necessary to find the minimum level of each input. As is true for DEA, this frontier estimation method assumes that a composite DMU can be used to identify a point on the efficient frontier that is attainable by an actual DMU. Therefore, we can estimate the point frontier for one unit of output when inputs are nonsubstitutable from the DMUs using the minimum amounts of each input, that is, from the DMUs establishing the boundaries of the right-angle reference set frontier. In Figure 1, DMU A uses the least of input 1 (1.16) and DMU E uses the least of input 2 (0.13), so a fully efficient composite DMU would use 1.16 units of 7

8 input 1 and 0.13 units of input 2, as shown by point F on the graph. Of course, if a particular DMU is the most efficient in the use of both inputs, then that one DMU alone would determine the point frontier. For example, if point F represented an actual DMU instead of a composite DMU, then that actual DMU would reflect the point of maximum efficiency. This point frontier also can be estimated with the same linear programming methodology used by DEA. For DEA with substitutable inputs, when output for all DMUs is equal, the most efficient level of each input is estimated based on the lowest observed value of one input given the observed values of the other inputs. As stated in Cooper, Seiford and Tone (2007b, p. 7), no point on this [Pareto-Koopmans] frontier line can improve one of its input values without worsening the other. In the general case, suppose that for each DMU ( = 1,..., J ) there are data on M inputs M xm = ( x1,... x M ) R + and on N outputs,..., ) N y = ( y 1 y R +. If there are two inputs (M = 2) and n N only one output (N = 1), and if that output is equal for all DMUs, then DEA scores can be computed with the linear program 1-4, where DMU k is the target. This model is the conventional CCR model adapted for one equal output. minθ λ (1) subect to y 1 = 1 = 1,2,..., J (2) λ 0 = 1,2,..., J (3) J x mλ θxkm m = 1, 2 (4) = 1 One point on the Pareto-Koopmans efficient subset frontier is estimated by the inputs x, 1 x of any DMU k with θ = 1 and no slack for either input. Now, let us rewrite linear program 1-4 by expanding equation set 4 to the individual equations 8 and 9. Thus, linear program 5-9 more explicitly illustrates the fact that the minimum θ attainable for either input is influenced by the value of the other input. Because of DEA s convexity assumption, for inputs defining the Pareto-Koopmans efficient subset frontier with output constant, the lower the value of one input, the higher must be the value of the other. That is, if one 8 k k2

9 input is decreased then it is necessary to substitute the other input in sufficient quantities to keep output constant. And, of course, it still is true that if a DMU sθ = 1, then its inputs estimate one point on the efficient frontier if neither contains slack. minθ λ subect to y 1 = 1 1,2,..., λ 0 1,2,..., (5) = J (6) = J (7) J x 1λ θ xk1 (8) = 1 J x 2λ θxk2 (9) = 1 If, however, inputs are not substitutable and output is fixed, then the efficient value for one input is not conditioned on the value of the other input. Consequently, the points on the Pareto-Koopmans efficient frontier must be estimated separately for each input. To estimate the value for the first input, linear program 5-9 would be run J times excluding equation 9. The input value corresponding to the efficient target DMU(s) would be the value of the first coordinate of the point frontier. Similarly, to estimate the value for the second input, linear program 5-9 would be run J times excluding equation 8. The input value corresponding to the efficient target DMU(s) would be the value of the second coordinate of the point frontier. No slack could occur, so these two input values define coordinates of the sole efficient point on the reference set frontier. 2.2 Comparison of efficiencies from the two frontier estimates As can be seen from Figure 1, if the inputs are truly nonsubstitutable then DEA over-estimates the efficiency of DMUs A-E by identifying them as 100 percent efficient when they are not. The five DMUs on the substitutable (piecewise-isoquant) frontier are substantially less efficient if they are measured against F on the nonsubstitutable (point) frontier. Thus, we need to develop metrics to measure efficiency when a fixed factor technology is present. We do so later in this paper, and then apply the metrics to a sample of electricity generating plants. Before developing the metrics, therefore, we will present our sample and statistically test it for substitutability of inputs. 9

10 3. Statistical testing for input substitutability The data set is a matched sample of 70 electricity generating plants for the five years , gathered from the Federal Energy Regulatory Commission s Form 1 annual reports (2008). To assure that the technologies being used were roughly the same, we limited our sample to plants that obtained at least 95 percent of their energy from coal in at least one of the five years, and obtained all of their remaining energy from gas and oil. We eliminated plants burning any amount of other fuels, such as tires or garbage, and of course eliminated plants with energy sources such as nuclear, wind and hydroelectric. We only used plants that supplied data for all of the necessary variables for all five years, and also eliminated all plants that reported data that clearly were incorrect. We started with the entire set of plants reporting data on Form 1, and, after applying the preceding screening criteria, 70 plants remained. Because by far the most common configuration of inputs in previous papers were installed generator capacity, number of employees, and BTUs, we use these three as measures of capital, labor and fuel. And, as almost universal in previous papers, we use annual mwh as our output measure. If the inputs are substitutable, then, holding output constant, the relationships between them will be negative to a statistically significant degree. If the inputs are nonsubstitutable, then, holding output constant, the relationships between them will be either not statistically significant, or positive to a statistically significant degree. Therefore, in order to identify the relationships, we need to conduct two regressions, in both cases holding output constant. In the first, we regress employment on capacity and BTUs, holding output (mwh) constant. In the second, we regress capacity on employment and BTUs, holding output constant. We can analyze these data with statistical panel data analysis (PDA) because we have five years of matched plant data available. We estimate the unobserved effects with a random effects model instead of a fixed effects model because of its greater power to detect differences, and because the lack of changes in installed generating capacity over the five years involved would make it impossible to estimate the 10

11 effect of capital otherwise. If any biases are present, they would generally make hypothesized relationships appear to be stronger than is actually that case, and this would become a concern only if there were negative relationships among the independent variables. The regression results reported in Tables 1 and 2 are not surprising. The number of employees is positively related to the installed generating capacity to a statistically significant degree, which would be expected. The number of employees is positively related to BTUs of fuel but the relationship is not statistically significant. This is not surprising because we know that staffing is directly related to installed generator capacity, not to the particular level of fuel consumption. Further, we would not expect the level of inefficiency in the use of labor to necessarily be the same as the level of inefficiency in the use of fuel. (If the levels of inefficiency for the two inputs were identical except for random variations, then they would show a statistically-significant positive relation in the regression.) Finally, the installed generator capacities of plants are positively related to the number of BTUs consumed to a statistically significant degree, which, since we are holding output constant, is indicative of some positive correlation between fuel and capital inefficiencies. Most important, there are no negative relationships, statistically significant or otherwise, and negative relationships would be necessary if the factors could be substituted for each other, holding output constant. It would be possible but is unnecessary to develop more complex and econometrically sophisticated models. Not only are the outcomes what were expected, but also it is unlikely to the extreme that the applicable regression coefficients would change signs from positive to negative and be statistically significant in the negative direction. Using a different proxy for capital might make some difference, but we have chosen to demonstrate the case with the measure of capital most commonly used in DEA analyses of electricity generating plants. In sum, none of the inputs are substitutable for each other, because none reflect negative relationships of any form when output is held constant. Some of the inputs are complements (positive relationships) to a statistically significant degree and others show non-statistically significant positive relationships. 11

12 Therefore we can assume a fixed factor proportion technology is present as we compare the reported efficiencies using conventional DEA models and fixed factor proportion models. Next, we develop a metric that we call the Fixed Proportion Additive (FPA) measure, because its efficiency estimates can be compared directly to those produced by the DEA Additive measure. 4. Additive efficiency measure to deal with nonsubstitutability 4.1 Indicator development In order to estimate each DMU s efficiency relative to the point frontier in Figure 1, one simple possibility would be a variation of the DEA Additive model (Cooper et al., 2007b, p. 94). With the Additive model, the efficient point for a target DMU is the furthest point on the piecewise isoquant frontier where neither of its inputs has increased and its output has not decreased. The rectilinear distance between the target DMU and that point measures the DMU s inefficiency. We call our variation the Fixed Proportion Additive (FPA) model, because it assumes that the inputs and outputs occur in fixed proportions. Like the Additive model, the degree of inefficiency is measured as the rectilinear distance between the target DMU and the efficient point. But, for the FPA model, the efficient point is the point frontier rather than a point on a piecewise isoquant frontier. Significantly, the only difference between the two models is the location of the point from which inefficiency is measured. This can be seen in Figure 2, which employs piecewise isoquant and point frontiers identical to those in Figure 1. As a result, direct comparisons of the two models efficiency scores can be made, because both models efficiency metric is exactly the same. Therefore, valid estimates can be made of the bias in DEA scores when inputs and outputs are nonsubstitutable. As can be seen in Figure 2, the Additive model indicates that DMUs A, B, C, D and E are efficient, while the FPA model indicates that none of these DMUs is efficient, as would be expected based on the Additive and FPA-estimated frontiers. For the target DMU, the Additive model s inefficiency score is 0.10 and the FPA model s inefficiency score is 0.22, more than twice as high. 12

13 The input-oriented FPA efficiency score for the target DMU k can be obtained for each target DMU k from a set of = 1,2,..., J DMUs by the use of equation 10. M FPA Ik = ( xkm / yk1) Min( x m / y 1) m= 1 We next apply equation 10 to electricity generating plant sample data involving three inputs and one output, and compare the resulting efficiency estimates to those obtained with the Additive model Comparing the efficiencies reported by the additive and FPA models So that the scores of the Additive and the FPA models will be directly comparable, we first divide each DMU s three inputs by its output. Therefore, the values used in the FPA model (equation 10) and the Additive model (equations 11-14) are identical, so the resulting sums of the slacks for the target DMU k are directly comparable. (10) Add k M = max s (11) i= 1 i J x mλ sm xkm Subect to + = m= 1,2,..., M (12) = 1 y = = 1,2,... J (13) 1 1 λ 0 (14) For 2007, using the FPA scores as the base, the Additive model reported efficiencies that were 42.4 percent greater on the average, ranging from 3.6 percent greater to 100 percent greater. The two models measure efficiency the same way and only differ on their identification of efficient points based on whether or not the inputs are substitutable. In this case we know that the FPA model is correct because the inputs are not substitutable. Thus, if the Additive model is (inappropriately) applied to these data, it overestimates true efficiency by over 40 percent on the average, but the efficiency of some DMUs is only slightly overestimated while the overestimation is very large for others. In short, in the presence 13

14 of nonsubstitutable inputs, the conventional Additive efficiency estimates are extremely biased and exhibit strikingly low precision. 5. Ratio measure to deal with nonsubstitutability We developed the FPA measure primarily because it would be completely comparable to a conventional DEA measure, but it is of little value in practice. A much more valuable measure would be one akin to a radial measure, because it would identify the proportional efficiency of target DMUs. In this section, we develop such a measure, the Fixed Proportion Ratio (FPR) measure, to deal with nonsubstitutability. 5.1 Efficiency of a single output/input combination In order to measure a DMU s relative degree of inefficiency in the use of an input to produce an output, the indicator needs to be normalized by some base. Our motivation was a comparison between the Additive and ERM measures. The Additive model is based on the value of slacks, while the ERM model normalizes the slacks by utilizing a DMU s output slack divided by its output, and its input slack divided by its input. Although we do not use slack directly, we do use the concept of normalizing actual values in order to develop a measure of relative inefficiency. Thus, for each output/input combination, we compute the normalized output/input ratio by dividing the target DMU s output/input ratio by that of the DMU that is the most efficient for that particular output/input ratio. Equation 15 illustrates the efficiency of target DMU k s input m and output n. The input and output in the numerator are from the target DMU k, and the input and output in the denominator are from the DMU that has the maximum output/input ratio for that specific output/input combination. eff kmn ykn / xkm = Max( y n / x m) (15) Because a fixed ratio of the various types of input is needed to produce output, all inputs must be positive. Also, to avoid division by zero, at least one y n must be positive. So, the range of efficiency scores for each output/input pair is [0,1], and at least one DMU will achieve an efficiency score of 1. 14

15 5.2 Overall efficiency of the target DMU Because the target DMU s efficiency would usually be different for each output/input combination, its average efficiency can be computed as the mean of its individual efficiencies. Thus, for each of target DMU k s output/input ratios, equation 15 is used to compute the normalized efficiency measure for that ratio. If there are m inputs and n outputs, then there will be m n efficiency measures of the form. eff kmn So, for each DMU k in a set of J DMUs, the mean of its m n efficiency measures is computed, which yields a partially normalized efficiency measure for that DMU. Then, each DMU k s partially normalized efficiency measure is divided by the maximum partially normalized efficiency measure, which yields a normalized efficiency measure in [0, 1]. We call this the Fixed Proportion Ratio (FPR) measure: FPR k = (1 / MN ) m= 1 n= 1 M N Max (1 / ) MN eff m= 1 n= 1 M N eff kmn mn (16) Because of the nature of the formula s construction, it is easy to determine which of the target DMU s output/input pairs are the sources of inefficiency. Further, if an analyst wished to weight some pairs more heavily than others, the formula could easily be modified to accommodate this. 5.3 FPR characteristics All inputs must be positive, but this is not a constraint because no output can be produced by a fixedproportion technology otherwise. Likewise, because of the fixed-proportion output technology, all outputs would be positive for a fully efficient DMU, but, for an inefficient DMU, outputs would only have to be non-negative. The inputs and outputs are units-invariant but not translation invariant; however, for a fixed-proportion technology, translation could not be contemplated in any case. As already noted, FPR has a range over [0, 1]. Technical, mix and scale efficiencies are included in the FPR measure. Finally, the measure is monotone decreasing in each input excess and output shortfall. FPR envelops the data. Each individual output/input ratio of a target DMU is divided by the maximum ratio for that particular output/input pair, which yields its efficiency relative to the most efficient pair. The 15

16 mean of each DMU s individual ratios is divided by the maximum mean ratio, yielding its mean efficiency relative to the maximum mean efficiency. As a result, the formula complies with the DEA convention of basing efficiency on the most efficient observations in the set of units being analyzed. 5.4 Application to electricity generating plant efficiency measurement Next, we compare scores of the FPR efficiency indicator with those of the Charnes-Cooper-Rhodes (CCR) measure and the Enhanced Russell Measure (ERM) with constant returns to scale. (ERM and its variations are also known as the Russell Measure, the Färe/Lovell Efficiency Index and the Slacks Based Measure. But, as Cooper, Huang, Li, Parker and Pastor (2007a) note, ERM is the most firmly embedded in the literature.) Because the FPR measure incorporates both technical and scale efficiencies, we compare it to conventional DEA models that also do so, rather than considering models that assume variable returns to scale and therefore measure only technical efficiency. Recall that our first measure, the FPA indicator, is directly comparable to the ADD model because both use the same metric, with the only difference being to location of the efficient point. When comparing these two measures, the ADD mean estimate was 142 percent of the FPA mean estimate, compelling evidence that the ADD measure indeed biases efficiency scores upward when inputs or outputs are nonsubstitutable. However, the FPR, CCR and ERM indicators each compute their own unique efficiency metric, so none of the three s efficiency estimates are directly comparable to the others. In particular, differences in the average efficiency estimates among the three are not too meaningful. Much more important would be less-than-high correlation between the indicators, and substantial differences in DMU rankings based on indicator values. If these symptoms are present, it indicates that the relative efficiencies of DMUs are affected by the presence of fixed proportion technologies, making comparisons between DMUs invalid when indicators assuming substitutability are utilized. 16

17 The data set used is the sample of 70 electricity generating plants, with output again being annual kilowatt hours and the three inputs again being installed generator capacity, employees, and BTUs of fuel. The three efficiency measures are computed from 2007 data. As can be seen in Figure 3, FPR and ERM track closely until their DMU efficiency estimates reach 70 percent, but from that point on there is significant variation with ERM always being higher and usually substantially higher. This probably is caused by the availability of substitutions in the ERM measure that allow a DMU to compare more favorably with its benchmarks, and such favorable comparisons are more likely to exist when the target DMU is more efficient in general. The CCR measure is much higher than the other two at all of the reported efficiency levels except for the highest few. Moreover, it shows much greater variation than the other two, and, although not shown, has extremely large slacks. This might be expected since CCR measures only weak efficiency (that is, it does not consider inefficiency reflected in slacks), while ERM measures strong efficiency (which does incorporate all inefficiencies including slacks). Unlike the CCR which estimates efficiency much higher than the FPR at all efficiency levels, the ERM efficiency estimates are lower than those of the FPR for several of the lowest efficiency levels and higher than those of the FPR for higher efficiency levels. For all 70 DMUs, the R-square value between FPR and ERM is 0.92, but, for the 24 DMUs with efficiency scores above 0.70, the R-square is only The R-square value between FPR and CCR is 0.83 for all 70 DMUs, and 0.33 for the 24 DMUs with highest efficiency. Also, for the highest 24, the Spearman rank coefficients between FPR and ERM is 0.76 and between FPR and CRR is 0.63, with the Kendall rank coefficients being 0.61 and 0.50 respectively. Perhaps of more concern is the fact that the rank order of some DMUs FPR scores is substantially different from their CRR and ERM ranks. The ERM ranks ranged from 15 higher to 7 lower than the FPR ranks, while the CCR ranks ranged from 29 higher to 24 lower. Thus, based on the results of this sample, it appears that CCR is an unacceptable efficiency measure when inputs are nonsubstitutable because of its very large upward bias and very low precision at 17

18 all efficiency levels. And, while ERM is a reasonably good measure at lower efficiency levels, at higher levels its estimates become increasingly biased upward and decreasingly precise when compared to FPR. Unfortunately, almost all DEA efficiency studies of electricity generating plants have used radial measures, sometimes involving a Joint Production Model when undesirable outputs are considered, with only a few using slacks-based measures. Thus, it is likely that most publications to date report electricity generating plant efficiency estimates that are significantly biased, imprecise, and report inaccurate efficiency rankings. Given the energy and environmental crises we are facing, this problem is of even greater concern if such studies are used for policy or operating decisions. 6. Methods permitting the use of conventional DEA models There are several methods that permit the use of conventional DEA models without suffering the bias and precision problems described in this paper. One method is to aggregate nonsubstitutable variables using their prices as weights. We believe that this solution is a good one if prices are available and can be adusted for price differences over time and among DMUs. These models also deal with the problem of a DMU being more efficient in its production of one type of output than another, or more efficient in the use of one type of input than another. For electricity generation, this solution would amount to using total operating costs plus depreciation as the sole input variable. Another benefit of using monetary value rather than physical inputs is that it would also account for the efficiency with which the plant commits its various generators to production (the unit commitment problem), and the levels at which they are operated (Powell et al., 1977). Another solution is to use conventional DEA models but utilize only one of the nonsubstitutable inputs. Because fixed-ratio variables are nonsubstitutable, they will increase and decrease together, so one can serve as a rough proxy for all. The problem with this approach is that it does not account for differences in a DMU s efficiency in producing different outputs or in using different inputs. But, in the absence of comparable prices, it may be the best choice available if one wishes to use conventional DEA 18

19 models. For electricity generation, it would seem that the best sole input variable would be fuel, measured either in heat units or physical units, given that labor and capital basically are fixed. 7. Conclusions This paper identifies the effects on efficiency estimates when conventional DEA models are applied to electricity generating plants, which employ a fixed factor proportion technology. Using electricity generating plant data, we found that inputs do occur in fixed proportions. As a result, the Additive and CCR efficiency estimates are substantially biased upward and provide little precision in comparison to the Fixed Proportion Additive and Fixed Proportion Ratio measures. ERM seems somewhat more robust to fixed input proportion relationships at low efficiency levels, and at the lowest efficiencies it underestimates true efficiency. At higher efficiency levels, ERM overestimates true efficiency, and provides increasingly imprecise estimates and improperly ranked DMUs as efficiency increases. Based on our sample data, therefore, it would appear that conventional DEA models should not be used to measure the efficiency of electricity generation plants using the typical proxies for capital, labor and energy inputs. Conventional DEA models should be used only if the nonsubstitutable inputs are aggregated or if only one of the nonsubstitutable inputs is utilized. We develop two new indicators, most importantly the FPR measure, for use when a fixed proportion technology is present. The primary theoretical strength of the FPR indicator is that it avoids the DEA requirement that inputs be substitutable. Its use would not be appropriate if substitutions were possible, however, because it ignores substitutability and therefore would bias true efficiencies downward. The FPR measure may have shortcomings which we have not discovered, and, hopefully, other researchers will seek to improve it and to develop additional indicators for measuring efficiency under fixed proportion technologies. But, as already noted, the use of DEA measures is of doubtful validity when the three conventional inputs of labor, capital and energy are utilized, and cannot be depended on to make precise and unbiased estimates with reasonably valid efficiency rank ordering. 19

20 References Banker, R.D., Charnes, A., and Cooper, W.W. (1984), "Some models for estimating technical and scale inefficiencies in data envelopment analysis", Management Science, Vol. 30 No. 9, pp Beattie, B.R. and Taylor, C.R. (1985), The Economics of Production, Wiley, New York. Charnes, A., Cooper, W.W., and Rhodes, E. (1978), "Measuring the efficiency of decision making units", European Journal of Operational Research, Vol. 2 No. 6, pp Cooper, W.W., Huang, Z., Li, S.X., Parker, B.R., and Pastor, J.T. (2007a), "Efficiency aggregation with enhanced Russell measures in data envelopment analysis", Socio-Economic Planning Sciences, Vol. 41 No. 1, pp Cooper, W.W., Seiford, L.M., and Tone, K. (2007b), Data Envelopment Analysis: A Comprehensive Text With Models, Applications, References and DEA-Solver Software, Springer, New York. Färe, R., Grosskopf, S., and Lovell, C.A.K. (1994), Production Frontiers, Cambridge University Press, Cambridge, England. Farrell, M.J. (1957), "The measurement of productive efficiency", Journal of the Royal Statistical Society, Series A, Vol. 120 No. 3, pp Federal Energy Regulatory Commission (2008), "Form 1 - Electric Utility Annual Report", available at: ftp://eforms1.ferc.gov/f1allyears/ (accessed 31 December 2008). Petersen, N.C. (1990), "Data Envelopment Analysis on a relaxed set of assumptions", Management Science, Vol. 36 No. 3, pp Powell, J., Gleason, J., and Burton, J. (1977), "Power generator scheduling by dynamic programming", European Journal of Operational Research, Vol. 1 No. 3, pp Welch, E.W. and Barnum, D.T. (2009), "Joint environmental and cost efficiency analysis of electricity generation", Ecological Economics, In press, Woodruff, E.B., Lammers, H.B., and Lammers, T.F. (2005), Steam Plant Operation, McGraw-Hill, New York. Zhou, P., Ang, B.W., and Poh, K.L. (2008), "A survey of data envelopment analysis in energy and environmental studies", European Journal of Operational Research, Vol. 189 No. 1, pp

21 Figure 1. Substitutable and Nonsubstitutable Input Best-Practice Frontiers, Equal Output for All Input 2 DMU Inputs DEA Frontier Point Frontier 0.4 A: 1.16, B: 1.19, 0.22 C: 1.28, 0.18 F: 1.16, 0.13 D: 1.48, 0.14 E: 1.79, Input 1 21

22 Figure 2. Graphical Representation of Additive and FPA measures 0.35 Labor/Output 0.30 A: 1.16, B: 1.19, , 0.24 (target) Additive rectilinear distance = 0.10 FPA rectilinear distance = 0.22 C: 1.28, F: 1.16, 0.13 D: 1.48, 0.14 E: 1.79, Capital/Output

23 Figure 3. Comparison of CCR, ERM and FPR FPR ERM CCR DMUs in FPR Order 23

24 Table 1. Regression of Employees on Installed Generator Capacity and BTUs, Holding Output Constant Employees Coef. Std. Err. z P> z Electricity Output Installed Generator Capacity BTUs Constant Table 2 Regression of Installed Generator Capacity on Employees and BTUs, Holding Output Constant Installed Generator Capacity Coef. Std. Err. z P> z Electricity Output Employees BTUs Constant