Common Errors: How to (and Not to) Control for Unobserved Heterogeneity

Transcription

1 Unversty of Pennsylvana ScholarlyCommons Fnance Papers Wharton Faculty Research 014 Common Errors: How to (and Not to) Control for Unobserved Heterogenety Todd A. Gormley Unversty of Pennsylvana Davd A. Matsa Follow ths and addtonal works at: Part of the Fnance Commons, and the Fnance and Fnancal Management Commons Recommended Ctaton Gormley, T. A., & Matsa, D. A. (014). Common Errors: How to (and Not to) Control for Unobserved Heterogenety. Revew of Fnancal Studes, 7 (), Ths paper s posted at ScholarlyCommons. For more nformaton, please contact repostory@pobox.upenn.edu.

2 Common Errors: How to (and Not to) Control for Unobserved Heterogenety Abstract Controllng for unobserved heterogenety (or common errors ), such as ndustry-specfc shocks, s a fundamental challenge n emprcal research.ths paper dscusses the lmtatons of two approaches wdely used n corporate fnance and asset prcng research: demeanng the dependent varable wth respect to the group (e.g., ndustry-adustng ) and addng the mean of the group's dependent varable as a control. We show that these methods produce nconsstent estmates and can dstort nference. In contrast, the fxed effects estmator s consstent and should be used nstead. We also explan how to estmate the fxed effects model when tradtonal methods are computatonally nfeasble. Dscplnes Fnance Fnance and Fnancal Management Ths ournal artcle s avalable at ScholarlyCommons:

3 Common Errors: How to (and Not to) Control for Unobserved Heterogenety A Todd A. Gormley B and Davd A. Matsa C August 3, 013 Abstract Controllng for unobserved heterogenety (or common errors ), such as ndustry-specfc shocks, s a fundamental challenge n emprcal research. Ths paper dscusses the lmtatons of two approaches wdely used n corporate fnance and asset prcng research: demeanng the dependent varable wth respect to the group (e.g., ndustry-adustng ) and addng the mean of the group s dependent varable as a control. We show that these methods produce nconsstent estmates and can dstort nference. In contrast, the fxed effects estmator s consstent and should be used nstead. We also explan how to estmate the fxed effects model when tradtonal methods are computatonally nfeasble. (JEL G1, G, G3, C01, C13) Keywords: unobserved heterogenety, group fxed effects, ndustry-adust, bas A Computer code and addtonal resources related to the ssues dscussed n ths paper are posted onlne at For helpful comments, we thank Mchael Anderson, Joshua Angrst, aver Groud, Chrstan Hansen, Drk Jenter, Sandy Klasa, Jonathan Lewellen, Alexander Lungqvst, Mtchell Petersen, Nagpurnanand R. Prabhala, Mchael Roberts, Nck Souleles, Mchael R. Wagner, Ton Whted, and Jeffrey Wooldrdge, as well as the semnar partcpants at Hong Kong Unversty of Scence and Technology, Massachusetts Insttute of Technology (Sloan), Nanyang Technologcal Unversty, Natonal Unversty of Sngapore, Sngapore Management Unversty, Unversty of Arzona (Eller), Unversty of Pennsylvana (Wharton), the Amercan Fnance Assocaton annual meetng, the Fnancal Intermedaton Research Socety conference, and the Rothschld Caesarea Center Conference. Matthew Denes, Chrstne Dobrdge, Jnglng Guan, Jllan Popadak, and Kans Saengchote provded research assstance. Gormley thanks the Rodney L. Whte Center for Fnancal Research Brandywne Global Investment Management Research Fellowshp and the Cyntha and Bennett Golub Endowed Faculty Scholar Award for fnancal support. B The Wharton School, Unversty of Pennsylvana, 360 Locust Walk, Sute 400, Phladelpha, PA, Phone: (15) Fax: (15) E-mal: tgormley@wharton.upenn.edu. C Kellogg School of Management, Northwestern Unversty, 001 Sherdan Road, Evanston, IL Phone: (847) Fax: (847) E-mal: dmatsa@kellogg.northwestern.edu.

4 Controllng for unobserved heterogenety s a fundamental challenge n emprcal fnance research because asset prces and most corporate polces depend on factors that are unobservable to the econometrcan. If these factors are correlated wth the varables of nterest, then wthout proper treatment, omtted varables bas nfects the estmated parameters and precludes causal nference. In many settngs, mportant sources of unobserved heterogenety are common wthn groups of observatons. For example, unobserved rsk factors, whch affect both stock returns and corporate decsons, are often common to frms of smlar sze. Potental unobserved factors abound: unobserved dfferences n local economc envronments, management qualty, and the cost of captal, to name a few. Although the emprcal fnance lterature uses varous estmaton strateges to control for unobserved group heterogenety, there s lttle understandng of how these approaches dffer and under whch crcumstances each provdes consstent estmates. Our paper examnes ths queston and shows that some commonly used approaches provde nconsstent estmates and can dstort nference. We focus on two popular estmaton strateges that are appled when there are a large number of groups and the number of observatons per group s small relatve to the number of groups (e.g., frmpanel data that s grouped nto ndustry-years). The frst estmaton strategy, whch we refer to as adusted-y (AdY), demeans the dependent varable wth respect to the group before estmatng the model wth ordnary least squares (OLS). A common example s when researchers ndustry-adust ther dependent varable so as to remove common ndustry factors n a frm-level analyss. A second approach, whch we refer to as average effects (AvgE), uses the mean of the group s dependent varable as a control n the OLS specfcaton. A common mplementaton of AvgE uses observatons state-year mean to control for tme-varyng dfferences n local economc envronments. Both AdY and AvgE are wdely used n emprcal fnance research. Artcles publshed n top fnance ournals, ncludng n the Journal of Fnance, Journal of Fnancal Economcs, and Revew of Fnancal Studes, have used both approaches snce at least the late 1980s, and they contnue to be used today. 1 Among artcles publshed n these three ournals n , we found over 60 artcles, splt about evenly between corporate fnance and asset prcng, that employed at least one of the two 1 The exact orgn of the two estmators n fnance s unclear; we suspect they were adapted from the event studes lterature, n whch stock returns are regressed on market-average returns. AdY may have been nspred by analyses of market-adusted returns, and AvgE by estmatons of the market model.

5 Common Errors: How to (and Not to) Control for Unobserved Heterogenety A Todd A. Gormley B and Davd A. Matsa C Abstract Controllng for unobserved heterogenety (or common errors ), such as ndustry-specfc shocks, s a fundamental challenge n emprcal research. Ths paper dscusses the lmtatons of two approaches wdely used n corporate fnance and asset prcng research: demeanng the dependent varable wth respect to the group (e.g., ndustry-adustng ) and addng the mean of the group s dependent varable as a control. We show that these methods produce nconsstent estmates and can dstort nference. In contrast, the fxed effects estmator s consstent and should be used nstead. We also explan how to estmate the fxed effects model when tradtonal methods are computatonally nfeasble. (JEL G1, G, G3, C01, C13) Keywords: unobserved heterogenety, group fxed effects, ndustry-adust, bas A Computer code and addtonal resources related to the ssues dscussed n ths paper are posted onlne at For helpful comments, we thank Mchael Anderson, Joshua Angrst, aver Groud, Chrstan Hansen, Drk Jenter, Sandy Klasa, Jonathan Lewellen, Alexander Lungqvst, Mtchell Petersen, Nagpurnanand R. Prabhala, Mchael Roberts, Nck Souleles, Mchael R. Wagner, Ton Whted, and Jeffrey Wooldrdge, as well as the semnar partcpants at Hong Kong Unversty of Scence and Technology, Massachusetts Insttute of Technology (Sloan), Nanyang Technologcal Unversty, Natonal Unversty of Sngapore, Sngapore Management Unversty, Unversty of Arzona (Eller), Unversty of Pennsylvana (Wharton), the Amercan Fnance Assocaton annual meetng, the Fnancal Intermedaton Research Socety conference, and the Rothschld Caesarea Center Conference. Matthew Denes, Chrstne Dobrdge, Jnglng Guan, Jllan Popadak, and Kans Saengchote provded research assstance. Gormley thanks the Rodney L. Whte Center for Fnancal Research Brandywne Global Investment Management Research Fellowshp and the Cyntha and Bennett Golub Endowed Faculty Scholar Award for fnancal support. B The Wharton School, Unversty of Pennsylvana, 360 Locust Walk, Sute 400, Phladelpha, PA, Phone: (15) Fax: (15) E-mal: tgormley@wharton.upenn.edu. C Kellogg School of Management, Northwestern Unversty, 001 Sherdan Road, Evanston, IL Phone: (847) Fax: (847) E-mal: dmatsa@kellogg.northwestern.edu.

6 Controllng for unobserved heterogenety s a fundamental challenge n emprcal fnance research because asset prces and most corporate polces depend on factors that are unobservable to the econometrcan. If these factors are correlated wth the varables of nterest, then wthout proper treatment, omtted varables bas nfects the estmated parameters and precludes causal nference. In many settngs, mportant sources of unobserved heterogenety are common wthn groups of observatons. For example, unobserved rsk factors, whch affect both stock returns and corporate decsons, are often common to frms of smlar sze. Potental unobserved factors abound: unobserved dfferences n local economc envronments, management qualty, and the cost of captal, to name a few. Although the emprcal fnance lterature uses varous estmaton strateges to control for unobserved group heterogenety, there s lttle understandng of how these approaches dffer and under whch crcumstances each provdes consstent estmates. Our paper examnes ths queston and shows that some commonly used approaches provde nconsstent estmates and can dstort nference. We focus on two popular estmaton strateges that are appled when there are a large number of groups and the number of observatons per group s small relatve to the number of groups (e.g., frmpanel data that s grouped nto ndustry-years). The frst estmaton strategy, whch we refer to as adusted-y (AdY), demeans the dependent varable wth respect to the group before estmatng the model wth ordnary least squares (OLS). A common example s when researchers ndustry-adust ther dependent varable so as to remove common ndustry factors n a frm-level analyss. A second approach, whch we refer to as average effects (AvgE), uses the mean of the group s dependent varable as a control n the OLS specfcaton. A common mplementaton of AvgE uses observatons state-year mean to control for tme-varyng dfferences n local economc envronments. Both AdY and AvgE are wdely used n emprcal fnance research. Artcles publshed n top fnance ournals, ncludng n the Journal of Fnance, Journal of Fnancal Economcs, and Revew of Fnancal Studes, have used both approaches snce at least the late 1980s, and they contnue to be used today. 1 Among artcles publshed n these three ournals n , we found over 60 artcles, splt about evenly between corporate fnance and asset prcng, that employed at least one of the two 1 The exact orgn of the two estmators n fnance s unclear; we suspect they were adapted from the event studes lterature, n whch stock returns are regressed on market-average returns. AdY may have been nspred by analyses of market-adusted returns, and AvgE by estmatons of the market model. 1

7 technques. The technques are used to study a varety of fnance topcs, ncludng bankng, captal structure, corporate boards, governance, executve compensaton, and corporate control. Artcles usng these estmaton methods have also been publshed n economcs, ncludng n the Amercan Economc Revew, Journal of Poltcal Economy, and Quarterly Journal of Economcs, and n accountng, ncludng the Accountng Revew, Journal of Accountng and Economcs, and Journal of Accountng Research. Our paper shows that, despte ther popularty, the AdY and AvgE estmators rarely provde consstent estmates; both estmators can exhbt severe bases (where bas here refers to the dfference between the probablty lmt of the estmate and the true parameter). The AdY estmator suffers from an omtted varable bas because t fals to control for the group average of the ndependent varables. Ths omsson s problematc when any explanatory varable s correlated wth ts group average, whch s lkely n practce. The AvgE estmator suffers from a measurement error bas because the sample mean of the group s dependent varable measures the true unobserved heterogenety wth error. AvgE s nconsstent when AdY s nconsstent and also when any ndependent varable s correlated wth the unobserved heterogenety. Even when the underlyng data structure exactly matches the AdY or AvgE specfcatons, both estmators are nconsstent. For both estmators, the bas can be large and complcated; tryng to predct even the sgn of the bas s typcally mpractcal because t depends on numerous correlatons. The shortcomngs of the AdY and AvgE estmators stand n stark contrast to the fxed effects (FE) estmator another approach avalable to control for unobserved group heterogenety. The FE estmator, whch nstead adds group ndcator varables to the OLS estmaton, s consstent n the presence of unobserved group heterogenety. When there s only one source of unobserved group heterogenety, the FE estmator s equvalent to demeanng all of the dependent and ndependent varables wth respect to the group and then estmatng the model usng OLS. The dfferences between the estmators are mportant because the AdY and AvgE estmators can lead researchers to make ncorrect nferences. We show that AdY and AvgE estmates can be more based than OLS and even yeld estmates wth the opposte sgn of the true coeffcent. AdY and AvgE can also be nconsstent even n crcumstances n whch the orgnal OLS estmates would be consstent. When estmatng a few textbook fnance models usng each of the dfferent technques to control for

8 unobserved heterogenety, we fnd large dfferences between the AdY, AvgE, and FE estmates and confrm that AdY and AvgE can exhbt larger bases than OLS and can yeld coeffcents of the opposte sgn as FE. These dfferences confrm the presence of unobserved group heterogenety n these settngs and of correlatons wthn these commonly used data structures that cause the AdY and AvgE estmators to be nconsstent and potentally qute msleadng n practce. Based on these fndngs, we argue that AdY, AvgE, and related estmators should not be used to control for unobserved group heterogenety. Any estmaton that transforms the dependent varable but not the ndependent varables typcally yelds nconsstent estmates. For example, subtractng the group medan, or the mean or medan of a comparable set of frms, from the dependent varable also yelds nconsstent estmates by falng to account for how the correspondng medan or mean of the ndependent varables affects the adusted dependent varable. Our fndngs apply to a dverse set of estmatons undertaken n the lterature. The practce of ndustry-adustng dependent varables s common n many corporate fnance papers. Even a smple comparson of ndustry- or benchmark-adusted outcomes before and after events as n many analyses of corporate control transactons, stock ssues, and other sets of 0/1 events does not reveal the true effect of the events. Corporate governance analyses of the effects of busness combnaton laws across U.S. states whle controllng for ndustry-year and state-year averages of the dependent varable are also not properly specfed. Our crtcsm also apples to estmators n some asset prcng studes. The method of characterstcally adustng stock returns n asset prcng subtracts the return of a benchmark portfolo contanng stocks wth smlar characterstcs, before sortng and comparng these stock returns across subsamples. Ths method s problematc because t does not control for how the varable used to sort the adusted stock returns vares across the benchmark portfolos. Our analyss also hghlghts related problems wth other estmators that are not desgned to control for unobserved heterogenety. For example, the omtted varable problem of AdY apples to any dependent varable that s constructed usng multple observatons. Our analyss also rases concerns about nstrumental varable (IV) estmators that nstrument for an endogenous ndependent varable usng ts group average. Ths estmator s excluson restrcton s volated whenever an unobserved group factor s correlated wth the regressor. 3

9 FE estmators should be used nstead of AdY or AvgE to control for unobserved heterogenety. FE estmators are consstent because they are equvalent to transformng both the dependent and ndependent varables so as to remove the unobserved heterogenety. For any AdY or AvgE estmator, there s a correspondng FE estmator that properly accounts for correlatons n the ndependent varables. For example, rather than ndustry-adustng a dependent varable or controllng for the ndustry mean of the dependent varable, researchers should nstead estmate a model wth ndustry fxed effects. Lkewse, rather than correlatng benchmark-portfolo-adusted stock returns wth an explanatory varable of nterest, a researcher should nstead estmate a model wth fxed effects for each benchmark portfolo. The FE estmator, however, also has lmtatons. Although the FE estmator controls for unobserved group heterogenetes, t s unable to control for unobserved wthn-group heterogenetes. FE estmaton also cannot dentfy the effect of ndependent varables that do not vary wthn groups and s subect to attenuaton bas n the presence of measurement error. We dscuss these lmtatons, how they can be addressed, and when FE estmaton s approprate. Fnally, we address another lmtaton of FE that has motvated some researchers to use AdY or AvgE rather than FE computatonal dffcultes that can arse when estmatng FE models that have multple types of unobserved heterogenety. As the sze and detal of datasets has ncreased, researchers are ncreasngly nterested n controllng for multple sources of unobserved heterogenety. For example, executve compensaton may be affected by unobserved manageral skll and by unobserved frm qualty (Graham, L, and Qu, 01; Coles and L, 011a). Lkewse, researchers who use frm-level data are ncreasngly concerned about both unobserved frm-level characterstcs and tme-varyng heterogenety across ndustres, such as ndustry-level shocks to demand. When there are multple sources of unobserved group heterogenety n an unbalanced panel, demeanng the data multple tmes s not equvalent to fxed effects. FE estmaton of such models requres a large number of ndcator varables, whch can pose computatonal problems. The computer memory requred to estmate these models can exceed the resources avalable to most researchers. We dscuss technques that provde consstent estmates for models wth multple, hgh- To help nterested researchers, we have also posted code and addtonal resources on our webste, to show how common mplementatons of AdY and AvgE can be transformed nto consstent FE estmators. 4

10 dmensonal group effects, whle avodng the computatonal constrants of a standard FE estmator. One approach s to nteract all values of the multple group effects to create a large set of fxed effects n one dmenson that can be removed by transformng the data. A second approach, whch helps to avod potental attenuaton bases and allows the researcher to estmate a larger set of parameters, s to mantan the multdmensonal structure but to make estmaton feasble by reducng the amount of nformaton that needs to be stored n memory. Ths can be accomplshed by usng the propertes of sparse matrces and/or by employng teratve algorthms. We dscuss the relatve advantages of each approach and how these technques can be mplemented easly n the wdely used statstcal software Stata. Overall, our paper provdes practcal gudance on emprcal estmaton n the presence of unobserved group heterogenety, whch s a pervasve dentfcaton challenge n emprcal fnance research. A small, but mpactful, set of recent artcles have addressed other challenges researchers face. For example, Bertrand, Duflo, and Mullanathan (004) and Petersen (009) recommend methods to account for correlaton across resduals n computng standard errors; Erckson and Whted (01) compare methods used to account for measurement error n nvestment regressons; and Fee, Hadlock, and Perce (011) evaluate the use of F-tests on ndcator varables n manageral style regressons. The remander of ths paper s organzed as follows. In Secton 1, we descrbe the underlyng dentfcaton concern of estmatng a model wth unobserved group heterogenety and why AdY and AvgE provde nconsstent estmates. In Secton, we contrast the AdY and AvgE estmators wth the FE estmator and dscuss why other related estmaton technques, whch are commonly used n the lterature, yeld nconsstent estmates. In Secton 3, we show that dfferences between the estmaton technques can be mportant n practce. In Secton 4, we dscuss the lmtatons of the FE estmator and descrbe when ts use s approprate. We conclude n Secton 5 and provde dervatons and proofs n the Appendx. 1. Estmaton Technques Used to Control for Unobserved Group Heterogenety Consder the case n whch an ndependent varable of nterest,, affects a dependent varable, y, whch also has unobserved group heterogenety that s possbly correlated wth. Specfcally, assume the data exhbts the followng structure: 5

11 y f,,, var( ), 0, var( ), var( f ) cov( f, ) 0, cov(, ) cov(, ) 0,,,, cov(, f ), f, f (1) where ndexes groups of observatons (e.g., ndustres) and ndexes observatons wthn each group (e.g., frms). There s a random sample of N groups (y,1,,y,j,,1,,,j ) wth J observatons per group. As s typcal n fnance regressons, we assume that J s small, N s large, and both the ndependent varable of nterest,, and the resdual, ε, are..d. across groups but not necessarly..d. wthn groups, and nether f nor the ndependent varable of nterest,, covary wth the resdual, ε. For ease of exposton, we assume that the ntercept and the means of both the group term and the ndependent varable are zero (.e., μ = μ f = 0); these assumptons smplfy the analyss but have no effect on the estmate of β under the dfferent estmaton technques we analyze. The model n Equaton (11) can be augmented to reflect more complcated sources of unobserved heterogenety wthout affectng our subsequent analyss. For example, a model wth two types of unobserved heterogenety, such as frm and year group effects n panel data, can be captured by addng an addtonal heterogenety term to Equaton (1). Tme-varyng omtted factors (such as ndustry shocks that vary over tme) can be captured by addng an addtonal subscrpt t to each varable, ncludng the unobserved heterogenety f. It s well known that usng OLS to estmate, the effect of on y, yelds an nconsstent estmate when there exsts a nonzero covarance, f, between the unobserved heterogenety and ndependent varable of nterest. 3 OLS estmates the followng specfcaton: The OLS estmate s OLS OLS y u. (),,, ˆ OLS f. (3) By falng to control for the group term, ˆ OLS conflates the effects of and f on the dependent varable, y. 3 Throughout the paper, we use the standard large-sample approach to determne an estmate s consstency by takng the number of groups, N, to nfnty, whle holdng group sze, J, constant. 6

12 The bas, /, represents the standard omtted varable bas n a unvarate regresson; t equals the f coeffcent from a regresson of the omtted varable, f, onto the ncluded varable,, multpled by the coeffcent on the omtted varable n the true model, whch n ths case s 1 [see Angrst and Pschke (009), Secton 3.., for a dervaton of the omtted varable bas formula]. Because OLS s nconsstent when f 0, researchers must rely on other estmaton technques. Two popular approaches are adusted-y (AdY), whch demeans the dependent varable wth respect to the group before estmatng the model wth OLS, and average effects (AvgE), whch uses the group s mean of the dependent varable as a control n an OLS specfcaton. In ths secton, we descrbe the AdY and AvgE estmates and dscuss why they typcally lead to nconsstent estmates of the coeffcent of nterest,. The source of the bas extends to models wth more complcated data structures. 1.1 Adusted-Y estmaton The AdY estmator attempts to remove the nfluence of the group term from the dependent varable by demeanng the dependent varable wthn each group. AdY estmaton s appled, for example, at the ndustry level n frm-panel datasets by subtractng the ndustry-mean from the dependent varable. When ths adustment s appled at the ndustry or ndustry-year level, researchers typcally refer to the dependent varable as beng ndustry-adusted. More specfcally, the researcher calculates the group mean, y, as 1 1 y y,, f, J J k k k kgroup kgroup (4) and estmates the followng model usng OLS: 4 AdY AdY y y u. (5),,, The AdY estmaton, however, does not provde a consstent estmate of because the estmaton suffers from an omtted varable problem. To see ths, t s helpful to re-express the dependent varable s sample group mean as y f, (6) 4 Another common mplementaton of AdY s to demean the dependent varable usng the sample group s mean after excludng the observaton at hand. In other cases, the medan s used. Redefnng the group mean, y, to reflect these other mplementatons of AdY does not affect our subsequent propostons; both of these approaches also yeld nconsstent estmates. We dscuss these estmators n Secton. 7

13 where and are the correspondng sample group means for and ε. In the presence of unobserved group heterogenety, as n Equaton (11), the dependent varable n AdY estmaton can thus be wrtten as y y. (7),,, Comparng the AdY estmaton (Equaton (55)) to the true data structure (Equaton (77)), we see that AdY fals to control for. Ths leads to a based estmate for f s correlated wth the AdY ndependent varable,. In other words, the covarance between, and u, s nonzero when the correlaton between and, s nonzero. AdY also fals to control for, but ths does not bas the estmate for β under the data structure assumed n Equaton (11). As shown n Proposton 1, lettng represent the covarance between, and ts group mean,, we can derve the sgn and magntude of the bas n the AdY estmate for β. Proposton 1. In the presence of unobserved group heterogenety, as n Equaton (11), the AdY estmator yelds an nconsstent estmate for β. Specfcally, ˆ AdY. Smlar to the OLS estmate, the bas of AdY follows the standard omtted varable bas formula. The bas,, equals the coeffcent from a regresson of the omtted varable,, on the ncluded / varable,,, multpled by the omtted varable s coeffcent n the true underlyng model, whch s (see Equaton (77)). If co-vares postvely wth ts group mean, then the AdY estmate for exhbts an attenuaton bas. And, nversely, f s negatvely correlated wth ts group mean, then the AdY estmate for s based away from zero. 5 In practce, postve covarance between and ts group mean s common, causng AdY 5 In practce, the bas of ˆ AdY s typcally attenuatng because t s unusual for to be negatvely correlated wth ts group mean when there s also a group component, f, that has nonzero correlaton wth (as n Equaton (11)). Specfcally, the covarance matrx for the underlyng data structure mpled by Equaton (11) s postve defnte only f ( 1)/( 1), J, f J, where, s the correlaton between,, and,-, and f s the correlaton between, and f. Whle ths condton places a lower bound on the / term of the AdY bas, t s possble for ths bound to be less than zero, n whch case ˆ AdY overestmates the magntude of β. 8

14 estmates to be nconsstent. For example, consder a standard frm-level captal structure estmaton, where leverage s regressed onto multple ndependent varables, such as the return on assets, bankruptcy rsk, and the market-to-book rato. Because frms n the same ndustry are subect to common demand and technology shocks, ther leverage, return on assets, bankruptcy rsk, and the other regressors typcally covary wth ther ndustry averages. By ndustry-adustng only leverage, the AdY approach removes the unobserved heterogenety n leverage, but fals to account for the covarance between the ndependent varables and ther ndustry averages n the transformed data. For ths reason, AdY estmaton (ncludng so-called ndustry-adustng ) s nconsstent. The bas n the AdY estmaton s present even wth very large groups and even when standard OLS estmates are consstent. Because the AdY estmator suffers from an omtted varable bas, ncreasng group sze does not elmnate the dentfcaton problem the estmaton s error term stll contans the omtted group average of the ndependent varable. Moreover,, and are typcally correlated even when, and f are not, so AdY s nconsstent even when the OLS estmate s consstent. In ths case, AdY ntroduces a new omtted varable problem n ts attempt to control for a nonexstent omtted varable problem n the orgnal OLS specfcaton. As shown n Proposton, the bas of the AdY estmator becomes consderably more complex when there s more than one ndependent varable of nterest. Suppose the true model s as follows: y Z f,,,, var( ), 0, var( ), var( f ) var( Z ), cov( f, ) 0, cov(, ) cov(, ) 0,,,, cov( Z, ) cov( Z, ) 0,,,, cov(, f ), f cov(, Z ) cov( Z,,, Z, f Z f Zf ). (8) There s stll ust one type of unobserved group heterogenety, f, but there are two ndependent varables of nterest, and Z. As before, assume that the ndependent varables,, and Z,, and the unobserved 9

15 group effect, f, do not co-vary wth the error, ε,. The two ndependent varables, however, do co-vary wth each other and wth the unobserved group heterogenety, f. For ease of exposton, we agan assume wthout loss of generalty that the ntercept and the means of both the unobserved heterogenety and the ndependent varables are zero (.e., μ = μ Z = μ f = 0). Proposton. In the presence of unobserved group heterogenety and two ndependent varables, as n Equaton (88), the AdY estmator yelds nconsstent estmates for both β and. Specfcally, ˆ Z Z Z Z ˆ AdY Z ZZ Z AdY Z Z Z Z Z Z ZZ Z Z, where s the covarance between Z ZZ, and Z, s the covarance between Z Z, and,, and Z s the covarance between, and Z. These expressons show that predctng the drecton and magntude of the bas n the AdY estmator s not straghtforward, and the sgn of ˆ AdY may not even match the sgn of the true. The sgn and magntude of the bas depends on both of the underlyng coeffcents, and, the relatve varances of and Z, and the covarance of and Z wth each other and the omtted varables, and Z Average effects estmaton AvgE estmaton approaches the problem of unobserved heterogenety dfferently. Instead of adustng the dependent varable, AvgE uses a proxy, the group s sample mean, y, to control for the unobserved varaton, f. A common mplementaton of AvgE uses observatons state-year mean to control for tme-varyng dfferences n local economc envronments. When there s one ndependent 6 The bas n Proposton follows the standard omtted varable bas formula for a multvarate regresson. See Secton 8.4. of Greene (00) or page 61, footnote 14, of Angrst and Pschke (009) for detals. 10

16 varable, the AvgE approach estmates the followng regresson: AvgE AvgE AvgE y y u. (9),,, Proposton 3 shows that, n the presence of unobserved group heterogenety, as n Equaton (11), AvgE estmaton s nconsstent. The underlyng problem s that AvgE suffers from measurement error. As seen from Equaton (66), f y ; thus, the sample dependent varable s group means, y, measure the unobserved varaton, f, wth error. Measurement error bases the coeffcent on the msmeasured regressor, along wth the coeffcents on all other regressors. Moreover, because the measurement error co-vares wth both the msmeasured varable and the errors, ths s not classcal measurement error, and the bas on ˆ AvgE s not necessarly attenuatng. Proposton 3. In the presence of unobserved group heterogenety, as n Equaton (11), the AvgE estmator yelds an nconsstent estmate for β. Specfcally, ˆ AvgE f f f f f f f where s the varance of, s the varance of and s the covarance between and., f s the covarance between f and,, The sgn and magntude of the bas for ˆ AvgE s consderably more complcated than for ˆ AdY. The bas s complcated because the measurement error,, can co-vary wth both the msmeasured varable, f, when 0 and wth, when 0. 7 As shown n Proposton 3, ether f a non-zero f or a non-zero causes the AvgE estmate of β to be nconsstent. These condtons cause the AvgE estmator to be nconsstent n most applcatons. For example, n a frm-level AvgE estmaton, where a researcher uses the state-year average to control for tmevaryng dfferences n local economc envronments, any covarance between the ndependent varables, 7 It s possble, however, to put bounds on the bas. For example, as we dscuss n the Appendx, a researcher can use the methods n Erckson and Whted (005) and assumptons about the correlaton between y and f to determne whether the sgn of an AvgE estmate s correct. See also Krasker and Pratt (1986), Erckson (1993), and Hu (006). 11

17 such as the return on assets, and ther state-average causes AvgE to be nconsstent. Such correlatons are common because frms located n the same state are subect to smlar local demand shocks and nvestment opportuntes. 1.3 Relatve performance of OLS, AdY, and AvgE To better understand the sources of the bas and to compare the relatve performance of OLS, AdY, and AvgE, t s helpful to re-express the key covarance term that contrbutes to each estmate s bas. To ths end, t s helpful to separate, nto ts group and dosyncratc components. Assume that, x w,, where the group means x are..d. wth mean zero and varance x, the dosyncratc components w, are dstrbuted wth mean 0 and varance w, and cov( x, w, ) 0. It can be shown that J 1 x w,, w, J J w, (10) where w w s the covarance between w, and w, (proof n the Appendx).,, The expresson n Equaton (1010) sheds addtonal lght on when the AdY and AvgE estmators are nconsstent. As shown n Propostons 1,, and 3, AdY and AvgE are nconsstent when 0. Equaton (10) shows that 0 whenever observatons ether have dfferent means across groups (such that x 0 ) or are not ndependent wthn groups (such that w w ). In most fnance 0,, applcatons, observatons have dfferent means across ndustres, geographes, or other groupngs. Even f all groups had the same mean, would stll be non-zero f the observatons wthn groups are not ndependent, whch s also common n practce. For example, tme-seres observatons are often serally correlated. Fnally, even f observatons are..d. wthn and across groups, s stll not equal zero because ncludes,, whch creates a mechancal, nonzero correlaton between an observaton and ts sample group average for fnte J. Defnng the group average for each observaton to exclude the observaton at hand (as s sometmes done n practce), such that 1

18 y 1,, k J 1 kgroup k y, (11) smplfes the expresson for but does not resolve the underlyng dentfcaton problem. Although excludng the observaton at hand when calculatng group means removes the mechancal correlaton between, and ts sample group average, the covarance between, and ts group average s stll nonzero whenever observatons have dfferent means across groups or are not ndependent wthn groups; specfcally, x w,, w (proof n the Appendx)., In some cases, AvgE and AdY wll provde a less-based estmate of than does OLS, and n other cases, they wll be more based and possbly even have the wrong sgn. Some examples of ths are provded n Table 1. 8 Under certan parameters, OLS wll actually be less based then both AdY and AvgE. But as shown n Table 1, there are also cases n whch OLS s less based than AvgE but more based then AdY and other cases n whch OLS s less based than AdY but more based than AvgE. There are also cases n whch the AvgE estmate actually has the opposte (and ncorrect) sgn. In the case of ust one ndependent varable, as n Table 1, the AdY estmator cannot ncorrectly flp the sgn of β. But as shown n Proposton (and our later applcatons n Secton 3), ths s no longer true when there s more than one ndependent varable. In ths case, both the AdY and AvgE estmators can return an nconsstent estmate wth the ncorrect sgn whle the OLS estmate has the correct sgn. The correlaton between the ndependent varable of nterest,, and the unobserved group heterogenety, f, has a large effect on the relatve performance of each estmator. Fgure 1 graphs OLS, AdY, and AvgE estmates of Equaton (11) as functons of varous parameter values when β = 1. Each panel shows the effect of varyng a specfc parameter n the data structure, whle holdng the rest constant. When not beng vared, the default parameter values are as follows: f 0.5 ; / 0.5 x w ;, 0.5 ; / 1 f ; / 1; and J 10. Panel A plots the mpact of the correlaton between w, w, the ndependent varable and the unobserved group heterogenety f on each estmator. AdY s less 8 To focus on the key determnants of the bases, we assume n Table 1 (and later n Fgure 1) that groups are defned as n Equaton (1111); ths elmnates the bas that arses from the mechancal correlaton between, and ts sample average, whch s less nterestng. To smplfy the analyss, we also assume that the errors are..d. Analytcal solutons for the AdY and AvgE estmates for β, expressed n terms of f, x / w, w,,, /, /, w, f and J, are provded n the Appendx. 13

19 based than the OLS only when the absolute magntude of the correlaton between and f s large. Ths s because the AdY bas s unaffected by f n contrast, s nonlnear n f low values of f, whereas the OLS bas ncreases lnearly f. The AvgE bas,. Under these parameters, AvgE s more based than both OLS and AdY for, whereas for hgh values of f, AvgE s less based than both OLS and AdY. The relatve varaton across versus wthn groups, x / w, and the correlaton between observatons wthn a group, w, w, also affect the relatve performance of the AdY and AvgE, estmators. As shown n Panel B of Fgure 1, the sgn and magntude of the bas for both AvgE and AdY ncrease n x / w, whereas the OLS estmate s unaffected. Ths s because AdY and AvgE bases depend on, whch ncreases n x, whereas the bas n OLS does not. For the same reason, correlaton between observatons n a group, w, w, ncreases the bas of AdY and AvgE but not OLS,, as shown n Panel C. The extent of unobserved heterogenety also has dfferent mplcatons for the varous estmators. Panel D of Fgure 1 shows the OLS, AdY, and AvgE estmates as a functon of /, the relatve varaton of the unobserved group heterogenety to that of the ndependent varable. The AdY estmate s unaffected by the extent of unobserved heterogenety, whereas the magntude of the bas ncreases n / for both OLS and AvgE, albet n opposte drectons. f Lastly, a reducton n relatve nose, /, or an ncrease n the number of observatons per group, J, does not necessarly mprove the performance of the varous estmators. As shown n Panels E and F of Fgure 1, the OLS and AdY bases are unaffected by f / and J (when demeanng the dependent varable after excludng the observaton at hand). Relatve nose and the number of observatons per group affect AvgE s bas, but the estmate asymptotes to dfferent based coeffcents, whch under these parameters have opposte sgns, for small and large values of / (see Panel E). Moreover, under these parameters, an ncrease n number of observatons per group actually ncreases the bas of AvgE (see Panel F). In sum, whether AdY or AvgE provdes an mprovement over OLS when β 0 depends on the exact parameter values, but regardless, all three estmates are nconsstent under most parameters. 14

20 . Fxed Effects Estmaton and Addtonal Implcatons Comparng the AdY and AvgE estmators to the well-known fxed effects (FE) estmator provdes further nsght nto why AdY and AvgE are nconsstent. The comparson also hghlghts why other related estmaton methods that are commonly used n the lterature yeld nconsstent estmates..1 Fxed effects estmaton Although the OLS, AdY, and AvgE estmates are all nconsstent n the presence of the unobserved heterogenety n Equaton (11), the FE estmator s consstent. FE estmaton nserts an ndcator varable for each group drectly nto the OLS equaton, thereby allowng the predcted mean of the dependent varable to vary across each group. Ths estmaton, whch s also referred to as least squares dummy varable (LSDV) estmaton, s consstent because t controls drectly for the unobserved heterogenety, f, n Equaton (11). The FE estmate s also consstent when f 0 and the orgnal OLS estmate s consstent. Equvalently, the FE estmator can be mplemented by transformng the data to remove the unobserved heterogenety. Ths transformaton s mplemented by demeanng all of the varables both the dependent and ndependent varables wth respect to the group and then estmatng OLS on the transformed data. Specfcally, fxed effects (FE) estmates FE FE y y u. (1),,, Comparng the FE estmator to the true data structure (see Equaton (77)), we can see that the FE estmator s consstent, gven our assumpton n Equaton (11) that does not covary wth. 9 Although the estmates are consstent, the standard errors must be approprately adusted to account for the reduced degrees of freedom. Typcally, the degrees of freedom s adusted downward (.e., the estmated standard errors are ncreased) to account for the number of fxed effects removed n the wthn transformaton. However, when estmatng cluster-robust standard errors (whch allows for heteroscedastcty and wthn-group correlatons), ths adustment s not requred as long as the fxed effects swept away by the wthn-group transformaton are nested wthn clusters (meanng all the 9 Frst dfferencng s another way to remove the unobserved group heterogenety. Although both are consstent, the frst-dfference and fxed-effects estmators dffer n ther assumptons about the dosyncratc errors. See Wooldrdge (010, p. 31) for detals on the relatve effcency of the two estmators. 15

21 observatons for any gven group are n the same cluster), as s commonly the case (e.g., frm fxed effects are nested wthn frm, ndustry, or state clusters). Statstcal software programs that estmate FE specfcatons make these adustments automatcally. 10 See Wooldrdge (010, Chapters 10 and 0), Arellano (1987), and Stock and Watson (008), for more detals.. Understandng AdY and AvgE n the context of FE The wthn-group transformaton of the FE estmator hghlghts the key problem of the AdY and AvgE estmators: they fal to account for the relaton between the group mean of the ndependent varable,, and the group mean of the ndependent varable, y. Comparng the FE estmaton n Equaton (11) wth the true underlyng structure of the demeaned dependent varable n Equaton (77), we see that the FE estmator correctly controls for the ndependent varable mean,, and restrcts ts coeffcent to equal β. The AdY estmator, however, fals to account for, and the AvgE estmator makes the same mstake and also fals to restrct the coeffcent on y to equal one (see Equaton (99)). Another way to understand why AdY and AvgE provde nconsstent estmates s to compare them to a regresson of Y onto two ndependent varables and Z. As s well known, a researcher nterested n the effect of on Y controllng for Z can dentfy ths effect by regressng the resduals from a regresson of Y on Z onto the resduals from a regresson of on Z. Partalng out the effect of Z from both and Y before regressng Y on s equvalent to regressng Y on controllng for Z [see Greene (000, pp ) for more detal]. Ths s the same reason why the wthn-group transformaton mplementaton of the FE estmator s equvalent to least squares dummy varable estmaton. The wthngroup transformaton s smply the result of partalng out the collecton of ndcator varables (the Z n ths case) from both the ndependent and dependent varables. The AdY estmaton, however, s equvalent to partalng out the effect of Z from only the dependent varable, Y, whch s not equvalent to regressng Y on controllng for Z. The AvgE approach s equvalent to regressng Y on and the ftted values from a regresson of Y on Z, whch s also not the same as regressng Y on and Z. By falng to transform the ndependent varable,, both estmators are nconsstent. 10 In the current verson of Stata, for example, these standard errors are reported by the TREG command. The AREG command, however, reports larger cluster-robust standard errors that nclude a full degrees of freedom adustment. More nformaton on dfferences between AREG and TREG cluster-robust standard errors s avalable on our webste, 16

22 .3 Other related estmaton technques are also nconsstent Other varants of the AdY and AvgE estmators are also problematc. In ths secton, we dscuss four related estmaton strateges that are nconsstent and suggest alternatves..3.1 Other dependent varable adustments or controls. Estmatng varants of AdY or AvgE that substtute other group summary statstcs for the group mean smlarly results n nconsstent estmates. For example, some papers adust the dependent varable n AdY-type estmaton by subtractng a group medan or subtractng a value-weghted group mean. These estmates suffer an omtted varable bas because they fal to account for how the correspondng group medan or value-weghted group mean of the ndependent varables affects the adusted dependent varable. Lkewse, usng a group medan or a value-weghted group mean n AvgE-type estmaton s problematc because both measure the unobserved heterogenety wth error. In both cases, the fxed effects estmator s the proper way to control for unobserved heterogenety. The omtted varable problem underlyng AdY also apples to other types of estmatons. Any dependent varable that s constructed from multple observatons wthout smlarly adustng the ndependent varables s problematc, even when ths s not done to control for unobserved heterogenety. For example, consder the case where a lnear combnaton of other observatons s used to adust the dependent varable. Falng to control for the ndependent varables of all observatons used to construct the dependent varable can cause an omtted varable bas. Adustng a dependent varable of nterest by frst regressng t onto other covarates and then usng the resduals as the adusted dependent varable s smlarly problematc. As dscussed n Secton., ths approach s not equvalent to controllng for the varables ncluded as covarates n the frst step regresson. To control for these varables properly, a researcher must ether partal them out from the ndependent varables or smply control for them drectly n the man specfcaton..3. Characterstcally adusted stock returns. AdY-type estmators also can be found n many asset prcng artcles. One approach commonly used to remove the nfluence of common rsk factors s to adust frms stock returns usng the average return of comparable frms. Ths method was frst proposed by 17

23 Danel et al. (1997) and has snce been adopted wdely n dverse settngs. A researcher frst sorts the data nto varous benchmark portfolos based on frm-level characterstcs, such as sze, book-to-market ratos, and momentum, and then adusts the ndvdual stock returns by demeanng them usng the average return of other frms n the same benchmark portfolo. From an econometrc perspectve, there s nothng wrong wth usng the adusted return as a measure of stocks performance, as proposed by Danel et al. (1997); t accurately summarzes a portfolo s performance relatve to a benchmark return. Problems arse, however, f the adusted return s then correlated wth other (unadusted) stock or frm characterstcs. For example, t s common for researchers to calculate adusted stock returns, sort them nto portfolos based on an ndependent varable that s thought to affect stock returns, and then compare the adusted returns across the top and bottom portfolos as a test of whether the ndependent varable affects stock returns. Ths sort, however, s equvalent to AdY estmaton where the adusted returns are regressed onto ndcators for each ndependent varable portfolo whle excludng the constant term from the regresson. Smlar to other AdY estmators, ths approach provdes nconsstent estmates because the specfcaton fals to control for how the average ndependent varable of other frms n the portfolo nfluences the adusted stock return. 11 As an example, consder analyses of research and development (R&D) ntensty and stock returns. Although R&D and returns are postvely correlated, t s possble that dfferences n frm sze confound ths relatonshp. Larger frms are assocated wth lower stock returns (n part for reasons presumably unrelated to R&D ntensty). If R&D ntensty s also correlated wth frm sze, then the correlaton between R&D and returns may be attrbutable to frm sze rather than to R&D. Usng szematched benchmark portfolos to adust returns but not adust R&D does not adequately control for sze, because t does not account for average dfferences n R&D ntensty across the benchmark portfolos. The average R&D of frms n a stock s benchmark portfolo affects ts adusted stock return but ths fact s overlooked when one compares adusted returns across portfolos sorted on frms unadusted R&D ntensty. To control for unobserved rsk factors across portfolos approprately, one needs to adust the 11 Other papers regress adusted stock returns on contnuous varables. For example, some studes regress frms adusted returns on changes n ther cash balances and nterpret the senstvty as the frms nternal value of cash. Such AdY estmatons are also nconsstent as they fal to control for dfferences n cash changes across the benchmark portfolos. 18

24 ndependent varable beng analyzed n addton to adustng returns. One way to accomplsh ths s to double-sort returns on the benchmark portfolo and the ndependent varable and then to compare returns across the ndependent varable wthn each benchmark portfolo. In the R&D example, ths wthnbenchmark-portfolo comparson properly controls for dfferences n average R&D ntensty across frm sze. The average dfference between the top and bottom ndependent varable portfolos across all of the benchmark portfolos summarzes the average effect of the ndependent varable on stock returns whle properly controllng for the characterstcs used to construct the benchmark portfolos. Ths quantty s equvalent to a fxed effects estmator; the dentcal estmate can be obtaned by regressng returns onto ndcators for each ndependent varable portfolo (excludng the bottom portfolo) and benchmark portfolo fxed effects. In practce, the double-sort s cumbersome to report when there are many benchmark portfolos (e.g., 5 sze x 5 book-to-market x 5 momentum = 15 portfolos) and systematc patterns may be dffcult to eyeball. The FE estmator accurately summarzes these patterns and s easy to mplement. In Secton 3.4, we provde an example of ths alternatve fxed effects approach and show that the more conventonal characterstcally adusted stock returns estmator s nconsstent..3.3 Comparsons of adusted varables. Even smple comparsons of an adusted varable, such as n the tme perods surroundng an event of nterest, can yeld ncorrect nferences. In many applcatons, ncludng studes of mergers and acqustons and leveraged buyouts, researchers compare the means of an adusted varable for frms across two tme perods. The varable of nterest, however, s frst adusted by subtractng the mean of some benchmark group, whch sometmes conssts of all frms from the same ndustry, all frms of smlar sze, and/or all frms of smlar past performance. As shown n Proposton 4, ths pre versus post comparson, however, does not reveal the true effect of the event beng analyzed. Ths s because the comparson ncorrectly removes part of the event s effect on the dependent varable when t demeans the outcome of nterest usng an average that ncludes both affected and unaffected frms. Proposton 4. A pre- versus post-event comparson of an adusted outcome varable does not reveal the effect of the event. Specfcally, ˆ AdY =, where s the average fracton of affected frms n an affected frm s benchmark group. 19