Performances of Ordinary and Generalized Least Squares Estimators on Multiple Linear Regression Models with Heteroscedasticity

Transcription

1 Performances of Ordnary and Generalzed Least Squares Estmators on Multple Lnear Regresson Models wth Heteroscedastcty Adebowale Olusola Adejumo 1* ; Obalowu Job 1 ; Ttlayo Dorcas Isaac ; and Benjamn Agboola Oyejola 1 1 Department of Statstcs, Unversty of Ilorn, Ilorn, Ngera. Department of Mathematcs, Federal Unversty Lafa, Nassarawa, Ngera.. E-mal: aodejumo@unlorn.edu.ng* ABSTRACT Ths paper focuses on the mpact of heteroscedascty on the estmate and varance of model parameters by studyng the performances of Ordnary Least Square (OLS) and General Least Square (GLS) estmators n multple lnear regresson models wth two ndependent varables only and error term characterzed wth dfferent magntudes of heteroscedastcty related to predctors at dfferent sample szes. Ths research explored the patterns of the varance estmates offered by the two estmators when data s front wth heteroscedascty. Also the studes explored the stuatons where concepts of substtute, complementary or jont demands/supply may reflect n the stochastc characterzaton of the error term. From Monte Carlo smulaton studes usng R-package the GLS estmator mantans ts superorty over the OLS n multple lnear regresson models. (Keywords: heteroscedastcty, ordnary least squares, generalzed least squares, stochastc error term, magntude, characterzaton, Monte Carlo, smulaton) transformaton n the data, bearng n mnd that the macroeconomc data publshed by the government are often aggregates. Regresson analyss s a statstcal tool used for testng hypothess about the lnear relatonshp between a dependent varable Y and explanatory varable (X) and for vald predcton. Regresson analyss s concerned wth the study of the dependence of one varable (dependent varable) on one or more of the explanatory varables, wth a vew to estmatng and/or predctng the populaton mean(average value )of the former n terms of the known or fxed values of the latter. In lnear regresson models, the dsturbance term I s a surrogate for all those varables that are omtted from the model occasoned by msspecfcaton but collectvely affect values of Y. Error term s often ncluded n catchall models as a substtute for all the excluded varables from the model. The dsturbance term s assumed normal wth mean zero, INTRODUCTION E( ) 0 and Var ( ), Data may be avalable for economc research though t s often plagued wth some problems. The valdty of any analyss ultmately depends on the avalablty of approprate data. Just as tme seres data do encounter non-statonary problems, cross-sectonal data too may portray especally the problems of heteroscedastcty, autocorrelaton and/or, multcollnearty. Thus n practce, regresson analyss depends on the avalablty of approprate data. Therefore t s very germane that researchers state clearly the sources of the data used n the analyss, defntons, methods of collecton, and any gaps or omssons n the data as well as any whch descrbes the homoscedastcty assumpton. When the assumpton of homoscedastcty s beng volated,.e.. Var ( ) for every, heteroscedastcty sets n. Heteroscedastcty s lkely to be more predomnant n cross sectonal data than n tme seres data. Cross-sectonal data can be collected on famly expendture, ncome sze, demand, producton, supply, etc. by randomly drawng sample sze (n) from the populaton (N). The Pacfc Journal of Scence and Technology 68 Volume 17. Number 1. May 016 (Sprng)

2 Accordng to Berry and Feldman (1985), slght heteroscedastcty has lttle effect on sgnfcances test, however when heteroscedastcty s marked, t can lead to serous dstorton of fndngs and serously weaken the analyss, thus ncreasng the probablty of type I error. Heteroscedastcty does not destroy the unbasedness and consstency propertes of the OLS estmator except that they are no longer effcent. Ths lack of effcency makes the usual hypothess testng procedure dubous, snce the estmated standard error s ether under or overestmated n ether case resultng n ncorrect nferences. And f we persst n usng the usual testng procedures despte heteroscedastcty, whatever conclusons we draw or nferences we make, may be msleadng therefore remedal measures become necessary. We refer to a Monte Carlo study conducted by Davdson and Macknnon (1993). They consdered the smple lnear model that follows: Y 1 X (1) They assume that 1 and ~ N(0, X h ) 1. As the last expresson shows, they assume that the error varance s heteroscedastc and s related to the value of the regressor X wth power h. If for example h=1, error varance s proportonal to the value of X: f h=, error varance s proportonal to the square of the values of X, and so on. Based on 0,000 replcatons and allowng for varous values of h, they obtan the standard errors of the two regresson coeffcents usng OLS, OLS allowng for heteroscedastcty and GLS estmators. The most strkng feature of these results s that OLS wth or wthout heteroscedastcty, consstently overestmates the true standard error obtaned by GLS procedure, especally for large values of h, thus establshng the superorty of GLS estmator. These results also showed that f we do not use GLS and rely on OLS-allowng for or not allowng for heteroscedastcty- the pcture s mxed. The usual OLS standard errors are ether too large for the ntercept or too small for the slope coeffcent n relaton to those obtaned by OLS allowng for heteroscedastcty. In furtherance of the Monte Carlo experment conducted by Davdson and Macknnon (1993) ths research attempts to study multple lnear regresson models wth two ndependent varables only where the error term s assumed to be related to each ndependent varable separately and jontly up to a multplcatve constant. Ths research examned the performances of OLS and GLS estmators usng a multple lnear regresson model front wth heteroscedastcty of dfferent magntudes nherent n the error term n relaton to the predctor(s). METHODOLOGY Heteroscedastcty s potentally a serous problem and researchers need to ascertan ts presence n regresson models. If ts presence s detected, then correctve acton becomes mperatve. Methods of detectng heteroscedastcty n bvarate data set are many, but for ths study we employed Goldfeld and Quandt test to detect the lkely presence of heteroscedastcty n the regresson model. It s a formal method of detectng heteroscedastcty. Monte Carlo Smulaton Study Consder the general form of heteroscedastcty wth the error structure: h ) X j,where h 0 Var( Generally speakng homoscedastcty stuaton can be made heteroscedastc by changng the error structure n relaton to one of the regressors. Step 1: Generate X j wth mean x and varance x usng sample sze (n) of observatons. Step : Formulate the error structure by specfyng the magntude of heteroscedastcty, such that h=0.5, 1, 1.5,,.5, 3 The Pacfc Journal of Scence and Technology 69 Volume 17. Number 1. May 016 (Sprng)

3 Step 3: Fx Values for β j, β 0 = β 1 =β =1 Step4: Generate the values of the dependent varable (Y). Y 1X 1 X () Step 5: Carry out the OLS regresson of the dependent varable(y) on the predctor(x). Step 6: Test for the presence of heteroscedastcty usng Goldfeld Quandt test. Step 7: If heteroscedastcty s lkely n Step 6, go to Step 8 otherwse examne Steps 1 to 5 Step 8: X-ray the performance of OLS and GLS estmators n the presence of heteroscedastcty. Ordnary Least Square Estmators (OLS) The classcal ordnary least square method (OLS) s wdely used n regresson analyss. The estmator mnmzes the sum of squares error.it becomes unsutable to apply OLS on regresson models when predctor varables are hghly correlated or heteroscedastcty s beng detected. For smple lnear regresson model (SLRM): y 1 X, OLS estmator remans unbased. Heteroscedastcty does not destroy the unbasedness property of the OLS estmator, but t becomes less effcent. Hence the varance of the model parameters s no longer the mnmum varance n the class of lnear estmators. Recall that for homoscedastcty stuaton, when h=0: ˆ Var( ), x X X (3) x And for heteroscedastcty, when h=1 ˆ * Var( ) X x, x X X (4) Hence ˆ * Var ( ) var( ˆ ) Generalzed Least Square Estmator (GLS) The generalzed least square estmator (GLS) make use of nformaton avalable n the data and assgn dfferent weght to varables accordng to ther szes such that observaton comng from populaton wth greater varablty are gven less weght than populaton wth smaller varablty. To llustrate ths supposed: y 0 x0 1X1 X (5) Where x 1 for all =1,,.., n 0 Assumng that the heteroscedastcty varance term s known up to a multplcatve constant. We shall transform the model equaton by dvdng through by to make t homoscedastc: y x0 x1 X 0 1 Whch for ease of exposton we wrte as: * * 0 x0 * * 1 X * 1 X (6) y * * * It s remarkable to note that the transformed error term n the model becomes homoscedastc. * * 1 Var ( * ) =E ( ) E( ) (7) 1 E( ) 1 Ths showed that the varance of the transformed dsturbance term * s now homoscedastc. Snce we are stll retanng all other assumpton of the classcal model, the fndng that * s homoscedastc suggests that we can now apply OLS to the transformed The Pacfc Journal of Scence and Technology 70 Volume 17. Number 1. May 016 (Sprng)

4 data whch wll produce estmates wth mnmum varance. Ths affrms that GLS s OLS transformed (see McCullach and Nelder, 1983). For SLRM, recall that when homoscedastcty assumpton holds the OLS mnmzes: ˆ y 0 ( ˆ 1x ) (8) But GLS mnmzes the unequally weghted resduals, the expresson wrtten as: * ( ) w w ( ˆ * 0 0 ˆ y x * 1 x ) (9) Where 1 w Thus GLS mnmzes a weghted resdual sum of square wth 1 actng as the weght. w But OLS mnmzes an un-weghted or equally weghted resdual sum of square (RSS). Ths shows that the GLS technque assgns weghts to each observaton whch s nversely proportonal to t varance (Damodar, 011). Goldfeld and Quandt Test Ths test s applcable to a large sample snce the number of observatons must be at least twce as the number of parameters to be estmated. The test assumes normalty and serally ndependent stochastc error terms. If we assume that the heteroscedastcty varance s postvely related to one of the explanatory varables n the regresson model. Then ft model lke n Equaton (1), X Y 1. Goldfeld and Quandt procedures follow: () Rank the observaton accordng to the magntude of explanatory varables X, suppose two or more varables are ncluded n the model, a varable X whch s closely related to Y s chosen for rankng the data. If apror we are not sure whch X varable s approprate, we can conduct the park test on each of those X varables. () Select a certan number of central observaton, call t c whch we shall omt from the analyss accordng to Mont Carlo experment conducted by Goldfeld and Quandt, they suggested the number of omtted observaton c when n=30, c=8and when n=60, c=16. Accordng to Judge et al the number of omtted observatons when n=30, c=4 and when n=60, c=10, here c s specfed a pror and the remanng (n-c) observatons are dvded nto two sub-samples of equal n c number of observatons, the ratonale behnd ths s to sharpen the dfference between the small varance group and the large varance group. () Ft separate OLS regresson to each subsample value of x. The frst the last n c observatons. n c and The dea s to obtan the respectve resdual sum of squares RSS 1 and RSS small and large varance groups, respectvely. Each RSS has 1 (n c) k degree of freedom, where k s the number of parameters to be estmated ncludng the ntercept. (v) Compute the rato, * F Where ( 1 ) RSS ~ ( 1) Rss1 F r 1, r 1 v 1 v ( n c) k (10) If * F 1or1 ths mples homoscedastcty otherwse f F * 1 orf v 1,, (v) Reject H 0. e. heteroscedastcty s lkely present (Damodar, 011). The Pacfc Journal of Scence and Technology 71 Volume 17. Number 1. May 016 (Sprng)

5 ANALYSIS In ths study we ft a multple lnear regresson model wth heteroscedastc error term related to the regressor(s) n order to examne the mpact of heteroscedastcty on model parameters. For nstance n demand and supply studes demand for certan commodtes are compettve, complementary, derved or jont n nature. Ths study explores the demand and supply scenaro above. It s feasble that heteroscedastcty s beng related to one of the regressors or jont regressors n multple lnear regresson model (MLRM) as t may be typcal of jont demand/supply n economc stuatons. We conducted a test for homoscedastcty assumpton employng Goldfeld and Quandt test to affrm that the smulated data s plagued wth heteroscedastcty. Based on 1000 replcaton the we run the OLS and GLS regresson to obtan the standard error estmates of model parameters for dfferent value of h =0.5,1,1.5,,.5, 3, and sample szes(n)= 0, 5,30,35,45 and 50. The results for test of heteroscedastcty are summarzed n Table 1. From Table 1 we observed that heteroscedastcty s lkely present as depcted by p-values. The performances of OLS and GLS estmators vs-a-vs the varance estmates of model parameters when are summarzed n Table. Table 1: Goldfeld and Quandt Test When N G df1 df GQ p-value E E E E E E E The Pacfc Journal of Scence and Technology 7 Volume 17. Number 1. May 016 (Sprng)

6 Table : Standard Error of Estmates of Model Parameter When. Sample szes G 0) OLS 0) GLS 1) OLS 1) GLS ) OLS ) GLS E E E-16.3E E E E E E E E E E E E E E E E E E E-16.77E E E E E E E When, the followng observatons are feasble from Table : the 1) GLS s the least, followed by ) GLS. Next s the 1) OLS then ) OLS. The 0) GLS s smaller than 0) OLS. Also from the smulaton studes we observed that: 0) GLS < 0) OLS 34 out of 36 tmes (94.4%) s ). e( 1 OLS ) OLS 33 out of 36 tmes (91.7%) 1) GLS ) GLS 33 out of 36 tmes (91.7%) ) GLS 1) OLS 9 out of 36 tmes (80.6%) 1) GLS 0) OLS 36 out of 36 tmes (100%) For the case of when error term s contamnated wth heteroscedastcty of the form whch s up to a jontmultplcatve constant n the model y 1 x1 x. Data smulated was examned for heteroscedastcty usng Goldfeld-Quandt test. The results are The Pacfc Journal of Scence and Technology 73 Volume 17. Number 1. May 016 (Sprng)

7 summarzed n the Table 3. Snce the p-values are strctly less than α=0.05. It s lkely that heteroscedastcty s present. Agan the performances of the OLS and GLS estmators when the error term s related to the product of values of the two regressors up to a multplcatve constant n the model, y1x1 x are summarzed n Table 4. h When var( ) ( X 1* X ) the followng observatons are feasble from Table 4: The 1) GLS s the least, followed by ) GLS. Next s the 1) OLS then ) OLS. The 0) GLS s smaller than 0) OLS. Also from the smulaton studes we observed that: 0) GLS < 0) OLS 36 out of 36 tmes (100%) 1) OLS (100%) s ). e( 1 GLS ) OLS 36 out of 36 tmes 1) OLS 36out of 36 tmes (100%) 1) GLS ) GLS 30out of 36 tmes (83.3%) ) GLS 1) OLS 9 out of 36 tmes (80.6%) Table 3: Goldfeld and Quandt Test when n G df1 df GQ p-value E E E The Pacfc Journal of Scence and Technology 74 Volume 17. Number 1. May 016 (Sprng)

8 Table 4: Standard Error of Estmates of Estmators When. Sample szes g 0) OLS 0) GLS 1) OLS 1) GLS ) OLS ) GLS DISCUSSION OF RESULTS Emprcally, Goldfeld-Quandt test showed that the smulated data va Monte Carlo smulaton method s heteroscedastc n nature judgng by P -values that are strctly less than the pre-specfed level of sgnfcance, α=0.05. Also based on 1000 replcaton, varyng magntude of heteroscedastcty, h=0.5, 1, 1.5,,.5, and 3 respectvely and for dfferent sample szes (n), 5, 30, 35, 40 and 50. The followng observatons were deduced from the analyss: When error varance s related to the values of frst explanatory varable (X 1 ) at dfferent sample szes and dfferent magntude of heteroscedastcty, j) OLS s greater than. e( j GLS for j 0,1,. It was also observed s ) from the table that, s e( 1) GLS. gves the least standard error of estmate when compared to the 0) OLS and 1) OLS, ths may be as a result of the error term beng related to values of the explanatory varable X 1, n the multple lnear regresson model wth two ndependent varables only. Also t was observed that sample sze alone does not affect standard error of estmate but The Pacfc Journal of Scence and Technology 75 Volume 17. Number 1. May 016 (Sprng)

9 the magntude of heteroscedastcty. These results are n tandem wth that obtaned by Davdson and Macknnon (1993) usng SLRM. When the varance of error term relates to the product of values of varables X1 and X, n regresson models havng only two predctors, we observed that the correspondng standard error of estmates of, j 0,1, offered by the GLS j estmator s less than that of OLS estmator for all sample szes at dfferent magntude of heteroscedastcty. Specfcally 1) GLS s the least. Next s ) GLS.Also 0) GLS s less than 0) OLS. The 1) OLS s strctly less than ) OLS most of the tmes. Ths may be due to the error term relatng to the product of the values of two explanatory varables up to a jont-multplcatve constant n the multple lnear regresson models wth two ndependent varables only (see Fgures 1 and ). Graphcal Dsplay of s. e( j) when. Fgure 1 (a): For n=5 and Varyng Magntude of Heteroscedastcty. Fgure1 (b): For n=30 and Varyng Magntude of Heteroscedastcty. The Pacfc Journal of Scence and Technology 76 Volume 17. Number 1. May 016 (Sprng)

10 Graphcal Dsplay of s. e( j) When. Fgure (a): For n=5 and Varyng Magntude of Heteroscedastcty. Fgure (b): For n=30 and Varyng Magntude of Heteroscedastcty. CONCLUSION Based on the analyss carred out t was evdent that OLS wth heteroscedastcty over-estmate the true value of standard error of estmate obtaned by the GLS estmator when error term s related to one or two explanatory varables up to multplcatve constant only n a multple lnear regresson models hence establshng the superorty of GLS estmator. In our fndngs, the effect of the standard error of estmate when error term s related to two explanatory varables up to a multplcatve constant n the multple lnear regresson models s smlar to when error term s related to the frst regressor, except that the pcture s not clear for that of ( 1 ) OLS and ( ) OLS as we have n the results when error term s related to the product of the two explanatory varable. Hence we can conclude that heteroscedastcty affects the effcency The Pacfc Journal of Scence and Technology 77 Volume 17. Number 1. May 016 (Sprng)

11 property of OLS estmator. There s upward bas scenaro n the estmates of varances of the model parameters. Therefore, the magntude of heteroscedastcty on a partcular explanatory varable reduces ts standard error of estmate. Agan these results are n tandem wth the result obtaned by Davdson and Macknnon (1993). RECOMMENDATIONS When heteroscedastcty s marked n a research data, t can lead to serous dstorton of fndngs and serously weaken the analyss, whch may lead to msleadng concluson. Hence t s advsable to correct for heteroscedastcty so as to mprove precson and make relable concluson when modelng a multple lnear regresson fraught wth heteroscedastcty of any amount of magntude. SUGGESTED CITATION Adejumo, A.O., O. Job, T.D. Isaac, and B.A. Oyejola Performances of Ordnary and Generalzed Least Squares Estmators on Multple Lnear Regresson Models wth Heteroscedastcty. Pacfc Journal of Scence and Technology. 17(1): Pacfc Journal of Scence and Technology REFERENCES 1. Berry, W.D. and S. Feldman Multple Regresson n Practce. Sage Unversty Paper Seres on Quanttatve Applcaton n the Socal Scence, Seres no Sage: Newbury Park, CA.. Damodar, N.G Basc Econometrcs. Fourth Edton. McGraw-Hll: New York, NY. 3. Davdson, R. and J.U.G. Macknnon Estmaton and Inference n Econometrcs. Oxford Unversty Press: New York, NY. 4. Fomby, T.B., C.R. Hll, and S.R. Johnson Advanced Econometrc Methods. Sprnger-Verlag: New York, NY. 5. Goldfeld, S.M., and R.E. Quandt Nonlnear Methods of Econometrcs. North-Holland: Amsterdam: The Netherlands. 6. Gujarat, D Basc Econometrcs, Thrd Edton. McGraw-Hll: New York, NY. 7. Judge, G.G., C.R. Hll, W.E. Grffths, H. Lütkepohl, and T.C. Lee Theory and Practce of Econometrcs. John Wley & Sons: New York, NY. 8. Monte Carlo Expermentaton usng PC-NAÏVE Advances n Econometrcs (wth A. J. Neale). 9. McCullach, P. and J.A. Nelder Generalzed Lnear Models. Number 37 n Monographs on Statstcs and Appled Probablty. Chapman & Hall: New York, NY. The Pacfc Journal of Scence and Technology 78 Volume 17. Number 1. May 016 (Sprng)