ROBUST STATISTICS FOR SOFT SENSOR DEVELOPMENT IN CEMENT KILN. Bao Lin 1, Bodil Recke 2, Philippe Renaudat 2 Jørgen Knudsen 2, Sten Bay Jørgensen 1

Transcription

1 ROBUS SAISICS FOR SOF SENSOR DEVELOPMEN IN CEMEN KILN Bao Ln 1, Bodl Rece, Phlppe Renaudat Jørgen Knudsen, Sten Bay Jørgensen 1 1 CAPEC, Department of Chemcal Engneerng, DU, Denmar FLS Automaton, Valby 500, Denmar Abstract: hs paper presents a systematc approach of developng data-drven soft sensor usng robust statstcal technque. Data preprocessng procedures are descrbed n detal. Frst, a template defned wth a ey process varable s used to handle mssng data. Second, a unvarate, followed by a multvarate approach, prncpal component analyss (PCA), s used to detectng outlyng observatons. hen, regresson technque s employed to derve an nferental model. he proposed methodology s appled to a cement ln system for realtme estmaton of free lme, demonstratng mproved performance over a standard multvarate approach. Copyrght 005 IFAC Keywords: Regresson analyss, Soft sensng, Statstcs 1. INRODUCION Soft sensors have been developed as supplement to onlne nstrument measurements for process montorng and control. Both model-based and datadrven soft sensors have been developed. If a frst prncple model (FPM) descrbes the process suffcently accurately, a model-based soft sensor can be derved. However, a soft sensor based on detaled FPM s computatonally ntensve for real tme applcatons. Modern measurement technque enables a large amount of operatng data to be collected and stored, thereby renderng data-drven soft sensor development a vable alternatve. A data-drven soft sensor s an nferental model developed from process observatons. Multvarate regresson technques have been extensvely employed to develop an emprcal model. Multvarate lnear regresson (MLR) suffers from numercal problems as well as degraded models when a data set s strongly collnear. Both prncpal component regresson (PCR) and partal least squares (PLS) solve ths ssue by proectng the orgnal process varables nto a smaller number of orthogonal latent varables. Early wor on soft sensor development assumed that a process model was avalable. he nferental model s developed usng Kalman flter (Joseph and Broslow, 1978). In case the process mechansms are not well understood, emprcal models, such as neural networ (Qn and McAvoy, 199; Radharshnan and Mohamed, 000), multvarate statstcal methods are used to derve the regresson model (Kresta et al., 1994; Par and Han, 000; Zhao, 003). Process measurements are often contamnated wth data ponts that devate sgnfcantly from the real values due to human errors, nstrument falure or changes of operatng condtons. Snce outlyng observatons spol the regresson model, robust statstcal approaches have been developed to provde relable results n the presence of abnormal observatons. hs paper presents a systematc approach for buldng a soft sensor. he applcaton example s estmaton of free lme for cement lns usng robust multvarate methods.

2 he paper s organzed as follows. Secton descrbes both unvarate and multvarate approaches to detect outlyng observatons. he robust PCR and PLS approaches are presented n secton 3, followed by the llustratve applcaton on development of a free lme soft sensor for a cement ln.. DAA PREPROCESSING Outlers are commonly defned as observatons that are not consstent wth the maorty of the data (Pearson, 00; Chang et al., 003), ncludng mssng data ponts or blocs, and observatons that devate sgnfcantly from the normal values. Popular multvarate approaches of buldng data-drven soft sensor, such as PCR and PLS, assume a lnear relatonshp between varables. he performance of derved model deterorates even n the presence of a sngle abnormal observaton. Process data from cement plants are commonly contamnated, e.g. by abnormal values, whch may lead to model msspecfcaton. herefore, outler detecton consttutes an essental prerequste step for a datadrven soft sensor desgn. Fg. 1. Segment of operatng data from a cement ln A heurstc procedure has been mplemented n the paper to handle mssng data. Fgure 1 shows a segment of four process measurements from a cement ln. A systematc pattern of mssng data s observed, whch can be dentfed wth a template defned by a ey measurement n the dataset. In case a small bloc (.e., less than hour) of data s mssng, nterpolated values usng neghbor observatons wll be nserted. If a large segment of mssng data s detected, these blocs wll be mared and not be used to buld the soft sensor. Mssng data s one type of outlers. he second type denotes abnormal operaton condtons. For example, the malfuncton of process equpment mght cause a change n process measurements that may affect several successve samples. Both unvarate and multvarate approaches have been developed to detect these outlyng process observatons. A popular approach to detect outlers s the 3σ edt rule (Ratclff, 1993). hs method labels outlers when data ponts are three or more standard devatons from the mean. Unfortunately, ths procedure often fals n practce because the presence of outlers tends to nflate the varance estmaton, causng too few outlers to be detected. he Hampel dentfer (Daves and Gather, 1981) replaces the outler-senstve mean and standard devaton estmates wth the outler-resstant medan and medan absolute devaton from the medan (MAD). he MAD scale estmate s defned as: MAD medan x x (1) where x s the medan of the data sequence. he factor was chosen so that the expected value of MAD s equal to the standard devaton σ for normally dstrbuted data. Snce process measurements from the cement ln system are not ndependent from each other, detectng outlers usng unvarate dagnostcs s not suffcent, resultng n masng and swampng. Masng refers to the case that outlers are ncorrectly dentfed as normal samples; whle swampng s the case that normal samples are classfed to be outlers. Effectve outler detecton approaches are expected to be based on multvarate statstcal technques. Prncpal component analyss (PCA) s a multvarate analyss tool. Gven a data matrx X constructed by m observatons of n varables, PCA proects t to a lower dmensonal space that explans a large fracton of varablty n the orgnal data. X t1 p1 tp... tp E tp E () 1 where E s the resdual matrx. he orthogonal loadng vectors, p, are egenvectors of the scatter matrx, 1 ( X X). he score vector, t, s the m 1 proecton of X along the drecton of p. he loadng vectors correspondng to the largest egenvalues are retaned to optmally capture the varatons of the data and mnmze the effect of random nose. he ftness between data and the model can be calculated usng the resdual, e, defned as: e x xˆ x (1 P P ) (3) where e s the th row of E and P s the matrx of the frst loadng vectors. he magntude, Q ee, ndcates how well a sample fts to the PCA model. he sgnfcance level for the Q statstc s:(jacson and Mudholar, 1979) where Q a c h0 h0 ( h0 1) 1 1 (4) 1 1 n 1 h0 1 3, 1 3 1,, 3 1 h0

3 s the egenvalue c s the normal devate correspondng to the upper 1 percentle. Q-statstc measures the varaton of a sample outsde of the PCA model. Hotellngs statstc provdes an ndcaton of varablty wthn the normal subspace. he value of a sample s equal to the sum of squares of the adusted (unt varance) scores: t (5) 1 It ndcates the dstance that the estmated sample by the PCA model s from the multvarate mean of the data. he statstcal sgnfcance level for can be calculated wth F-dstrbuton:(Wse, 1991) ( m 1), m, F (6), m, m where s the standard normal devate. he combned Q- and -tests are used to detect abnormal observatons (Chang et al., 003). Measurements wth Q Q or are, m, classfed as outlers. In ths paper the sgnfcance level alpha has the same value n the two tests, however fndng a compromse between acceptng large modelled dsturbances and reectng large unmodelled behavours for outler detecton clearly needs further nvestgaton. 3. ROBUS SAISICS Scalng s an mportant step n PCA. Snce numercally large values are assocated wth numercally large varance, approprate scalng methods are ntroduced such that all varables wll have approxmately equal weghts n the PCA model. Autoscalng s commonly used n the absence of a pror nowledge about relatve mportance of process varables. Frst, each measurement s mean-centered by subtractng the mean value of the varable. hen the measurement s dvded by standard devaton to be scaled down to unt varance. As prevously mentoned, both the mean value and the standard devaton are nflated by the outlyng observatons. Autoscalng s not sutable n handlng data whch are especally nosy. For such cases, a robust scalng has been suggested (Chang et al., 003): where M x x x x MAD M (7) MAD x s the medan and x the MAD of the varable. hs paper apples robust scalng to cement ln data before performng PCA. here are two types of approaches for renderng PCA robust. he frst s detectng and removng outlers usng a unvarate approach then carry out a classc PCA on the new data set; the second s multvarate and s based on robust estmaton of covarance matrx. In ths study, ellposdal multvarate trmmng (MV) (Devln et al., 1981) approach s used. It teratvely detects bad data based on the squared Mahalanobs dstance: where d ( x x ) S 1 ( x x ) (8) x s the current robust estmaton of the locaton and S s the robust estmaton of the covarance matrx. 95% of measurements wth smallest Mahalanobs dstance wll be used for the estmaton of the covarance matrx n the next teraton. Snce the data set has been preprocessed wth a Hampel dentfer, 95% of data are retaned. he teraton proceeds tll both x and S converge. In ths paper, the teraton stops at the 10 th teraton such that at least 60% of the data s retaned for the estmaton of covarance matrx. Snce outlyng observatons dsrupt the covarance matrx from the ntalzaton step. Chang et al. (003) proposed the closest dstance to center (CDC) approach. he most consstent observatons are frst dentfed based on ther devaton from the center of the data. he m/ observatons wth smallest dstance are used to calculate the mean value, x. hus, a PCA model s developed usng the samples dentfed by the CDC/MV procedure. A regresson model s derved wth score vectors and free lme measurements from the lab. Prncpal component regresson (PCR) s a natural extenson of PCA to derve an nferental model. he measurement of free lme from lab analyss s assgned to be vector y. Gven the decomposton of PCA, y can be regressed aganst the matrx of score vectors, usng multvarate lnear regresson method. 1 b ( ) y (9) he coeffcents between orgnal data, X and y can be obtaned straghtforwardly as: c P b (10) PCA model dentfes outlyng observatons. Durng the regresson step, zero weghts are assgned to these outlyng observatons; a weght value of one s assgned to normal data. PLS s a multvarate statstcal approach for relatng nput data matrx, X and dependent data bloc y. PLS can be thought of as a smultaneous decomposton of the X and y matrces usng PCA. he X -matrx s proected onto a -dmensonal hyper-plane such that the coordnates are good predctors of y. he exstence of outlyng observatons can be the source of error n the PLS model that s based on the assumpton of lnear relatonshp between nput and output matrces. herefore, the outlyng measurements dentfed wth a PCA model s downweghted before PLS analyss.

4 he proposed approaches, robust PCR and weghted PLS, are appled to a data set collected from the log system of a cement ln. In summary, the systematc procedure of applyng robust statstcal technques for soft sensor development conssts of the followng steps: 1. Handle mssng data usng a template defned wth the ey process measurement;. Detect outlers wth a unvarate approach (Hampel dentfer) followed by a multvarate approach (robust PCA) usng Q and tests; 3. Derve regresson model wth weghted PLS; 4. Valdate the soft sensor on other process data. the perturbatons wthn the ln and the cooler, whch result n uncertan ndcaton of the average qualty. It s desrable to develop a soft sensor that s able to accurately predct the content of free lme n real tme, and can be employed for effectve qualty control. 5. CASE SUDY he data from a cement ln log system s used n ths study. here are 13 process measurements avalable, ncludng ln drve current, ln feed, fuels to calcner and ln, plus several temperature measurements wthn the ln system. Kln fuel s the 4. CEMEN KILN SYSEM he rotary ln s the most operatonally complex and energy consumng equpment n the cement ndustry. For most dry processes (as shown n Fgure ), the feed materals are preheated by hot gas from the rotary ln. A fuel combuston chamber, called precalcner, s ntegrated n the preheatng tower to mprove energy effcency. he mxture of preheated and precalcned materals enters the rotary ln, where fuel together wth ar enter from the opposte end. ton / hour Fg. 3. Kln fuel flowrate durng valdaton perod man manpulated varable for the ln system, whch vares from between 5 and 3.5 ton per hour. As shown n Fgure 3, ln fuel measurement s heavly contamnated by the mssng values and outlyng observatons. Followng the procedure mentoned prevously, a template s defned by the ln drve measurement to dentfy the pattern of mssng observatons, Fg.. A typcal modern dry ln system Several exothermc and endothermc reactons tae place n both sold and gas phases. he sold feed s heated to an extremely hgh temperature (about 1500 C) n the burnng zone such that raw materals react and form the nodular clner. he clner exts the ln at about 100 C, then s cooled down by cross-flowng ar n a separate clner cooler. Partal heat ntegraton s acheved by feedng part of the heated ar bac nto the ln and part to the precalcner. he operatng data of the precalcner and the ln are used to derve a soft sensor of free lme n the clner. he qualty of the ln product s ndcated by the amount of free lme. he drect measurement s generally only avalable wth a delay tme of about an hour. In addton, the measurement also suffers from he standard measurements are logged every 10 mn, whereas the laboratory analyss of free lme content of the clner s logged approxmately every hours. A data bloc of 000 samples s selected n ths study. the data bloc between 8000 and s used to derve the model and samples between and 1500 for valdaton. he one-step ahead predcton resdual sum of squared errors (PRESS) between the model and measured lme content s used to select the number of prncpal components (PCs): N V yˆ( ) y m ( ) PRESS (11) 1 where N s the total number of samples durng the V valdaton perod. It s calculated only when a new lab measurement s avalable. he PRESS of the PCR model usng 6 PCs has the mnmum PRESS (see Fgure 4). he 5th PC does not contrbute postvely to the predcton error, whch suggests that ths factor s not relevant for predctve ablty of the model.

5 PRESS Another regresson model s derved wth PLS approach for the same bloc of operatng data. he relaton between PRESS and the number of latent varables (LV) s shown n Fgure 7. Fg. 4. PRESS of robust PCR model Improved predctve ablty s acheved by ncludng the 6 th prncpal component at the expense of an ncreased nose level. he valdaton of PCR models wth 6 PCs (PRESS = ) s shown n Fgure 5. free lme (%) Number of PCs Fg. 5. Valdaton of robust PCR model wth 6 PCs ( - lab measurements; sold lne - robust PCR model) Gven the PCA decomposton, zero weghts are assgned to abnormal ponts to downweght these observatons before a regresson model s derved between score vectors and free lme results. 95% sgnfcance level s commonly used for Q- and - tests. he lower the sgnfcance level, the hgher the chance to reect outlyng ponts, at the rss of reectng essental process dynamcs. hus, the optmal value of sgnfcance level for Q- and - tests should be selected consderng the qualty of the data. Fgure 6 shows the PRESS of robust PCR model usng 6 PCs wth vared Q- and - test sgnfcance level, varyng from 100% to 95%. It s obvous that downweghtng outlyng observatons mproves the predctable ablty of the regresson model (see Fgure 6). he mnmum (PRESS = 38.36) s obtaned when the sgnfcance level s selected as 97.5%. free lme (%) PRESS Fg. 7. PRESS of PLS model he PLS analyss shows a mnmum of PRESS at LVs. PLS fnds LVs that descrbe a large amount of varaton n X and are correlated wth dependent varables, Y, whle the PCs n PCR approach are selected only on the amount of varaton they explan n X. herefore, PLS model s able to capture more of the relevant nformaton than PCR model wth a smaller number of LVs. Comparsons of the PLS model wth lab measurements durng modelng and valdaton perods are shown n Fgure 8 and 9 respectvely. Fg. 8. PLS model wth LVs versus lab measurements durng modellng perod ( - lab measurements; sold lne - PLS model) free lme (%) Number of LVs Fg. 9. PLS model wth LVs versus lab measurements durng valdaton perod ( - lab measurements; sold lne - PLS model) Fg. 6. PRESS of robust PCR model of 6 PCs wth sgnfcance level vared from 100% to 95% he study wth robust PCR models shows enhanced performance usng downweght scheme based on the PCA analyss of the data matrx, X. Snce PLS model s based on the assumpton of lnear relatonshp, the result of PLS analyss s affected by outlyng observatons n the dataset. Downweghtng these data ponts s benefcal for the PLS model.

6 Fg. 10. PRESS of weghted PLS model wth LVs sgnfcance level vared from 100% to 95% hen, the weghtng vectors obtaned durng the PCR analyss s appled to the PLS model. he weghtng vectors vary wth the selecton of sgnfcance level. Fgure 10 shows the PRESS of weghted PLS model versus the change of sgnfcance level from 100% to 95%. he mnmum s obtaned wth the sgnfcance level 99.5% wth a PRESS of 41.36, about 10% less than that of the standard PLS model (PRESS = ). Results of applyng standard and robust PCR, standard and weghted PLS are summarzed n able 1. Wth a larger number of prncpal components employed n the development of regresson model, sum of squared errors (SSE) durng the modelng perod of PCR approaches are comparable to those of PLS models. Among all the approaches nvestgated n ths paper, standard PLS model s worst regardng predctve ablty. Due to the contamnaton of outlyng observatons, PRESS of standard PLS (45.004) s hgher than the weghted model (41.36) derved utlzng the weghtng vector obtaned by PCA analyss. he robust PCR model (38.36) performs slghtly better than the PLS model at the cost of usng 4 more prncpal components. able 1. Comparson of PCR and PLS models Approach No. of PCs SSE PRESS Std. PCR Robust PCR Std. PLS Weghted PLS CONCLUSIONS hs paper presents a systematc approach to buld a soft sensor usng robust statstcal technques. he proposed methodology s appled to predct free lme of cement ln systems. Due to the low sgnal-tonose rato n operatng data, data preprocessng demonstrates to be an essental step for development of data-drve soft sensor. A case study demonstrates the mproved performance of robust PCA model n the detecton outlers. he real-tme estmaton of free lme can be obtaned wth the data-drven soft sensor, whch shows some potental to be used n closed loop control. However, the performance of soft sensor depends substantally on the data used to buld the nferental model. Due to the contamnaton of process measurements n operatng data, downweghtng outlyng observatons s benefcal to enhance the predctve ablty of a regresson model. he case study ndcates the exstence of a optmal downweghtng vector determned by the sgnfcance level of Q- and - statstcs. he ssue of fndng the optmal sgnfcance levels for regresson model and ntegratng the nformaton from rregularly-sampled low qualty measurements nto the weghtng vector needs to be performed n the future. REFERENCES Chang, L.H., R.J. Pell and M.B. Seasholtz (003). Explorng process data wth the use of robust outler detecton algorthms. Journal of Process Control, 13, (5): Daves, L. and U. Gather (1981). he dentfcaton of multple outlers. Journal of the Amercan Statstcal Assocaton, 88, Devln, S.J., R. Gnanadesan and J.R. Kettenrng (1981). Robust estmaton of dsperson matrces and prncpal components. Journal of the Amercan Statstcal Assocaton, 76, Jacson, J.E. and G.S. Mudholar (1979). Control procedures for resduals assocated wth prncpal component analyss. echnometrcs, 1, (3): Joseph, B. and C. Broslow (1978). Inferental control of processes. AIChE J., 4, Kresta, J.V.,.E. Marln and J.F. MacGregor (1994). Development of nferental process models usng PLS. Computers and Chemcal Engneerng, 18, (7): Par, S. and C. Han (000). A nonlnear soft sensor based on multvarate smoothng procedure for qualty estmaton n dstllaton columns. Computers and Chemcal Engneerng, 4, Pearson, R.K. (00). Outlers n process modelng and dentfcaton. IEEE ransactons on Control Systems echnology, 10, (1): Qn, S.J. and.j. McAvoy (199). Nonlnear PLS modelng usng neural networs. Computers and Chemcal Engneerng, 16, (4): Radharshnan, V.R. and A.R. Mohamed (000). Neural networs for the dentfcaton and control of blast furnace hot metal qualty. Journal of process control, 10, (6): 509. Ratclff, R. (1993). Methods for dealng wth reacton tme outlers. Psychologcal Bulletn, 114,: Wse, B.M. (1991). Adaptng multvarate analyss for montorng and modelng dynamc systems, PhD hess, Unversty of Washngton. Zhao, Y.-H. (003). A soft sensor based on nonlnear prncpal component analyss. 003 Internatonal Conference on Machne Learnng and Cybernetcs.