Ian David Lockhart Bogle and Michael Fairweather (Editors), Proceedings of the 22nd European Symposium on Computer Aided Process Engineering, 17-20 June 2012, London. 2012 Elsevier B.V. All rights reserved Investigation in to the Application of PLS in MPC Schemes Oliver K. Onel, a Barry Lennox, b a Oliver K. Onel, The University of Manchester, School of Elecrical Engineering, Manchester M13 9PL, UK b Barry Lennox, The University of Manchester, School of Elecrical Engineering, Manchester M13 9PL, UK Abstract Since its introduction in 1975, Partial Least Squares (PLS) has become a standard tool for developing regression models when data sets contain highly correlated variables. In this paper, the implementation of PLS models within a Model Predictive Control (MPC) scheme is investigated. A significant problem when developing an MPC application is generating suitable data to identify an accurate model of the process. To address issues that can result when the data available for identifying a model is not sufficiently exciting, several researchers have recently suggested using PLS, rather than more traditional identification algorithms when developing predictive models. This paper shows that caution should be exercised when using models identified with PLS in an MPC algorithm. It is shown that if the data available for modelling is not highly correlated then traditional techniques can produce models that provide improved predictive capabilities. However, when limited data is available then there are benefits in using standard PLS. Furthermore, the paper proposes several methods that can be applied to generate unbiased models with varying structures. Keywords: partial least squares, model predictive control, system identification, excitation, unbiased. 1. Introduction In industrial processes where a high number of variables and disturbances are present, the data can be collinear, leading to ill conditioned data matrices. Latent variable methods (LVM) are used to overcome the effects of collinearity [Flores-Cerillo et al, 2005] and recently PLS has been proposed to be included into MPC applications for multi-step ahead predictions [Lauri, 2010]. Traditional methods such as output error methods provide good multistep ahead predictions [Ljung, 1999] as long as the process data is sufficiently exciting. However, if the structure of the disturbance changes with time then traditional methods can fail to provide suitable models. This is especially true with limited amounts of data, as may be required if an adaptive scheme is to be used. Least squares methods are equipped to deal with these scenarios. In this work we investigate exactly when PLS is needed in the context of updating a model for use in MPC. Two simulated case studies are presented in this work; the first is a single input, single output (SISO) system, proposed by Zhu (Zhu, 2001) and the second is the benchmark multiple input, multiple output (MIMO) distillation column, proposed by Wood and Berry (Wood et al, 1973).
2 O. K. Onel et al In both simulations Random Gaussian Signals (RGS) and Generalised Binary Noise (GBN) are employed with different lengths and band pass levels. GBN is by nature highly exciting. For RGS, excitation depends on the band pass level. The suitability of biased and unbiased PLS to provide an accurate model is analysed, together with the order of excitation, the condition number of the data matrix and the power spectrum of the input signal. Alongside this, identification with traditional methods is conducted and the multistep ahead prediction results are compared to the LVM. Additionally, the structure of the disturbance is changed to Moving Average (MA) and Integrated Moving Average (IMA) in the second case study in order to understand the effects this has on the models. 1.1. Numerical Methods The modelling methods used in this paper are PLS, recursive least squares (RLS), unbiased recursive least squares (urls) (Åström, 2008), Output Error method (OE) with autoregressive exogenous input (ARX) structures and Prediction Error Method (PEM) and the ARMAX method for Box Jenkins (BJ) structured models (Ljung, 1999). The PEM and ARMAX techniques are used to generate empirical models with the following general Box Jenkins structure; B( z ) C( z ) y( = u( + ξ( A( z ) D( z ) Where y( t ) is the output, u( t ) is the input; and ξ( is random white noise. The identification algorithm used for the ARMAX models automatically chooses between Gauss Newton and the Levenberg-Marquardt algorithms for minimisation. PEM uses a slightly modified cost function but the same minimization methods. The OE and the least squares techniques are used to generate ARX models, the structure is given below;. A( z ) y( = B( z ) u( + ξ( The ARX structure is not able to fully characterise the noise, which causes an increase in the prediction error of the model. This is shown in the second case study. 1.1.1. Unbiased PLS Unbiased PLS models can be identified using the coefficients of other prediction error, output error and urls methods. The benefit of unbiased models is that the disturbances are not correlated with the equation error, leading to more consistent estimations. Using the coefficients of the traditional methods in the PLS gives unbiased models. This combines the advantages of the PLS algorithm with the noise structure modelling benefits of the other methods. In this work the coefficients from the urls method were used to construct upls models. This is achieved by instead of using the output data ^ matrix Y in the PLS algorithm the predicted Y is used, where Y = X * θ. urls 1.2. Selection of Latent Variables & Order Selection The number of latent variables for the least squares methods were selected using 2-fold cross validation (Burman, 1989). Using this technique the number of latent variables that resulted in the minimum multi-step ahead prediction error was specified. The order of the models were also confirmed using the same validation data.
Investigation in to the Application of PLS in MPC Schemes 3 2. Case Studies 2.1. Narrow Band Coloured Noise - SISO System This example is taken from Zhu (Zhu, 2001). The transfer function is given as below; 2 ( q 0.5q 1 y = + G( q) u( + υ ( = u( + 2 1 1.5q + 0.7q 1 0.9q e( This case study is used to demonstrate how modelling methods are affected by the quality of the data. Initially, a 500 sample long band limited RGS that provides low excitation is used as the input signal for identification. Following this, the excitation is increased and the results are compared. The pass band for the RGS is expressed as fractions of the Nyquist frequency and is between zero and an upper limit, in the first example 0.07. The GBN is a PRBS like signal and is expressed in terms of average switching time in samples. The performance of the identified transfer function models is assessed with the overall mean Coefficient of Determination (CoD) plots and the mean squared error (MSE), from one to thirty step ahead predictions. Table 1 shows the MSE values when each model is used to estimate the step response of the process. Each run with the given sample length is repeated 100 times and the mean of the squared error is taken. Table 1; Mean Squared Error values of various signal types and lengths Signal Type RGS(band) GBN(time) Samples Mean Squared Error at 30 steps ahead used to identify model PLS upls RLS urls BJ RGS (0.01) 500 157.4 43.3 112.7 511.3 1010 RGS(0.01) 2500 174.6 142.7 88.8 214.2 5059 RGS (0.075) 500 31.7 18.7 154.03 92.7 559.6 RGS (0.3) 500 57.9 9.5 55.1 9.3 1.7 GBN(8) 500 43.3 9.2 42.3 7.1 1.8 GBN(8) 2500 31.9 2.3 31.1 1.4 0.3 GBN(64) 500 31.2 10.2 31.1 10.7 3.9 GBN(64)* 500 130.4 56.1 125.4 58.5 24.5 GBN(64) 2500 30.5 2.9 30.1 2.9 0.9 GBN(64)* 2500 108.5 13.5 101.9 15.0 5.4 The signals shown with * indicator are where the level of noise was increased to higher levels. The results confirm that the more exciting the input signal is, the better the obtained models are. The low excitation RGS signal gives large errors for PEM, but small errors for the least squares techniques. When the excitation of the RGS is increased the error becomes smaller. However in the case of shorter less exciting signals, the case for using PLS and unbiased PLS is justified, as the Box Jenkins type model structures tend to result in poor quality models as a consequence of the limited data. The increase of noise also increases the overall magnitude of the error across all models.
4 O. K. Onel et al 2.2. Wood and Berry Distillation Column MIMO System This system (Wood et al, 1973) is non-linear, MIMO and has been used as a benchmark process in various studies (Lauri et al, 2010). In this example the aim is to build BJ and ARX models of the process to predict multiple steps ahead as applied in MPC. The identification signals are RGS and GBN signals. The signal to noise ratio of the disturbance signal with regards to the input is 10dB. 2.2.1. Original Disturbances Figure 1 and Figure 2 show the CoD plots of both of the outputs, y1 and y2, identified with 500 sample long RGS with a pass-band of 0.01. The prediction error methods in this case do not perform well, and only the least squares methods produce reliable models. Y1 is modelled best by using RLS and Y2 with upls. When the excitation of the signal is increased and the type of signal is changed to the more exciting GBN the results shown in Table 2 are obtained for Y2. Figure 1; CoD Plot of Y1 Figure 2; CoD Plot of Y2 Table 2; Mean Squared Error values for various signal types and lengths for Y2 Signal Type Overall Mean Squared Prediction Error Y2 RGS(band) Samples GBN(time) PLS upls RLS urls PEM RGS (0.01) 500 372 1.0e+5 3.4e+3 1.0e+5 8.2e+15 RGS(0.01) 2500 137 3.1e+4 432.2 3.1e+4 2.6e+10 GBN(150/170) 500 437 1.7e+3 727 1.7e+3 2.8e+3 GBN(64) 500 1.1e+3 2.4e+3 572 2.5e+3 4.0e+3 GBN(64) 2500 458 408 220 418 151 GBN(64) 10000 449 168 145 170 28 In this structured noise scenario, the prediction error method performs well when the signal is well conditioned with GBN(64) as the identification signal and when there is sufficient data present. When the data is short or not highly exciting, then least squares methods perform better, as seen in the case for GBN(64) 500 sample long, highly exciting but short data set. When the signal has low-medium excitation, it is found that relying on a general mean squared error analysis may not be ideal, as from run to run the best performing method changes. This shows that it is best to analyze the results of several modelling methods at every update instance when updating the model in the MPC algorithm.
Investigation in to the Application of PLS in MPC Schemes 5 Figure 3;1 to 30 Ahead Prediction over a range of signal lengths (200 10000) to build models 2.2.2. Modified disturbances To analyse the effects of the noise structure on the model s capabilities, the previous exercise was repeated with both MA and IMA noise sequences. In this situation the BJ structures are able to model the system well, as illustrated in Figure 3, which shows the 30-step ahead prediction of the ARMAX model for Y1. However, because the structure of the model is now incorrect, the PLS models do not capture the dynamics of the process as well. In Figure 4 the disturbance is MA and the identification signal is GBN of switching period 150. Only the least squares methods give consistent results out of 100 repeated runs and the upls method performs better than traditional PLS. Figure 4; 30 Step ahead prediction over a range of identification signal lengths. 3. Conclusion In the studies conducted, PLS was found to be the most accurate modelling technique when the data was ill-conditioned or highly excited but of limited size. As the amount of data that is available increased, more traditional modelling techniques, such as ARMAX and PEM tend to provide improved models. However, in general applications it is expected that the most suitable modelling technique will be problem dependent, and for a given application it is sensible to compare the capabilities of PLS, unbiased PLS and a more traditional approach, such as PEM. References K. J.Åström, and B. Wittenmark. 2008. Adaptive Control. Dover Publications P. Burman, 1989, A comparative study of ordinary cross-validation, r-fold cross validation and the repeated learning-testing methods, Biometrica, 76, 3, 503-14 J.Flores-Cerillo, J. MacGregor, 2005, Latent variable MPC for trajectory tracking in batch processes, Journal of Process Control, 15, 651-663 D. Laurí, M. Martínez, J.V. Salcedo, and J. Sanchis. 2010. PLS-based model predictive control relevant identification: PLS-PH algorithm,chemometrics and Intelligent Laboratory Systems 100 (2) (February 15): 118-126. D. Laurí, J.A. Rossiter, J. Sanchis, and M. Martínez. 2010. Data-driven latent-variable modelbased predictive control for continuous processes, Journal of Process Control 20 (10) (December): 1207-1219. L. Ljung, 1999, System Identification; Theory for the user R. K. Wood, M. W. Berry, 1973. Terminal composition control of a binary distillation column. Chemical Engineering Science 28 (9), 1707-1717 Y. Zhu, 2001. Multivariable system identification for process control. Elsevier, October 1.