ASI-CG Client Conference Proceedings rd ASI-CG 3 Annual Client Conference Celebrating 27+ Years of Clients' Successes DETROIT Michigan NOV. 4, 2010 ASI Consulting Group, LLC 30200 Telegraph Road, Ste. 100 Bingahm Farms, MI 48025 (248) 530-1395 www.asiusa.com
Bill Leisenring Control-Tec, LLC Tecumseh, MI AUTOMATED ROBUST ASSESSMENT ASI-CG 3 rd Annual Client Conference Tim Gernant Ford Motor Company Ann Arbor, MI ABSTRACT Powertrain Systems development is entering a period of unprecedented challenge driven by the convergence of many factors: increasing government regulations for both tailpipe emissions and fuel economy, increased competition, reduced workforce, and tighter program budgets. This has resulted in timing compression and resource reduction that stress a typical Design-Build-Test development practice. The application of telematics and information technology to engineering development can provide the efficiency gains required for engineers to deliver a robust powertrain system. By automating the evaluation of a system s robustness, engineers can focus their time on problem areas during their normal development process and launch with quality. This paper will detail how this methodology was jointly applied by Control-Tec and Ford Motor Company to identify and improve system performance before production.
INTRODUCTION ASI-CG 3 rd Annual Client Conference Reduced staffing and testing resources have made powertrain development more difficult in recent years. The requirements have also been getting more difficult due to increased competition, increased fuel economy regulation and tighter emissions standards. The Design-Build-Test iterative development model is no longer sustainable. The development process must change in order to effectively deliver robust solutions to demanding consumers. The application of affordable innovations in telematics and information technology to enhance the development process is one potential method to enable engineers to launch quality products. INDUSTRY OVERVIEW The Transportation Industry is undergoing unprecedented change due to significant increases in regulation for both fuel economy and tailpipe emissions, Figure 1. Corporate Average Fuel Economy (CAFE) is being increased until it reaches 60 miles-per-gallon in the year 2025. During this period, LEV III tailpipe emissions regulations are reducing Non-Methane Hydrocarbons (NMHC) and Particulate Matter (PM) by 73% from current levels. Previous emissions reductions were done when CAFE was constant. Designing a robust powertrain system is now more difficult as solutions for improved fuel economy and emissions are, minimally, independent of each other and, in some cases, can be counter-productive. All of these solutions are required to be diagnosed via On-Board Diagnostics (OBD) as mandated by the California Air Resources Board (CARB) and the Environmental Protection Agency (EPA). This requires that the Check Engine or Malfunction Indicator Light (MIL) be illuminated for any hardware failure before exceeding 1.5 times the emissions standard. Failure to do so may result in a recall for the affected vehicle line. With the low emissions levels planned for in LEV III, the OBD margin will be very thin and a challenge to obtain with the new technology being driven by the fuel economy and emissions regulations. Complicating the task is the fact that a company cannot afford to turn on the MIL when a malfunction is not present as this creates unnecessary warranty expense. Dissatisfied customers who typically will come back to the dealership multiple times as no hardware issue is found often result. U.S. OEMs and Parts Suppliers are spending more than $13 Billion on warranty annually, Figure 2. Despite numerous Quality programs by these companies, warranty costs remain the same on a volume-adjusted basis, as reactive approaches cannot keep up with the increasing complexity of vehicles, Figure 3.
Robust Engineering ASI-CG 3rd Annual Client Conference Figure 1: Light Duty Emissions and Fuel Economy Standards Figure 2: U.S. Based Automotive Companies Warranty Expenses
ASI-CG 3 rd Annual Client Conference Figure 3: U.S. Based Automotive Companies Warranty Expenses Analyzing a parallel industry sheds some light on what may happen to the Light Duty industry over the next fifteen years as the new regulations are implemented. The Heavy Duty Transportation has faced similar change as new NMHC regulations were implemented in 2004, Nitrous Oxide (NOx) emissions regulations were made more stringent in 2007, and OBD phase-in started in 2010 and continues to 100% compliance by 2016. Figure 4 shows the warranty performance of a Diesel Engine manufacturer during this period. Each one of these regulation changes has resulted in increased warranty. Per J.D. Power and Associates, the problem rate for Heavy Duty vehicles in 2010 is twice as high as that in 2004 and that over 50% of owners have experienced some type of problem. True development process innovation is required for the Light Duty industry to avoid the increased warranty demonstrated by their Heavy Duty counterparts. Figure 4: Warranty for a Diesel Engine Manufacturer
NEW VALIDATION PROCESS ASI-CG 3 rd Annual Client Conference Light Duty companies typically test vehicles during development and consider no identified failures a pass but typically know little about the performance of the powertrain system. Engineering feedback is often limited to a contact with a fleet supervisor when the system fails, Figure 5. However, no information is known other than the fact that a failure occurred. The engineer is now forced to try to reproduce the issue in order to collect the data necessary to solve the problem and has no idea if this issue was a one off or is indicative of a potentially major problem. Figure 5: Reactive Validation Approach Instead, the validation process can take a proactive approach by measuring the parameters that characterize subsystem performance every time the system is exercised, Figure 6. The robustness of the subsystem can now be understood and engineers can more readily solve the issue. Sensitivity to the noise space can be analyzed as well by studying the performance of the subsystem under the different operating conditions recorded along with the subsystem metrics. Figure 6: Proactive Validation Approach
ASI-CG 3 rd Annual Client Conference The powertrain system conducts several hundred transactions during a typical drive cycle that can be measured to assess the robustness of the system. For example, one engine start produces a handful of metrics that can characterize the robustness of that start such as: the time taken for the engine to start after the ignition was commanded to start, the peak engine speed observed during the start, the time it took to reach the desired engine speed, and more, Figure 7. Also, these measures are usually a function of the operating conditions during the start, which can also be measured and recorded: the engine coolant temperature, the ambient air temperature, barometric pressure, etc. All major powertrain subsystems can be measured to assess engine start performance, idle quality, transmission shift quality, diagnostic robustness, and more. Advances in vehicle data recorders (VDR), wireless data transfer and data storage now make it possible to measure every powertrain system transaction in order to assess the robustness of the system. Figure 7: Engine Start Subsystem CASE STUDIES Control-Tec and Ford jointly implemented an automated robustness system utilizing a VDR that automatically downloaded vehicle data from an existing test fleet of vehicles to a server where the data was then processed and issues were identified. This enabled Ford Engineering to focus on these problem areas and correct the issues before launch. Three examples are presented here. In these examples, an automated statistical assessment utilizing the statistical process control metric C pk was used to identify potential issues. C pk is defined as:! "# where is the mean of the data andis the standard deviation. This result estimates process capability for a one-sided diagnostic and assumes an approximately normal distribution. Existing processes are recommended to have C pk values greater than 1.25 to be robust and new processes greater than 1.50. C pk in relation to process fallout is
ASI-CG 3 rd Annual Client Conference indicated in Table 1. For diagnostic systems, the Failure Rate needs to be analyzed based on the number of times the diagnostic is exercised. The useful life of vehicles mandated by LEV III is 150,000 miles. Assuming an average trip distance of 50 miles, a vehicle will have 3,000 drive cycles during its useful life. OBD diagnostics by CARB are mandated to make a decision on at least 33% of the drive cycles. Therefore, each diagnostic in a compliant vehicle will produce a minimum of 1,000 test results during its useful life. Warranty can now be forecasted based on C pk with assumptions for vehicle volume and repair cost. The warranty in Table 1 assumes a vehicle volume of 50,000 units and a repair cost of $200. This warranty compounds for non-robust diagnostics that are not fixed properly on the first visit resulting in repeat repairs and dissatisfied customers. Table 1: Cpk in Relation to Failure Rate and Warranty! "#!"#$%&'#()'$*+,#$-())./(*+ "0/*1#(203(42567778 90##0:3;4<8 Example One The first example is for an OBD Monitor on a new feature mandated by CARB. This new monitor was calibrated on a few development vehicles and was considered robust based on the data collected on that population of vehicles. However, data taken on a larger volume of vehicles indicated otherwise, Table 2. C pk data shows significant variety in robustness. Some vehicles were near failure while others were very robust. The development data lined up with Vehicles 20, 23, 26 and 28, as indicated by the bold-italic data in Table 2. Table 2: Monitor Performance Data with Initial Calibration
ASI-CG 3 rd Annual Client Conference The monitor threshold was then recalibrated by demonstrating the feature with a more failed part. This is a common method used in OBD development as thresholds for worse parts are typically larger and are more robust by moving the threshold away from the test results. During the later stages of development this is more readily done than recalibrating the feature to reduce its variation and/or shift the mean. However, the monitor still produced significant variation and did not meet Ford s internal robustness metrics on all test vehicles in the fleet. As a result, the data was then analyzed to determine the source of the test result variation. The test result is comprised of a series of individual tests during a drive cycle. Analyzing the data as a function of one of the test conditions, showed that 92% of the failed individual tests occurred when the test condition was 0.42 or less, Figure 8. The monitor was then re-calibrated with an increase to the minimum test condition to no longer operate the diagnostic in this region and a significant increase in robustness was achieved, Figure 9. Figure 8: Individual Failed Test Results as a Function of Test Condition
Figure 9: Vehicle 27 Diagnostic Performance ASI-CG 3 rd Annual Client Conference Example Two The next example is for an OBD diagnostic that is not new but varies significantly from one program to another. This requires detailed calibration for each program variant and can be quite extensive in order to test the wide range of operating conditions. This diagnostic was monitored in the test fleet on all build combinations and one variation, Variant 2, was identified as having several near misses where the diagnostic was very close to failing and the C pk was not ideal, Figure 10.
ASI-CG 3 rd Annual Client Conference Figure 10: Diagnostic Test Results for Three Program Variants The results from Variant 2 were then analyzed further as a function of the enable conditions. It was found that the robustness deteriorated significantly when the test was run at a value greater than 0.5 for a particular test condition, Figure 11. The C pk for the diagnostic was significantly different when the test was run at a test condition less than 0.5 compared to greater than 0.5. This corresponds to a particular portion of the calibration for this diagnostic. The monitor was then recalibrated in this region and the results are now robust and consistent with the results of Variants 1 and 3 shown earlier in Figure 10.
ASI-CG 3 rd Annual Client Conference Figure 11: Variant Two Test Results as a Function of a Test Condition Example Three The final example is for another OBD diagnostic on an all-new program that featured significant new content from a powertrain systems perspective. In this example, the C pk for the diagnostic was adequate when compared to the diagnostic threshold but not when assessed against Ford s internal targets, Figure 12.
ASI-CG 3 rd Annual Client Conference Figure 12. Diagnostic Test Results Compared to Threshold and Internal Target Lessons learned from previous programs for this feature indicate that field warranty will be minimal if the monitor does not exceed the internal target. As a result, the calibration was modified before production with improved robustness, Figure 13. Figure 13: Diagnostic Results After Calibration Change This example demonstrates another point. Both calibrations were robust, C pk values > 1.25, when compared to the diagnostic threshold. With this new approach it could have been determined to release the initial calibration into production despite it failing the internal not to exceed target and focus resources on other program issues. CONCLUSION Advances in automated vehicle data measurement and analysis can be used to streamline the powertrain systems development process and lead management to allocating resources via data driven program assessments. Given the nature of today s stressed development environment, the examples presented in this paper may have gone to production and resulted in significant warranty costs in the absence of this automated robust assessment system. The development to date of this system has been focused on a quality measurement system with automated data processing and issue finding with C pk methodology. The next phase of the system development will be to apply more Design For Six Sigma principles to the data to perform automated sensitivity analysis with Signal-to-Noise ratios as a function of test conditions as well as exploring the Mahalanobis-Taguchi System (MTS) to predict warranty performance.