DATA QUALITY ASSURANCE AND PERFORMANCE MEASUREMENT OF DATA MINING FOR PREVENTIVE MAINTENANCE OF POWER GRID

Transcription

1 1 DATA QUALITY ASSURANCE AND PERFORMANCE MEASUREMENT OF DATA MINING FOR PREVENTIVE MAINTENANCE OF POWER GRID Leon Wu 1,2, Gail Kaiser 1, Cynthia Rudin 3, Roger Anderson 2 1. Department of Computer Science, Columbia University 2. Center for Computational Learning Systems, Columbia University 3. MIT Sloan School of Management, MIT KDD4Service 11 San Diego, California, USA August 21, 2011

2 2 Definitions Power grid is an electricity distribution and transmission system. Smart grid is an automated electric power system that monitors and controls grid activities, ensuring the two-way flow of electricity and information between power plants and consumers and all points in between. * * Federal Smart Grid Task Force, Smart grid basics, 2010

3 3 New York City s Power Grid Consolidated Edison (Con Ed) is a regulated utility providing electric, gas, and steam service in NYC, Westchester, NY, and northern New Jersey and northeastern Pennsylvania. * Source: Consolidated Edison Company of New York

4 4 Service and Maintenance of Power Grid A sustainable energy future depends on an efficient, intelligent, and reliable power grid. The smart grid of the future will have to operate efficiently to satisfy the increasing capacity demand, and should use the current legacy grid as much as possible to keep costs lower. Service and maintenance of the old electrical components in the legacy grid is critical to improve power grid system reliability.

5 Past Lesson: Cascading Electrical Component Failure In 2004, the U.S.-Canada Power System Outage Task Force released their final report on the 2003 U.S. Northeast blackout, placing the main cause of the blackout on some strained high-voltage power lines in Ohio that later went out of service, which led to the cascading effect that ultimately forced the shutdown of more than 100 power plants. 5

6 Power Grid Becomes Increasingly Unreliable as It Gets Smarter According to two data sets, one from the U.S. Department of Energy and the other one from the North American Electric Reliability Corp., the number of power outages greater than 100 Megawatts or affecting more than 50,000 customers in the U.S. almost doubled every five years in the past fifteen years, resulting in about $49 billion outage costs per year. 6 *Amin, S. Massoud. U.S. electrical grid gets less reliable. IEEE Spectrum, Jan 2011.

7 7 U.S. power outages in the past fifteen years * Amin, S. Massoud. U.S. electrical grid gets less reliable. IEEE Spectrum, pp. 80, January 2011.

8 8 Power Outage Problem In New York City Power outages have been a long-term problem in New York City. Factors lead to power outages in NYC Legacy systems with legacy components, especially, feeders, cables, joints, transformers. High utilization rate: 24x7, big apple is the city that never sleeps. Peak load and stress load: 18.9 million metro population plus businesses.

9 9 Preventive Maintenance Preventive maintenance is a schedule of planned maintenance actions aimed at the prevention of breakdowns and failures to improve long-term system reliability through replacing worn components before they actually fail. After U.S. northeast blackout in August 2003, Columbia University and Con Ed started to conduct collaborative research on how to improve the power grid reliability using intelligent preventive maintenance empowered by machine learning and data mining.

10 10 Feeder Ranking Systems In New York City, underground primary feeders are one of the most failure-prone types of electrical components. MartaRank and ODDS are two online machine learning and data mining-based feeder ranking systems that rank the feeders according to their susceptibility to the impending failure. The feeders are then proactively inspected and/or repaired in order of their estimated susceptibility to failure.

11 11 Problem Statement How can we evaluate the online machine learning and data mining model running in the live environment, the quality of the data input and output, and the consequential benefits, i.e., physical system improvements, after the actions recommended by the machine learning and data mining have been taken?

12 12 Our Approach NOVA (Neutral Online Visualization-aided Autonomic evaluation framework) is a comprehensive multi-stage online evaluation framework that is able to provide such an evaluation objectively, effectively and efficiently. NOVA has three stages provided through a unified user interface: evaluation of input data quality, evaluation of machine learning and data mining results, and evaluation of the reliability improvement of the power grid.

13 13 NOVA System Design and Workflow Data Collection From Power Grid User Interface Visualization Alert Data Preparation 1 Evaluation Machine Learning Output Results 2 Evaluation Recommended Actions Taken on Power Grid Data Collection From Power Grid 3 Evaluation

14 14 Step Evaluation Target Techniques 1 Input data Sparkline graph, data checks and constraints 2 Machine learning results ROC curve, AUC time series 3 Physical system improvements Duane plot, MTBF, failure rate, linear regression Unified user interface Dashboard, charts, warning messages, alert s Table 1. Summary of main techniques. Figure 2. Sparkline graph for attributes data. Figure 3. AUC cyclicity graph. Figure 4. Cumulative outages versus time log-log chart. Figure 5. MTBF versus time and linear regression.

15 15 Experimental Results NOVA was able to evaluate the MartaRank and ODDS feeder ranking systems and the reliability improvements of the grid system effectively and efficiently. Both feeder ranking systems have comparable overall performance in ranking feeders susceptibility to the impending failure. Another experimental result we concluded from the evaluation using NOVA is the increasing MTBF (Mean Time Between Failures), i.e., lower failure rate and better system reliability, for most networks using these feeder ranking systems.

16 16 Related Work Langley s seminal paper Machine Learning as an Experimental Science made empirical study an indispensable aspect of machine learning research. * Japkowicz reviewed shortcomings in current machine learning evaluation methods. * P. Langley. Machine learning as an experimental science. Machine Learning, 3(1): 5 8, N. Japkowicz. Why question machine learning evaluation methods (an illustrative review of the shortcomings of current methods). In 2006 AAAI Workshop Evaluation Method for Machine Learning. AAAI, 2006.

17 17 Conclusion I presented the Neutral Online Visualization-aided Autonomic evaluation framework (NOVA) for evaluating machine learning algorithms for preventive maintenance on the electrical grid. A prototype version of NOVA has been deployed for the power grid in New York City, and it is able to evaluate machine learning and data mining systems objectively, effectively and efficiently.

18 18 Acknowledgements U.S. Department of Energy awarded Con Ed $45 million of funding for its Secure Interoperable Open Smart Grid Demo Project, of which Columbia is a proposal co-author and sub-recipient, one of 16 projects selected by DOE nationwide. (2009) Columbia s Creation of a Columbia Machine Learning System to Optimize the Recharging of Electric Delivery Vehicles in Large Urban Cities was chosen as the only Winning University Program in General Electric s Ecomagination Innovation Challenge of (2010)

19 19 Thank You Q & A Contact Information Leon Wu leon@ccls.columbia.edu