Trends in Experiment Design

Transcription

1 Trends in Experiment Design Univ. Prof. Dr. Stefan Jakubek Technische Universität Wien Stefan Jakubek 1

2 Introduction Design of experiments (DOE) is the design of any information-gathering exercises where variation is present, whether under the full control of the experimenter or not. However, in statistics, these terms are usually used for controlled experiments. Planned experimentation is often used in evaluating physical objects, chemical formulations, structures, components, and materials. In 1747, while serving as surgeon on HMS Salisbury, James Lind carried out a controlled experiment to develop a cure for scurvy. As with other branches of statistics, experimental design is pursued using both frequentist and Bayesian approaches: In evaluating statistical procedures like experimental designs, frequentist statistics studies the sampling distribution while Bayesian statistics updates a probability distribution on the parameter space evon up 2 date 2014 Stefan Jakubek 2

3 Early Appearance of Experiment Design: The Charting of the South Terra Australis (meaning "South Land") is one of the names given to a hypothetical continent which appeared on European maps between the 15th and 18th centuries evon up 2 date 2014 Stefan Jakubek 3

4 Early Appearance of Experiment Design: The Charting of the South Phase one: Exploration of the driveable area Hollandia Nova, 1659 map prepared by Joan Blaeu evon up 2 date 2014 Stefan Jakubek 4

5 Early Appearance of Experiment Design Phase Two: Utility-Driven Inward DoE evon up 2 date 2014 Stefan Jakubek 5

6 Traditional Approaches to DoE The Bayesian Utility Approach In evaluating statistical procedures like experimental designs, frequentist statistics studies the sampling distribution while Bayesian statistics updates a probability distribution on the parameter space. where is the marginal probability density in observation space The expected utility of an experiment with design can then be defined evon up 2 date 2014 Stefan Jakubek 6

7 Traditional Approaches to DoE Example: Maximin based DoE Maximin designs assign the distance to the applied points as the informative value of each design point and the design maximises the minimum distance between each two design points evon up 2 date 2014 Stefan Jakubek 7

8 Exploration vs. Exploitation in the Real World Competing Models result from Evolving Model Structures evon up 2 date 2014 Stefan Jakubek 8

9 Model Based DoE Idea: To optimise inputs/experiments with respect to a certain objective Widespread: The Fisher Information: Optimize w.r.t. a certain measure, such as And adhere to various contraints, such as evon up 2 date 2014 Stefan Jakubek 9

10 Dynamic Model-Based Experiment Design The Foundations of Model-Based Experiment Design for Dynamic Systems have been around for a long time: evon up 2 date 2014 Stefan Jakubek 10

11 Dynamic Model Based Experiment Design A constrained optimisation problem: evon up 2 date 2014 Stefan Jakubek 11

12 An Example Excitation of Engine Exhaust Temperature The Effect of Model Based DoE on the Model Quality: Christoph Hametner, Markus Stadlbauer, Maxime Deregnaucourt, Stefan Jakubek, Thomas Winsel Optimal experiment design based on local model networks and multilayer perceptron networks Engineering Applications of Artificial Intelligence, Available online 19 June 2012, ISSN , evon up 2 date 2014 Stefan Jakubek 12

13 Online DoE & Online Training evon up 2 date 2014 Stefan Jakubek 13

14 Online DoE & Online Training Dynamic Exhaust Temperature revisited evon up 2 date 2014 Stefan Jakubek 14

15 Online DoE & Online Training Dynamic Exhaust Temperature revisited evon up 2 date 2014 Stefan Jakubek 15

16 Online DoE & Online Training Dynamic Exhaust Temperature revisited evon up 2 date 2014 Stefan Jakubek 16

17 Online DoE & Online Training Benefits (a) Control of Limit Violations (b) Improved Model Quality Christoph Hametner, Markus Stadlbauer, Maxime Deregnaucourt, Stefan Jakubek "Incremental optimal process excitation for online system identification based on evolving local model networks" Mathematical and Computer Modelling of Dynamical Systems, Taylor & Francis DOI: / evon up 2 date 2014 Stefan Jakubek 17

18 Model Accuracy on demand Custom Output Range Exploration (CORE) DoE evon up 2 date 2014 Stefan Jakubek 18

19 CORE DoE Accuracy on demand evon up 2 date 2014 Stefan Jakubek 19

20 CORE DoE Accuracy on demand Benefits Tremendous Increase of Accuracy in the COR! evon up 2 date 2014 Stefan Jakubek 20

21 Outlook into the near future: Pareto-Optimal DoE y 2 (A) Estimation of a model to calculate objective improving input directions (B) Classification of drivable, objective improving directions (C) Algorithm stops when C-optimality is reached y evon up 2 date 2014 Stefan Jakubek 21

22 Danke für Ihre Aufmerksamkeit! Technische Universität Wien Stefan Jakubek 22