Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 8
|
|
- Jared Barker
- 5 years ago
- Views:
Transcription
1 Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 8 Slides adapted from Jordan Boyd-Graber, Justin Johnson, Andrej Karpathy, Chris Ketelsen, Fei-Fei Li, Mike Mozer, Michael Nielson Machine Learning: Chenhao Tan Boulder 1 of 53
2 HW1 grades Most people did well! We are extra lenient this time Submit only your code in a zip file, with no folder structure Machine Learning: Chenhao Tan Boulder 2 of 53
3 Final projects Team formation Machine Learning: Chenhao Tan Boulder 3 of 53
4 Quiz 1 Which of the following statements is true? (Suppose that training data is large.) A. In training, K-nearest neighbors takes shorter time than neural networks. B. In training, K-nearest neighbors takes longer time than neural networks. C. In testing, K-nearest neighbors takes shorter time than neural networks. D. In testing, K-nearest neighbors takes longer time than neural networks. Machine Learning: Chenhao Tan Boulder 4 of 53
5 Quiz 2 How many parameters are there in the following feed-forward neural networks (assuming no biases)? x 1 x 2... x 100 h 1... h 50 A. 100 * * * 5 h 1... h 20 B. 100 * * * o 1... o 5 Machine Learning: Chenhao Tan Boulder 5 of 53
6 Quiz 3 How many parameters are there in the following convolutional neural networks? (assuming no biases, convolution with 4 filters, max pooling, ReLu, and finally a fully-connected layer) input image (10*10) 4@6*6 4@3*3 5*1 4 filters 5*5, stride 1 Max pooling 2*2, stride 2 Machine Learning: Chenhao Tan Boulder 6 of 53
7 Quiz 3 How many ReLU operations are performed on the forward pass? (assuming no biases, convolution with 4 filters, max pooling, ReLu, and finally a fully-connected layer) input image (10*10) 4@6*6 4@3*3 5*1 4 filters 5*5, stride 1 Max pooling 2*2, stride 2 Machine Learning: Chenhao Tan Boulder 6 of 53
8 Overview History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 7 of 53
9 History lesson Outline History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 8 of 53
10 History lesson History lesson 1962: Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms First neuron-based learning algorithm Could learning anything that you could program Machine Learning: Chenhao Tan Boulder 9 of 53
11 History lesson History lesson 1962: Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms First neuron-based learning algorithm Could learning anything that you could program 1969: Minsky & Papert, Perceptron: An Introduction to Computational Geometry First real complexity analysis Showed, in principle, many things that perceptrons can t learn to do Shut down any interest in neural networks Machine Learning: Chenhao Tan Boulder 9 of 53
12 History lesson History lesson 1986: Rumelhart, Hinton & Williams, Back Propagation Overcame many difficulties raised by Minsky et al. Neural networks wildly popular again (for a while) Machine Learning: Chenhao Tan Boulder 10 of 53
13 History lesson History lesson Shift to Bayesian Methods Best ideas from neural networks Direct statistical computing Support Vector Machines Nice mathematical properties Kernel tricks A few people still playing with NNs Bengio Hinton LeCun Machine Learning: Chenhao Tan Boulder 11 of 53
14 History lesson History lesson Core group continues to make improvements Various tricks to make NNs practical 2010-present BOOM! Machine Learning: Chenhao Tan Boulder 12 of 53
15 History lesson AlexNet Krizhevsky et al. [2012] Machine Learning: Chenhao Tan Boulder 13 of 53
16 History lesson History lesson Question: Why? What made neural networks amazing again? Massive datasets Computing power Algorithmic improvements Machine Learning: Chenhao Tan Boulder 14 of 53
17 History lesson History lesson Machine learning has a short history, but seems cyclic. What is next? Machine Learning: Chenhao Tan Boulder 15 of 53
18 Deep learning in practice Outline History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 16 of 53
19 Deep learning in practice Improve stochastic gradient descent Outline History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 17 of 53
20 Deep learning in practice Improve stochastic gradient descent Gradient descent Gradient descent w t+1 = w t η f (w t ) Machine Learning: Chenhao Tan Boulder 18 of 53
21 Deep learning in practice Improve stochastic gradient descent AdaGrad Not all features are created equal! Machine Learning: Chenhao Tan Boulder 19 of 53
22 Deep learning in practice Improve stochastic gradient descent AdaGrad Not all features are created equal! Gradient descent w t+1 = w t η f (w t ) Machine Learning: Chenhao Tan Boulder 19 of 53
23 Deep learning in practice Improve stochastic gradient descent AdaGrad Not all features are created equal! Gradient descent w t+1 = w t η f (w t ) Adagrad [Duchi et al., 2011] cache = cache + ( f (w t )) 2 1 w t+1 = w t η cache f (w t) Machine Learning: Chenhao Tan Boulder 19 of 53
24 Deep learning in practice Improve stochastic gradient descent Momentum Gradient descent w t+1 = w t η f (w t ) Machine Learning: Chenhao Tan Boulder 20 of 53
25 Deep learning in practice Improve stochastic gradient descent Momentum Gradient descent w t+1 = w t η f (w t ) Machine Learning: Chenhao Tan Boulder 21 of 53
26 Deep learning in practice Improve stochastic gradient descent Momentum Gradient descent w t+1 = w t η f (w t ) Physical interpretation: Imagine a object is falling, but it does not accumulate any velocity. Machine Learning: Chenhao Tan Boulder 21 of 53
27 Deep learning in practice Improve stochastic gradient descent Momentum Gradient descent w t+1 = w t η f (w t ) Physical interpretation: Imagine a object is falling, but it does not accumulate any velocity. Let us fix that! Machine Learning: Chenhao Tan Boulder 21 of 53
28 Deep learning in practice Improve stochastic gradient descent Momentum Gradient descent w t+1 = w t η f (w t ) Physical interpretation: Imagine a object is falling, but it does not accumulate any velocity. Let us fix that! Momentum v t+1 = βv t f (w t ) w t+1 = w t + ηv t+1 Machine Learning: Chenhao Tan Boulder 21 of 53
29 Deep learning in practice Improve stochastic gradient descent Momentum Image credit: Alec Radford Machine Learning: Chenhao Tan Boulder 22 of 53
30 Deep learning in practice Improve stochastic gradient descent More variations Adam [Kingma and Ba, 2014] RMSProp Machine Learning: Chenhao Tan Boulder 23 of 53
31 Deep learning in practice Improve stochastic gradient descent Dropout layer "randomly set some neurons to zero in the forward pass" [Srivastava et al., 2014] Machine Learning: Chenhao Tan Boulder 24 of 53
32 Deep learning in practice Improve stochastic gradient descent Dropout layer Forces the network to have a redundant representation Machine Learning: Chenhao Tan Boulder 25 of 53
33 Deep learning in practice Improve stochastic gradient descent Dropout layer Another interpretation: Dropout is training a large ensemble of models Machine Learning: Chenhao Tan Boulder 25 of 53
34 Deep learning in practice Unstable gradients Outline History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 26 of 53
35 Deep learning in practice Unstable gradients Unstable gradients x h 1 h 2... h L o Machine Learning: Chenhao Tan Boulder 27 of 53
36 Deep learning in practice Unstable gradients Unstable gradients x h 1 h 2... h L o L b 1 = σ (z 1 ) w 2 σ (z 2 ) w 3... σ (z L ) L a L Machine Learning: Chenhao Tan Boulder 27 of 53
37 Deep learning in practice Unstable gradients Unstable gradients sigmoid derivative Machine Learning: Chenhao Tan Boulder 28 of 53
38 Deep learning in practice Unstable gradients Vanishing gradients If we use Gaussian initialization for weights, w j N (0, 1), w j < 1 w j σ (z j ) < 1 4 L decay to zero exponentially b1 Machine Learning: Chenhao Tan Boulder 29 of 53
39 Deep learning in practice Unstable gradients Vanishing gradients ReLu ReLu derivative Machine Learning: Chenhao Tan Boulder 30 of 53
40 Deep learning in practice Unstable gradients Exploding gradients If w j = 100, w j σ (z j ) k > 1 Machine Learning: Chenhao Tan Boulder 31 of 53
41 Deep learning in practice Data preprocessing Outline History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 32 of 53
42 Deep learning in practice Data preprocessing Another subtle issue of activation function If all inputs x are positive, the gradients on w are either all positive or all negative. Machine Learning: Chenhao Tan Boulder 33 of 53
43 Deep learning in practice Data preprocessing Another subtle issue of activation function If all inputs x are positive, the gradients on w are either all positive or all negative. Zero-center the inputs! Machine Learning: Chenhao Tan Boulder 33 of 53
44 Deep learning in practice Data preprocessing Data preprocessing Original data Machine Learning: Chenhao Tan Boulder 34 of 53
45 Deep learning in practice Data preprocessing Data preprocessing Original data Zero-centered data (X X.mean(axis = 0)) Machine Learning: Chenhao Tan Boulder 34 of 53
46 Deep learning in practice Data preprocessing Data preprocessing Original data Zero-centered data (X X.mean(axis = 0)) Normalized data (X/ = np.std(x, axis = 0)) Machine Learning: Chenhao Tan Boulder 34 of 53
47 Deep learning in practice Data preprocessing Data preprocessing Original data Zero-centered data (X X.mean(axis = 0)) Normalized data (X/ = np.std(x, axis = 0)) PCA, whitening Machine Learning: Chenhao Tan Boulder 34 of 53
48 Deep learning in practice Data preprocessing Batch normalization Why only for the input data? [Ioffe and Szegedy, 2015] Consider a batch of activations at some layer. Make each dimension unit gaussian: â k = ak E[a k ] Var[a k ] Machine Learning: Chenhao Tan Boulder 35 of 53
49 Deep learning in practice Data preprocessing Batch normalization Reduces internal covariant shift Reduces the dependence of gradients o the scale of the parameters or their initial values Allows higher learning rates and use of saturating nonlinearities Reduce the need for dropout (maybe) Machine Learning: Chenhao Tan Boulder 36 of 53
50 Deep learning in practice Data preprocessing Batch normalization During training, use batch mean and batch variance; during testing use empirical mean and variance on training data Machine Learning: Chenhao Tan Boulder 36 of 53
51 Deep learning in practice Data preprocessing Batch normalization Add batch normalization before nonlinear activation or after nonlinear activation? master/batchnorm.md Machine Learning: Chenhao Tan Boulder 37 of 53
52 Deep learning in practice Weight Initialization Outline History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 38 of 53
53 Deep learning in practice Weight Initialization Non-convexity Machine Learning: Chenhao Tan Boulder 39 of 53
54 Deep learning in practice Weight Initialization Weight initialization Old idea: W = 0, what happens? Machine Learning: Chenhao Tan Boulder 40 of 53
55 Deep learning in practice Weight Initialization Weight initialization Old idea: W = 0, what happens? There is no source of asymmetry. (Every neuron looks the same and leads to a slow start.) Machine Learning: Chenhao Tan Boulder 40 of 53
56 Deep learning in practice Weight Initialization Weight initialization Old idea: W = 0, what happens? There is no source of asymmetry. (Every neuron looks the same and leads to a slow start.) δ L = a LL σ (z L ) # Compute δ s on output layer For l = L,..., 1 L = δ l (a l 1 ) T # Compute weight derivatives W l L b l = δl # Compute bias derivatives δ l 1 = ( W l) T δ l σ (z l 1 ) # Back prop δ s to previous layer Machine Learning: Chenhao Tan Boulder 40 of 53
57 Deep learning in practice Weight Initialization Weight initialization First idea: small random numbers, W N (0, 0.01) Machine Learning: Chenhao Tan Boulder 41 of 53
58 Deep learning in practice Weight Initialization Weight initialization Var(z) = Var( i w i x i ) = nvar(w i )Var(x i ) Machine Learning: Chenhao Tan Boulder 42 of 53
59 Deep learning in practice Weight Initialization Weight initialization Xavier initialization [Glorot and Bengio, 2010] W N (0, 2 n in + n out ) Machine Learning: Chenhao Tan Boulder 43 of 53
60 Deep learning in practice Weight Initialization Weight initialization Xavier initialization [Glorot and Bengio, 2010] Does not work for ReLU W N (0, 2 n in + n out ) Machine Learning: Chenhao Tan Boulder 43 of 53
61 Deep learning in practice Weight Initialization Weight initialization He initialization [He et al., 2015] W N (0, 2 n in ) Machine Learning: Chenhao Tan Boulder 44 of 53
62 Deep learning in practice Weight Initialization Weight initialization This is an actively research area and next great idea may come from you! Machine Learning: Chenhao Tan Boulder 45 of 53
63 Deep learning in practice Model Architecture Outline History lesson Deep learning in practice Improve stochastic gradient descent Unstable gradients Data preprocessing Weight Initialization Model Architecture Machine Learning: Chenhao Tan Boulder 46 of 53
64 Deep learning in practice Model Architecture ResNet How to train a neural network with 100 layers? [He et al., 2016] Machine Learning: Chenhao Tan Boulder 47 of 53
65 Deep learning in practice Model Architecture ResNet Why is it hard to train a large number of layers? Machine Learning: Chenhao Tan Boulder 48 of 53
66 Deep learning in practice Model Architecture ResNet Simple solution: Machine Learning: Chenhao Tan Boulder 49 of 53
67 Deep learning in practice Model Architecture ResNet Machine Learning: Chenhao Tan Boulder 50 of 53
68 Deep learning in practice Model Architecture ResNet Machine Learning: Chenhao Tan Boulder 51 of 53
69 Deep learning in practice Model Architecture References (1) John Duchi, Elad Hazan, and Yoram Singer. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. J. Mach. Learn. Res., 12: , URL Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Yee Whye Teh and Mike Titterington, editors, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages , Chia Laguna Resort, Sardinia, Italy, May PMLR. URL Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In The IEEE International Conference on Computer Vision (ICCV), December Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages , Lille, France, Jul PMLR. URL Machine Learning: Chenhao Tan Boulder 52 of 53
70 Deep learning in practice Model Architecture References (2) Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In Proceedings of ICLR, Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15: , URL Machine Learning: Chenhao Tan Boulder 53 of 53
CSE 40171: Artificial Intelligence. Artificial Neural Networks: Neural Network Architectures
CSE 40171: Artificial Intelligence Artificial Neural Networks: Neural Network Architectures 58 Group projects are due 12/12 at 11:59PM. Check the course website for guidance. 59 Course Instructor Feedback
More informationA General Artificial Neural Network Extension for HTK
A General Artificial Neural Network Extension for HTK Chao Zhang & Phil Woodland University of Cambridge 15 April 2015 Overview Design Principles Implementation Details Generic ANN Support ANN Training
More informationRegularized Linear Models in Stacked Generalization
Regularized Linear Models in Stacked Generalization Sam Reid and Greg Grudic Department of Computer Science University of Colorado at Boulder USA June 11, 2009 Reid & Grudic (Univ. of Colo. at Boulder)
More informationPredicting Solutions to the Optimal Power Flow Problem
Thomas Navidi Suvrat Bhooshan Aditya Garg Abstract Predicting Solutions to the Optimal Power Flow Problem This paper discusses an implementation of gradient boosting regression to predict the output of
More informationStatistical Learning Examples
Statistical Learning Examples Genevera I. Allen Statistics 640: Statistical Learning August 26, 2013 (Stat 640) Lecture 1 August 26, 2013 1 / 19 Example: Microarrays arrays High-dimensional: Goals: Measures
More informationVehicle's Velocity Time Series Prediction Using Neural Network
21 Vehicle's Velocity Time Series Prediction Using Neural Network A. Fotouhi, M. Montazeri-Gh and M. Jannatipour Systems simulation and control Laboratory, School of Mechanical Engineering, Iran University
More informationText Generation and Neural Style Transfer
Text Generation and Neural Style Transfer S. Singhal, K. Siddarth, P. Agarwal, A. Garg Mentor: N. Asnani Department of Computer Science and Engineering IIT Kanpur 22 nd November 2017 Introduction Text
More informationIntelligent Subset Selection of Power Generators for Economic Dispatch
Intelligent Subset Selection of Power Generators for Economic Dispatch Biswarup Bhattacharya University of Southern California Los Angeles, CA 90089. USA. Email: bbhattac@usc.edu Abhishek Sinha Adobe Systems
More informationSPEED IN URBAN ENV VIORNMENTS IEEE CONFERENCE PAPER REVIW CSC 8251 ZHIBO WANG
SENSPEED: SENSING G DRIVING CONDITIONS TO ESTIMATE VEHICLE SPEED IN URBAN ENV VIORNMENTS IEEE CONFERENCE PAPER REVIW CSC 8251 ZHIBO WANG EXECUTIVE SUMMARY Brief Introduction of SenSpeed Basic Idea of Vehicle
More informationData Mining Approach for Quality Prediction and Improvement of Injection Molding Process
Data Mining Approach for Quality Prediction and Improvement of Injection Molding Process Dr. E.V.Ramana Professor, Department of Mechanical Engineering VNR Vignana Jyothi Institute of Engineering &Technology,
More informationComputer science Case study: your (autonomous) taxi awaits you
M&N18/4/COMSC/HP3/ENG/TZ0/XX/CS Computer science Case study: your (autonomous) taxi awaits you For use in May 2018 and November 2018 Instructions to candidates y Case study booklet required for higher
More informationVOLTAGE STABILITY CONSTRAINED ATC COMPUTATIONS IN DEREGULATED POWER SYSTEM USING NOVEL TECHNIQUE
VOLTAGE STABILITY CONSTRAINED ATC COMPUTATIONS IN DEREGULATED POWER SYSTEM USING NOVEL TECHNIQUE P. Gopi Krishna 1 and T. Gowri Manohar 2 1 Department of Electrical and Electronics Engineering, Narayana
More informationAdaptive Fault-Tolerant Control for Smart Grid Applications
Adaptive Fault-Tolerant Control for Smart Grid Applications F. Khorrami and P. Krishnamurthy Mechatronics/Green Research Laboratory (MGRL) Control/Robotics Research Laboratory (CRRL) Dept. of ECE, Six
More informationMulti Core Processing in VisionLab
Multi Core Processing in Multi Core CPU Processing in 25 August 2014 Copyright 2001 2014 by Van de Loosdrecht Machine Vision BV All rights reserved jaap@vdlmv.nl Overview Introduction Demonstration Automatic
More informationThe State of Charge Estimation of Power Lithium Battery Based on RBF Neural Network Optimized by Particle Swarm Optimization
Journal of Applied Science and Engineering, Vol. 20, No. 4, pp. 483 490 (2017) DOI: 10.6180/jase.2017.20.4.10 The State of Charge Estimation of Power Lithium Battery Based on RBF Neural Network Optimized
More informationSimulating Autonomous Vehicles
School of Electrical, Electronic & Computer Engineering Faculty of Engineering, Computing & Mathematics The University of Western Australia GENG5512: Engineering Research Project Part 2 Simulating Autonomous
More informationAutonomous inverted helicopter flight via reinforcement learning
Autonomous inverted helicopter flight via reinforcement learning Andrew Y. Ng, Adam Coates, Mark Diel, Varun Ganapathi, Jamie Schulte, Ben Tse, Eric Berger, and Eric Liang By Varun Grover Outline! Helicopter
More informationOptimal Vehicle to Grid Regulation Service Scheduling
Optimal to Grid Regulation Service Scheduling Christian Osorio Introduction With the growing popularity and market share of electric vehicles comes several opportunities for electric power utilities, vehicle
More informationLECTURE 6: HETEROSKEDASTICITY
LECTURE 6: HETEROSKEDASTICITY Summary of MLR Assumptions 2 MLR.1 (linear in parameters) MLR.2 (random sampling) the basic framework (we have to start somewhere) MLR.3 (no perfect collinearity) a technical
More informationOnline Learning and Optimization for Smart Power Grid
1 2016 IEEE PES General Meeting Panel on Domain-Specific Big Data Analytics Tools in Power Systems Online Learning and Optimization for Smart Power Grid Seung-Jun Kim Department of Computer Sci. and Electrical
More informationCapacity-Achieving Accumulate-Repeat-Accumulate Codes for the BEC with Bounded Complexity
Capacity-Achieving Accumulate-Repeat-Accumulate Codes for the BEC with Bounded Complexity Igal Sason 1 and Henry D. Pfister 2 Department of Electrical Engineering 1 Techion Institute, Haifa, Israel Department
More informationOnline Learning and Optimization for Smart Power Grid
1 2016 IEEE PES General Meeting Panel on Domain-Specific Big Data Analytics Tools in Power Systems Online Learning and Optimization for Smart Power Grid Seung-Jun Kim Department of Computer Sci. and Electrical
More informationIntelligent Fault Analysis in Electrical Power Grids
Intelligent Fault Analysis in Electrical Power Grids Biswarup Bhattacharya (University of Southern California) & Abhishek Sinha (Adobe Systems Incorporated) 2017 11 08 Overview Introduction Dataset Forecasting
More informationLearning Objectives:
Topic 5.5 High Power Switching Systems Learning Objectives: At the end of this topic you will be able to; recall the conditions under which a thyristor conducts; explain the significance of the following
More informationA Data Driven Health Monitoring Approach to Extending Small Sats Mission
A Data Driven Health Monitoring Approach to Extending Small Sats Mission Fangzhou Sun 1, Abhishek Dubey 2, Chetan S. Kulkarni 3, Nagbhushan Mahadevan 4 and Ali Guarneros Luna 5 1,2,4 Institute for Software
More informationTECHNICAL REPORTS from the ELECTRONICS GROUP at the UNIVERSITY of OTAGO. Table of Multiple Feedback Shift Registers
ISSN 1172-496X ISSN 1172-4234 (Print) (Online) TECHNICAL REPORTS from the ELECTRONICS GROUP at the UNIVERSITY of OTAGO Table of Multiple Feedback Shift Registers by R. W. Ward, T.C.A. Molteno ELECTRONICS
More informationRubber Band Car. Tommy Stewart Corey Marineau John Martinez
Tommy Stewart Corey Marineau John Martinez Rubber Band Car PURPOSE: Create a rubber band propelled car that will travel three meters. Then create a regression line using the data that represents how the
More informationRisk-Based Collision Avoidance in Semi-Autonomous Vehicles
Independent Work Report Spring, 2016 Risk-Based Collision Avoidance in Semi-Autonomous Vehicles Christopher Hay 17 Adviser: Thomas Funkhouser Abstract Although there have been a number of advances in active
More informationHow and why does slip angle accuracy change with speed? Date: 1st August 2012 Version:
Subtitle: How and why does slip angle accuracy change with speed? Date: 1st August 2012 Version: 120802 Author: Brendan Watts List of contents Slip Angle Accuracy 1. Introduction... 1 2. Uses of slip angle...
More informationSyllabus: Automated, Connected, and Intelligent Vehicles
Page 1 of 8 Syllabus: Automated, Connected, and Intelligent Vehicles Part 1: Course Information Description: Automated, Connected, and Intelligent Vehicles is an advanced automotive technology course that
More informationACCELERATING THE RACE TO SELF-DRIVING CARS. Jen-Hsun Huang, Co-Founder & CEO, NVIDIA Jan. 4, 2016
ACCELERATING THE RACE TO SELF-DRIVING CARS Jen-Hsun Huang, Co-Founder & CEO, NVIDIA Jan. 4, 2016 SELF-DRIVING IS A MAJOR COMPUTER SCIENCE CHALLENGE SOFTWARE SUPERCOMPUTER DEEP LEARNING 2 NVIDIA DRIVE PX
More informationCapacity-Achieving Accumulate-Repeat-Accumulate Codes for the BEC with Bounded Complexity
Capacity-Achieving Accumulate-Repeat-Accumulate Codes for the BEC with Bounded Complexity Igal Sason 1 and Henry D. Pfister 2 Department of Electrical Engineering 1 Techion Institute, Haifa, Israel School
More informationROTOR POSITION OF SWITCHED RELUCTANCE MOTOR USING SENSORLESS METHOD
ROTOR POSITION OF SWITCHED RELUCTANCE MOTOR USING SENSORLESS METHOD 1 RAMESH PALAKEERTHI AND 2 Dr. P. SUBBAIAH 1 Research scholar, Dr. MGR Educational and Research Institute University, Chennai, INDIA
More informationAI Driven Environment Modeling for Autonomous Driving on NVIDIA DRIVE PX2
AI Driven Environment Modeling for Autonomous Driving on NVIDIA DRIVE PX2 Dr. Alexey Abramov, Christopher Bayer, Dr. Claudio Heller, Claudia Loy Chassis & Safety Agenda 1 2 3 4 5 6 7 Introduction Autonomous
More informationFuzzy based Adaptive Control of Antilock Braking System
Fuzzy based Adaptive Control of Antilock Braking System Ujwal. P Krishna. S M.Tech Mechatronics, Asst. Professor, Mechatronics VIT University, Vellore, India VIT university, Vellore, India Abstract-ABS
More informationFUZZ-ART NEURAL NETWORKS FOR PREDICTING CHI-CHI EARTHQUAKE INDUCED LIQUEFACTION IN YUAN-LIN AREA
Journal of Marine Science and Technology, Vol. 10, No. 1, pp. 21-32 (2002) 21 FUZZ-ART NEURAL NETWORKS FOR PREDICTING CHI-CHI EARTHQUAKE INDUCED LIQUEFACTION IN YUAN-LIN AREA S. G. Chern*, R. F. Hu**,
More informationECE 550D Fundamentals of Computer Systems and Engineering. Fall 2017
ECE 550D Fundamentals of Computer Systems and Engineering Fall 2017 Digital Arithmetic Prof. John Board Duke University Slides are derived from work by Profs. Tyler Bletch and Andrew Hilton (Duke) Last
More informationPower Matching Strategy Modeling and Simulation of PHEV Based on Multi agent
Power Matching Strategy Modeling and Simulation of PHEV Based on Multi agent Limin Niu* 1, Lijun Ye 2 School of Mechanical Engineering, Anhui University of Technology, Ma anshan 243032, China *1 niulmdd@163.com;
More informationCRSM: Crowdsourcing based Road Surface Monitoring
CRSM: Crowdsourcing based Road Surface Monitoring Kongyang Chen 1, Mingming Lu 2, Guang Tan 1, and Jie Wu 3 1SIAT, Chinese Academy of Sciences, 2 Central South University 3Temple University Nov. 15 th,
More informationApplication of Artificial Neural Networks for Emission Modelling of Biodiesels for a C.I Engine under Varying Operating Conditions
Application of Artificial Neural Networks for Emission Modelling of Biodiesels for a C.I Engine under Varying Operating Conditions R.Manjunatha Assistant Executive Engineer, Irrigation Department, GBC
More informationA Study of Individual Characteristics of Driving Behavior Based on Hidden Markov Model
Sensors & Transducers 2014 by IFSA Publishing, S. L. http://www.sensorsportal.com A Study of Individual Characteristics of Driving Behavior Based on Hidden Markov Model Xingjian Zhang, Xiaohua Zhao, Jian
More informationInvestigation in to the Application of PLS in MPC Schemes
Ian David Lockhart Bogle and Michael Fairweather (Editors), Proceedings of the 22nd European Symposium on Computer Aided Process Engineering, 17-20 June 2012, London. 2012 Elsevier B.V. All rights reserved
More informationPERFORMANCE ANALYSIS OF D.C MOTOR USING FUZZY LOGIC CONTROLLER
PERFORMANCE ANALYSIS OF D.C MOTOR USING FUZZY LOGIC CONTROLLER Mohammad Muktafi Ali Khan 1, Zafar Khan 2, AbulSaeed Azad 3 1 Student, M.Tech, 2,3 Associate Prof., Department of Electrical & Electronics
More informationtime in seconds Amy leaves diving board
1 Amy dives from the high diving board at a swimming pool. Look at the graph of her motion. speed in m / s 15 10 Amy enters water P Q 5 0 0 0.5 1.0 1.5 2.0 2.5 time in seconds Amy leaves diving board (a)
More informationData envelopment analysis with missing values: an approach using neural network
IJCSNS International Journal of Computer Science and Network Security, VOL.17 No.2, February 2017 29 Data envelopment analysis with missing values: an approach using neural network B. Dalvand, F. Hosseinzadeh
More informationEstimation of Vehicle Parameters using Kalman Filter: Review
Review Article International Journal of Current Engineering and Technology E-ISSN 2277 4106, P-ISSN 2347-5161 2014 INPRESSCO, All Rights Reserved Available at http://inpressco.com/category/ijcet Sagar
More informationSPEED AND TORQUE CONTROL OF AN INDUCTION MOTOR WITH ANN BASED DTC
SPEED AND TORQUE CONTROL OF AN INDUCTION MOTOR WITH ANN BASED DTC Fatih Korkmaz Department of Electric-Electronic Engineering, Çankırı Karatekin University, Uluyazı Kampüsü, Çankırı, Turkey ABSTRACT Due
More informationProblem Set 3 - Solutions
Ecn 102 - Analysis of Economic Data University of California - Davis January 22, 2011 John Parman Problem Set 3 - Solutions This problem set will be due by 5pm on Monday, February 7th. It may be turned
More informationArtificial-Intelligence-Based Electrical Machines and Drives
Artificial-Intelligence-Based Electrical Machines and Drives Application of Fuzzy, Neural, Fuzzy-Neural, and Genetic-Algorithm-Based Techniques Peter Vas Professor of Electrical Engineering University
More informationAI RC Car Racing. 1 Introduction
AI RC Car Racing Martin Liivak martin.liivak@gmail.com Markus Loide markusloide@gmail.com Meri Liis Treimann mltreima@ut.ee Sebastian Värv sebastian.varv@gmail.com Abstract Autonomous or self-driving racing
More informationCOMP 776: Computer Vision
COMP 776: Computer Vision Basic Info Instructor: Svetlana Lazebnik (lazebnik@cs.unc.edu) Office hours: By appointment, SN 219 Textbook: Forsyth & Ponce, Computer Vision: A Modern Approach Class webpage:
More informationABSTRACT. Keywords Neural network, forecasting, diesel fuels.
Neural Network Model for Forecasting the Cetane Number in the Diesel Fuels Petar Halachev Department of Informatics, University of Chemical technology and Metallurgy, Bulgaria, Sofia, ABSTRACT The cetane
More informationSmall Scale-Wind Power Dispatchable Energy Source Modeling
Small Scale-Wind Power Dispatchable Energy Source Modeling Jordan Cannon, David Moore, Stephen Eason, Adel El Shahat Department of Electrical Engineering, Georgia Southern University, USA Abstract Due
More informationIndian Journal of Science and Technology Vol. 5 No.3 (Mar 2012) ISSN:
Radial basis function neural networks in prediction and modeling of diesel engine emissions operated for biodiesel blends under varying operating conditions R. Manjunatha 1*, P. Badari Narayana 2, K. Hemachandra
More informationSimulation and Analysis of Vehicle Suspension System for Different Road Profile
Simulation and Analysis of Vehicle Suspension System for Different Road Profile P.Senthil kumar 1 K.Sivakumar 2 R.Kalidas 3 1 Assistant professor, 2 Professor & Head, 3 Student Department of Mechanical
More informationSPEED CONTROL OF FOUR QUADRANT PMDC MOTOR DRIVE USING PI BASED ANN CONTROLLER
SPEED CONTROL OF FOUR QUADRANT PMDC MOTOR DRIVE USING PI BASED ANN CONTROLLER Visakh Murali 1, Anju G Pillai 2 and Vijai Jairaj 3 1 PG Student [Electrical Machines], Department of EEE, Sree Buddha College
More informationEuropean Conference on Nanoelectronics and Embedded Systems for Electric Mobility. An Insight into Active Balancing for Lithium-Ion Batteries
European Conference on Nanoelectronics and Embedded Systems for Electric Mobility ecocity emotion 24-25 th September 2014, Erlangen, Germany An Insight into Active Balancing for Lithium-Ion Batteries Federico
More informationAn Autonomous Braking System of Cars Using Artificial Neural Network
I J C T A, 9(9), 2016, pp. 3665-3670 International Science Press An Autonomous Braking System of Cars Using Artificial Neural Network P. Pavul Arockiyaraj and P.K. Mani ABSTRACT The main aim is to develop
More informationSupervised Learning to Predict Human Driver Merging Behavior
Supervised Learning to Predict Human Driver Merging Behavior Derek Phillips, Alexander Lin {djp42, alin719}@stanford.edu June 7, 2016 Abstract This paper uses the supervised learning techniques of linear
More informationAdaptive Power Grids: Responding to Generation Diversity
Short Course on Future Trends for Power Systems, The University of Sydney, 12 th October, 2009 Adaptive Power Grids: Responding to Generation Diversity David J Hill Research School of Information Sciences
More informationIn-Place Associative Computing:
In-Place Associative Computing: A New Concept in Processor Design 1 Page Abstract 3 What s Wrong with Existing Processors? 3 Introducing the Associative Processing Unit 5 The APU Edge 5 Overview of APU
More informationDeep Unordered Composition Rivals Syntactic Methods for Text Classification
Deep Unordered Composition Rivals Syntactic Methods for Text Classification Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III University of Maryland, College Park University of Colorado,
More informationUnmanned autonomous vehicles in air land and sea
based on Ulrich Schwesinger lecture on MOTION PLANNING FOR AUTOMATED CARS Unmanned autonomous vehicles in air land and sea Some relevant examples from the DARPA Urban Challenge Matteo Matteucci matteo.matteucci@polimi.it
More informationDeep Fault Analysis and Subset Selection in Solar Power Grids
Deep Fault Analysis and Subset Selection in Solar Power Grids Biswarup Bhattacharya University of Southern California Los Angeles, CA 90089. USA. Email: bbhattac@usc.edu Abhishek Sinha Adobe Systems Incorporated
More informationCatCharger: Deploying Wireless Charging Lanes in a Metropolitan Road Network through Categorization and Clustering of Vehicle Traffic
CatCharger: Deploying Wireless Charging Lanes in a Metropolitan Road Network through Categorization and Clustering of Vehicle Traffic Li Yan, Haiying Shen, Juanjuan Zhao, Chengzhong Xu, Feng Luo and Chenxi
More informationBiologically-inspired reactive collision avoidance
Biologically-inspired reactive collision avoidance S. D. Ross 1,2, J. E. Marsden 2, S. C. Shadden 2 and V. Sarohia 3 1 Aerospace and Mechanical Engineering, University of Southern California, RRB 217,
More informationBuilding Fast and Accurate Powertrain Models for System and Control Development
Building Fast and Accurate Powertrain Models for System and Control Development Prasanna Deshpande 2015 The MathWorks, Inc. 1 Challenges for the Powertrain Engineering Teams How to design and test vehicle
More informationAn Application of Artificial Neural Network for Predicting Engine Torque in a Biodiesel Engine
American Journal of Energy Research, 2014, Vol. 2, No. 4, 74-80 Available online at http://pubs.sciepub.com/ajer/2/4/1 Science and Education Publishing DOI:10.12691/ajer-2-4-1 An Application of Artificial
More informationModeling, Design and Simulation of Active Suspension System Frequency Response Controller using Automated Tuning Technique
Modeling, Design and Simulation of Active Suspension System Frequency Response Controller using Automated Tuning Technique Omorodion Ikponwosa Ignatius Obinabo C.E Evbogbai M.J.E. Abstract Car suspension
More informationAn Example using the Scientific Explanation Tool within the New Paradigm
An Example using the Scientific Explanation Tool within the New Paradigm Inattentional Blindness is a "Good" explanation and Conspicuity is a "Bad" explanation Explanation A good explanation is difficult
More informationThe pathway to self-driving vehicles: Disconnects between human capabilities and advanced vehicle systems?
The pathway to self-driving vehicles: Disconnects between human capabilities and advanced vehicle systems? Bryan Reimer, Ph.D. MIT AgeLab & New England University Transportation Center JITI Self-Driving
More informationOn Using Storage and Genset for Mitigating Power Grid Failures
1 / 27 On Using Storage and Genset for Mitigating Power Grid Failures Sahil Singla ISS4E lab University of Waterloo Collaborators: S. Keshav, Y. Ghiassi-Farrokhfal 1 / 27 Outline Introduction Background
More informationSegway with Human Control and Wireless Control
Review Paper Abstract Research Journal of Engineering Sciences E- ISSN 2278 9472 Segway with Human Control and Wireless Control Sanjay Kumar* and Manisha Sharma and Sourabh Yadav Dept. of Electronics &
More informationEarly Stabilizing Feature Importance for TensorFlow Deep Neural Networks
Early Stabilizing Feature Importance for TensorFlow Deep Neural Networks IJCNN 2017, Jeff Heaton Steven McElwee James Cannady James Fraley Problem This study addresses the problem that there are not existing
More informationA Short History of Real World Testing; What have we learnt?
A Short History of Real World Testing; What have we learnt? September 25, 2013 MIRA Ltd 2013 Real World Testing My view of durability and development testing attempts to replicate the real world. Introduce
More informationWhat s new. Bernd Wiswedel KNIME.com AG. All Rights Reserved.
What s new Bernd Wiswedel 2016 KNIME.com AG. All Rights Reserved. What s new 2+1 feature releases in the last year: (3.0), 3.1, 3.2 Changes documented online 2016 KNIME.com AG. All Rights Reserved. 2 What
More informationHuman-Learning Steering Control Based-on Artificial Neural Network and Visual Servo
Human-Learning Steering Control Based-on Artificial Neural Network and Visual Servo Arthit Srikaew 1, Prayoth Kumsawat 2, Kitti Attakitmongcol 1 and Bannakit Khitthuk 1 Abstract This article presented
More informationThe Application of UKF Algorithm for type Lithium Battery SOH Estimation
Applied Mechanics and Materials Online: 2014-02-06 ISSN: 1662-7482, Vols. 519-520, pp 1079-1084 doi:10.4028/www.scientific.net/amm.519-520.1079 2014 Trans Tech Publications, Switzerland The Application
More informationANN Peak Load Shaver.
ANN Peak Load Shaver. ANN Peak Load Shaver. ANN Based Electricity Load Forecasting By: Ameya Deoras [1] For short term operations and long term planning for utilities, accurate forecasts are critical.
More informationPredictive Control Strategies using Simulink
Example slide Predictive Control Strategies using Simulink Kiran Ravindran, Ashwini Athreya, HEV-SW, EE/MBRDI March 2014 Project Overview 2 Predictive Control Strategies using Simulink Kiran Ravindran
More informationModeling Strategies for Design and Control of Charging Stations
Modeling Strategies for Design and Control of Charging Stations George Michailidis U of Michigan www.stat.lsa.umich.edu/ gmichail NSF Workshop, 11/15/2013 Michailidis EVs and Charging Stations NSF Workshop,
More informationMODELING AND SIMULATION OF SMART AUTOMOTIVE COOLING SYSTEM. Karpagam College of Engineering, Coimbatore.
Volume 118 No. 22 2018, 1671-1676 ISSN: 1314-3395 (on-line version) url: http://acadpubl.eu/hub ijpam.eu MODELING AND SIMULATION OF SMART AUTOMOTIVE COOLING SYSTEM 1 R.Arul Murugan, 2 R.Mahesh, 3 C.Krishnaraj,
More informationAnalysis of Big Data Streams to Obtain Braking Reliability Information July 2013, for 2017 Train Protection 1 / 25
Analysis of Big Data Streams to Obtain Braking Reliability Information for Train Protection Systems Prof. Dr. Raphael Pfaff Aachen University of Applied Sciences pfaff@fh-aachen.de www.raphaelpfaff.net
More informationSECTION 8 BEVEL GEARING
SECTION 8 BEVEL GEARING For intersecting shafts, bevel gears offer a good means of transmitting motion and power. Most transmissions occur at right angles, Figure 8-1, but the shaft angle can be any value.
More informationStudy on State of Charge Estimation of Batteries for Electric Vehicle
Study on State of Charge Estimation of Batteries for Electric Vehicle Haiying Wang 1,a, Shuangquan Liu 1,b, Shiwei Li 1,c and Gechen Li 2 1 Harbin University of Science and Technology, School of Automation,
More informationFeatured Articles Utilization of AI in the Railway Sector Case Study of Energy Efficiency in Railway Operations
128 Hitachi Review Vol. 65 (2016), No. 6 Featured Articles Utilization of AI in the Railway Sector Case Study of Energy Efficiency in Railway Operations Ryo Furutani Fumiya Kudo Norihiko Moriwaki, Ph.D.
More informationProfessor Dr. Gholamreza Nakhaeizadeh. Professor Dr. Gholamreza Nakhaeizadeh
Statistic Methods in in Data Mining Business Understanding Data Understanding Data Preparation Deployment Modelling Evaluation Data Mining Process (Part 2) 2) Professor Dr. Gholamreza Nakhaeizadeh Professor
More informationMachine Learning for the Detection of Electricity Theft
Machine Learning for the Detection of Electricity Theft Patrick GLAUNER SnT - Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg February 21, 2018 Patrick GLAUNER ML
More informationModeling, Design and Simulation of Active Suspension System Root Locus Controller using Automated Tuning Technique.
Modeling, Design and Simulation of Active Suspension System Root Locus Controller using Automated Tuning Technique. Omorodion Ikponwosa Ignatius Obinabo C.E Abstract Evbogbai M.J.E. Car suspension system
More informationElectricity and Magnetism Module 2 Student Guide
Concepts of this Module Introducing current and voltage Simple circuits Circuit diagrams Background Electricity and Magnetism Module 2 Student Guide When water flows through a garden hose, we can characterize
More informationTrends in Experiment Design
Trends in Experiment Design Univ. Prof. Dr. Stefan Jakubek Technische Universität Wien Stefan Jakubek 1 Introduction Design of experiments (DOE) is the design of any information-gathering exercises where
More informationRule-based Integration of Multiple Neural Networks Evolved Based on Cellular Automata
1 Robotics Rule-based Integration of Multiple Neural Networks Evolved Based on Cellular Automata 2 Motivation Construction of mobile robot controller Evolving neural networks using genetic algorithm (Floreano,
More informationIdentify Formula for Throughput with Multi-Variate Regression
DECISION SCIENCES INSTITUTE Using multi-variate regression and simulation to identify a generic formula for throughput of flow manufacturing lines with identical stations Samrawi Berhanu Gebermedhin and
More informationEVS28 KINTEX, Korea, May 3-6, 2015
EVS28 KINTEX, Korea, May 3-6, 25 Pattern Prediction Model for Hybrid Electric Buses Based on Real-World Data Jing Wang, Yong Huang, Haiming Xie, Guangyu Tian * State Key laboratory of Automotive Safety
More informationThe Degrees of Freedom of Partial Least Squares Regression
The Degrees of Freedom of Partial Least Squares Regression Dr. Nicole Krämer TU München 5th ESSEC-SUPELEC Research Workshop May 20, 2011 My talk is about...... the statistical analysis of Partial Least
More informationNeuro-Fuzzy Controller of a Sensorless PM Motor Drive for Washing Machines
4 th Intr. Conf. On Systems, Signals & Devices 19-22 March 2007 Hammamat, Tunisia Neuro-Fuzzy Controller of a Sensorless PM Motor Drive for Washing Machines Paper No.: SSD07-SAC-1117 Dr. Kasim M. Al-Aubidy,
More informationDesign, Theory, Calculations
Heavy-Duty Wheeled Vehicles: Design, Theory, Calculations By Boris N. Belousov and Sergei D. Popov INTERNATIONAL. Warrendale, Pennsylvania, USA Table of Contents Preface xi Acknowledgments xiii Symbols
More informationJAXA's electric propulsion systems
JAXA's electric propulsion systems Akira Nishizawa Emission free aircraft section Innovative Aircraft Systems Research Aircraft Systems Research Team Next Generation Aeronautical Innovation Hub Center
More informationThe Session.. Rosaria Silipo Phil Winters KNIME KNIME.com AG. All Right Reserved.
The Session.. Rosaria Silipo Phil Winters KNIME 2016 KNIME.com AG. All Right Reserved. Past KNIME Summits: Merging Techniques, Data and MUSIC! 2016 KNIME.com AG. All Rights Reserved. 2 Analytics, Machine
More informationRoehrig Engineering, Inc.
Roehrig Engineering, Inc. Home Contact Us Roehrig News New Products Products Software Downloads Technical Info Forums What Is a Shock Dynamometer? by Paul Haney, Sept. 9, 2004 Racers are beginning to realize
More information