Rule-based Integration of Multiple Neural Networks Evolved Based on Cellular Automata

Transcription

1 1 Robotics

2 Rule-based Integration of Multiple Neural Networks Evolved Based on Cellular Automata 2

3 Motivation Construction of mobile robot controller Evolving neural networks using genetic algorithm (Floreano, 1996) Genetic programming (Nordin, 1997) Fuzzy (Cho, 1996) Integration of multi-modules Solving complex task Simpler components or subtasks Learn each module by separate systems Combine them to solve goal task 3

4 Behavior-based Robot : Khepera Light Source Battery Recharge Area 8 distance sensors ( 0 ~ 1023 ) 8 light sensors ( 50 ~ 500 ) 2 motors ( -10 ~ 10 ) Battery level sensor ( 0~2500 ) Floor-brightness sensor 4

5 Evolved NNs based on CA Generate initial population Develop NNs based on CA from chromosome Apply it to a problem Evaluate fitness of each NN Satisfied NN found? No Select good NNs in population Maniplate them with genetic operater Generate new population Yes Stop 5

6 CA-Based Neural Networks Growth phase Signal phase Chromosome Input Output Make neural network structure with chromosome Transmit signals from input cells to output cells 6

7 Applying to Control a Mobile Robot Evolution Fitness value Chromosome Sensor value Velocity of wheels Neural Networks Growth phase Signaling phase Khepera Simulator

8 Cell States Blank Empty space No participate in any cell interaction during signaling Neuron Collect neural signals from surrounding dendrite cells Send them to surrounding axon cells Axon Distribute signals to neighborhoods originating from neuron Dendrite Collect signals from neighborhoods Pass them to neuron cells 8

9 Growth Phase y 1 x 1 x 2 x 3 x 4 y 2 y 3 y 4 (a) (b) (c) (d) (e) (f)

10 Signaling phase Collecting signals Dendrite Distributing signals Neuron Inhibitory Axon Excitatory Axon Inhibitory signal Excitatory signal

11 Basis Behaviors Battery Recharge If a robot arrives at battery recharge area, battery is recharged. Go Ahead If there is nothing around the robot, it goes ahead. Follow Light The robot goes to stronger light. Avoid Obstacles If the obstacles exist around the robot, it avoids obstacles without bumping against them. 11

12 Battery Recharge Programmed Module Enables it to live for a long time Battery Recharge area Black area in environment Light source exists 12

13 Go Ahead Programmed module The velocity of left : 5 The velocity of right : 5 Make it to move continuously without stopping 13

14 Follow Light (1) The robot goes to stronger light => Go to Battery Recharge area Evolved module Evolving CAM-Brain to follow light Fitness Function S : Average speed of the two wheels V : Difference between the velocity of two wheels c : 1 (if the robot does not bump against obstacles) 1/2 (if the robot bumps) D : Distance from the robot to the goal position 14

15 Follow Light (2) Best Fitness Average Fitness 15

16 Follow Light (3) 0 degree 90 degrees 180 degrees 270 degrees 16

17 Avoid Obstacles Evolved module Evolving CAM-Brain to avoid obstacles without bumping Using CAM-Brain module evolved incrementally in previous work 17

18 Incremental Evolution Single system learns a succession of tasks Task {t1, t2, t3,., tn} tn : goal task ti is easier than ti+1 for all i: 0 < i <= n Each task is derived by transforming a goal-task 18

19 Environments (a) (b) (c) (d) (e) (f) 19

20 Fitness Incremental Evolution Direct Evolution 20

21 Architecture of NNs

22 22 Applying to Other Environments

23 Rules of Combining (1) IF ( Battery Recharge area) Battery Recharge module ELSE IF (Battery sensor < α) AND (Minimum value of light sensors γ) IF (Maximum value of distance sensors β 1 ) Follow Light module ELSE Avoid Obstacles module ELSE IF (Maximum value of distance sensors β 2 ) Go Ahead module ELSE Avoid Obstacles module 23

24 Rules of Combining (2) α : If battery sensor value is less than α, battery is needed to recharging. β 1, β 2 : If the maximum values of distance sensors are larger than β 1, β 2, the robot recognize as obstacles. γ : If the minimum values of light sensor is less than γ, the robot recognizes as light. 24

25 Results of Combining Moving 5000 step Moving step Moving step 25

26 26 Coordination of Multiple Behavior Modules Evolved on CAM-Brain

27 Motivation Construction of mobile robot controller Evolving neural networks using genetic algorithm (Floreano, 1996) Genetic programming (Nordin, 1997) Fuzzy system evolved by genetic algorithm(cho, 1996) Integration of multi-modules by action selection mechanism Solving complex task Simpler components or subtasks Learn each module by separate system Combine them to solve goal task by action selection mechanism 27

28 Action Selection Mechanism Proposed by Maes (1989) Idea : Integrate high-level behavior into a system using lower-level behaviors Distributed, non-hierarchical network Two waves of input to the network Sensors of external environment Motivations or goals Different types of links encoding various relationships Components of ASM : Nodes, internal links and external links 28

29 29 Example

30 Nodes INPUT PREDECESSOR LINKS SUCCESSOR LINKS CONFLICTOR LINKS GOALS ENVIRONMENT PRECONDITIONS ADD LIST DELETE LIST ACTIVATION OUTPUT PREDECESSOR LINKS SUCCESSOR LINKS CONFLICTOR LINKS EXECUTABLE CODE 30

31 Internal Links A B is active Predecessor link If (p = false) (p precondition of A) (p add list of B) Successor link If (p = false) (p add list of A ) (A is executable) (p precondition of B) Conflictor link If (p = true) (p precondition of A ) (p delete list of B) 31

32 External Links p A is active From sensors of the environment If (p = true) (p precondition of A) From goals If (p 0 ) (p add list of A) From protected goals If (p 0 ) (p delete list of A) 32

33 Action Selection Procedure Activation of a node is updated by excitation coming in from the environment and the motivations By the type of internal links, activation of a node is exchanged between two nodes Normalize the node activation AVG(SUM( (activation of node1) + (activation of node2).)) = π If (all preconditions of a node are true) and (activation of a node θ) then the node is executable If (executable node exist ) then execute the node else θ is reduced by 10% and repeat cycle 33

34 Experiment Environments Light Source Battery Recharge Area Light Source Battery Recharge Area 34 Simple Chaotic

35 Basic Behaviors Recharging battery If a robot is in battery recharge area, then battery is recharged Following light Robot goes to stronger light Avoiding obstacles If obstacles exist around the robot, then robot avoids them without bumping Going straight Robot goes ahead 35

36 Sensors and Goals Sensors of environment (Binary value) In battery recharge area Near battery recharge area In shade area Nothing around the robot Obstacles are near Goals (Real or binary value) Battery is not zero Battery is not below the half 36

37 Precodition and Add List of Nodes Recharging battery Following light Precondition In battery recharge area In shade area Near battery recharge area Add list Battery is not zero Battery is not below the half In battery recharge area Avoiding Obstacles Going Straight Obstacles are near Nothing around the robot Nothing around the robot Obstacles are near In battery recharge area Near battery recharge area 37

38 Internal Links between Nodes Predecessor Links (Recharging battery Following light) (Recharging battery Going straight) (Following light Going straight) (Going straight Avoiding obstacles) (Avoiding obstacles Goinging straight) Successor Links (Following light Recharging battery) (Going straight Following light) (Going straight Recharging battery) (Going straight Avoiding obstacles) (Avoiding obstacles Going straight) 38

39 Model of ASM In battery recharge area Battery is not zero Near battery recharge area Following light Recharging battery 39 In shade area Nothing around the robot Obstacles are near Going straight Avoiding Obstacles Battery is not below the half

40 Simulation Results Simple Chaotic 40

41 Action Sequence (1) = Recharging Battery 2 = Following Light 3 = Avoiding Obstacle 4 = Going Straight

42 Action Sequence (2) 5 A B C D

43 43 A Framework of Evolvable Systems and Measures for Intelligent Agents

44 Need for Agents Conventional information processing Inference, planning, and commands directed by users Increase of the amount of information Need for an agent that works for a user 44

45 Intelligent Agent Intelligent agent: achieve user s goals autonomously instead of users Properties of intelligent agents Autonomy Reactivity Proactivity Reasoning and learning social ability Cooperation Communication Smart agents Collaborative learning agents Cooperate Learn Autonomous Collaborative agents Interface agents A category of intelligent agents 45

46 Related Works: Application of Soft Computing Techniques Soft computing techniques Neural networks, fuzzy, probabilistic inference, evolutionary computation Flexible inference/random searching capability Type 1: application of each soft computing technique To Simple problems Difficult to tune internal parameters Require expert s knowledge/time and effort Type 2: combination with evolutionary algorithms Tuning of internal parameters by evolutionary algorithms 46

47 Problems of Type 2 Can the same results be obtained? Adaptive evolution( 1 ) What properties are genetically preferred? Adaptive behaviors( ) How the solutions are formed? Evolutionary pathways to the solutions( 3 ) Behavioral properties? Illustration of emergence( ) 4 2 Emergence Desirable Evolutionary Causes and Effects 2 4 Good Solution High probability Low probability High Evolvability 1 Adaptive Evolution Non- Adaptive Evolution Low Evolvability 47 3 Adaptive Behavior Bad Solution

48 Research Objectives A framework for intelligent agents easy to represent expert s knowledge analyze evolution Application to a real-world agent to show the usefulness 48

49 A Soft Computing Framework Research goals How to achieve the goals What we can get or show Agent Construction Rule-Based System Hardware Agent Software Agent Evolutionary Algorithms Evolution Analysis Evolutionary Activity Statistics Adaptive Evolution Schema Anaysis Adaptive Behavior Analysis of Evolution Emergence Behavior Analysis Observational Emergence Evolutionar y Pathways 49

50 Rule-based Systems Ruse Based Systems Working Memory (Facts) Inference Engine Knowledge Base (Rules) Working memory: represents current facts Inference engine: infer appropriate actions based on current facts Knowledge base: a set of rules for inference IF (condition) THEN (action). IF (condition) THEN (action) 50 Merits Easy to represent expert s knowledge Easily understood by humans Demerits Need knowledge base

51 Genetic Algorithm (GA) Genetic Algorithm Parameters Candidate Solutions Encoding of chromosomes Test Terminate? Y Crossover Mutation Reproduce Types of algorithms: Simple GA, Overlapped GA Genetic operations: Mutation/Crossover Others N End 51

52 Evolutionary Activity Statistics Attach a counter to each component (e.g., alleles, individuals, ) Accumulate counter values over generations Need to define increment function for the counter Evolutionary Activity Mean Activity New Activity Overall adaptability of a system Adaptability of each component Adaptive innovations 52

53 Schema Definition A similarity template representing a subset of strings with similarity at certain string positions ( Holland 1968) Composed of a character set and a don t care character Examples Character set = {0,1}, don t care=# #0000 {10000, 00000} #111# {01110, 01111, 11110, 11111} 53

54 Observational Emergence creation of new properties Morgan, C.L., Emergent Evolution, Williams and Norgate, 1923 Observational emergence Proposed by Bass, N.A. S : structure - system, organization, organism, machine, P : a property of S observed by observational mechanism Obs 54

55 Application to a Robot Agent Robot Agent Khepera mobile robot: 8 proximity sensors, 2 motors 55

56 Research goals How to achieve the goals What we can get or show Rule-Based System Agent Construction Fuzzy Logic Controller Evolutionary Algorithms Genetic Algorithm Evolution Analysis Evolutionary Activity Adaptive Evolution Schema Anaysis Adaptive Behavior Analysis of Evolution Emergence Behavior Analysis Observational Emergence Evolutionary Pathways to Solutions 56

57 Fuzzy Rule-based System Eight proximity Fuzzy Working Memory Inference Engine sensors Min-Max Correlation Centroid Defuzzification Output: Two motors Rule-based Knowledge Base 20 rules maximum System Others Membership functions Number of fuzzy sets 57

58 Genetic Algorithm Parameters Encoding of chromosomes Types of algorithms: Simple GA, Overlapped GA Genetic operations: 변수의퍼지집합코딩 Mutation/crossover Overlapped GA (50%) Two point crossover (50%) Mutation (1%) Others 50 individuals 58

59 Adaptive Evolution 120 Fuzzy Model Neutral Shadow Fuzzy Model Neutral Shadow Mean Activity New Activity Generation Generation 59

60 Adaptive Behaviors

61 61 Evolutionary Pathways

62 Best (816th generation) M C Mutation Crossover 1 ~ 5 Tags for information M th individual (780) C 5th individual (775) 27th individual (777) M 9th individual (600) M C M 25th individual (578) 6th individual (578) 24th individual (605) C C 47th individual (576) 12th individual (574) 12th individual (603) 6th individual (599) M M M M 21th individual (1) 33th individual (8) 27th individual (0) 6th individual (6)

63 Observational Emergence: Turning Around Behavior Int First order structures Obs 1 Obs The interactions of the three first-order structures S2, S5, S7 make Obs2(S2) different from the Obs1( 1 ), of the first-order This implies that S i {2,5,7 } i 1 Therefore, we can conclude that Obs2(S2) is observationally emergent behavior 63