Evolutionary acquisition of neural topologies

Evolutionary acquisition of neural topologies (EANT/EANT2) is an evolutionary reinforcement learning method that evolves both the topology and weights of artificial neural networks. It is closely related to the works of Angeline et al.^[1] and Stanley and Miikkulainen.^[2] Like the work of Angeline et al., the method uses a type of parametric mutation that comes from evolution strategies and evolutionary programming (now using the most advanced form of the evolution strategies CMA-ES in EANT2), in which adaptive step sizes are used for optimizing the weights of the neural networks. Similar to the work of Stanley (NEAT), the method starts with minimal structures which gain complexity along the evolution path.

Contribution of EANT to neuroevolution

Despite sharing these two properties, the method has the following important features which distinguish it from previous works in neuroevolution.

It introduces a genetic encoding called common genetic encoding (CGE) that handles both direct and indirect encoding of neural networks within the same theoretical framework. The encoding has important properties that makes it suitable for evolving neural networks:

It is complete in that it is able to represent all types of valid phenotype networks.
It is closed, i.e. every valid genotype represents a valid phenotype. (Similarly, the encoding is closed under genetic operators such as structural mutation and crossover.)

These properties have been formally proven.^[3]

For evolving the structure and weights of neural networks, an evolutionary process is used, where the exploration of structures is executed at a larger timescale (structural exploration), and the exploitation of existing structures is done at a smaller timescale (structural exploitation). In the structural exploration phase, new neural structures are developed by gradually adding new structures to an initially minimal network that is used as a starting point. In the structural exploitation phase, the weights of the currently available structures are optimized using an evolution strategy.

Performance

EANT has been tested on some benchmark problems such as the double-pole balancing problem,^[4] and the RoboCup keepaway benchmark.^[5] In all the tests, EANT was found to perform very well. Moreover, a newer version of EANT, called EANT2, was tested on a visual servoing task and found to outperform NEAT and the traditional iterative Gauss–Newton method.^[6] Further experiments include results on a classification problem ^[7]

References

^ Peter J Angeline, Gregory M Saunders, and Jordan B Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5:54–65, 1994. [1]
^ NeuroEvolution of Augmented Topologies (NEAT) by Stanley and Miikkulainen, 2005 [2]
^ Yohannes Kassahun, Mark Edgington, Jan Hendrik Metzen, Gerald Sommer and Frank Kirchner. Common Genetic Encoding for Both Direct and Indirect Encodings of Networks. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), London, UK, 1029–1036, 2007.[3]
^ Yohannes Kassahun and Gerald Sommer. Efficient reinforcement learning through evolutionary acquisition of neural topologies. In Proceedings of the 13th European Symposium on Artificial Neural Networks (ESANN 2005), pages 259–266, Bruges, Belgium, April 2005. "Archived copy" (PDF). Archived from the original (PDF) on 2007-06-13. Retrieved 2008-02-11.{{cite web}}: CS1 maint: archived copy as title (link)
^ Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun and Frank Kirchner. Performance Evaluation of EANT in the RoboCup Keepaway Benchmark. In Proceedings of the Sixth International Conference on Machine Learning and Applications (ICMLA 2007), pages 342–347, USA, 2007 [4]
^ Nils T Siebel and Gerald Sommer. Evolutionary reinforcement learning of artificial neural networks. International Journal of Hybrid Intelligent Systems 4(3): 171–183, October 2007. [5]
^ Nils T Siebel and Gerald Sommer. Learning Defect Classifiers for Visual Inspection Images by Neuro-evolution using Weakly Labelled Training Data. Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2008), pages 3926–3932, Hong Kong, China, June 2008. [6].

External links

BEACON Blog: What is neuroevolution?

[1] Peter J Angeline, Gregory M Saunders, and Jordan B Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5:54–65, 1994. [1]

[2] NeuroEvolution of Augmented Topologies (NEAT) by Stanley and Miikkulainen, 2005 [2]

[3] Yohannes Kassahun, Mark Edgington, Jan Hendrik Metzen, Gerald Sommer and Frank Kirchner. Common Genetic Encoding for Both Direct and Indirect Encodings of Networks. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), London, UK, 1029–1036, 2007.[3]

[4] Yohannes Kassahun and Gerald Sommer. Efficient reinforcement learning through evolutionary acquisition of neural topologies. In Proceedings of the 13th European Symposium on Artificial Neural Networks (ESANN 2005), pages 259–266, Bruges, Belgium, April 2005. "Archived copy" (PDF). Archived from the original (PDF) on 2007-06-13. Retrieved 2008-02-11.{{cite web}}: CS1 maint: archived copy as title (link)

[5] Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun and Frank Kirchner. Performance Evaluation of EANT in the RoboCup Keepaway Benchmark. In Proceedings of the Sixth International Conference on Machine Learning and Applications (ICMLA 2007), pages 342–347, USA, 2007 [4]

[6] Nils T Siebel and Gerald Sommer. Evolutionary reinforcement learning of artificial neural networks. International Journal of Hybrid Intelligent Systems 4(3): 171–183, October 2007. [5]

[7] Nils T Siebel and Gerald Sommer. Learning Defect Classifiers for Visual Inspection Images by Neuro-evolution using Weakly Labelled Training Data. Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2008), pages 3926–3932, Hong Kong, China, June 2008. [6].

[1]

[2]

[3]

[4]

[5]

[6]

[7]