Reinforcement Learning, Logic and Evolutionary Computation

Reinforcement learning (RL) consists of methods that automatically adjust behaviour based on numerical rewards and penalties. While use of the attribute-value framework is widespread in RL, it has limited expressive power. Logic languages, such as first-order logic, provide a more expressive framework, and their use in RL has led to the field of relational RL. This thesis develops a system for relational RL based on learning classifier systems (LCS). In brief, the system generates, evolves, and evaluates a population of condition-action rules, which take the form of definite clauses over first-order logic. Adopting the LCS approach allows the resulting system to integrate several desirable qualities: model-free and "tabula rasa" learning; a Markov Decision Process problem model; and importantly, support for variables as a principal mechanism for generalisation. The utility of variables is demonstrated by the system's ability to learn genuinely scalable behaviour - behaviour learnt in small environments that translates to arbitrary large versions of the environment without the need for retraining.

Autorentext

Drew Mellor completed his PhD in Computer Science at theUniversity of Newcastle, Australia. He is currently a ResearchAssociate at the Centre for Bioinformatics, Biomarker Discoveryand Information-Based Medicine, Newcastle. His interests centre on simulation and prediction of complex, natural systems through computational methods.