Discovering Hierarchy in Reinforcement Learning

We are relying more and more on machines to performtasks that were previously the sole domain ofhumans. There is a need to make machines more self-adaptable and for them to set their own sub-goals.Designing machines that can make sense of the worldthey inhabit is still an open research problem.Fortunately many complex environments exhibitstructure that can be modelled as an inter-relatedset of subsystems. Subsystems are often repetitivein time and space and reoccur many times ascomponents of different tasks. A machine may be ableto learn how to tackle larger problems if it cansuccessfully find and exploit this repetition.Evidence suggests that a bottom up approach, thatrecursively finds building-blocks at one level ofabstraction and uses them at the next level, makeslearning in many complex environments tractable.This book describes a machine learning algorithmcalled HEXQ that automatically discovershierarchical structure in its environment purelythrough sense-act interactions, setting its own sub-goals and solving decision problems usingreinforcement learning.

Autorentext

Bernhard Hengst is Research Group Manager at NICTA in Sydney and
holds a conjoint appointment at the University of New South
Wales. His research interests include machine intelligence and
robotics. Previously Dr Hengst was General Manger at Ferntree
Computer Corporation in Australia. He holds a PhD from the
University of New South Wales.

Klappentext

We are relying more and more on machines to perform
tasks that were previously the sole domain of
humans. There is a need to make machines more self-
adaptable and for them to set their own sub-goals.
Designing machines that can make sense of the world
they inhabit is still an open research problem.
Fortunately many complex environments exhibit
structure that can be modelled as an inter-related
set of subsystems. Subsystems are often repetitive
in time and space and reoccur many times as
components of different tasks. A machine may be able
to learn how to tackle larger problems if it can
successfully find and exploit this repetition.
Evidence suggests that a bottom up approach, that
recursively finds building-blocks at one level of
abstraction and uses them at the next level, makes
learning in many complex environments tractable.
This book describes a machine learning algorithm
called HEXQ that automatically discovers
hierarchical structure in its environment purely
through sense-act interactions, setting its own sub-
goals and solving decision problems using
reinforcement learning.