The development of action repertoires for humanoid robot behavior
Keywords
No Thumbnail Available
Authors
Issue Date
2019-02-21
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The predictive processing account of neuroscience views the brain as a prediction-making
and hypothesis-testing machine based on a hierarchically-organized series of generative models.
The brain makes predictions about the world, and then seeks to resolve the error between them
and observed reality. One of the ways in which it may do so is denoted as active inference: by
acting on the world based on a model describing the actions’ effects such that the observed
state now matches a wanted state. How these models are formed and adapted during infancy,
however, is not yet known. In this exploratory study we investigated a possible computational
strategy for creating an action repertoire for use in a humanoid robotic agent, adapting it to
become more or less fine grained, whilst transferring previously learnt information. For this, we
used an implementation based on clustering and Q-learning, and transferring previously learnt
Q-values between clusterings of varying granularity. We tested its performance in a virtual
reaching task based on visual and motor data gathered from a NAO robot. Our final results
show that our proposed method of Remapping of Q-values ("ReQ-ing") allows the agent to
learn fine-tuned actions to its performance plateau faster, but at the same time, doesn’t allow
the agent to learn to a lower error over-all compared to those agent that do not use this method.
Other observations are that reward functions should also be properly adapted to predispose
a more fine-tuned model, and that the addition of additional axes of control should also be
reflected in clustering and an agent’s reward function.
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen