Open-Ended Learning Robots: Overview of ISTC-CNR Research in the Project GOAL-Robots

We overview the research carried out by the Laboratory of Computational Embodied Neuroscience, a research group of the Institute of Cognitive Sciences and Technologies, National Research Council of Italy (LOCEN-ISTC-CNR: www.istc.cnr.it), within the EU funded project Goal-based Open-ended Autonomous Learning Robots (GOAL-Robots: www.goal-robots.eu; Start: 11/2016; End: 10/2020). Open-ended learning [1] is inspired by learning in organ- isms [2], and in particular by learning in children [3], that aims to build robot controllers able to autonomously acquire skills (policies/motor trajectories) driven by mechanisms such as intrinsic motivations – IMs (social learning is also important but for the sake of focusing it is not considered by the project). IMs [1] [2] are mechanisms able to drive the acquisition of knowledge and skills without the need of human guidance. There are three main classes of IM mechanisms [4] [5]: (a) novelty-based IMs: these detect patterns that have been rarely or never experienced; (b) prediction-based IMs: these detect patterns that are ‘surprising’, i.e. violate the prediction of the agent’s world models; (c) competence-based IMs: these detect if the agent is improving its ability to accomplish a certain task. These mechanisms produce signals [6] usable to select behaviours that increase the signals themselves, hence driving the robot to learn novel or surprising patterns or to increase competence, or to guide the learning of such behaviours, in particular with reinforcement learning (RL). The concept of the GOAL-Robots project [7] is that robots should autonomously acquire skills not by directly using IMs to drive behaviour and learning, but rather to self-generate (intrinsic) goals, defined as the internal representation of a desired state (or trajectory) of the world, possibly in an abstract fashion. Once generated, goals can be used to generate learning signals that can be used to drive the learning of the skills leading to accomplish them. Suitable control architectures are needed to learn and re-use multiple goals and skills. Here we expand the issues of the autonomous acquisition of goals, the learning of skills with them, and the architectures needed to acquire and re-use multiple goals and skills.