A mechatronic platform for behavioral analysis on nonhuman primates

*Laboratory of Biomedical Robotics and Biomicrosystems Universit! a Campus Bio-Medico di Roma 00128 Rome, Italy †Biorobotics Laboratory, " Ecole Polytechnique F"ed"erale de Lausanne CH-1015 Lausanne, Switzerland ‡Masmec spa, 70026 Modugno (Ba), Italy Unit of Cognitive Primatology Institute of Cognitive Sciences and Technologies, CNR 00197 Rome, Italy Laboratory of Computational Embodied Neuroscience Institute of Cognitive Sciences and Technologies, CNR 00197 Rome, Italy ||Laboratory of Developmental Neuroscience Universit! a Campus Bio-Medico di Roma 00128 Rome, Italy **f.taffoni@unicampus.it


Introduction
The term behavioral sciences encompasses all the disciplines that explore the activities and the interactions among organisms in the natural world.It involves the systematic rigorous analysis of human and animal behavior through controlled experiments and naturalistic observations (Klemke et al., 1980).Behavior is anything that a person or an animal does which can be observed and measured.In particular, animal behavior analysis is the scientific study of the ways in which animals interact with each other, with other living beings, and with the environment.It includes topics such as how animals find and defend resources, avoid predators, choose mates, reproduce, and care for their offspring.
Among animal models, nonhuman primates show several highly complex behavioral patterns that share fundamental parallels with human primates.These parallels include highly developed cognitive abilities and complex social relationships.For this reason, they are often the subject of comparative studies on learning, memory, information processing, social behavior, sensory functioning, visual-motor coordination, and/or visuospatial orientation (Tomasello & Call, 1997).There are several approaches to study animal behavior (Martin & Bateson, 1998).Especially in the past, while psychologists focused on the proximate causation of behavior and general processes of learning in a few animal species, namely those better adapted to laboratory conditions, ethologists were typically interested in studying the ultimate causation of behavior; particularly in nature where spontaneous behavior and the role played by environment could be better appreciated.Nowadays, these two fields are more integrated and neuroscience successfully contributes to clarify the neural correlates of behavior (Wasserman & Zentall, 2006).All the above disciplines require precise methods and tools for quantitative assessment of behaviors, possibly monitoring different levels of analyses, so as to integrate them.
Several studies on nonhuman primates have used observational and experimental techniques.See Visalberghi et al. (2009) and di Sorrentino et al. (2010) for specific examples concerning capuchin monkeys, the target species of this article.This methodology is suitable for use in wild environments, however, it is time-consuming as it is often based on manual scoring of the observations.Moreover, it allows the encoding of only a subset of behaviors usually measured in terms of the number of such acts or the amount of time engaged in that behavior.To allow long-term monitoring of physical activity, wearable systems based on inertial technologies have been developed and used.Such systems are composed of omnidirectional wireless accelerometers, usually embedded into collars (Mann et al., 2005;Munoz-Delgado et al., 2004;Papailiou et al., 2008;Hunnell et al., 2007) and allow for effective quantification of whole body movement in monkeys but not for arm/hand movements which could alter the final result (Mann et al., 2005).
To study particular behaviors in a semi-automatic way, it is very common to develop ad-hoc apparatus or structure the environment with different kinds of sensors.In Levin et al. (1986) for example, an apparatus was constructed to study visual exploration in infant rhesus macaques.The developed apparatus consisted of an instrumented two-chamber box with a peephole at each end.The position of the monkey in the test chamber was monitored by contact relay circuits wired to the stainless bars on the floor, while time spent looking out of the peephole was measured by infrared sensors mounted so that whenever the monkey looked out its head would interrupt the photo-beam.In Yamamoto & Tanaka (2010), an apparatus to test selfish versus prosocial behaviors has been developed.The apparatus consists of three buttons equipped with three flash bulbs of different colors: one button was the start key, flashing green, and the other two, flashing red and orange, were used for delivering food rewards to their partner and themselves, or only to themselves (buttons that flashed red and orange were assigned either to the selfish option or to the prosocial option).These systems allow the study of particular behaviors but they cannot be easily reconfigured.
To allow reprogrammability of semi-automatic system for behavioral analysis, computerized apparatus are often used.The Language Research Center's Computerized Test System (LRC-CTS) (Richardson et al., 1990) for example, was originally devised to provide individually housed rhesus monkeys with 24-h access to computerized tasks (the equipment was contained within clear Lexan enclosures).The test system has since been used to study many psychological processes, including attention, categorization, memory, numerical judgment, spatial cognition, self-control, and uncertainty monitoring (Washburn & Rambaugh, 1992), and it has also proved to be usable with socially housed nonhuman primate species (Evans et al., 2008;Fagot & Bont, 2010).It comprises a general-purpose computer, a color display monitor, a digital gamepad/joystick, external speakers, and a pellet dispenser linked to a digital I/O board within the computer through a solid-state relay board.All tasks and utilities are written in QuickBasic language and can be modified or added to.
Even if computerized systems allow a certain level of reprogrammability, they limit the possibility of interaction: subjects can interact with the apparatus interfaces (joystick and buttons), but they are outside the cage or mounted in such a way to avoid any possible improper interaction.It is not easy to modify or change the affordance of the interface and it is necessary to have knowledge of a programming language to change the experimental protocol.In this work, we present a new mechatronic platform for measuring behavior of nonhuman primates thus allowing high reprogrammability and providing several possibilities of interactions with subjects.Its modularity and reprogrammability makes this platform a multipurpose experimental set up.Even if it was designed for semiautomatic testing of capuchin monkeys (Cebus apella), it can be used for behavioral studies of other nonhuman primates and children.

The Mechatronic Platform
Mechatronics is a natural stage in the evolutionary process of modern engineering design.A mechatronic system is defined as the synergistic integration of mechanical engineering, with electronics and intelligent computer control in the design and manufacturing of products and processes (Harshama et al., 1996).In this section, we discuss the details of the design and development of a mechatronic platform for behavioral analysis on nonhuman primates.A similar version with different dimensions and materials has also been developed for children to allow comparative studies.
The main advantage of a mechatronic approach is the possibility to change and reprogram the platform to satisfy different experimental requirements.In particular, we have focused our attention on the possibility to change how the platform responds to the interaction with monkey (actionÀoutcome relationship) to investigate learning mechanisms based on intrinsic motivations, and action recall.Intrinsic motivations (IM) have been first described by psychologists (Harlow, 1950) to explain motivational and learning processes that could not be accounted for on the basis of the behaviorist framework of homeostatic regulations, drives, and extrinsic rewards (e.g., food, pain, sex).For example, IM can explain why animals persevere in solving puzzles in the absence of extrinsic rewards (Harlow, 1950), why they engage longer with complex, unexpected, or in general surprising objects (Berlyne, 1966), or why they can be motivated to perform actions that have a strong impact (effectance) on the environment (White, 1959).In general, as argued in detail in Berlyne (1966), IM have the function of driving the acquisition of generalpurpose knowledge and skills that can later be used to accomplish fitness-enhancing useful tasks (impacting the visceral body and its homeostatic regulations), although these fitness enhancements are not present at the moment of the acquisition of the skills and knowledge themselves.Notwithstanding the importance of IM, there is still a lack of understanding of how in detail they drive the acquisition of new skills and knowledge and how these are exploited in a later stage.
It seems that a crucial role in learning processes is played by dopamine which promotes exploration and it is related to the level of curiosity and interest (Panksepp, 1998).Dopamine is thought to influence behavior and learning through two, somewhat decoupled, forms of signal: phasic (bursting and pausing) responses and tonic levels (Grace, 1991).What is important is that a set of experimental evidence shows that dopamine activity can result from a large number of arousing events including novel and unexpected stimuli (Hooks & Kalivas, 1994;Schultz, 1998;Fiorillo, 2004).

Functional and technical specifications
The mechatronic platform for behavioral analysis of nonhuman primates should be modular and easily reconfigurable, allowing us to customize the experimental setup according to different protocols and to deliver novel and unexpected stimuli.For this reason, it should be provided by instrumented interchangeable objects (mechatronic modules) eliciting different kinds of manipulative behaviors (e.g., rotations, pushing, pulling, repetitive hand movements, button pressing, etc.).These objects should allow to record synchronized multimodal information for behavioral analysis and provide different kinds of complex stimuli: visual, acoustic, and tactil.According to typical experimental protocols, the platform should be also provided by a mechanism for food dispensing (reward mechanism).Finally, it should be made by materials, mechanisms, and electronic components robust enough to resist typical monkey actions (e.g., hitting, rubbing, biting) and avoiding any potentially dangerous interaction.
To easily reconfigure the experimental setup responding to the requirements detailed above, a hierarchical three-level control architecture was chosen (see Fig. 1).The physical level is made by interfaces the subjects can directly interact with: modules and rewarding mechanisms.This level is mechanically and electronically decoupled by other higher levels allowing, on one hand, an easy change of mechatronic modules, while on the other hand, an improvement of the robustness of the apparatus.The microcontroller-based middleware level control manages low level communication with mechatronic modules, reward mechanisms, and audiovisual stimuli while the high level control is a control program running on a remote laptop which allows supervising the acquisition and programming the arbitrary association between action and outcome.

Hardware and software development
The mechatronic board is composed of two main parts: (i) a planar base, into which a set of interchangeable mechatronic modules is plugged; (ii) a reward releasing unit.The two parts are independent and can be easily separated to facilitate their transport.The current version is shown in Fig. 2.
The planar base (overall dimensions: 800 Â 600 Â 200 mm) is provided of three slots where different mechatronic modules (in this version, three simple pushbuttons), identified by a unique hardware address, can be easily plugged in.Each module has a specific set of optical sensors which separate electronics from moving parts, allowing a safety recording of quantitative data on interaction.
The reward releasing unit (800 Â 200 Â 400 mm) is mounted on the back area of the planar base and contains the reward boxes where small objects or food reward are placed by the experimenter by means of an opening on the rear face.Boxes are closed in the frontal part by a sliding door made of transparent material so that the subjects can always see what is inside them.The reward system is conceived such that the subject can retrieve the reward only when he/she performs the correct action on the mechatronic module(s), otherwise the box remains closed (see Fig. 3).
Several sources of multimodal stimuli (acoustic and visual) are distributed on the board to provide various sensory feedbacks associated with the manipulation of mechatronic objects.The stimuli come both from the mechatronic objects (object stimuli) and from the reward releasing boxes (box stimuli).The acoustic stimuli can be chosen among a database of both natural and artificial sounds and delivered from six different independent sources.The visual stimuli consist of a set of 21 independent multicolored lights: red, white, and blue.The actions on the mechatronic objects activate the audio-visual stimuli and the opening of the reward box(es), as defined by the experimental protocol.A local wide-angle camera fixed on the top of the reward releasing unit, allows recording videos of the workspace during the experiments.
The actionÀoutcome association can be reprogrammed by the experimenter with a high level interface (see Fig. 4).The programming window is logically split in three parts.In the upper part on the left, experimenters can select the action and the slot where the action is performed whereas on the right, the experimenter can select the outcome and where the outcome will be delivered.The selected actionÀoutcome relationships are listed in the bottom part of the window.In the example reported in Fig. 4, a button pressed action on the module plugged in slot 1 was selected and the three central lights of the reward-releasing units programmed to be switched on when the selected action is performed.
The described interface is part of a control software developed in LabVIEW which allows to manage and control the experimental variables and the acquisition of behavioral data.Data gathered with this software allow automatic scoring of: latency to first exploration of each stimulus and latency to first exploration of each affordance; task persistence (i.e., the time each participant manipulates the object); richness of investigation (i.e., number of different actions performed on the objects as well as the number of times an effect À À À e.g., sound, light À À À is produced).Moreover, additional information were collected by the videocamera embedded in the board synchronized with an external camera: subject orientation (extent to which the subject draws the face near to the boxes), total time in physical contact with the board, use of mouth and hands to explore the board and frequency of behavioral measures of arousal (i.e., scratching).All the above variables provide a complete and fine-grained picture of the subject's exploration of the board affordances and its problem-solving learning abilities.This complex set of stimuli and the possibility to change their relationship with subjects' actions enable the investigation of intrinsic motivation learning.In particular, it is possible to test the effect of multimodal stimuli on learning processes intrinsically motivated performing two phases protocols: in a first training phase, subjects are exposed to the board without any food reward, they simply have to explore the board and their exploration should be promoted by the \novelty effect".The goal in this phase is to learn the relationship between modules and boxes.How much animals learn this relation is tested in a second test phase, where they have to apply the learnt relationship to retrieve a food reward from the boxes.
Even if the board is currently equipped with a set of push-button modules, it is also possible to use new additional objects.We have designed and manufactured, for example, a set of three complex mechatronic modules that we are going to use for comparative studies with children and monkeys.The first \complex" mechatronic module, called Circular Tap (see Fig. 5(a)), assesses rotations and vertical translation.In particular, the latter action should be very natural for monkeys because usually it is performed to break nuts.The second one called Fixed Prism (see Fig.  be direct if the subject rotates the central cylinder or translates it using the horizontal handle, or mediated by a inner mechanism, with the rotation of a lateral wheel that is converted to a horizontal translation of the cylinder along its main axis (see Fig. 5(b)).

Preliminary Experiments with the Mechatronic Platform
Here, we provide an example of in-field use of the above mechatronic board with a New World primate species, the tufted capuchin monkey (Cebus apella).The example The pilot study is aimed at checking the functioning of the board with capuchin monkeys, a species well known to be manipulative when dealing with objects and food items (Fragaszy et al., 2004).During the pilot, systematic data were collected on the monkeys initial response to the mechatronic platform and the time spent manipulating the buttons (see above).
This pilot study is part of the research project Intrinsically Motivated Cumulative Learning Versatile Robots (IM-CLeVeR) that aims to develop a new methodology for designing robots controllers that can cumulatively learn new efficient skills through autonomous development based on intrinsic motivations, and reuse such skills for accomplishing multiple, complex, and externally assigned tasks.The data presented here refer to the button condition that preceded the use of the mechatronic objects and whose actionÀoutcome associations were assumed to be less demanding for monkeys to learn, as compared to mechatronic modules which present more affordances.

Experimental protocol
The subjects of the pilot study were three adult capuchin monkeys hosted at the Primate Centre.Capuchins were tested individually in an indoor enclosure (5 m 2 Â 2:5 m high).Each subject was separated from the group solely for the purpose of testing, just before her/his testing session.Subjects were not food deprived and water was freely available at all times.The board had three buttons of different colors (white, black, and red), placed at about 25 cm apart from one another along the same line, that could be discriminated by trichromatic and dichromatic subjects (capuchin monkeys male are all dichromats, whereas females could be either dichromats or trichromats, (Jacobs, 1998)).The pressure of each button produces a specific combination of audio and visual stimuli along with the opening of one of the three boxes.The pilot experiment included two phases.In Phase 1, the correct action performed by the subject (i.e., pressing a button at least once) produced a specific combination of audio and visual effects together with the opening of one box.The box did not contain any reward.Phase 1 lasted for 20 min.In Phase 2, the reward (one peanut kernel) was located in one of the three boxes in clear view of the subject.The reward could be obtained by pressing the associated button.Each subject received nine trials and the reward position was balanced across boxes.Phase 2 ended after nine trials or when 40 min elapsed, whichever came first.
For all subjects, the white button (WB) opened the central box (CB), the black button (BB) the left box (LB) and the red button (RB) the right box (RB) (see Fig. 6).Thus, the spatial relation between button and associated box was crossed for WB and BB and frontal for RB.The pilot experiment was videotaped by a camera (Sony Handycam, DCR-SR35) and by the camera embedded in the board.The ELAN software allowed synchronizing the videos obtained by the two cameras.

Phase 1
Two subjects contacted the board within a few sec (subject 1, 6 s and subject 3, 37 s) whereas subject 2 took much longer (6 min and 27 s).Subject 1 performed her first pressing directed toward a button 1 min and 15 s after the beginning of the trial, whereas the other subjects never did it.Subject 1 pressed all the buttons at least twice, for a total of 14 pressings.Her average time during which she held the button pressed was 0.17 s (SE 0.008).The overall mean time in contact with the board was 5 min and 5 s and the value varied across subjects (subject 1: 10 min and 38 s; subject 2: 3 min and 55 s; subject 3, 3 min and 11 s).Boxes close distance exploration (within 10 cm) never occurred for subject 1, whereas subject 2 did it once and subject 3 eight times.

Phase 2
Seeing a reward in one of the boxes prompted the subject's attention towards it and increased his/her motivation to manipulate the board.Capuchins readily visually explored the baited box; this behavior was much more frequent than in the previous phase (subject 2, 170 times; subject 3, 132; and subject 1, 20).Table 1 shows, for the three box-button associations, the number of times each button is pushed, the mean of incorrect responses before pushing the correct button, and the mean holding time of each button.Overall, the frontal association (right box-red button) had a mean number of errors similar to the left box-black button crossed association, whereas the other crossed association (central box-white button) scored a higher level of errors (see also Fig. 7).The black button located in the central position (operating the left box) was pressed almost twice the number of times the other two buttons, thus increasing the probability to open the left-box.Consequently, the comparison between frontal and crossed associations should be carried out by  Electronic version of an article published as Journal of Integrative Neuroscience, Vol.11(1), 2012, 87-112, DOI: 10.1142/S0219635212500069 © Imperial College Press http://www.worldscinet.com/jin/comparing the performances in the right and in the central box.Since the mean number of errors per trial per subject was 1.2 (right box) and 3.7 (central box), we suggest that spatial proximity plays a primary role in learning an association between action and outcome.

Conclusion
In this work, we present a new mechatronic platform for semi-automatic assessment of behavior.The presented platform is the result of a multidisciplinary design process involving bio-engineers, developmental neuroscientists, primatologists, and roboticians.To define the main characteristics of this new platform, a state-of-theart analysis of the main techniques available for nonhuman primate behavioral studies was carried out and discussed in the Introduction.The use of only observational and experimental techniques have been challenged because they are timeconsuming and allow the encoding of only a subset of behaviors usually manually scored in terms of the number of such acts or the amount of time engaged in that behavior.To measure a higher number of behaviors using semi-automatic system of data scoring, wearable system and structured environment are also often used.All these systems do not allow to define an easily reconfigurable experimental protocol: a promising techniques in this sense seems to be represented by computerized apparatus.Despite state-of-the-art computerized apparatus allowing a high level of reprogrammability which partially addresses the main drawbacks of the other nonhuman primates behavioral analysis technique, the possibility of interaction are reduced due to the small set of animal-interfaces available for test (usually joystick or buttons) and the difficulties to modify the experimental protocol without language programming knowledge.This is not the case of the presented mechatronic platform.It has been designed to guarantee the higher possible level of reprogrammability both hardware and software: user interfaces can be easily changed and the actionÀoutcome matrix modified by means of a graphical user interface which does not require any programming language knowledge.Moreover, the mechatronic platform could be put inside the test cage, promoting a more natural interaction with respect to the other computerized systems.
A detailed discussion on main features of the platform has been reported and an example of its in-field use with a New World primate species, provided.Preliminary data seems to suggest that this platform can be effectively used for nonhuman primates behavioral analysis.Despite the pilot study being carried out using a platform equipped with pushbutton modules, more challenging mechatronic objects with different possibility of interaction and affordances have been designed and will be used with monkeys and children for comparative studies.In the design of the platform, we paid great attention in choosing commercial components that could be easily found to ease replicability.Detailed mechanical and electrical drawings as well as software and firmware are available upon contacting the corresponding author.

Fig. 1 .
Fig. 1.Functional concept of the mechatronic platform: Reward/Stimuli (R/S) modules are physically separated by instrumented objects on the base.Relationship between objects and reward/stimuli modules are managed by a local reprogrammable control unit.

Fig. 2 .
Fig. 2. The mechatronic board equipped with three pushbuttons.The circular holes under reward boxes and in front of each pushbuttons are for acoustic (black ones) and visual stimuli.
5(b)), allows to assess horizontal rotation (rubbing) and translation.The third one, called three-degree-of-freedom cylinder (3 Dof cylinder), allows interaction with three different affordances (see Fig. 5(c)).The effect of interaction can

Fig. 4 .
Fig. 4. Action-outcome relationship window: in the upper part experimenter can select the action (on the left) and the related outcome (on the right).

Fig. 5 .
Fig. 5. Mechatronic modules.(a) Circular tap: overall layout and a detail of encoder electronics for rotation measurement.(b) Fixed prism: the frontal wall has been removed allowing to see inner mechanism.(c) 3 Dof cylinder: overall layout on the left, degree of freedoms on the right.

February
Fig. 6.(a) Disposition of buttons and their association with boxes from the embedded camera perspective.(b-c) Experimental trial: (b) exploration, (c) reward dispensing.

Fig. 7 .
Fig. 7. Mean number of incorrect pushes (per subject per trial) performed while the reward was in the left box, in the central box, and in the right box (x-axis).

Table 1 .
Association between boxes and buttons.