(Embodied) Artificial Intelligence: making robots curious

The following video describes how, active vision and embodied artificial inteligence contribute to the ROMI platform. Funded by EU Grant 773875.

Videos are available in:

• Hi-res (1080p Apple ProRes)
• Mid-res (1080p H265)

Video script:

(MINCHIN) At Humboldt University we see how ‘embodied artificial intelligence’, ‘computer vision’ and ‘active vision’, have been used to create feedbacks between an environment and the robotic tools in motion.

(HAFNER) Our big goal is to understand intelligence, and to understand the principles of intelligence in natural systems like humans and other animals, and to extract these principles and put them into algorithms and put and test them in robots. We think that not everything can be pre-programmed into the robot, but the robot has to learn and make experiences by the interaction of the real world. So I can show that here, so the robot is like a child randomly moving its arm around, and at the same time learning the correlation between the motor commands here and the position of the hand as the robot can see, by its own, by its own camera. That that approach is called ‘embodied artificial intelligence’ this adaptive approach is very much important for the ROMI project where we're partners in, because we can't program everything just by hand because the the conditions change you have difference, you don't know how the how the area looks like the maybe even the robot hardware changes, the motor changes and so we can use our adaptive methods and apply them to the ROMI robots so that they can really interact with the with the plants and extract information from the plants.

(SCHILLACI) We have a system that implements a sort of artificial curiosity. The artificial curiosity drives the movement of the robot towards interesting locations. For instance one of our tasks is to support the 3d reconstruction of plants, and instead of using predefined movements of the 3d scanner around the plants, our system instead is trying to to make more intelligent movements to to discover parts of the object that perhaps are more interesting to look at.

So we have the learning system running at the moment, so we have the artificial curiosity explorer based exploration that sends motor commands and at the same time this, so all the information that are recorded from from the robot which are images which you can see here, and the positions of the robot are sent to the model which is going to be trained. So you probably have a goal that is like; look at the tracks, like put the tripod at the centre of the image so that's probably one of the goal. And the idea is that the next step will be like to have to move around this goal so as soon as your system is well trained, so this goal could be could be changing, for instance because the plant is growing, so you need to to move these goals around to make the system adaptive. Tt's coming back again to the tripod this one.

(HAFNER) In ROMI it's a very challenging task because we have to we have to cope with this these real systems and we have to cope with with wind and with weather and the plants growing unexpectedly and so it's very challenging, but it's also very fascinating.

(SCHILLACI) So the next step will be having more movement capabilities, so you will be able to segment out parts of the images that you don't need. Antonio is actually working on segmenting out the interesting objects from the from the scene, and in particular also to avoid the arm to crash into plants. So you you take first a screenshot from from a camera that is located at the top of the robot and then so you segment out all the objects that you don't need, and the object also where the robot could crash so you your trajectories will be going around plans do not crash onto them.