H. Kimura (National University of Electro-Communications, Japan)
How does robotics contribute to biomechanics and neuroscience beyond the engineering limitations?

Recently, a lot of researchers support the concept such that locomotion could be generated emergently by the non-linear dynamic system constructed by the neural system and the musculo-skeletal system through the interaction with the environment. In this concept, the dynamic properties of the coupled non-linear dynamic system should been investigated while considering the dynamic properties of both neural system and musculo-skeletal system.

Roughly speaking, the neuroscience and the biomechanics have been investigating the neural system and the musculo-skeletal system, respectively, with less collaboration. But once the neuroscience and the biomechanics collaborate and try to verify their hypotheses by computer simulation, those studies can be involved in robotics if we ignore the engineering limitations. In this sense, the robotics could be a cross-bridge between the neuroscience and the biomechanics. Since the studies of the neuroscience and the biomechanics should be faithful to the biological data, it seems to be difficult to succeed even in simulation of simple walking. On the other hand, it is impossible to realize the neuro-musculo-skeletal system faithful to the biological data under the current engineering limitations about actuators, sensors, mechanism, and so on. Therefore, what we can or should do in robotics at this moment could be to ignore the details, implement the concepts which seem to be important, and investigate whether those concepts work well or not.

The most of people who support the concept of emergent locomotion employ CPG (Central Pattern Generator) and reflexes for motion generation and adaptation at the lower level. In this talk, I would like to introduce several studies on emergent legged locomotion, and also introduce biologically inspired hypotheses about legged locomotion from the view point of robotics while considering the basic knowledge in the neuroscience and the biomechanics.

S. Schaal (University of Southern California, USA):
Learnable pattern generators for limb control.

Since more than 100 years ago, researchers have accumulated neuroscientific, behavioral, and theoretical evidence that motor control in biology is organized in terms of dynamic pattern generators. While this idea has found largely acceptance for insects and lower vertebrates, it has remained hard to extend the pattern generator concept to limb movements in primates. In this talk, we describe a line of research conducted over the past year, that provides behavioral, imaging, and modeling evidence that the concept of pattern generators can indeed be extended to limb movement. Among the key finding was the development of a model of learnable pattern generator, i.e., a principled method to adjust parameters of a nonlinear dynamic systems to form almost arbitrary attractor
landscapes. We introduce this model in the context of imitation learning, movement recognition, and reinforcement learning. Several synthetic and humanoid robotics examples will demonstrate the viability of the pattern generator hypothesis. This is joint work with Auke Ijspeert and Jun Nakanishi.

A. Ijspeert & A. Crespi (EPFL, Switzerland):
Neurorobotic models of lower vertebrate locomotion: towards a salamander-like swimming and walking robot.

In this talk, we will present recent results of a project involving the construction of an amphibious salamander-like robot. The goal of the project is to investigate the spinal mechanisms underlying gait generation and modulation in the salamander. The salamander is an amphibian capable of swimming and walking, and represents, among vertebrates, a key animal in the evolution of legged locomotion. Based on recent findings on the salamander nervous system, we develop models of the salamander's central pattern generator for locomotion. In particular, we develop models based on coupled neural oscillators capable of producing the salamander's typical swimming and walking gaits. Similarly to the real animals, the models require only simple input signals for generating coordinated movements of all degrees of freedom and for modulating the speed and direction of locomotion, as well as the type of gait. The models are tested in a dynamic simulation of the salamander body as well as in a robot currently under construction. The robot is a central element in the project since it allows us (1) to test whether the models are effectively capable of producing locomotion in water and on ground and (2) to investigate how sensory feedback affects the pattern generation.

A. Arleo, C. Boucheny, T. Degris, N. Brunel, & S. Wiener
(College de France, CNRS, University of Paris):
Head direction cells and spatial orientation in rats: Experimental findings and computational modeling.

Head direction (HD) cells constitute a likely substrate for the rat's sense of direction and are anatomically and functionally coupled to the hippocampal place cells, neurons that are critically involved in spatial learning. We study the HD cell system both experimentally (i.e. extracellular recordings) and theoretically (i.e. bio-inspired computational modeling). Two series of electrophysiological experiments are presented. The first focuses on the temporal aspects of state transitions of the HD cell system following reorientation of static visual cues. The second tests the hypothesis that HD cells might rely on dynamic visual signals (like motion parallax) to select
anchoring visual landmarks. Then, we present a model of the HD cell system based on a continuous attractor-integrator network and study how the self-sustained neural activity of HD cells might originate and how it might be dynamically updated based on self-motion inertial signals as well as visual information. The model, that permits to reproduce the presented experimental observations, has also been implemented on a mobile robotic platform.

.
P. Gaussier, N. Cuperlier, M. Quoy, J.P. Banquet, B. Poucet, & and M. Save
(ENSEA, France)
Understanding the interactions between the hippocampus, the prefrontal cortex and the basal ganglia: Facts and models.

Roboticians have always tried to find new architectures allowing autonomous robots to behave in a more "intelligent" manner for a given task or set of tasks. In this context, the discovery of correlations between the activity of neurons in the rat hippocampus (HS) and the location of the animal in a given environment has given rise to numerous question from roboticians and specialists in Artificial Intelligence: Is the hippocampus an equivalent of the cartesian maps used in most of the robot navigation systems ? How is it compatible with the ideas defended by the behavior base approach saying that the world is its own best memory ?

Besides, studies on human have shown HS is also very important in tasks requiring a declarative memory. Hence both neuroscientists and roboticians evoke the possibility HS may have different functional roles according to the different species or the nature of the tasks for a given species. In our lab. we have used mobile robots equipped with an active vision system to try to model neurons having "place cell" like activity. We were surprised to see how a simple mechanism merging the "what" and "where" informations extracted from the visual flow allows high-performance homing behaviors in simple open environments (arena like environments). We showed learning to associate the recognition of 3 places in the neighborhood of a given goal location to their corresponding actions to reach that goal was sufficient to create a kind of attraction basin allowing our robot to reach the goal from any starting point in the experimental room. Hence the question was: why does the rat need to learn so many places in an open environment and why the place fields are so small as compared to those found in the robotic experiments? Navigation experiments in maze like environments have shown us the necessity to differentiate "place cells" from "place transition cells" that can be an important element to link "place cell" activity, with action plans and with the final triggering of a specific motor action.

An ongoing projects with neurobiologists to test this idea and how it is connected to planning capabilities does not allow to conclude definitively about the correctness of the model but has risen new interesting questions allowing to refine the model and to build new biological experiments. In this talk, we will discuss some of the issues regarding our model and the related neurobiological experiments. We will try to propose a single model explaining the different data and showing how the hippocampus (HS), the prefrontal cortex (PFC) and the basal ganglia (BG) may be connected in a network allowing to solve a wide variety of tasks. We will question the different kinds of loops that have been found in HS itself (recurrences in the CA3 region) or between HS regions (loop between the entorhinal cortex, the CA regions and the subiculum) and the circuit connecting HS, PFC and BG. We will show our model can scale up both to sparse coding and complex behaviors modelization and conclude about the remaining open questions.

J. Krichmar, J. Fleischer, D. Nitz, A. Seth, & G. Edelman
(The Neurosciences Institute, USA)
Spatial and episodic memory in a real-world device containing a model of hippocampal-cortical interactions.

K. Doya (ATR, Japan):
Cyber Rodents: Exploration of adaptive mechanisms for self-preservation and self-reproduction.

Animal reward and aversive systems are tightly bound with the fundamental requirements for biological systems, namely, self-preservation and self-reproduction. Different rewards and aversions can often be in conflict or trade-off, necessitating sophisticated decision and learning mechanisms. The goal of our Cyber Rodent project is to explore the adaptive mechanisms required for self-preservation and self-reproduction using artificial agents. Cyber Rodents are two-wheel driven rodent-like robots that can recharge
from battery packs scattered in the field and can copy their 'genes' (programs or parameters) through IR communication ports.

We report our prelimiary results of our simulations and experiments, such as evolution of meta-parameters of reinforcement learning, learning of 'mating' behaviors, and evolution of reward and cost functions.

R. Chavarriaga & W. Gerstner (EPFL, Switzerland):
Reward-based selection of navigation strategies.

Animals can adopt different navigation strategies according to the environment and the task they have to solve. Autonomous robots should also be able to scale up from simple reactive behaviours, to more complex navigational strategies (ie involving the use of a cognitive map). Neurobehavioral studies in rats support the idea that the process of selecting the appropriate navigation strategy (the one which maximises the reward) in mammals involves multiple parallel pathways working in parallel in a competitive way according to the situation in which learning occurs. The hippocampus has been thought to be the potential basis for a first type of navigation strategy in which a representation of the space is required. This representation uses spatially tuned neurons (place cells) found in the rat hippocampus. A second, different, navigation strategy that involves the dorsal striatum in the basal ganglia, is used by rats if the target can be identified by a visible cue. We present a computational model of the basal ganglia and its interactions with the hippocampus and sensory cortices, able to reproduce experiments regarding self-localisation, and automatic selection of appropriate navigation strategies.

W. Alexander (Indiana University, USA):
Mutual influences of environment and behavior on the development of a neural model

In this talk, a neural model of neuromodulatory systems implicated in theprocessing of appetitive and aversive events is discussed in the context of the ongoing interactions of environment, behavior, and the development of synaptic connections in the model. Under conditions in which the occurrence of rewarding events is controlled by the experimenter, the model learns to reproduce salient features of the mammalian dopamine system. During autonomous behavior, structural properties of the environment influence the development of the neural model. Over the course of extended experimental runs, the interaction of the agent with the environment produces changes in these environmental properties, resulting in alterations in behavior and patterns of synaptic development. The model's ability to reproduce the characteristic activity of the dopamine system is observed to depend both on properties of the environment as well as the agent's behavior within the environment.

O. Sporns & M. Lungarella (Indiana University, University of Tokyo):
Measuring the structure of sensory and motor data in neurorobotic systems

Organisms and robots are situated in specific environments that are sampled by their sensors and within which they carry out motor activity. Their control architectures, nervous systems or brains, receive, analyze and process streams of sensory data and ultimately generate sequences of motor actions. Sensory and motor activities are intricately linked through a continuous dynamic coupling to the surrounding environment. The quality of the sensory data relayed to the brain is critical for enabling appropriate developmental processes, perceptual categorization, adaptation, and learning. In this talk, we argue that the ability to actively structure and generate statistical regularities in their inputs represents a major functional rationale for the evolution and design of embodied systems. We introduce a set of quantitative univariate and multivariate statistical measures that can be used to characterize the structure of sensory and motor data. We illustrate the application of these measures in computer simulations and in a simple active vision robotic system. We discuss the potential importance of these measures for understanding sensorimotor coordination in organisms and for robot design.

S. Potter (Georgia Tech., USA)
Hybrots: Hybrids of living neurons and robots, for studying distributed neural dynamics

We grow cultures of neurons and glia from mouse cortex on multi-electrode arrays (MEAs). We have developed the hardware and software necessary for real-time feedback, to these neuronal networks, of stimuli that are based on the nets' own activity. We are interested in studying learning and memory in vitro, where it will be easier to observe the cellular and network-level changes that underlie information processing and storage, than in living animals. The thing that 'learns' in our case is a 'hybrot', or system comprised of the living net controlling a robotic body. By re-embodying the
network in a closed-loop system, we bring in vitro studies closer to those done in animals. We apply 2-photon laser-scanning microscopy to observe morphological dynamics at the minutes to days time scale. And we use both optical and MEA recordings to observe functional dynamics at the milliseconds to hours time scale.

http://www.neuro.gatech.edu/groups/potter/potter.html