Bayesian modeling of a sensori-motor loop
An application to handwriting
PhD thesis of Estelle Gilet (2009)
How can the motor system influence perception ?
How can the same movements be performed with different effectors ?
In order to examine these questions, which are relevant in cognitive sciences, we studied the sensorimotor system involved in tasks of reading and writing isolated handwritten cursive letters.
Our contribution is a formal model of this sensorimotor loop (BAP – Bayesian Action Perception) , that takes into account experimental findings as well as state-of-the-art theories.
Our main hypothesis is the existence of an internal encoding of letters that is common to both perception and action. This encoding is assumed to be independent of the space of the effectors’ joints. It acts as a pivot between perception and action models, and is made of sequences of geometrically salient control point. These are extracted from trajectories of handwritten letters and allow to recover part of the kinematic information.
The formalism we used in this work is called Bayesian Programming. It allows to represent and manipulate incomplete and uncertain information using probabilities ; models are defined by specifying joint probability distributions.
Given the joint probability distribution, the BAP model can solve several cognitive tasks. Each of these corresponds to a probabilistic question that is automatically solved by Bayesian inference : reading, reading with an internal simulation of the writing gesture, trace copy and letter copy, writer recognition, writing using different effectors. We have implemented and simulated each of these probabilistic inferences.
Our results reproduce observations from psychophysics experiments. Our model also suggests new experimental protocols, which we defined.