Feedforward emergence of perceptual gestalts in the human visual cortex

How does our visual system combine the features or parts in a complex display to provide a percept of a whole? Intense behavioral work in Gestalt psychology has described a number of phenomena where this whole is not equal to a mere sum of its parts. However, precise unifying principles of such Gestalt effects have remained elusive. Current neuroscientific models suggest that these phenomena are the consequence of the interplay between bottom-up, feedback, and lateral connections in the hierarchical visual system. Here we approach this question from a different perspective by investigating the neural basis of Gestalt formation. In our functional magnetic resonance imaging (fMRI) paradigm, we utilized a configural superiority effect, where a visual search task for the odd element in a display of three lines oriented at 45 degrees and one at 135 degrees (parts) is facilitated by adding an irrelevant corner to each of the lines, forming three arrows and a triangle (whole shapes). To assess the extent of grouping in early and higher visual areas, we compared multivariate pattern analysis performance on the same task, detection of the odd element, using the fMRI activity pattern in the primary visual cortex (V1) and shape-selective lateral occipital complex (LOC). The behavioral advantage for searching among shapes rather than lines was reflected in a better classification performance in LOC but not in V1, where decoding of parts was more pronounced. This suggests that the configural superiority effect may arise by predominantly feedforward processes. Moreover, even when we trained the classifier on whole shapes in LOC, its decoding performance on parts remained as poor as when we trained on those parts, indicating that the representation of a whole shape bears no similarity to the representation of its parts at these higher stages of visual processing. Simulations confirm that these findings are consistent with a feedforward model of vision, HMAX. Taken together, these results show how at least some Gestalt phenomena in vision are consistent with and caused by the feedforward processing of visual shape.