Decoding noncommutative 3D perceptual consequences of commutative 2D swiping movements

Abstract

Interaction with objects is a skill that humans master. Rapidly developing touchscreen technology adds a new computational challenge to this expertise by allowing us to interact with virtual three-dimensional (3D) objects. On a touchscreen reversing the order of swiping movements does not change the end point of the finger on the screen (commutative property of swiping movements). However, when interacting with a virtual 3D object, augmented by the screen, reversing the order of swiping movements leads to a different end orientation of the object (noncommutative property of rotations). The aim of our study was to unravel neural circuitry that transforms commutative actions into noncommutative visual consequences. Using functional magnetic resonance imaging (fMRI), we measured brain activity while participants performed swiping movements to rotate a virtual 3D object on a touchscreen. By manipulating the order of the swipes and the starting orientation of the object, we studied commutative and noncommutative processes. We used representational similarity analysis to find brain regions with similar activation patterns to our modeled prediction. Whilst our main results did not reach significance they provide a tentative insight into brain regions that could be involved in coding perceptual commutative and noncommutative properties. They suggest that right inferior parietal lobule may be involved in noncommutative processes. Clusters in the right precunus, cuneus, lingual gyrus, superior frontal gyrus respond following the commutative model.