A biologically plausible phosphene simulator for the optimization of visual cortical prostheses

Abstract

Blindness affects millions of people around the world, and is expected to become an increasingly prevalent condition in the years to come. For some blind individuals, cortical visual prosthetics provide a promising solution to restoring vision, by converting camera input to electrical stimulation of the cortex to bypass part of the impaired visual system. Electrical stimulation in the primary visual cortex has been found to produce dots of light in the subject’s vision, called phosphenes. By evoking phosphenes in the right patterns, prosthesis wearers can be shown a representation of the outside world. As this representation has a limited resolution, visual prosthetics will need to rely on intelligent image processing algorithms that filter meaningful information from the visual surroundings. To optimise these processing strategies, non-invasive simulated prosthetic vision (SPV) can be used with sighted subjects or computational models. However, most SPV studies use highly simplified models of phosphene generation, limiting their validity for real-life applications. In this project, we developed a fast and fully differentiable phosphene simulator that transforms electrode stimulation patterns into biologically plausible representations of what the prosthesis wearer is expected to see. To achieve this, the simulator includes several computational models that take into account the visuotopic organisation of the cortex and the spread of activation in cortical tissue to determine phosphene locations and sizes. Several stimulation parameters are taken into account to model phosphene brightness and threshold values. Temporal dynamics are incorporated to allow for a realistic simulation over time. All models are parameterised and can be conveniently adapted to model empirical observations. Our results show the usability of the simulator for both computational applications as well as behavioural experiments. Keywords: Phosphene vision, bionic vision, visual prosthesis, blindness, computational modelling, deep learning, neurotechnology, cortical stimulation