ViSAS: Visual Sequencing for Audio Serialisation

Sighted people have access to large amounts of visual information simultaneously; blind people, however, experience an audio translation in a single, linear stream. Although these two experiences initially appear quite different, in fact they share the concept of the “locus of attention”; a single source or location of sensory input that a person attends to at a given time. If we can determine how sighted people move their attention through visual information, we may be able to predict a sequential path through a parallel visual resource; effectively, linearizing visual content.

We propose a unified investigation centred around collecting eye movement, biometric and observational/interview data from people viewing artworks, and using this to determine the serial `path’ taken through the work. Our objective is to understand the characteristics of visual serialisation including its extent, variability and limitations, then use this understanding to predict a person’s path through a visual resource, thereby allowing the content to be accurately and effectively moved from a visual to an auditory presentation. In this way we can increase our understanding of how people experience art, and support any person who does not have access to the displayed resource.

A pilot study has been performed at Manchester City Art Gallery, involving eye-tracking members of staff and the public who looked at a series of paintings. This fun and interesting experiment has provided some interesting results, suggesting avenues for future research, and captured the attention of the media.