I see what you hear

A collaboration between Mette Ramsgard Thomsen, CITA, the University of Brighton, School of Architecture and Design and the Department of Computer Science, University College London

I see what you hear is a dynamic interactive environment for intuitive interaction. The installation merges the users' sound output [talking, singing and making noises] with the dynamic production of a 3 dimensional space generated as clouds of matter that the user 'blows' into being. Swarms of insects, leaves, fish, circles, numbers, create a spatial density around the presence and action of the user. Like a snowstorm, the space is a condition rather than a limitation, defined by the quality and deliberations of interaction. The individual textures are mapped onto the pitch level creating bands of matter within the installation space. The users' sound production scaling across these bands as well as the gravity of particle stream mixes the textures together into worlds of their own.

As the users learn to control the parameters of interaction they learn to generate spaces of singular objects or mix together different objects of choice. The environment is presented stereoscopically allowing the user to be sited within the spatial density.

I see what you hear is the making of an Emobodied Interface. Embodied Interfaces addresses the physicality of the users sense of presence. Where traditional interfaces augment our actions in the way of a tool [e.g. mouse], locating our means of interaction in a distinct body part [e.g. hand], embodied interfaces track an aspect of the user’s physical presence in space that in turn becomes input for the computed environment. The research hypothesises that embodied interfaces challenge and extend the way that the senses operate, proposing that the construct of our embodiment, as well as the senses that confirm it, is essentially malleable. Embodied interfaces change the co-ordination of sensory input challenging our body image and allowing the development of a new sense of self as well as a new sense of presence. Embodied interfaces address the core conception of our embodiment. Rather than perceiving technology as a tool with which we engage in an augmented reality as augmented beings, they reiterate the possibility of presence, redefining and re-constructing our selves in a reality which mixes, joins and grafts the dimension of the physically present and the computed.

Initially designed for the CAVE, users enter the environment wearing a wireless microphone as well as stereoscopic shutter glasses and the CAVE head tracker. Users are asked to engage with the environment through sound and movement interaction. As users interact by different vocal (talking, singing and making noises) as well as physical sounds (clapping, stamping and clicking) vividly coloured spline surfaces appear in the display environment.

In I see what you hear the digital is instantiated in the present, coming into being through the moment of interaction. When the user is still, or quiet, the presence of the digital dissolves, returning the user to the normative containing environment. The digital becomes an enacted dimension, a durational space taking place across the temporal extensions of experience.

Venue
I see what you hear was exhibited at Embodied Interfaces at the Grand Parade Gallery, University of Brighton, December 2002.

Support
I see what you hear is developed in collaboration with and through support by University of Brighton, School of Architecture and Design and Department of Computer Science, University College London.