Bill Seaman and Ingrid Verbauwhede

Poly-sensing Environment

Toward the Development of an Integrated Distributed Technology Exploring Poetic/ Informational Grammars of Attention and Functionality.

The Foundation is supporting The Poly-Sensing Environment, a virtual imaging environmental project proposed by Bill Seaman and Ingrid Verbauwhede. Bill Seaman is the Head of the new Digital Media Program at Rhode Island School of Design, Graduate Studies. He also continues his research at UCLA, collaborating with Ingrid Verbauwhede in EE. Ingrid Verbauwhede is Associate Professor at the Electrical Engineering Department at the University of California in Los Angeles (UCLA).

The notion of interface is generally understood as the connection point between the actions of a user of technology and the consequences of such actions at a digital or virtual level. Although interfaces may combine or overlap, there are basically two types: the transparent interface, which functions according to easily-identifiable modalities between input and output (e.g. the console of an electronic game), and the opaque interface, which is spatially diffuse, generating effects that are hard to associate with a cause (e.g. interactive environments using sensory devices).

Recent advances in sub-micron and nano-scale semiconductor technologies have allowed the integration of multiple heterogeneous sensors onto a single chip. Using this poly-sensing technology, Seaman and Verbauwhede are creating interfaces that users can control according to set parameters, but whose hardware disappears entirely, thus creating a relationship with technology that is more cognitive than instrumental.

Giulio Camillo's 16th-century Memory Theatre, described by Frances Yates in The Art of Memory, serves as a model for the complex technological environment that Seaman and Verbauwhede have in mind. This portable, wooden mnemotechnic structure, in which the viewer would be seated, was to be filled with objects/artefacts/signs/images evoking the various microcosmic and microcosmic levels of knowledge (from vegetal to mineral to universal). Among other things, the Memory Theatre was designed to house files of speeches and writings, all related mnemonically to the subjects suggested by the diverse icons and artefacts.

In the poly-sensing environment that Seaman and Verbauwhede are proposing, (1) cordless sensors are placed on all surfaces of the room, including walls, tables, chairs, clothes and other objects. Like observation posts simultaneously monitoring numerous aspects of the same phenomenon, each sensor matrix coordinates a set of heterogeneous data, which it memorizes in real time in the physical site (motion, sound, pressure, heat, chemical data from the participant's body, etc.). A radio technology of the GPS (Global Positioning System) variety will allow the transfer of data from one matrix to another; a server then displays the results of these exchanges via media events (projection of composite images or video/sound environment) in a linked virtual space. These events do not necessarily stem from the real-time processing of the data measured in the space of the participant; a bank of pre-programmed media collections (2D and 3D images, objects and text objects, digital video, sound) is also linked to the sensing parameters of the space. A dynamic matrix tool enables a series of if/then relationships connecting the physical with the virtual.

This interweaving of chains of apparently unconnected signifiers is reminiscent of Seaman's interactive installations -e.g. Exquisite Mechanism of Shivers (1991) and Red Dice (2000) -in which a participant constructs a series of recombinant meanings based on a finite number of images and texts stored in a database. Here, however, an action or decision on the part of the participant does have repercussions in the virtual space (the effects of an action won't produce one field of meaning, but rather several series of disseminated media events).

With The Poly-Sensing Environment, Seaman and Verbauwhede hope to generate a cybernetic environment modelled on cognitive functions. The various polysensors process and relate a set of heterogeneous signals, filtering extraneous and/or non-changing data from significant data, in the same way human cognition evaluates information from the numerous stimuli in the external world.

Several modes for apprehending the environment are offered. To process information in a restricted semantic register, participants make a series of functional choices which activate and deactivate certain sensors. (2) For a wider field, they are able to trigger all of the sensors monitoring the physical space. Finally, in a retrospective mode, participants are able to consult nodal maps and grids created by previous users.

A later phase of the project will explore the possibilities of a networked space, in which participants in different locations would simultaneously observe or navigate the virtual environment.

This collaboration between Seaman and Verbawhede will extend over a period of three years. A functioning prototype of the environment will be the basis of a new installation by Seaman, provisionally entitled The Quiescent Space of Epiphany.

---

(1) See the schematic below, which presents all environmental components and interactive modes: http://users.design.ucla.edu/~fwinkler/poly_sensing/

(2) For example, it will be possible to probe the environment for such phenomena as ambient humidity.