At the recent Venice Biennale of Architecture I presented a work entitled "Seen". It was part of "Next Memory City", the Canadian entry to the Biennale. Rather than focus on trophy buildings, "Next Memory City" looked at the public space that exists in the spaces between buildings and in particular, ways of visualizing the flows of people passing through and hanging out in these spaces.



In 'Seen' as in many of my other works, I use the video camera as a sensing device. A video camera has about 300,000 pixels, and each individual pixel can be considered an individual sensor. A video camera effectively projects 300,000 sensitive beams into the space it is observing. (This is strangely reminiscent of the ancient greek notion that the eye emitted rays of perception that reached out and touched the seen objects, reporting back to the eye what they had found.) I tend to apply relatively simple algorithms to each pixel or local set of pixels and render the results of these algorithms as processed video images. The images are generally processed live and in real-time, although a variety of issues precluded using live footage in Venice.



The source video stream for 'Seen' are shots of Piazza San Marco taken from a variety of positions at various elevations above the square and from different angles. Four distinct processes are applied to this source material. The first process extracts the moving elements (people and of course pigeons) from the background of static buildings, pavement and souvenir kiosks. In this stream, people float in an empty black space isolated from physical context. The fourth stream is the conceptual inverse of the first, showing only what is not moving: the context with only the most static of pedestrians visible. The two streams are mutually exclusive. Something cannot exist in both images at the same time.

The third stream uses the first as its source. This image is looped back upon itself so that each person is multiplied into a moving procession of themselves, crossing the piazza single-file in a Marais-like motion study. As these traces build up, a map of the ways that people occupy the piazza develops with high traffic areas clearly differentiated from those that are avoided or rarely crossed.

The fourth stream shows, for each pixel, the amount of time since the last movement was seen in that particular location. If a movement is sensed at a pixel, the processed pixel becomes an intense red. After movement has stopped at that pixel, the colour slowly fades. While in 'Seen' I chose to represent only the last 8 seconds of movement history for visual clarity, the process maintains the history of each pixel as a floating point number, meaning that it can represent an almost infinite amount of time so, for example, it would be possible to determine from the data that a specific location had last seen movement several years ago. The resulting image is striking, dynamic and eloquent. The trajectory of every single pigeon and every single pedestrian in the piazza is tracked and leaves a fading trail that defines the direction and speed of their movement.

These visualizations of movement through public space are ultimately intended for the human eye and perceptual system. We have a highly developed visual system that outperforms computers at many tasks involving large correlated fields of data. The computer however, is capable of shifting phenomena into the range of our perception allowing us to use our own highly refined abilities. This is especially true of cross-temporal phenomena that constitute flow: movement patterns that happen too quickly or too slowly for us to properly register with our eyes.

Ideally this represents a balance of human and machine. Attaining that balance requires a comprehensive awareness of what machine and human do best. In particular, it requires an awareness of the limitations of computers and software beyond the easily grasped limits of gigabytes and gigahertz. The first stage in achieving this sort of balance comes from making the tools themselves highly interactive so that we can also use our powerful cognitive capabilities to tune the behaviour of the human-machine system for maximum symbiosis.

But perhaps even more important is a recognition of another sort of flow that exists here. Any interaction between humans and machines involves a feedback loop. In a rock concert, the feedback loop between the microphones and the loudspeakers must be carefully tuned by the audio engineer to prevent squealing high-pitched feedback. Sound captured by the microphones is amplified back out through the speakers, which is picked up again by the microphone etc. The flow through any feedback loop (a snake swallowing its own tail) is a complex dynamic system that variously inhibits and reinforces what is flowing through it.

When feedback loops exist between humans and machines, interesting factors come into play. It has been determined that consciousness tends to operate at a delay of about 1/10 of a second. Computers tend to respond in much less than 1/30th of a second. As a result, the feedback between human and machine can creep under the level of consciousness, invisibly reinforcing and attenuating various aspects of the complex stream flowing through the loop. Such feedback systems have their own synergetic characteristics. And because the fastest responding element of the system is usually the computer, what is most reinforced through the loop is often defined more by the computer than the human.



I find that this even manifests itself in familiar systems like e-mail. The potential speed at which email dialogs can progress tends to reinforce issues that can be instantly resolved with straightforward answers. Meanwhile, at least in my experience, my inbox accumulates a huge pile of unanswered but more interesting e-mails that can't properly be addressed in the rapid cycle that e-mail encourages.

As we move into a scenario where more-and-more less-and-less conspicuous computing devices populate our lives, we need to pay careful attention to what is being reinforced and what is being discouraged in our relationships with these devices. The prospect of pervasive computing poses the difficult challenge of guaranteeing a pervasive humanity flowing through these systems.