BILL BUXTON

Draft of April 12, 1993

Abstract

Input/output transducers (and their physical form) have a large impact on end users' mental models of computation. As we move towards ubiquitous and "intimate" computing, this will become even more the case. One of the key properties of systems will be their ability to provide a seamless interface between the objects and information in physical space and those in electronic space.

Since there are many types of such objects and information, there will be many different types of input/output transducers in the UbiComp repertoire. Devices will be tailored to have transducers appropriate to the types of activities and artifacts encountered in the context in which they are found or located. UbiComp will be characterized by many devices of different sizes. It will also be distinguished by the ability to "absorb" information and artifacts of the physical world into the electronic domain, and to "squeeze out" into the physical domain that which originated in the electronic domain.

One implication of this view is an incentive to place greater emphasis on exploring alternate input/output transducers on the UbiComp devices that we are building, prototyping, or even thinking about.

A Musical Anecdote

Around 1978 I directed a project at the University of Toronto which developed one of the first all digital interactive systems for music composition, sound design, synthesis, and performance. Essentially, it had the functional properties of a Macintosh with a graphical score editor, voicing program, and MIDI synthesizer. Musicians from about ten different countries, with no computer experience, came to realize compositions with this system. After an hour or two, they typically were able to work on their own. The system ran on a PDP-11/45, with some custom input and output devices, as well as a custom digital synthesizer.

In the same room where this operated, there was a PDP-11/40, that had a card reader, a tape drive, and D/A converters. Students from the Faculty of Music who were studying electronic and computer music used this latter system for their course work. They would punch up a stack of cards defining their job, bring them to the I/O clerk who would then feed the job into the 11/40. If all went well, the music software would grind out the samples of the resulting music and write them onto the tape drive. This tape would then be "played back" through the D/A converters and recorded onto audio tape. Turn around was typically overnight. (Incidentally, this procedure was typical for computer music at the time.)

Realizing the discrepancy between these two music systems, I approached the dean of the Faculty of Music, who also directed the electronic music program. I volunteered that his students would be welcome to use my system for their course work, rather than the batch system. Effectively, his response was, "No, because if they use your system they will not be learning to use a computer, and what they do learn will have no use to them when they graduate."

What this response says to me is that his view of a computer was completely dominated by the process of interaction and the input/output transducers through which this interaction took place. This is despite the fact that both systems were running on virtually identical CPUs. Now this was a highly intelligent person who was very literate in technology. Rather than making his response more surprising, I see this as emphasizing the weight of the influence of the I/O transducers and method of interaction in shaping users' models of the nature of computation.

A Little Audience Participation

There is a simple exercise that one can undertake in order to push on this point a little harder. With a class or some other audience, undertake the following exercise:

In 15-20 seconds, draw a computer.

Do this yourself before reading further.

Most likely, what most people will draw will be a keyboard, a display, and possibly a mouse. What many will omit will be the box that holds the computer itself. What I suspect nobody will draw is a schematic of an arithmetic/logic unit (ALU), and/or a set of data registers interconnected by a data and an address bus.

Having done the drawings, we can ask ourselves two revealing questions:

1. Which drawing actually represents what actually does the computation? Those which show the I/O devices or that which shows the ALU, registers and busses?

2. Which drawings stand the test of time? If the same exercise was done 15-20 years ago, those which were based on the I/O devices would be very different than they are today.

This exercise emphasizes two points. First, it shows - yet again - how deeply what users see and touch (the input/output transduces) shapes their model of the system. Secondly, it shows that the input/output transducers are "accidents of history" that are peripheral to computation proper (albeit central to its delivery).

Projecting into the future, this perspective opens the door to our considering yet different I/O transducers for systems of the future, and the recognition that these new transducers will, likewise, have a profound affect on how users view the systems into which they are integrated.

Moving Away from the "Henry Ford" School of Design

One of the key observations to make from the exercise above is the consistency that one finds among the various drawings produced. What this says to me is that current computer design follows what I call, the Henry Ford School of Design. Just as Henry Ford is purported to have said of his automobiles, "you can have it in any colour you want as long as it is black," so do current computer manufacturers say, "You can have it in any form you want as well as it has a keyboard, display and a mouse."

From twenty paces, all desktop machines running a GUI are indistinguishable, regardless whether they are PC's running Windows, UNIX boxes running Motif, or Macintosh's running the Finder. Furthermore, portables are simply shrunken-down desktop machines. There are no fundamental, or first-order design differences across the bulk of the currently installed base of systems.

Central to the concept of UbiComp is that there are a large number of "computers" and that these vary both is size and form. Consistent with the inverse law of strength vs. generality, UbiComp devices are strong and specific rather than weak and general purpose. It is largely because of their specific nature that we need numerous devices: each is tailored to afford particular types of activity. One size will no longer fit all.

Diversity in Sponges

Let us recall the influence that input/output transducers have on shaping our perception of systems. Our thesis is that it is mainly the diversity and the design of these transducers that will shape the "strong specific" characteristics of UbiComp devices.

As we move towards more "intimate" computation, it will become increasingly important for systems to directly handle the diverse objects and information that we encounter and handle in our situated day-to-day activities. The diversity of UbiComp systems and their input/output transducers will be driven by the diversity of these objects and information, and the contexts in which they are found. Systems will be tailored to "absorb" information and artifacts from the physical world into the electronic domain, and to "squeeze out" into the physical domain that which originated in the electronic domain. Towards this end, a much broader range of technologies will be used, including papertronics, videotronics and audiotronics.

Conversations, Proximal Sensing and the Responsive Environment

UbiComp simply means a family of devices variously integrated with scanners, microphones, and printers, for example. The input/output properties of UbiComp go beyond what they can absorb and squeeze out at the physical level. It implies some fundamental change from current mainstream computation that goes much deeper.

Traditional input/output can be characterized as primarily a foreground activity. It is mainly an intentional activity that is conversation-like.

Integral to the UbiComp model is the notion that input/output takes place at a background level as well. That is, some input and output may take place without (or in parallel with) any foreground conversational activity. For example, much like remote sensing, where satellites collect data in the background in order to monitor various phenomena on the earth, so will certain UbiComp devices collect and monitor aspects of their environment, including the objects and activities within it. By analogy to its remote sensing counterpart, I call this background collection of environmental and contextual data proximal sensing.

The behavior of the systems will be influenced by what is sensed by these background processes. This may be in the form of data collection. Or, like a thermostat, it may be in the form of the system exercising some effect on the environment. In this case, there is background output as well as input, which is embodied in the notion of the reactive environment.

Perhaps the most interesting and unique implication of this background, or peripheral interaction, occurs when it is used to support foreground conversational-type interactions. It opens the door to systems which can now begin to interpret user actions and requests in terms of the physical and social context in which they occur.

Proximal sensing shares some architectural implications with remote sensing. Mainstream interaction with computers has been largely dominated by output. The bandwidth of input, as manifest in typing, using a mouse, or even speaking, is greatly exceeded by the output bandwidth, especially as manifest in contemporary graphics displays. With UbiComp, this imbalance may well be leveled, or even reversed. Like remote sensing, the amount of data collected may be overwhelming, and dealing with this is one of the key challenges of UbiComp architectures.

"Multimedia," UbiComp and Ecological Design

As we delve into the topic of UbiComp, it is inevitable that the question of how it relates to "multimedia" will arise. Both owe much of their genesis to the ongoing convergence of telecommunications, computer and consumer electronic (A/V) technologies. Nevertheless, there are distinctions to be made.

Broadly speaking, in multimedia, I see the convergence of these technologies manifest in a drive towards a "super-appliance" that can speak, hear, let you script and watch movies, video conference and access large volumes of information from CD ROMs or on-line data bases. Depending on your perspective, multimedia is either the multi-modal/multi-channel documents that such appliances handle, or the appliances themselves. Regardless, this convergence towards such super-appliances runs the strong risk of falling into the trap of the weak-general "one size fits all" approach of current systems.

UbiComp is an ecological rather than appliance based approach to design. The ubiquity and transparency aspect of UbiComp demand distributed strong specific systems. These will be integrated into the environment in an architectural sense. The box into which solutions are fit is not some appliance on your desk; rather, it is the box in which you work: your room, or the desk itself.

Because there are multiple devices and they are distributed, one does not have to contend with channeling all activities through a single super appliance. A basic tenet of UbiComp is that it is as inappropriate to channel all of one's computational activities though a single "workstation" as it is to try to build a desk with a Swiss army knife. Even if all of the needed tools were collected in the knife, they are still more effective when they are not integrated into a single package.

Summary and Conclusions

We have argued that perceptions of technologies are dominated by what we see and touch: primarily, the devices used for entering things into the system and getting things out. Current design is hampered by at least two problems. First, the limited repertoire of input/output devices restricts our ability to provide a seamless bridge between the objects and artifacts in the everyday physical world, and those in the electronic domain of information systems. Second, by their "one size fits all" general approach, current systems are inherently weak.

This second point brings up the law of the inverse relationship between strength and generality. Existing systems are weak and general. Individually, UbiComp devices are strong but specific, and are tailored for specific classes of task, object or context. What is significant about UbiComp is that generality is achieved through these elements working in concert as an ensemble. Consequently, what one achieves is a system which is strong and general.

In terms of future work, the argument of this essay is that we need to invest more resources into expanding the range of devices in the UbiComp family. In particular, we need to push harder on providing that seamless link between the artifacts and information of the electronic and physical worlds. In order to do so, without spreading ourselves too thin, involves two things: (1) developing an underlying architecture that facilitates the integration of new classes of device into the system, and (2) exploiting the fact that strong specific systems are limited by definition, and therefore can be contained within finite tractable boundaries.

However, if we want to get to the core of what distinguishes UbiComp, we need to prioritize our activities to emphasize that which is not going to be undertaken by other groups which are pursing more mainstream architectures. Consequently, while UbiComp will rely heavily on marking (pencentric) and voice interfaces, our work in this area should be mainly be in the form of integration.

Where others are not working is in integrating the concepts of proximal sensing, and the responsive environment into the architecture, especially in such a way that compliments the more conventional foreground activities.

We have to choose our problems well, and invest our resources carefully. By doing so, we will accelerate the rate at which these ideas are developed, deployed and adapted.

Acknowledgments

The ideas expressed in this essay have developed over the past year or so during conversations with a wide range of people. Mainly, they were cultivated during discussions with colleagues at Xerox PARC and the Ontario Telepresence Project. In addition, they were stimulated and influeneced by participation in two workshops on UbiComp, one at MIT and one sponsored by Xerox PARC. The contribution of these interactions is gratefully acknowledged.

The research underlying this essay has been supported by Xerox PARC, the Ontario Telepresence Project and the NAtural Sciences and Engineering Research Council of Canada.

Position Paper:
Xerox Ubiquitous Computing Workshop
Los Altos, CA
April 5 & 6, 1993
University of Toronto & Xerox PARC