What I would like to do today is to give you a sense of where I think we are going in multi-media, and maybe give some ideas. I have just a few introductory remarks, which I think are very important for people to remember. We think we are designing easy-to-use interfaces for everyday people, but in fact it turns out that there are some terribly bad metaphors and very poor design examples out there. I think that we ignore difficulties in designing well, particularly for the computer media.

The other thing about designing multi-media is that we know, that media are changing but the user's capabilities are not. Most of us still have difficulties with VCRs and telephones, and although we now have technologies so-called "increasing in capability", most of the VCRs in America still flash midnight all over the country because no one knows how to change the setting of the clock.

At the moment most multi-media is a series of interruptible linear sequences. A series of cascading text folders, that bring you sequentially to the real chapter of a particular book. You can click on things and watch them. I think we have to be very careful that we don't look at things like books and just replicate what a book is into a computer and assume it must be better. There are some good examples - Bob Stein showed you some brilliant examples of good multi-media - but the kind of thing that we are seeing a lot of, which poisons people to the future of it, is bad multi-media. They have added sound and pictures, and because it has got more stuff in it, it is supposed to be better. We have moved from text to pictures, but in fact what we really should be doing is look at how we use dynamics of time, and this is missing in most of these things, particularly in sound.

Most of what has gone on in terms of designing, is that it has not been user-driven. Designers haven't thought about what people are actually going to do and use this work for. One of the nice things Apple did, a couple of years ago, was that we began to broaden our horizon of who we should put in charge of interesting projects. For Quicktime we hired someone on sabbatical from New York University, who actually had a background in telecommunications and film. Quicktime is a software in which you can play back time events, which could be either sound or video, and it's an extension to the Macintosh system software, which means, you don't need any hardware to do this.

This prototype was built in the early days of Quicktime to convince our vice presidents that you might want video in a Mac. Why would you want to do that? The wonderful thing about 'why' is... because people do that sort of thing, and so when you design an interface, you have to design an interface to support that.

The important part of this story is that the computer people were very specialised in what they already did, and they didn't necessarily see any need for video. However, when they showed the users the simulation, people were driven to push beyond the text barrier and bring in dynamic sources of data, particularly in this case video. The interesting thing is that we didn't know what would happen. As soon as you enable people to see something they haven't seen before, other things happen, and you can't control that.

The other part of this story, which is an important design lesson for those people who may not be familiar with multi-media yet in their design world, is in a hand-controller. The hand-controller was developed by people with a background in film, to shuttle through video-footage. However the end of the story was that the average user just wanted to cut and paste video, they didn't want to draft-shuttle through the video. It took the designers almost a year to get rid of their wonderful controller. The important part was that they didn't need to have designed it in the first place. You don't have to solve these problems yourself. You just need to listen to the users. And I think that's a very important lesson.

The other thing that I think designers need to be careful of when they design is to actually store examples of past designs, and possibly their results. The problem in industry often is, that they don't know what good design is, and if you just show them your answer, they won't see what's involved.

What we did for the next generation of Quicktime was, we turned it over to yet another student from NYU, who also had a film background, and he decided that the best thing to do was to attach a Macintosh to a camera, and provide a virtual experience for people to go to a place they may not be able to go to. And what is wonderful about this is that it has enabled a whole bunch of new content providers, such as photographers, to imagine being able to create these kinds of interactive movies. Now, that would not have come to pass unless we had been able to turn something like Quicktime over to someone like Dan O'Sullivan, who had an unusual background in doing interactive television shows.

After they have seen lots of these movies, people say, 'What does it sound like in these spaces?' In fact, we don't know very much about sound. Some of us maybe are somewhat inexperienced or illiterate, in visual design, but we are also very inexperienced in audio design. So we have also been looking a bit into how to make sound more accessible, and this was driven by user input. We found that nobody uses tape recorders, because they can't find anything afterwards, and we wanted to know how we could find a way of making recorded sound more accessible to people. Another problem with sound is that you really want it to go faster. Our goal was to move it into the software realm, where you can actually find these tones.

If you have made an audio summary, you want a much more pictorial form of that, so you can see where, and who spoke at what point. So, the reason I put this up is to encourage people to actually look at how sound should look on the computer, and how I might find things I want. As I think we have made very little progress in this overall area.

So I'll show another application we wrote, called The Transition Factory, because once you have a video on your Mac you're now going to want to cut and paste pieces of video together. Anyone of you who has done video editing realises that there's lots of fancy tools for doing that and a lot of people get paid a lots of money for editing. However, we also know that users become more literate over time, and they're going to want to do a little of this themselves. So our challenge here was how we might design an interface that was actually easy enough for everyday people to use. The point is however, that the audio and the video can actually be merged together in a direct-manipulation way.

As you can see, the beginning of this chain is that you begin with Quicktime, give it to some people who know other things, and so it evolves into some unusual and very compelling directions. You saw a lot of media today, where you just click on it. But the thing here is, now you're going to not just click on it, you can move around in it, and you can move it around you. So now you need to have a visual language that communicates to the user immediately at the interface, 'What can I do?' So you need to make apparent to people what they can actually do when they're at the interface.

Finally, I want to say a few words about where I think the media will go in the future. I think that the forms that it's been in are arcane and not very attractive. What we really need is to have a computer that you would want to go home to play with. My feeling is that I still don't know why I want to use a computer. It doesn't feel very good, whatever it's doing. I also think that there's a lot of work out there that we need to pay attention to, and look at how the industries that are merging are going to take advantage of not just computing but also genres such as television. So you can imagine a world now where immediately the media becomes much more useful and is far more interactive than the sort of thing that you've seen up to now.