| D O O R S O F P E R C E P T I O N 5 | |
 |
|
| visual musical installations |  |
| PDF link for printable transcription | Toshio Iwai |
The last time I needed a interpreter for my presentation, but today I want to try to speak in English myself. Just a reminder: I would like to show one of my works, Musical Insects, which I showed at the end of my last presentation at the first Doors of Perception. This is a sort of musical painting software which I created on Super Nintendo several years ago. And you can choose one of the colours using a mouse and you can do something on the screen. There are four small bugs walking across on the screen. And when the bugs meet the colour, they play a sound. And the difference of the colour means the difference of the key. And when you do something landing on the screen they play a sort of music. And each bug has a different instrument Here I have some techniques to create a more musical melody. Using a white colour, when the bug meets the white colour it goes back, and this red bug is a drummer and instead of playing notes, he plays the percussion sound. Using a grey colour it means 'turn right'. So you can make sort of loops. The dark grey means 'turn left'. When the bug escapes you can use the eraser. And this colour is rainbow, so you can make a sort of melody. And each small creature has an instrument. If you like to change the instrument, you can simply choose other bug. I would like to tell you why I wanted to create this kind of visual, musical software or installation. And to tell you the truth, when I was a child I got the worst grade in my music class when I was in primary school and actually I liked playing instruments or I liked singing a song, but I didn't get the idea of why I needed a musical score. I cannot understand. One day I found a very nice digital toy at the shop, and I was very encouraged. I brought this toy, and I would like to introduce you to it a little. This is a digital music box. Usually in a music box you can see a tiny metal drum with a lot of little pins. But here for this music box we can use these paper strips with many holes. Now I will play one of the paper strips. I hope you can hear this subtle sound (Happy Birthday). Happy birthday Doors! When I found this little music box, I realised this paper strip is also a musical score but instead of having some complicated notes on a traditional score, here we have just many holes. It shows actually the structure of the music. The distance between the holes represents the interval of the notes. And it's really easy to understand. And gradually, looking at this paper, I thought this was kind of an abstract picture. And actually, I like to look at the structure of the music. I thought it was very beautiful. And if I see this is a sort of picture, maybe it could be possible to play it the other way around. So now I would like you to listen the reversed Happy Birthday. This is a little bit 'unhappy birthday'. But I think this is very beautiful music. I don't know who has a copyright of this music. Anyway, when I met this little music box, I was very comfortable about understanding a structure of music. Then I started working on a computer to make a sort of expansion of this kind of traditional musical instrument. So I would like to go back to the video. This is very simple software which is also running on Super Nintendo. Here it has the same function as this little music box, but instead of making lot of holes on a paper, you can use a mouse to point at the stars in the sky. The name of the software is Starfly, like firefly, and you control this little box. You can change the speed of the music. You can reverse the direction. Or even you can grab this line and then make a sort of stretch. There are a number of different instruments. And when you change the instruments, the light effect on the star is also changed. I designed these light patterns very carefully because for me, the combination of the animation, image and the sound is very important. Here are some examples which my friends created or I created. This is a good example. Here's typical Japanese music. Here I would like to show another possibility of using this kind of tool. If you push the button, automatically the Starfly modifies your music. And when you create nice music by yourself then you can walk or take a nap, then gradually the music will be modified by itself and you can listen to all sorts of imitation of your music later. The next example of my work is a piano as image media which I created in Germany in 1995. Here you can see the similarity of this music box and this installation and here I attached seats and screens on the grand piano. And you can sit down on the chair. You can do something on a horizontal screen. And immediately when the images on the horizontal screen approach the keyboard. It plays another piano, and another three dimensional image flies out from the keyboard. So instead of composing music with the computer I wanted to have a more real-time experience, like playing a musical instrument. I'm not able to play the piano, but in this way I could play the piano with moving images. And here I could combine the other acoustic sounds and even the movement of the piano keyboard. So it was a sort of big leap from using just a sampled sound or synthesiser sound with the computer. And after I created this installation, I went to Japan from Germany and I had an idea to make a concert with this kind of piano which plays images as well as music. And instantly I met Riuchi ('Riu') Sakamoto, who is a pianist and well known composer, a composer of The Last Emperor score. And we made a collaboration at the end of 1996 in Japan. I installed it to a concert around the piano's on a stage and I hang the real big semi-transphere on the screen pianos. The piano is a sort of midi piano which generates a midi signal when Riu plays the piano, then my program (running on a Silicon Graphics workstation) transforms all of the information of the real-time music into the visual movement. It's a MIDI signal which is being generated by the piano, combining the notes, length of the notes and the velocity or loudness of the notes. I made a program which responds to all of the subtle information of the music. This is another program which I also used during the concert. There are 88 lines which is the same number of the piano keys. The 88 lines made a kind of circle in the air. Then each of the lines correspond to the piano key. And now Ru is making a sort of dynamic image. But here he has to keep the quality of his music. So he's trying to make both the imaginary and the musical performance. And the basic idea of the concert is having the piano as an image medium on the stage for the concert, but during the discussion with Riu, I realised there are a number of possibilities to combine images and music. And here is my performance and I'm using a computer mouse on the stage, putting numbers of white dots on the piano. And all of the dots are bouncing on the piano and they're making a sound. So the dots are moving on the piano and making a different sound. Next is another part of the performance. Here I took the little camera on the stage. And now Riu is moving his hand and body. The video image of his body is transmitted to the computer and digitalised on the screen. And then this image plays the piano. Here's another idea using a chess-like interface. Me and Riu made a real co-operation on this part, and here we have just one computer mouse, sharing the one mouse and putting the white dots on this chess-like interface. And Riu is approaching as a musician, making some nice music or pattern. And I'm approaching it as a visual artist on the same interface. And after we'd created a nice musical pattern, then Riu collaborates with that pattern. Here I'm using two piano's. And when Riu plays the piano on the left side and the image would fly out from this piano and make a parabola and reach to the right piano. And the right piano is played by image. Actually it's a little bit difficult to see but the left side of the right piano there is a video image of Riu which is delayed the actual timing of this image leads to the next piano. Here it looks similar to the previous one, but when Riu plays a simple melody, the whole of the notes are noted in a computer memory. And it's repeated again and again. Then Riu is trying to make other notes in this group. Gradually the image and the sound gets more complex. The intermediate of this performance will be an experiment on the Internet. Many musicians use Internet broadcasting for live performances, but always this kind of live broadcasting is just a low quality version of the television or something. We thought that as far as the Internet is interactive media we wanted to make a sort of interactive performance with the Internet. Here Riu is playing one key and the number shows the position of the piano key. And all of the viewers of the theatre site and the Internet can see this screen on Real Video or something. Now we're asking to input the same number on the browser. We made a special program using Java. Now Riu said 'please input number one', which is the lowest key of the piano. The next one is 88, the highest key. It's a little bit difficult to see but the highest key is moving. We got lots of viewers at the same time. All of them are trying to input the same number. Now we said please input from 1 to 10. The lowest of the keys are moving randomly. And finally we said, you can input any number. Then Riu made a collaboration with them, with nearly 100 people. And the next performance was the end of last year. We did the same performance for children. Here is a new idea. It looks like a video game. We have the game controller and the robots walk on the piano and when the robot walks it plays a piano. Here we have a top instead of robots. It makes a different sound on the piano. And the last videotape is one of my latest work which I created this year and is a sort of 'music table' onto which computer-generated images are projected. So I wanted to mix this virtual image and the physical interface at the same time. And when you push the button, you can change the direction of the arrow. And you can make a sort of puzzle changing a part of the light moving. And there are four different colours, like my Musical Insects. And each one has a different instrument and a different speed. So this is a sort of multi-player interface. When you keep pushing the button, the arrow automatically rotates. So you can add some random elements to the music. So this video is the end of my presentation. And as I told you, I had a very bad grade when I was a child in the music class. But I'm trying to make such visual or musical works using the new technologies. I hope everyone tries to find your hidden talent and expand a new possibility using the new technology and play with it. |
|
|
url: DOORS OF PERCEPTION |
|