Wireless technology in gesture controlled

computer generated music

Leonello Tarabella
Graziano Bertini
computer ART lab of CNUCE/IEI, Area della Ricerca di Pisa
via Moruzzi 1 - 56126 Pisa, Italy
l.tarabella@cnuce.cnr.it g.bertini@iei.pi.cnr.it
http://www.cnuce.pi.cnr.it/tarabella/cART.html
Abstract
We here describe the experience of researchers and artists involved in the activities of the
computer ART lab (cART lab) of the Italian National Council of Research (C.N.R.) in Pisa,
regarding the wireless technology developed for controlling in real-time interactive multimedia
performances. The most relevant devices and systems developed at cART lab for gesture
recognition to be used for giving expression to interactive multimedia performances are here
reported. Also, a real-time powerful music language based on C language able to perform audio
synthesis and processing is introduced.

1 Introduction
We hereby report some general considerations about
the relationships between science and music that can
help young researchers of MOSART network for
understanding the philosophy of this fascinating topic
of computer music. This is an excerpt of the
introduction written for Interface as Guest editor of
the special issue on Man-Machine Interaction,
Interface, Journal of New Music Research [1].
Besides, some technical parts of this paper have been
already presented in other conferences, here properly
updated and extended [2].
The history of Mathematics, Physics and Technology
have played a crucial role in the history of Music as
regard the evolution of the language reference syntax
(notes and modern harmony) and the shape and the
mechanical functionalitys of musical instruments.
We know that for centuries musical scales has been
constructed by notes whose pitches are related by
simple ratios and that only at the end of 17th century,
the well-tempered schema, based on logarithms
introduced in the realm of mathematics some decades
before (1612), has been proposed. [3].
It may be that the evolution from the plucking system
used in the spinetta and in the clavicembalo to the
striking-hammer system used in the piano, should not
be considered as a true result of progresses in
Science but, rather, as a consequence of the smartness
and the craftsmanship of the Italian musical
instruments manufacturer Bartolomeo Cristofori. But
its highly probable that the terminal part of wind

instruments also called bell, evolved from the

conical shape derived from ancient Romans
clarines and war-horns to the modern exponentialcurve-based shape as a consequence of a precise
awareness and knowledge on the physics of sound.
This kind of curve allows a loud and brilliant acoustic
response and gives the trumpet and the other horns,
an elegant profile.
As a counter proof, what follows is what did not
happened. The number of combinations of the three
on/off valves of the trumpet are 7 rather than 8 as
expected from binary arithmetic culture. This derives
from the length of the by-passes activated when the
valves are depressed which are related by ratios 1,2,3.
It may be that if at the time the trumpet reached its
final shape and mechanical evolution, binary
arithmetic had been so popular as nowadays, the
length of the 3 by-passes should have be chosen to
have the ratios 1,2,4 so gaining a further combination
which should have produced a different fingering.

2 Technology and expressiveness

Whatever the history, the shape and the mechanical
interface between the human body and the musical
instrument, playing a piece of traditional music
means generating sounds with suitable tools
controlled by one or more parts of the body: mouth,
arms, hands, feet. From a physiological point of view
playing an instrument entails activating and
coordinating a specific set of muscles which in turn
activates those parts of the body in contact with that
instrument. To be sure, making music is something

more than merely activating sound generators;

musicians usually need a number of degrees of
freedom to put a work their capabilities on their
instrument in order to communicate their emotions,
and the huge quantity of information downloaded
onto the instrument assures complete control and
high level of artistic expressiveness.
When analog electronics began to be used to create
music, it was natural to design a simple architecture
derived from that of the piano, i.e. an array of
switches assembled with the same layout as a piano
keyboard, and a sound generator based on oscillators
linked to the switches. When analog electronics was
replaced by digital electronics, the functionalitys
were greatly improved both from the point of view of
control and of sound generation, but the same
architecture was observed. Additional expressive
controllers, such as pitch-bend or pitch-wheel, were
added to the basic piano-like keyboard. Then the
MIDI standard was introduced for interfacing a
variety of different products released by different
manufacturers.
All that biased toward, and in some way inherited by,
traditional music based on the concept of note as
the building block for composing and performing;
and more, all that bearing in mind consumer music
(rock music, dance music and advertising music) as
main targets. However, in contemporary music, and
more generally in contemporary art, composers
consider sound in a wide sense and outside of
whatever reference language as the building block for
composition. In this context the set of MIDI
controllers are no more suitable.
2.1 Electronic sound under gesture control
Modern technology permits high quality sound/music
compositions; however, musicians complain that
computers do not allow the direct and complete
control of sound and colors as do traditional media:
traditional music is played by manipulating real tools,
(manipulation derives from mani, Latin for hands)
and traditional painting involves manipulating colors
on the canvas. The vestibular correspondence
(feedback) between action and art, plays an important
role for the artist's cognitive awareness, and
consequently for his/her creativeness.
Gesture, the successive postures of the hands, head,
mouth and eyes, has a very important role in human
communication specially when seen as a parallel
language for enriching the semantic content of speech
or as an alternative way to communicate basic
concepts and information between people of different
cultures and mother tongues. During the last decade
many researchers and musicians have focused
attention on the possibilities offered by this new
approach to making music with computers, by
designing and implementing original man-machine
interfaces controlled by gesture [4,5,6].

Due to the daily increase in computers power and

electronics systems able to sense the presence, the
shape, the distance and the position of objects, a new
field of investigation and implementation has been
started in the last few years: computer recognition of
human gesture [7,8]. As a result, the human body
itself can now be considered as a natural and
powerful expressive interface able to give feeling
to performances based on computer generated
electro-acoustic music and computer generated
visual-art. Modern human computer interfaces are
extremely rich, incorporating traditional interface
devices such as keyboard and mouse and a wealth of
advanced media types: sound, video, animated
graphics. The term multi-modal is often associated
with such interfaces to emphasize that the combined
use of multiple modes of perception is relevant to the
users interface [9].
2.2 Research on gesture
A significant panorama on the state-of-the-art of the
intenational research results can be found in [10]
Trends in Gestural Control of Music edited by
Marcelo M. Wanderley and Marc Battier at Ircam Centre Pompidou, Paris, which also includes the
research results of our Lab at CNR, Pisa. This web
page contains information about research in the field
of gesture capture, interfaces, and applications to
sound synthesis and performance.
Six different main topics are indicated:
1)
2)
3)
4)
5)
6)

Virtual musical Instruments

Gesture, Instrumental, empty-handed, etc.
Acquisition, Gestural capture techniques
Different mapping strategies
Sensory feedback, haptic systems, etc.
Interface examples

Some significant excerpts from linked literature are

here reported.
1)
In "Virtual Musical Instruments: Accessing
the Sound Synthesis Universe as a Performer", 1994,
Axel Mulder's, School of Kinesiology, Simon Fraser
University, Burnaby, B.C., V5A 1S6 Canada,
considers a Virtual Musical Instruments (VMI) as an
instrument capable of mapping any type of physical
gesture or mouvement to any class of sounds and
writes: with current state-of-the-art human
movement tracking techology it is possible to
represent in real-time most of the degrees of freedom
of a (part of the) human body. This allows for the
design of a virtual musical instrument (VMI),
analogous to a physical musical instrument, as a
gestural interface, that will however provide for
much greater freedom in the mapping of movement
to sound. A musical performer may control therefore
parameters of sound synthesis systems that in real-

time performance situations are currently not

controlled to their full potential or simply not
controlled at all. In order to decrease the learning
and adaptation needed and avoid injuries, the design
must address the musculo-skeletal, neuro-motor and
symbolic levels that are involved in the
programming and control of human movement. The
use of virtual musical instruments will likely result in
new ways of making music and new musical styles
[11].
2)
In Toward an Understanding of Musical
Gesture 1996, Teresa Marrin, Media Arts and
Sciences, Massachusetts Institute of Technology,
writes: recent work done in developing new digital
instruments and gestural interfaces for music has
revealed a need for new theoretical models and
analytical techniques. Interpreting and responding to
gestural events -- particularly expressive gestures for
music, whose meaning is not always clearly defined - requires a completely new theoretical framework.
The tradition of musical conducting, an existent
gestural language for music, provides a good initial
framework, because it is a system of mappings
between specific gestural cues and their intended
musical results [12].
3)
As regard Gestural capture techniques,
Depalle et alii, Serveur IRCAM - CENTRE
POMPIDOU Paris, 1997, say: if we focus on the
case of instrumental gestures, one can divide gestural
capture techniques according to the following
capture strategies: One can use different sensors in
order to capture different gestures or movements.
These sensors may or may not need physical contact
to transduce the gesture(s) into electrical signals. If
physical contact is involved, they can be called
Haptic; otherwise, Non-Haptic. The sensors can, in
turn, be classified as to whether the sensor outputs
are continuous or discrete values of the variable
sensed. We consider these techniques as direct
gestural acquisition, since one is capturing the actual
physical gesture or movement through the sensors
[13].
4)
Mapping: J. Rovan, M. Wanderley, S.
Dubnov, and P. Depalle, from Serveur IRCAM CENTRE POMPIDOU Paris,
in Instrumental
gestural mapping strategies as expressivity
determinants in computer music performance write:
a common complaint about electronic music is that
it lacks expressivity. In response to this, much work
has been done in developing new and varied
synthesis algorithms. However, because traditional
acoustic musical sound is a direct result of the
interaction between an instrument and the
performance gesture applied to it, if one wishes to
model this espressivity, in addition to modeling the
instrument itself - whatever the technique/algorithm -

one must also model the physical gesture, in all its

complexity. Indeed, in spite of the various methods
available to synthesize sound, the ultimate musical
expression of those sounds still falls upon the capture
of gesture(s) used for control and performance. We
propose a classification of mapping strategies into
three groups: One-to-One Mapping : Each
independent gestural output is assigned to one
musical parameter, usually via a MIDI control
message. This is the simplest mapping scheme, but
usually the least expressive. It takes direct advantage
of the MIDI controller architecture. - Divergent
Mapping : One gestural output is used to control
more than one simultaneous musical parameter.
Although it may initially provide a macro-level
expressivity control, this approach nevertheless may
prove limited when applied alone, as it does not
allow access to internal (micro) features of the sound
object. - Convergent Mapping : In this case many
gestures are coupled to produce one musical
parameter. This scheme requires previous experience
with the system in order to achieve effective control.
Although harder to master, it proves far more
expressive than the simpler unity mapping. [14]
5)
Sensory Feedback Claude Cadoz and his
group at ACROE - Grenoble, France where since the
late 1970's they have been studying and developing
interaction systems using physical modelling
synthesis (CORDIS/ANIMA system) and force
feedback devices (Modular Feedback Keyboard).
Cadoz write in different places: many studies have
shown the importance of tactile-kinesthetic feedback
for an expert player, in comparison to the relative
importance of visual feedback for beginners. In an
instrumental context, it is therefore interesting to try
to simulate these effects in order to profit from the
expert technique developed for acoustic instruments..
[15]
this entry points to many
6)
Interfaces:
different academic (including the cART lab), private
organization groups and individuals which and who
realized a great variety of interfacing systems and
devices.

3 The cART lab project

The activity of the cART Lab (computer Art lab of
CNUCE/CNR) is characterized by the design and the
realization of systems and devices for gesture control
of real-time computer generated music in order to
give expression to interactive electroacoustic
performances, as it happens in traditional music. The
wireless technology paradigm has been taken into
consideration.
Main targets of the research consist of the
implementation of models and systems for detecting
gesture(s) of the human body that becomes the

natural interface able to give feeling and

expressiveness to computer based multimedia
performances. The term multi-modality is often
related to these typologies of interfaces just for
emphasizing that combining different modes of
perception becomes relevant for the performer and
for the audience that, at the end, is the final user of
the performance [16].
At the cART lab attention has been focused in
designing and developing new original generalpurpose man-machine interfaces taking into
consideration the infrared beams and the real-time
analysis of video captured images wireless
technologies. Specific targets of the research consist
of studying models for mapping gesture to sound
because in electro-acoustic music nothing is preestablished as in traditional music and instruments
where there exist precise and well consolidated
timbric, syntactic (harmony) and fingering systems of
reference.
The basic idea consists of remote sensing gesture of
the human body considered as a natural and powerful
expressive interface able to get as many as possible
information from the movements of naked hands.
Other related targets concern - the analysis and the
representation of audio signal by means of timbre
models based on additive (Fourier) technique and
more recent models [17] the implementation of
languages for music composition and interactive
performance - the realization of algorithms for
synthesis
and
processing
(e.g.
coloring,
spatialization) of sound. The research has been
motivated by real artistic expressive needs coming
from artists outside and inside the lab.

accurate in respect to the irregularity, in shape and

color, of the hands palms. Voltage analog values
coming from the Rxs are converted into digital
format and sent the computer about 30 times/sec. The
computer then processes data in order to reconstruct
the original gesture [18].

Fig. 1 Different positions of the hands

Its so possible to detect positions and movement of

the hands such as height and side and/or front
rotations; besides, depending on the specific piece of
music composed to be played, its possible and
necessary to properly interpret data for giving
meaning to other kind of movements of the hands
such as flying on the device in different directions
at different heights as shown in fig.1.
The new version of this interface has on board a
microprocessor for pre-processing sensor data with
calibration, linearization and other ad hoc routines. It
meets the standalone specifications and is equipped
with a MIDI OUT port.
This device is stable and responsive; as a
consequence, the generated sound provokes on the
performer the sensation of touching the sound; this
is a sort of a psychological feedback which greatly
contributes to give expression to computer generated
electro-acoustic music.

4 Infrared beams controller

This device consists of two sets of sensing elements
that create two zones of the space, i.e. the vertical
edges of two square-based parallelepiped, or virtual
towers. At the time the
first
prototype
was
realized (1995), the
shape of the beams
suggest us the profile of
the Twin Towers in New
York. Starting form
then, the device has been
presented in conferences and concerts with this name;
but after september 11 we decided to use no more
this name. However, since we recently carried out a
new versions of this device consisting of up to 8
groups of 4 elements
(towers) with special
mechanical supports which allow different
arrangements such as a drum set, already we had
decided to look for a new one.
The measurements of distance of the different zones
of the hands are performed by the amount of reflected
light captured by the receivers (Rxs) and are quite

5 Handle
Based on real-time analysis of video captured images,
a system for recognizing shape, position and rotation
of the hands has been developed: the performer
moves his/her hands in a video-camera capture area,
the camera sends the signal to a video digitizer card
and the computer processes the mapped figures of the
performers hands and produces data concerning x-y
positions, shape (that is, posture)
and angle of rotation of both the
hands. Data extracted from the
image analysis of every frame are
used for controlling real-time
interactive computer music and
computer graphics performances.
Both the hands are taken into consideration.
For each hand the following steps are executed: - the
barycenter is computed and then used for constructing a one-period-signal using distances
from the barycenter to the points along the contour of

the shape of the hand taken on radii at a predefined

angular steps; -the resulting one-period-signal is then
processed

Fig. 2 One-period-signal of the open-fingers hands

with FFT algorithm and the resulting spectrum is

compared with previously stored spectra referring to
specific posture of the hands; - the "closer" spectrum
in terms of vector distance corresponds to the
recognized shape. The program finally produces data
concerning x-y positions, shape (posture) and angle
of rotation of both the hands [19].
On the basis of the functionality of this application
two relevant systems have been implemented: the
Imaginary Piano and the PAGe (Painting by Aerial
Gesture) system.
5.1 Imaginary Piano
In the Imaginary Piano a pianist sits as usual on a
piano chair and has in front nothing but a CCD
camera few meters away pointed on his hands. There
exists an imaginary line at the height where usually
the keyboard lays: when a finger, or a hand, crosses
that line downward, handle systems report proper
information regarding the key number and a
specific messages issued in accordance of "where"
and "how fast" the line has been crossed. Messages
are used for controlling algorithmic compositions
rather than for playing scored music.
5.2 PAGe system
Another application based on the video captured
image analysis system is PAGe, Painting by Aerial
Gesture, which allows video graphics realtime
performances. It has been inspired and proposed by
visual artist Marco Cardini (www.marcocardini.com)
after the experience of the Italian artist Lucio Fontana
who, cutting the canvas indicated the way for
trespassing the "limits" of the canvas itself, during
years '93 and '94 suggested the idea of a system able
to give the possibility of "..painting in the air.." and to
introduce a new dimension to painting: time.

Figure 3: PAGe system configuration

PAGe comes after two previous trial experiences:

Shine Hands and the Aerial Painting Hands. PAGe
has been designed by L.Tarabella and developed by
Dr. Davide Filidei as argument for his thesis
discussed at the Computer Science faculty, University
of Pisa. D.Filidei is still maintaining and updating
PAGe following the expressive needs of artist
Cardini. The system permits Cardini, who also
performs on stage, to paint images projected onto a
large video screen by moving his hands in the air.

6 Mapping
The different kind of gestures such as continuos or
sharp movements, threshold trespassing, rotations
and shifting, are used for generating sound event
and/or for modifying sound/music parameters.
For classic acoustic instruments the relationship
between gesture and sound is the result of the physics
and mechanical arrangement of the instrument itself.
And there exist one and only one relationship.
Using a computer based music equipment, its not so
clear what and where is the instrument; from
gesture interfaces such as the infrared beam
controller or Handle, to loudspeakers which actually
produce sound, there exist a quite long chain of
elements working under control of the computer
which performs many tasks simultaneously:
management of data streaming from the gesture
interfaces, generation and processing of sound,
linkage between data and synthesis algorithms,
distribution of sound on different audio channels, etc.
This means that a music composition must be written
in term of a programming language able to describe
all the components including the modalities for
associate gesture to sound, also said how to map
gesture to sound. The mapping makes therefore part
of the composition.

7 A new realtime music language

In order to put at work the facilities offered by the
gesture interfaces we realized, at first we took into
consideration the most popular music languages:
MAX/DSP and Csound.

Unfortunately, both languages resulted not precisely

suited for our purposes mainly because Csound was
not so realtime as declared and Max was not so
flexible for including video captured images analysis
code. Using two computers (first one for managing
interfaces, second one for audio synthesis) connected
via MIDI, resulted awkward and inefficient.
We started writing basic libraries for processing
sound and for driving the gesture interfaces bearing
in mind the goal of a programming framework where
to write a piece of music in terms of synthesis
algorithms, score and management of data streaming
from gestural interfaces [20]. On the long run the
framework became a very efficient, stable and
powerful music language based on pure C
programming, that is pure-C-Music, or pCM.
pCM falls in the category of the embedded
language and needs a C compiler in order to be
operative; at first this may appear an odd news, but
consider the good news of getting a compiled code of
a synthesis algorithm which generate and process
sound running many times faster in respect to Csound
which, on the counter part, is an interpreted language.
7.1 pCM, pure-C-Music language
PCM has been implemented using one of the most
popular C compiler or better, multiplatform
development system: Metrowerks Code Warrior. As
a result a pCM composition consists of a CW project
which includes all the necessary libraries including,
once again, a DSPlib consisting of a number of
functions (at the moment about 50) able to implement
in realtime the typical synthesis and processing
elements such as oscillators, envelope shapers, filters,
delays, reverbs, etc..
The composition itself a C program consisting of four
functions(void): Init(), Orchestra(), Score() and
Finish() properly invoked by the main program which
controls the whole machinery; for pratical
problems its a good idea the four functions make
part of the same file which also includes declaration
of the variables visible by the all functions and
expecially from Orchestra() and Score(). Everything
here is written following C syntax; synthesis
algorithms, score and declaration of variables and
data structures; everything here is compiled into
machine code and running at CPU speed.
The Init() function includes everything regarding
initialization and/or loading such as envelopes, tables,
samples, delays, reverbs, variables and data
structures. Usually it includes calls for opening Midi,
Audio and/or Video Input channels and is called once
at the beginning of the composition. As a counter part
the Finish() function is called at the end and is used
for closing channel and dispose previously allocated
memory.

7.2 Instruments in pCM

An instrument is defined in the Orchestra() function
and consists of code for sound synthesis and
processing; it is continuously called at audio
sampling rate, that is 44100 times per second. An
instrument is defined in terms of an ordinary C
program (with all the programming facilities such as
for, do-while, and if-then-else control structures)
which calls functions belonging to the DSPlib;
assigning the results of the whole computation to two
predefined system variables outL and outR, sound
is generated. Look at the following simple example.
.
sig = ampli*Env(1)*OscSin(1,freq);
outL = sig*pan;
outR = sig*(1.-pan);

where
- sig is a local variable;
- ampli, freq and pan are global variables filled in by
Score();
- Env(1) and OscSin(1,freq) belong to the DSPlib
and sounds like that:
float OscSin (int nOsc, float freq)
{
float pos;
pos = oscFase[iN][nOsc] + freq;
if (pos>=tabLenfloat)
{pos=pos-tabLenfloat;}
if (pos<0) {pos=pos+tabLenfloat;}
oscFase[iN][nOsc] = pos;
return Tabsen[(long)pos];
}
float Env (int nEnv) // One-shot envelope
{
float vval,vv,pos;
long
ntabEnv =envNum[iN][nEnv];
pos = envPos[iN][nEnv];
vval = *(envTable[ntabEnv] + (long)pos);
if((long)pos<envLenght[ntabEnv]-1.0)
envPos[iN][nEnv]=pos+1.0;
return vval;
}

7.3 The Score

Once again the Score() is a C function which
prepares parametric values and fills the global
variables (such as ampli, freq and pan) used in the
instrument active in Orchestra().
There exist
different modalities for writing a score: following the
algorithmic composition approach, writing sequences
of predefined events, getting values coming from the
external gesture interfaces and, finally combining in
different ways these techniques.
Suppose we want to control in real-time a very
simple instrument
sig = OscSin(1,frq);
outL = sig*pan;
outR = sig*(1.-pan);

using movements of the mouse and linking the

vertical position to frequency and the horizontal

position to left-right panning. In the MacOs

environment, the mouse position is returned invoking
the GetNextEvent(..) tool-box function which leaves
the x,y position values can be found in the
Event.where.- variable and used as follows:
.
pan= Event.where.h/1023.;
frq= 800-Event.where.v;

These two lines make part of the Score() which is

automatically and repeatedly called by the main
mechanism. Since the mouse spans between 0 and
1023 horizontally and from 0 to 767 vertically, the
variable pan and frq communicate proper values to
the instrument for changing frequency and panoramic
position.
Its also possible to generate (offline or in realtime
under control of data streaming from gesture
interfaces) sequences of events automatically
activated by the Scheduler(), a special mechanism
which triggers sounds at the right times and change
parametric values in the instrument.
Being too long to explain here the potentialities of the
Scheduler() and the other facilities offered by pCM,
we stop here on the paper and promise well give a
satisfying demo during the meeting.

developing software for audio synthesis and

processing and hardware related to the gesture device
and systems; special thanks are also due to artist
Marco Cardini who greatly contributed and still is
contributing to the artistic life of cART lab of CNR in
Pisa and to Dr. Davide Filidei who, as a student first
and as a professional programmer now, put and puts
at work his skillfulness and love for Art.

References
[1] Tarabella L. - Guest Editor of the Special Issue on
Man-Machine Interaction in live Performance Interface, Journal of New Music Research, Swets &
Zeitlinger B.V. Vol.22 n.3 (1993)

[2] Tarabella L., Bertini G.,

Giving expression to
multimedia performances ACM Multimedia 2000,
Workshop Bridging the Gap: Bringing Together New
Media Artists and Multimedia Technologists Los
Angeles, 2000

[3] Desmond K., A timetable of Inventions and

Discoveries, M. Evans & Company, Inc. New York
(1985)

[4] Paradiso J., Electronic music: new ways to play. In

IEEE Spectrum, Vol. 34(12), pages 18-30. IEEE
Computer Society Press, (1997)

[5] Tarabella L., Magrini M., Scapellato G., Devices for

8 Conclusions
In the introduction we reported a small but
significative panorama of the state-of-the-art on the
different problems, solutions and results of people
and research centers around the world active on the
gesture interface topic. In particular we described our
approach and the results reached.
The systems and devices we realized have been used
several times in the last years for
demos,
conferences, lectures and concerts, always well
accepted with interest and enthusiasm by the
audience from both scientific and artistic points of
view.
From the experience gained after deeply using the
gesture devices and systems during our academic and
artistic activity, we realized that its time now for
better formalizing mapping strategies and for
classifying sets of gestures [21].
After all, this research activity finds now the right
place in the MOSART project for what concerns in
particular the involvement of young researchers of
the network.

9 Acknowledgments
Special thanks are due to Massimo Magrini and
Gabriele Boschi who, as professional programmers
and electronic designers, greatly contributed in

interactive computer music and computer graphics

performances , IEEE Computer Society Press,
(1997).

[6] Rowe R., Interactive Music Systems - Machine

Listening and Composing. MIT Press, 1993.

[7] Rowe, R., Machine Musicianship Cambridge: MIT

Press. March 2001 ISBN 0-262-18296-8

[8] Wexelblat A., An Approach to natural gesture in

virtual environment in ACM ToCHI, pp.179-200
ACM Press, (1996)

[9] Nigay L., Coutaz J., A generic platform for

addressing the multimodal challenge. In Procs of
ACM CHI95 pagg. 98-105, ACM Press

[10] M.Wanderly,<http://www.ircam.fr/equipes/analysesynthese/wanderle/Gestes/Externe/>

[11] Mulder A:, ``Virtual musical instruments: Accessing

the sound synthesis universe as a performer,'' in
Proceddings of the First Brazilian Symposium on
Computer Music, 1994.

[12] T. A. Marrin and R. Picard, ``A Methodology for

Mapping Gestures to Music Using Physiological
Signals.'' Presented at the International Computer
Music Conference(ICMC'98), Ann Arbor, Michigan,
1998.

[13] C. Cadoz and C. Ramstein, ``Capture, representation

and composition of the instrumental gesture,'' in
Proc. Int. Computer Music Conf. (ICMC'90), pp. 5356, 1990.

[14] P.

Depalle, S. Tassart, and M. Wanderley,

Instruments Virtuels Resonance, pp. 5-8, Sept. 1997.

[15] J. Rovan, M. Wanderley, S. Dubnov, and P. Depalle,

``Instrumental gestural mapping strategies as
expressivity determinants in computer music
performance'' in Proceedings of the Kansei - The
Technology of Emotion Workshop, (Genova - Italy),
Oct. 1997.

[16] R. Bargar, ``Multi-modal synchronization and the

automation of an observer's point of view,'' in
Proccedings of the 1998 IEEE International
Conference on Systems, Man, and Cybernetics
(SMC'98), 1998.

[17] Bertini G., Tarabella L., Magrini M., Spectral data

management
tools
for
additive
synthesis,
Proocedings of DAFX (Digital Audio Effects),
Universit di Verona, dicembre 2000

[18] Tarabella L., Bertini G., Sabbatini T., The Twin

Towers: a remote sensing device for controlling liveinteractive computer music. In Procs of 2nd
international workshop on mechatronical computer
system for perception and action, SSSUP, Pisa, (1997)

[19] Tarabella L., Magrini M., Scapellato G., A system for

recognizing shape, position and rotation of the hands
in in Proceedings of th Internationl Computer Music
Conference 97 pp 288-291, ICMA S.Francisco,
(1997)

[20] Tarabella L., Bertini G., Boschi G., A data streaming

based controller for realtime computer generated
music, Proceedings of ISMA2001, International
Symposium on Musical Acoustics, Fondazione Cini,
Venezia.

[21] R. Bargar, ``Multi-modal synchronization and the

automation of an observer's point of view,'' in
Proccedings of the 1998 IEEE International
Conference on Systems, Man, and Cybernetics
(SMC'98), 1998.