|
Synthesis of Shape Dependent Sounds with Physical Modeling |
|
|
Abstract:We describe a general framework for the simulation of sounds produced by colliding physical objects in a real time graphics environment. The framework is based on a physical model of the vibration dynamics of bodies. The computed sounds depend on the material of the body and its shape. A key contribution of this work is that the sounds also depend on the location of impact on a struck object. This adds important realism to virtual environments. The framework has been implemented in a Sonic Explorer program, which simulates a room with several objects such as a chair, tables, and rods. After a preprocessing stage, the user can hit the objects at different points to interactively produce realistic sounds. We also have created an online demo of the sound synthesis, written in Java.
IntroductionIn this paper we show how to synthesize realistic sounds of colliding objects from physical models of the geometry and material properties of objects. While previous work has considered some shape dependent properties (Gaver, 1993a),(Gaver, 1993b), we believe this is the first work to incorporate impact location dependent sonic information in a virtual environment. Due to space limitations, we can only give an outline of our work, which is described more completely in (van den Doel et al., 1996). The cognitive importance of realistic sounds is well known. As simulations become more interactive, for instance in large architectural walkthroughs (Airey, J. et al., 1990) and virtual reality, synthesizing realistic object sounds directly from physical models and rendering them in real time will be increasingly important. The generation of sounds can be characterized as shown in Figure 1, which depicts the process as a pipeline similar to the sound rendering pipeline of (T. Takala et al.,1992).
While this is a simplification, it indicates the major computational tasks in going from a collision event to the sound heard by a human ear. The focus of the paper is the initial stage of this pipeline: the computation of the sounds produced by vibrations of the object, which depend on the geometry of the object, its material properties, and the characteristics of the impact. Synthesis by physical modeling is presently a very active topic of research in computer music. Musical instruments are extremely complicated objects, which is the reason that most physical modeling in computer music has been based on phenomenological models rather than on strict physical laws. We are interested in much simpler objects, for which a more realistic model can be constructed.
General ApproachWhen an object is struck, the energy of impact causes deformations to propagate through the body, causing its outer surfaces to vibrate and emit sound waves. The resulting sound field propagates through and also interacts with the environment before reaching the inner ear where it is sensed. Real sounds therefore provide an important "image" of various physical attributes of the object, its environment and the impact event, including the force (or energy) of the impact, the material composition of the object, the shape and size, the place of impact on the object, and finally the location and environment of the object. The shape of the object and its structure (such as a plate versus a membrane) determines a characteristic frequency spectrum. This frequency signature facilitates the recognition of an object by its sound. A metal bar, for example, rings with a sparse non-harmonic spectrum, of which the higher modes decay rapidly. This is how we recognize its sound. The sound of an impact also depends on where an object is hit, and this also provides useful information about the environment. Though the frequency signature is the same over the object, the relative amplitudes change. Generally, an object sounds brighter (i.e. more upper partials are excited) when struck near an edge, than when struck near the center. Recall for example Sherlock Holmes, who taps on the walls to find a secret compartment.
Based on material and shape properties, we show that, within a linear model,
the acoustic response of an object is completely characterized by
its frequency spectrum Within this model, the sound of an impact on an object consists of a discrete set of exponentially decaying sine waves. The frequencies are determined by the geometry and material properties (such as elasticity) of the object, the initial amplitudes are determined by the location of the impact, and the decay rates are determined by the internal friction parameter. Since the resulting sound is described as a sum of sine waves, each with an exponential envelope, additive synthesis (Depalle et al., 1990) can be used to render the sounds. However, because all amplitudes decay exponentially, different synthesis algorithms that take advantage of this specific type of decay, as discussed in (Gaver, 1993b) can be used.
Results
We have considered systems that can be described by a linear wave equation
of the form
This method is very general, and allows the computation of the vibrations under impact of any object governed by a linear partial differential equation. We do a precomputation of the characteristic frequencies of each object and we divide the boundary of the object into small regions and determine the amplitudes of the excitation modes on this grid. After the preprocessing, a sound map is attached to an object, which allows us to render sounds resulting from impacts on the body. The sound map can consist of precomputed samples, or we can synthesize the sounds on the fly from the amplitudes, using the algorithm given in (Gaver, 1993a). This is similar to the tesselation of a surface for graphics rendering. The whole procedure is analogous to assigning a color to a surface and rendering it with some shading model.
The frequency spectrum
In general one has to resort to numerical methods. For membranes, the problem reduces to the solution of the Laplace equation on a given domain, which is a well studied problem. We mention the method of particular solutions (Fox et al., 1967), which we have adapted for the example of the L-shaped membrane. For plates, the differential equation is of fourth order, and a more general finite element method can be used. See for example (Johnson, 1987). We have constructed a testbed application, called the "Sonic Explorer", which demonstrates the level of reality that can be achieved within this model. The Sonic Explorer is currently set up to precompute the impact sounds of several types of objects, incorporate them in a real time interactive simulation, and render them in real time, using the audio hardware. A picture of a virtual room environment is given in Figure 3, and the sounds can be heard in the accompanying video.
We have also created an application in Java, which allows the user to generate objects with specific vibrational properties and interactively explore the sounds they make under various stimuli. The demo is available here.
ConclusionsThe testbed application has been succesful in simulating the sounds of simple objects. We demonstrated the testbed informally and found that people recognized the sounds as being "correct". However, much work needs to be done on quantifying how "good" the sounds are and what aspects need improvement most. To compute the sounds for complicated (curved) shapes requires a lot of numerical work through finite element modeling. An alternative would be to measure the model parameters (the frequency spectrum, the internal friction parameter, and the mode shapes) for real objects. An extension to more complicated sounds like scraping and rolling is relatively straightforward. When the eigenfrequencies and the eigenmodes are known, the vibration response of the object to any kind of external force can be computed in principle. The main challenge in this field is to find a real time synthesis algorithm. For these types of sounds a good model of the surface roughness of the objects is another direction for further research. Another possible direction for future research is to compute the effects of the material and shape of the striking object. We have assumed here that the interaction can be characterized by an impulsive force located at a single point.
ReferencesDepalle, P. and Rodet, X (1990). Synthèse additive par FFT inverse. Technical report, IRCAM, Paris.
About this document ...Synthesis of Shape Dependent Sounds with Physical Modeling The original of this document was generated using the LaTeX2 HTML translator Version 95 (Thu Jan 19 1995) Copyright © 1993, 1994, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
The translation was initiated by Kees van den Doel on Tue Apr 8 18:38:53 PDT 1997
Author Information
Kees van den Doel and Dinesh K. Pai |
, its normal mode shapes
(the eigenmodes 
of the vibration equation),
and a single material property, the
internal friction parameter 
and the eigenmodes are 
,
we have shown 
on the surface can be obtained from an
additive synthesis of sine-waves with frequencies
and initial amplitudes
and the eigenfunctions
can be computed analytically in a number of
cases. In Figure 2 we show a plot of the initial amplitudes
against frequency for a square membrane of unit length.
We have taken the lowest frequency to be 500 Hz and
taken the first 400 modes into account.
