Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
  • H04R1/00—Details of transducers, loudspeakers or microphones
  • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
  • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
  • H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
  • H04R1/403—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers loud-speakers

Definitions

• This invention relates in general to multi-channel audio production, editing and playback systems.
• the present invention discloses a method and apparatus for a dynamic, adaptive mapping of three-dimensional aural phenomenon to a two- dimensional sound reproducing surface comprised of sound pixels that are discrete, addressable output locations.
• actual or synthetic recorded audio material is decomposed into discrete sound sources which are then mapped to the sound pixels.
• This mapping occurs during the production phase by means of computer-aided design (CAD) system.
• CAD computer-aided design
• the CAD system guides the sound designer in the implementation of complex acoustical designs by automating the most complex and computationally intensive functions.
• the present invention extends the mapping function to integrate real-time human interaction, ultimately creating a participatory, interactive virtual sound environment.
• Figure 1 is a block diagram illustrating the basic components of the present invention
• FIG. 2 is a dataflow diagram that further describes the functions of the computer-aided design (CAD) system in the present invention
• Figure 3 is a block diagram illustrating the components of the sound reproducing surface and its electronics, wherein playback occurs in a static mode
• Figure 4 is a block diagram illustrating the components of the sound reproducing surface and its electronics, wherein playback occurs in a static mode with responsive inputs
• Figure 5 is a block diagram illustrating the components of the sound reproducing surface and its electronics, wherein playback occurs in a dynamic mode with responsive inputs and real-time audio;
• Figure 6 is a block diagram illustrating the components and interconnections of a plurality of sound reproducing surfaces
• Figure 7 is a block diagram illustrating the components of a static sound pixel using global digital signal processing
• Figure 8 is a block diagram illustrating the components of a dynamic sound pixel using local digital signal processing
• Figure 9 is a block diagram illustrating me components of a satellite network for sound pixels
• Figure 10 is a block diagram illustrating the components of a satellite sound pixel with analog signal processing
• Figure 11 is a block diagram illustrating the components of a satellite sound pixel with analog signal processing
• Figure 12 is a block diagram illustrating the components of a satellite sound pixel utilizing digital processing and having a MIDI interface
• Figure 13 is a block diagram illustrating the components of a satellite sound pixel utilizing digital processing and having analog output to other satellite sound pixels
• Figure 14 is a block diagram illustrating the components of a satellite sound pixel utilizing digital processing and having digital output to other satellite sound pixels;
• Figures 15A-15D are block diagrams illustrating various perspective views of a sound bubble constructed according to the teachings of the present invention.
• Figures 16A and 16B show the left panel and right panel of the sound bubble;
• Figure 17 shows the rear panel of the sound bubble;
• Figure 18 shows the top panel of the sound bubble
• Figure 19 is a block diagram of the electronic components used to control the operation of the sound bubble.
• Figure 1 is a block diagram illustrating the basic components of the present invention.
• the present invention is intended to have applications in several markets, ranging from special venue entertainment markets such as video arenas and theme parks, to more commonplace markets such as concert halls, theatres and video games.
• a two-dimensional planar surface 10 comprised of sound reproducing elements called sound pixels 12 is used to reproduce 3-dimensional aural phenomenon.
• the sound pixels 12 together produce an "aural image” or "sonic tapestry".
• the sound pixels 12 each comprise a discrete, addressable output location, wherein audio material is decomposed into a plurality of discrete sound data streams that are mapped to the sound pixels 12.
• the data streams that control the sound pixels are created using a computer- aided design (CAD) system 14 that guides a sound designer through the modeling of the acoustical playback environment.
• the CAD system 14 is used by the sound designer to create a virtual acoustical environment (VAE) modeled on the actual acoustical environment, of the playback venue to automate the capture or creation of propagating acoustical energy within the modeled environment, and to transform this captured acoustical energy into digital data streams for playback purposes.
• VAE virtual acoustical environment
• the CAD system 14 allows the sound designer to simulate real- world ambience and re-create the tangible events that form the basis for spatial perception of sound. Physical conditions associated with the VAE, including temporal and spectral characteristics, can be modeled by the CAD system 14.
• Audio material is "painted" onto a digital representation of the sound reproducing surface 10 in the CAD system 14, rather like an artist lays brush strokes on a canvas.
• Another way to look at the present invention is as a discrete-time digital sound stage, wherein the sound designer assumes the role of stage director, placing objects in areas on a stage and orchestrating their interaction.
• the CAD system 14 re-creates the interaction of multiple sound objects with one another, so that the additive nature of real- world sonic waveform interaction can be reproduced. Thereafter, the sound objects can be propagated through the VAE to represent changes in location for the sound producing entity.
• the audio material is compiled and placed in a high density storage device 16, such as a hard disc drive or optical disc.
• the audio material typically comprises discrete data streams that are mapped onto or assigned to the sound pixels 12.
• the CAD system 14 creates the digital data streams for the sound pixels 12 with the same characteristics of real-world sound events in a real environment.
• the data streams created by the CAD system 14 are retrieved from the storage device 16, processed, and then transmitted to the sound pixels 12 to reproduce a 3-dimensional aural phenomenon.
• Each sound pixel 12 contains a full range audio transducer that reproduces sound energy under control of the data streams.
• the playback of the audio material can be accomplished in either static or dynamic modes.
• static mode the data streams that comprise the audio material are "static" in that it is not altered once completed.
• the operator simply activates the playback system 18 and the audio material is replayed through the sound reproducing surface 10, possibly in synchronism with an accompanying video program.
• Sound objects can move through space and locations during playback by the data streams sequentially addressing different pixels 12 in the surface 10.
• dynamic mode the present invention extends the implementation of static mode to include real-time interaction, thereby creating participatory sound environments.
• Computer-Aided Design System Figure 2 is a dataflow diagram that further describes the functions of the computer-aided design (CAD) system 14 in the present invention.
• the sound designer first creates a graphical representation of the VAE. This is accomplished in a manner similar to prior art CAD systems by specifying dimensions, characteristics and parameters describing the VAE, including information relating to the perspective of the audience. These values may be specified by selecting values from a library of various acoustical characteristics such as reflection and absorption, or by selecting environments from a library of VAEs such as "listening rooms,” “theaters,” “concert halls,” or other typical acoustical environments. In addition, these values may be derived empirically, for example, by sampling the frequency response of a similar space over time and frequency.
• the values specified for a VAE form the basis of the computations performed by the CAD system 14 that result in the data streams used to control the sound pixels in the sound reproducing surface 10.
• the CAD system 14 of the present invention comprises an operating system and user interface 20, application modules 22-38, and system disc management module 40.
• the operating system and user interface 20 accepts various types of data, including user input, SMPTE time codes, and MIDI data.
• the operating system and user interface 20 also interacts with the operator and displays various information on a screen. Initially, the operator specifies various parameters describing the VAE and the playback system. Model parameters identify the characteristics of the VAE, and are augmented by detailed information on the scene dimensions and features. Array features specify the configuration of a particular playback system. Venue features are provided to describe characteristics of the playback venue.
• Application module 22 is used by the operator to modify the VAE model.
• the VAE model can be configured in a number of ways, for example, analytically or empirically.
• the VAE model may include standardized parameters from a look-up table of previously stored or standardized VAE models.
• the look-up table may include numerous references to acoustical properties of spaces and materials.
• Application module 24 is used by the operator to manage MIDI data input the CAD system 14. Typically, application module 24 generates MIDI-based location data and MIDI-based effects for use within the VAE.
• the VAE model is used by application module 26 to transform parameters such as the scene dimension and scene features into an acoustical model.
• the application module 24 can display a visual representation of this acoustical model for the operator.
• Application module 28 accepts parameters for sound pixel array features and venue features to generate data representing venue acoustics and array geometry /dimensions.
• the array geometry /dimensions describe the configuration of the venue surfaces, which can be flat or circular surfaces, so that the acoustical energy can be accurately mapped onto the sound pixels 12.
• the geometry /dimensions describe the prospective of the audience in the venue.
• Venue acoustics may represent the frequency responses of the various materials in the venue.
• Application module 30 provides the general functions for managing sound objects. For example, when creating a new sound object, the application module 30 accepts data representing the object features, object movement, object location and duration.
• Application module 32 manages the time base for all activities in the VAE. For example, all movement and action of sound objects in the VAE are identified by a time line reference.
• Application module 32 also accepts SMPTE time codes to synchronize the sound objects to external events.
• Application module 32 produces virtual time increments for use by other modules in synchronizing and referencing activities within the VAE.
• Application module 34 translates the acoustical model into the DSP coefficients and programming necessary to drive the sound pixels. These DSP coefficients and programming include filter coefficients, 2nd order effects, and delay coefficients. Application module 34 uses the venue acoustics and geometry/dimensions to achieve this function.
• Application module 36 manages the location of sound objects in the VAE using location data from application module 30, virtual time increments from application module 32, and MIDI-based location data from application module 24. Application module 36 produces object virtual location data which is used to modify the delay coefficients.
• Application module 38 is used to place sound objects in the ambience of the VAE, i.e. , it maps sound objects to sound pixels 12 in accordance with the time line reference indicated by the virtual time increments.
• Application module 38 accepts filter coefficients, delay coefficients, 2nd order effects, virtual timing increments, object virtual locations, sound objects, and array dimensions to perform these functions.
• Application module 38 transforms this data to create PCM data for the sound pixels, which is stored as discrete data streams for later retrieval by system disc manager 40.
• System disc manager 40 also retrieves sound objects from storage for manipulation by application module 38.
• Figure 3 is a block diagram illustrating the components of the sound reproducing surface 10 and its electronics, wherein playback occurs in a static mode.
• static mode the energy dispersion profiles and object location parameters assigned by the CAD system 14 are fixed and do not respond to changes in real-time conditions.
• Static mode provides a bare-bones embodiment of the present invention, wherein the focus is on solving the basic system problems, providing a solid foundation for other embodiments, and yet creating very sophisticated sound reproduction.
• the sound reproducing surface 10 has a plurality of addressable sound pixels
• the pixels 12 are coupled via bus 42 to digital and audio and control signal drivers 44.
• a time code generator 46 is coupled to the drivers 44 as well as a digital audio storage device 48, which can be any commercial device for storing the data streams such as a CD-ROM or a hard disk.
• the data streams are read from the digital audio storage device 48 and passed to the drivers 44 to control the operation of pixels 12.
• the drivers 44 transmit digital audio signals to the pixels 12 in AES/EBU format and transmit control signals to the pixels 12 in RS485 format.
• the time code generator 46 provides a reference clock to both the drivers 16 and the storage device 18 for synchronizing the transmission of the data streams to the pixels 12.
• the pixels 12 decode and amplify the digital audio signals in accordance with the control signals.
• the sound pixels 12 provide a distributed active speaker array with integral digital-to-analog converters.
• the sound reproducing surface 10 is preferably deformable and thus the sound pixels 12 can be arranged into any 2-dimensional or 3-dimensional configuration.
• the surface 10 may be curved in an arc of 270 degrees or more to correspond to the curvature of a screen.
• the surface 10 may curved into a spherical shape.
• some installations may only need a surface 10 with a flat, planar shape. The point is that playback architecmre is flexible enough to accommodate a wide range of requirements.
• the number of sound pixels 12 comprising the sound reproducing surface 10 are also flexible. As with an imaging system, the larger the number of energy producing elements, the greater the resolution of the reconstructed "image. " In the present invention, some applications may require only a small number of pixels 12, while other applications may require large numbers of pixels 12. Typically, the number of sound pixels 12 will not be a function of physical constraints, but rather of cost and space considerations. In more advanced "virtual reality” systems, it is conceivable that there could be many thousands of sound pixels 12 incorporated into a sound reproducing surface that encloses participants in a 360 degree sphere for complete realism in sound reproduction.
• Figure 4 is a block diagram illustrating the components of the sound reproducing surface 10 and its electronics, wherein playback occurs in a static mode with responsive inputs.
• one embodiment of the invention would be to add input from remote sensors 50 which could be a control pad or joy stick, infrared sensors, ultrasound sensors, or other apparatus well known in the art.
• the inputs from these remote sensors 50 would be received at 52 and processed at 54 to provide select/enable signals for determining which data streams are retrieved from a data storage device 56.
• the data streams retrieved from the digital storage 56 would be combined with or controlled or modified by the data streams retrieved from the storage device 48.
• Figure 5 is a block diagram illustrating the components of the sound reproducing surface and its electronics, wherein playback occurs in a dynamic mode with responsive inputs and real-time audio.
• one embodiment of the invention would be to add digitized real-time audio 50 from one or more microphones. These real-time inputs 50, as well as the inputs from the remote sensors 50, would be received at 60 and processed at 62 to provide select/enable signals for determining which data streams are retrieved from the data storage device 56. Further, processor 62 can generate local DSP control data and P channels of real-time digital audio data streams. The data streams retrieved from the digital storage 48 and the storage device 56 could be combined with the real-time audio data streams.
• FIG. 6 is a block diagram illustrating the components and interconnections of a plurality of sound reproducing surfaces.
• a master processor 64 is interconnected to a plurality of playback systems and controls the interaction therebetween.
• the master processor 64 is coupled to a local processor 66 controlling a particular array of sound pixels 12.
• Each local processor 66 is coupled to digital audio and control signal drivers 44, and digital audio storage 48.
• FIG. 7 is a block diagram illustrating the components of a static sound pixel 12 using global digital signal processing.
• the sound pixel 12 receives digital audio data at a receiver 68 and control signals at a receiver 70.
• the digital audio receiver 68 processes and transforms the digital audio signals into pulse code modulation (PCM) digital audio and timing data.
• the control data receiver 70 processes the control signals, which are comprised of enable and synchronization data, and transmits the enable and synchronization signals to the D/A converter 72.
• the D/A converter 72 transmits a monaural analog audio signal to an amplifier and low pass filter (LPF) 74 for driving the transducer 76 of the sound pixel 12.
• LPF low pass filter
• FIG 8 is a block diagram illustrating the components of a dynamic sound pixel 12 using local digital signal processing.
• a local DSP 78 processes the digital audio data and control signals.
• the DSP 78 transforms the digital audio signals and control data into PCM digital audio and timing data, synchronization data, and enable data, for transmission to the D/A convenor 72.
• Figure 9 is a block diagram illustrating the components of a satellite network for sound pixels 12.
• digital audio and control signals transmitted to "master" sound pixels 12 are simultaneously provided to "satellite" sound pixels 12.
• Satellite sound pixels 12 include components 80 to alter or other process the data streams transmitted to the master sound pixel 12.
• the satellite sound pixel 12 may alter characteristics such as gain, delay, or perform filtering of the data streams.
• FIG 10 is a block diagram illustrating the components of a satellite sound pixel 12 with analog signal processing.
• the satellite sound pixel 12 receives analog audio signals from the master sound pixel at receiver 82.
• the analog audio signals are modified by a delay and volume control circuit 84.
• the delay and control circuit 84 is controlled by a manual adjust or by MIDI commands received by receiver 86.
• the output of the delay and volume control circuit 84 is amplified and filtered at 74 before driving the transducer 76.
• Figure 11 is a block diagram illustrating the components of a satellite sound pixel 12 with analog signal processing.
• a local DSP 78 processes the digital audio data and control signals transmitted to the master sound pixel.
• the DSP 78 transforms the digital audio signals and control data into PCM digital audio and timing data, synchronization data, and enable data, for transmission to the D/A convenor 72, amplifier and low pass filter 74, and transducer 76.
• Figure 12 is a block diagram illustrating the components of a satellite sound pixel 12 utilizing digital processing and having a MIDI interface.
• MIDI commands received by a MIDI receiver 86 are used to control the delay and volume control circuit 84.
• FIG. 13 is a block diagram illustrating the components of a satellite sound pixel 12 utilizing digital processing and having analog output to other satellite sound pixels 12.
• a local DSP 78 processes the digital audio data and control signals transmitted to the master sound pixel 12.
• the control signals include satellite data, filtering data, and timing data.
• the DSP 78 transforms the digital audio signals and control data into PCM digital audio and timing data, synchronization data, and enable data, for transmission to the D/A convenor 72.
• the output of the D/A converter 72 is used to drive the amplifier and low pass filter 74 and transducer 76, as well as being shared with other satellite sound pixels 12 via analog drivers 88.
• Figure 14 is a block diagram illustrating the components of a satellite sound pixel 12 utilizing digital processing and having digital output to other satellite sound pixels 12.
• the local DSP 78 outputs filtered PCM digital audio and timing data to an AES/EBU formatter 90 and RS485 driver 92, as well as the D/A converter 72.
• FIGS 15A-15D are block diagrams illustrating various perspective views of a sound bubble 94 or sound tent constructed according to the teachings of the present invention.
• the walls of the sound bubble 94 are formed by the sound reproducing surface 10 of the present invention.
• the surface 10 will comprise a fabric incorporating various numbers of sound pixels 12.
• the enclosure may include a monitor 96 for use in generating video images.
• the sound bubble 94 is primarily intended as an enhancement to video games. Another embodiment would incorporate the sound bubble 94 into a helmet or visor or other similar personal device.
• Figures 16A and 16B show the left panel 98 and right panel 100 of the sound bubble 94, which can include a soft speaker configuration.
• the left panel 96 includes a super tweeter 102 and sub woofer 104. Between the super tweeter 102 and sub woofer 104 are array elements or sound pixels 106-110 (also labeled as IL, 2L, and 3L).
• the right panel 100 also includes a super tweeter 112 and sub woofer 114 and array elements or sound pixels 116-120 (also labeled as 1R, 2R, and 3R).
• Figure 17 shows the rear panel 122 of the sound bubble 94, which has one array element or sound pixel 124 (also labeled as 1R) located between a super tweeter 126, a sub woofer 128, and two flexible electrical connection panels 130 and 132.
• Figure 18 shows the top panel 134 of the sound bubble 94, which has one array element or stand pixel 136 located between two flexible connection panels 138 and 140 on each respective side of top panel 134.
• the flexible connection panels 138 and 140 each have a front flap 142 and 144 respectively located at an end.
• FIG 19 is a block diagram of the electronic components used to control the operation of the sound bubble 94.
• the input typically comprises digital audio input received from, for example, a CD-ROM or cartridge game.
• the digital audio is fed into a digital receiver 146, which processes and transforms the digital audio input into left data, right data, and word clock/bit clock. All data processed by digital receiver 146 is sent to a decompression and oversampling device 148.
• the decompression and oversampling device 148 then processes and transforms the left data, right data, and word clock/bit clock into an 8 bit digital signal, which is sent to a routing device 150 which routes the 8 bit digital signals to a filter bank 152.
• the filter bank 152 generates eight signals for the array elements, four supersonic signals for the super tweeters, and three subsonic signals for the sub woofers.
• the eight signals for the array elements are sent to an amplifier and volume control array 154, which also receives user volume control input.
• the amplifier and volume control array 154 produces signals for the array elements or sound pixels 12.
• the four supersonic signals for the super tweeters and the three subsonic signals for the sub woofers are sent to super tweeter and sub woofer amplifier and volume control array 156, which also receives user volume control input.
• the super tweeter and sub woofer amplifier and volume control array 156 mixes the user volume control input, supersonic signals, and subsonic signals and sends the resulting signals to the super tweeters and sub woofers.

Landscapes

• Health & Medical Sciences (AREA)
• Otolaryngology (AREA)
• Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• Acoustics & Sound (AREA)
• Signal Processing (AREA)
• Stereophonic System (AREA)
• Electrophonic Musical Instruments (AREA)
• Processing Or Creating Images (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=22254792

Family Applications (1)

Country Status (4)

Cited By (1)

Families Citing this family (4)

Family Cites Families (5)

• 1996
  • 1996-02-29 EP EP96908616A patent/EP0885545A1/de not_active Withdrawn
  • 1996-02-29 JP JP09530899A patent/JP2000505627A/ja active Pending
  • 1996-02-29 AU AU51798/96A patent/AU5179896A/en not_active Abandoned
  • 1996-02-29 WO PCT/US1996/002853 patent/WO1997032449A1/en not_active Ceased

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events