WO2023035218A1 - Multi-channel audio processing method, system and stereo apparatus - Google Patents

Multi-channel audio processing method, system and stereo apparatus Download PDF

Info

Publication number
WO2023035218A1
WO2023035218A1 PCT/CN2021/117633 CN2021117633W WO2023035218A1 WO 2023035218 A1 WO2023035218 A1 WO 2023035218A1 CN 2021117633 W CN2021117633 W CN 2021117633W WO 2023035218 A1 WO2023035218 A1 WO 2023035218A1
Authority
WO
WIPO (PCT)
Prior art keywords
pair
channel
signals
surround
channel audio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/117633
Other languages
French (fr)
Inventor
Xiaocheng Li
Jianwen ZHENG
Shaofu SHIH
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harman International Industries Inc
Original Assignee
Harman International Industries Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harman International Industries Inc filed Critical Harman International Industries Inc
Priority to PCT/CN2021/117633 priority Critical patent/WO2023035218A1/en
Priority to CN202180101821.2A priority patent/CN117859348A/en
Priority to EP21956401.0A priority patent/EP4399883A4/en
Priority to US18/690,697 priority patent/US20240388862A1/en
Publication of WO2023035218A1 publication Critical patent/WO2023035218A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation 
    • H04S5/005—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation  of the pseudo five- or more-channel type, e.g. virtual surround
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S1/00—Two-channel systems
    • H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/01—Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Definitions

  • This present disclosure relates to a method for processing multi-channel audio signals, a system for processing multi-channel audio signals and a stereo apparatus comprising such a system.
  • multi-channel speaker systems are becoming increasingly popular for consumers. These technologies have their own multi-channel audio encoding techniques that provide multi-channel audio signals, each of which is intended to be provided to and played back by a respective speaker of the multi-channel speaker systems, to provide a good spatial audio resolution as well as a good immersive surround sound experience.
  • Multi-channel speaker systems are normally named after their speakers or audio channels, such as 5.1/7.1/9.1/5.1.2/7.1.2/9.1.2/5.1.4/7.1.4/9.1.4 speaker systems.
  • a 5.1.2 speaker system is a multi-channel speaker system, with the “5” referring to left, right, center, left surround and right surround speakers and their five corresponding channels, and the “1” referring to a woofer speaker and its corresponding channel, and the “2” referring to left top and right top speakers and their two corresponding channels.
  • a 5.1.4 speaker system is a multi-channel speaker system, with the “5” referring to left, right, center, left surround and right surround speakers and their five corresponding channels, and the “1” referring to a low frequency effect speaker and its corresponding channel, and the “4” referring to left top front, right top front, left top rear and right top rear speakers and their four corresponding channels.
  • the speaker system can better reproduce height effects and thus provide a better immersive surround sound experience.
  • the 5.1.4 or 7.1.4 speaker system will be able to achieve the full 360-degree surround experience, namely the helicopter completes the full circle with pinpoint accuracy.
  • the speaker system can only complete the front 180-degree surround sound.
  • the helicopter only flies left and right in front of the listener, rather than flies a full circle around the listener as in the 5.1.4/7.1.4 channel speaker systems.
  • speaker systems without top speakers such as 5.1/7.1 channel speaker systems may downmix the top channels to the front and surround channels, and thus may not produce the height effects and may deteriorate the surround sound experience.
  • a multi-channel audio processing method comprises: receiving multi-channel audio signals from an external audio source, the multi-channel audio signals comprises a pair of surround channel signals and a pair of top channel signals; applying a crosstalk cancellation process in consideration of a head-related transfer function to the pair of top channel signals, so as to produce a pair of processed top channel signals, the head-related transfer function is configured to provide an elevation angle; mixing the pair of processed top channel signals, respectively, with the pair of surround channel signals, so as to produce a pair of mixed surround channel signals; providing the pair of mixed surround channel signals, respectively, to a pair of surround speakers.
  • a multi-channel audio processing system comprises a processor for performing a multi-channel audio processing method.
  • a stereo apparatus comprises: an audio source; a speaker system comprising a plurality of speakers, wherein the plurality of speaker comprises a pair of surround speakers; and a multi-channel audio processing system, the multi-channel audio processing system being configured to receive multi-channel audio signals from the audio source.
  • Fig. 1 shows a schematic view of a stereo apparatus 100 according to one or more embodiments of the present disclosure
  • Fig. 2 is a block diagram showing an exemplary crosstalk cancellation (CTC) process
  • Fig. 3 is a schematic view showing the elevated sound effect of the head related transfer function
  • Fig. 4 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more embodiments of the present disclosure
  • Fig. 5 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more alternative embodiments of the present disclosure
  • Fig. 6 shows an exemplary calibration process according to one or more embodiments of the present disclosure
  • Fig. 7 shows a schematic view about the calculation of the angle b in Fig. 6;
  • Fig. 8 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more further embodiments of the present disclosure.
  • the phrases “at least one of ⁇ A>, ⁇ B>, ... and ⁇ N>” or “at least one of ⁇ A>, ⁇ B>, ... ⁇ N>, or combinations thereof” are defined by the Applicant in the broadest sense, superseding any other implied definitions herebefore or hereinafter unless expressly asserted by the Applicant to the contrary, to mean one or more elements selected from the group comprising A, B, ... and N, that is to say, any combination of one or more of the elements A, B, ... or N including any one element alone or in combination with one or more of the other elements which may also include, in combination, additional elements not listed.
  • the present disclosure provides for a multi-channel audio processing method, a multi-channel audio processing system and a stereo apparatus comprising a multi-channel audio processing system.
  • the multi-channel audio processing method comprises: receiving multi-channel audio signals comprising a pair of surround channel signals and a pair of top channel signals; applying a crosstalk cancellation process and a head-related transfer function to the pair of top channel signals, so as to produce a pair of processed top channel signals; mixing the pair of processed top channel signals with the pair of surround channel signals, and providing the pair of mixed surround channel signals, respectively, to a pair of surround speakers.
  • the head-related transfer function is configured to provide an elevation of 30-60 degrees.
  • the pair of mixed surround channel signals contains a pair of surround channel signals and a pair of processed top channel signals, and is provided to and played by the pair of surround speakers.
  • the sounds produced by playing back the pair of surround channel signals (hereafter referred to as surround channel sound) are coming from the physical positions of the pair of surround speakers.
  • the top channel signals since the top channel signals have been applied with the crosstalk cancellation process and the head-related transfer function, it has an elevated sound effect as if the sounds produced by playing back the pair of processed top channel signals (hereafter referred to as top channel sounds) are coming from elevated positions relative to the physical positions of the surround speakers. That is, the method of the present disclosure may provide a pair of virtual top speakers at an elevated position relative to the physical positions of the surround speakers.
  • the method of the present disclosure may provide a pair of virtual top rear speakers for a 5.1.2 or 7.1.2 channel speaker system. That is, the present disclosure may provide a sound effect of a 5.1.4 or 7.1.4 channel speaker system by using a 5.1.2 or 7.1.2 channel speaker system. Thus, the present disclosure may have an advantage of providing a better immersive surround sound experience with reduced cost.
  • the method of the present disclosure may be used to retrofit an existing speaker system that does not include a pair of top rear speakers, such as a 5.1.2 or 7.1.2 speaker system, to provide a sound effect of a 5.1.4 or 7.1.4 channel speaker system without rewiring or adding new speakers. This may be particularly advantageous when the existing 5.1.2 or 7.1.2 speaker system is provided in a room where redecoration or rewiring is undesirable.
  • the method of the present disclosure may be applied to a 5.1 or 7.1 channel speaker system to provide a pair of virtual top speakers, to provide a sound effect of a 5.1.2 or 7.1.2 speaker system.
  • the method of the present disclosure may be applied to a 5.1 or 7.1 channel speaker system to provide two pair of virtual top speakers, to provide a sound effect of a 5.1.4 or 7.1.4 speaker system.
  • the present disclosure may be applied to either wired speaker system or wireless speaker system.
  • the method further comprises: calibrating the crosstalk cancellation process by adjusting parameters of the crosstalk cancellation process. Calibrating the crosstalk cancellation process may be performed automatically. Calibrating the crosstalk cancellation process may be performed periodically and/or upon an actuation of the user.
  • the crosstalk cancellation process may be adjusted to accommodate an actual position of the listener so that the listener may still enjoy an improved sound effect even when he or she is away from the original sweet spot of the speaker system.
  • the calibration may be performed automatically and conveniently upon an actuation of the user.
  • Fig. 1 shows a schematic view of a stereo apparatus 100 according to one or more embodiments of the present disclosure.
  • the stereo apparatus 100 comprises a 5.1.2 speaker system.
  • the 5.1.2 speaker system comprises a left speaker 112, a right speaker 114, a center speaker 116, a woofer speaker 118, a left surround speaker 122, a right surround speaker 124 (the left surround speaker 122 and the right surround speaker 124 may be collectively referred to as a pair of surround speaker) , a left top front speaker 132 and a right top front speaker 134.
  • the stereo apparatus 100 further comprises an audio source (not shown in Fig. 1) for providing multi-channel audio signals each for a respective speaker.
  • the stereo apparatus 100 further comprises a multi-channel audio processing system (not shown in Fig.
  • the multi-channel audio processing system is configured to receive at least a left surround channel signal, a right surround channel signal (the left surround channel signal and the right surround channel signal may be collectively referred to as a pair of surround channel signals) , a left top rear channel signal and a right top rear channel signal (the a left top rear channel signal and the right top rear channel signal may be collectively referred to as a pair of top rear channel signals) from the audio source.
  • the multi-channel audio processing system applies a virtual algorithm including a crosstalk cancellation process and a head-related transfer function to the left top rear channel signal and the right top rear channel signal and then mix them respectively with the left surround channel signal and the right surround channel signal before providing the mixed audio signals respectively to the left surround speaker and the right surround speaker.
  • the virtual algorithm is configured to provide an elevated sound effect so that the sound is coming from the ceiling, i.e., as if there is a pair of top rear speakers. That is, the virtual algorithm functions to provide a pair of virtual top rear speakers, i.e., a virtual left top rear speaker 142 and a virtual right top rear speaker 144.
  • the stereo apparatus 100 shown in Fig. 1 has only two physical top speakers, i.e., the left top front speaker 132 and the right top front speaker 134
  • the stereo apparatus 100 can produce a sound effect of four top speakers, i.e., the left top front speaker 132 and the right top front speaker 134 and the virtual left top rear speaker 142 and the virtual right top rear speaker 144.
  • the present disclosure may achieve a 360-degree surround height effect.
  • the multi-channel audio processing system and method of the present disclosure is used in conjunction with a 5.1.2 speaker system.
  • the present disclosure is not limited thereto.
  • the multi-channel audio processing system and method of the present disclosure may be applied to any suitable speaker system, such as a 7.1.2, 9.1.2, 11.1.2, 5.1, 7.1, 9.1 or 11.1 channel speaker system.
  • Fig. 2 is a block diagram showing an exemplary crosstalk cancellation (CTC) process.
  • the input of the process may be any suitable audio input.
  • the input may be the left top rear channel signal and the right top rear channel signal.
  • C stands for a transfer function between the speaker (s) and the listener.
  • H stands for a crosstalk cancellation function.
  • the crosstalk cancellation function may be defined as below.
  • the superscript H stands for a conjugate transpose operation
  • the superscript -1 stands for an inverse operation
  • a head related transfer function may be utilized in connection with the crosstalk cancellation process to provide or enhance the elevated sound effect.
  • a modified transfer function in consideration of the HRTF may be rewritten as follows.
  • C HRTF stands for a measured HRTF configured to provide an elevation angle.
  • the C HRTF may be measured by using two microphones positioned in a dummy head.
  • the measuring method for a C HRTF is known in the art and the detailed description thereof is omitted.
  • the C HRTF may also be obtained by a numerical simulation.
  • Fig. 3 is a schematic view showing the elevated sound effect of the head related transfer function.
  • the plane of Fig. 3 is substantially perpendicular to the plane of Fig. 1.
  • the virtual left top rear speaker 142 has an elevation angle ⁇ relative to the left surround speaker 122. That is, an elevation angle (an angle in a vertical direction) ⁇ is formed between the line from the listener 352 to the virtual left top rear speaker 142 and the line from the listener 352 to the left surround speaker 122.
  • the virtual right top rear speaker 144 also has an elevation angle relative to the right surround speaker 124. That is, the head-related transfer function is configured to provide an elevation angle ⁇ .
  • the elevation angle may be 30-60 degrees. In one or more embodiments of the present disclosure, the elevation angle may be about 60 degrees.
  • the HRTF mainly functions to achieve an elevated sound effect. However, the present disclosure is not limited thereto. In one or more further embodiments of the present disclosure, the HRTF may also function to achieve a surround angle, i.e., an angle in the plane of Fig. 1, in addition to the elevation angle.
  • Fig. 4 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more embodiments of the present disclosure.
  • the process receives multi-channel audio signals from an external audio source.
  • the multi-channel audio signals comprises a left surround (Ls) channel signal, a right surround (Rs) channel signal, a left top rear (Ltr) channel signal and a right top rear (Rtr) channel signal.
  • Ls left surround
  • Rs right surround
  • Ltr left top rear
  • Rtr right top rear
  • the mixed surround channel signals are provided to the left surround (Ls) speaker and the right surround (Rs) speaker respectively.
  • the left surround channel signal and the right surround channel signal are delayed to synchronize with processed left top rear channel signal and the processed right top rear channel signal respectively.
  • the step of delay is optional in the present disclosure. In one or more embodiments of the present invention, the step of delay may be omitted.
  • Fig. 5 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more alternative embodiments of the present disclosure.
  • the multi-channel audio processing method shown in Fig. 5 further comprises a calibration process of calibrating the crosstalk cancellation process by adjusting parameters of the crosstalk cancellation process.
  • the adjusted parameters comprises a distance and an angle of an intended listener relative to the pair of the surround speakers.
  • the multi-channel audio processing method shown in Fig. 5 is similar to that shown in Fig. 4 except the calibration process, and detailed description thereof is omitted.
  • Fig. 6 shows an exemplary calibration process according to one or more embodiments of the present disclosure.
  • the calibration process may be performed automatically upon an actuation of the user.
  • the calibration process may be started when a button, such as a button on a remote control, is pressed down by a user.
  • the calibration process may be configured to obtain the distances and angles of an intended listener 652 relative to the surround speakers 122, 124, such as the distances Dis_Ls, Dis_Rs, Dis_LsRs and the angles c, d shown in Fig. 6, and then calibrate the crosstalk cancellation process by using the obtained distances and angles, so as to accommodate the position of the intended listener 652. In this way, the listener 652 may enjoy an improved sound effect even when he or she is away from an original sweet spot of the speaker system.
  • each of the surround speakers 122, 124 plays a sweep test signal and hence emits a sweep sound, and the microphones 662, 662 receives the sweep sounds.
  • the calibration process may obtain or calculate a time period it takes for the sound to travel from the speakers 122, 124 to the microphone 662, 664, and the time difference between the time when the two microphones 662, 664 receives the sound from one speaker.
  • the calibration process then may obtain or calculate the distances and angles of an intended listener 652 relative to the surround speakers 122, 124 based on the obtained time period and time difference, as well as a preset listening distance Dis_C.
  • the distance Dis_L, the distance Dis_R, the angle a and the angle b shown in Fig. 6 may be obtained or calculated by using the time period it takes for the sound to travel from the speakers 122, 124 to the microphone 662, 664 and the time difference between the time when the two microphones 662, 664 receives the sound of a speaker.
  • the angles of the intended listener 652 relative to the surround speakers 122, 124 may be obtained or calculated from the distance Dis_L, the distance Dis_R, the angle a and the angle b.
  • the distance Dis_R may be obtained or calculated by multiplying a sound speed and the time period it takes for the sound to travel from the speakers 124 to the microphone 662, 664.
  • the angle b may be obtained or calculated by using the time difference between the time when the two microphones 662, 664 receives the sound from the speaker 124 (the time difference corresponding to ⁇ d) and the distance D between the two microphones 662, 664, as shown in Fig. 7.
  • the Dis_L and angle a may be obtained in a similar way.
  • the distances Dis_Ls, Dis_Rs, Dis_LsRs and the angles c, d may be obtained or calculated from Dis_L, Dis_R, the angles a, b and the preset listening distance Dis_C by using a geometry method.
  • the listening distance Dis_C is a distance value preset by the user. In one or more other embodiments, the listening distance Dis_C may be obtained or calculated in a way similar to the distance Dis_R. In short, a speaker may be placed at the position of the intended listener, and then speaker may emit a sweep sound and the microphones 662, 664 receives the sweep sound. Then the listening distance Dis_C may be obtained or calculated in a similar way as the distance Dis_R.
  • the calibration process of the present disclosure can be started by the user very easily, such as by a simple press down action of the user on a button.
  • the calibration procedure may be performed completed automatically without any user’s intervention.
  • the user can start the calibration process whenever he or she wants, so that he or she may still be able to enjoy an improved sound effect even when he or she has changed his or her position.
  • Fig. 8 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more further embodiments of the present disclosure.
  • the multi-channel audio processing method shown in Fig. 8 is similar to that shown in Fig. 4 except that in the embodiments shown in Fig. 4, the process applies a crosstalk cancellation process and then applies a head-related transfer function to the pair of top rear channel signals, while in the embodiments shown in Fig. 8, the process applies a head-related transfer function and then applies a crosstalk cancellation process to the pair of top rear channel signals, and detailed description thereof is omitted.
  • the present disclosure is not limited thereto, and the crosstalk cancellation process and the head-related transfer function may be applied in any suitable combination.
  • the modified transfer function in consideration of the head-related transfer function may be defined as one of the followings.
  • H [C H C] -1 C H C HRTF ;
  • the calibration process shown in Figs. 5-9 may be applied to any of the combination of the crosstalk cancellation process and the head-related transfer function.
  • the present disclosure can be implemented as follows.
  • Item 1 a multi-channel audio processing method, comprising:
  • the multi-channel audio signals comprises a pair of surround channel signals and a pair of top channel signals;
  • the head-related transfer function is configured to provide an elevation angle
  • Item 2 the multi-channel audio processing method according to Item 1, wherein the head-related transfer function is configured to provide an elevation of 30-60 degrees.
  • Item 3 the multi-channel audio processing method according to any of Items 1-2, wherein the crosstalk cancellation process in consideration of the head-related transfer function may be defined as one of the followings:
  • H [C H C] -1 C H C HRTF ;
  • H stands for the crosstalk cancellation process in consideration of the head-related transfer function
  • C HRTF stands for the head-related transfer function
  • C stands for a transfer function between speaker (s) and a listener
  • the superscript H stands for a conjugate transpose operation
  • the superscript -1 stands for an inverse operation.
  • Item 4 the multi-channel audio processing method according to any of Items 1-3, further comprising:
  • calibrating the crosstalk cancellation process by adjusting parameters of the crosstalk cancellation process, wherein the parameters comprises a distance and an angle of an intended listener position relative to the pair of the surround speakers.
  • Item 5 the multi-channel audio processing method according to any of Items 1-4, wherein calibrating the crosstalk cancellation process is performed automatically.
  • Item 6 the multi-channel audio processing method according to any of Items 1-5, wherein calibrating the crosstalk cancellation process is performed upon an actuation of the user.
  • Item 7 the multi-channel audio processing method according to any of Items 1-6, wherein multi-channel audio signals comprises 5.1.4 or 7.1.4 channel audio signals, and the pair of top channel signals is a pair of top rear channel signals.
  • Item 8 the multi-channel audio processing method according to any of Items 1-7, further comprising:
  • Item 9 a multi-channel audio processing system, comprising a processor for performing a method according to any of Items 1-8.
  • Item 10 A stereo apparatus, comprising: an audio source; a speaker system comprising a plurality of speakers, wherein the plurality of speaker comprises a pair of surround speakers; and a multi-channel audio processing system according to Item 9, the multi-channel audio processing system being configured to receive multi-channel audio signals from the audio source.
  • Item 11 the stereo apparatus according to Item 10, wherein the speaker system is a 5.1.2 speaker system or a 7.1.2 speaker system,
  • the pair of top channel signals is a pair of top rear channel signals, and the speaker system does not comprises a top rear speaker.
  • aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc. ) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit, ” “module” or “system. ”
  • the present disclosure may be a system, a method, and/or a computer program product.
  • the computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.
  • the computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
  • the computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • a non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , a static random access memory (SRAM) , a portable compact disc read-only memory (CD-ROM) , a digital versatile disk (DVD) , a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or Flash memory erasable programmable read-only memory
  • SRAM static random access memory
  • CD-ROM compact disc read-only memory
  • DVD digital versatile disk
  • memory stick a floppy disk
  • a mechanically encoded device such as punch-cards or raised structures in a groove having instructions
  • a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable) , or electrical signals transmitted through a wire.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function (s) .
  • the functions noted in the block may occur out of the order noted in the figures.
  • two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Stereophonic System (AREA)

Abstract

A method for processing multi-channel audio signals, a system for processing multi-channel audio signals and a stereo apparatus comprising such a system. The multi-channel audio processing method comprises: receiving multi-channel audio signals from an external audio source, the multi-channel audio signals comprises a pair of surround channel signals and a pair of top channel signals; applying a crosstalk cancellation process in consideration of a head-related transfer function to the pair of top channel signals, so as to produce a pair of processed top channel signals, the head-related transfer function is configured to provide an elevation angle; mixing the pair of processed top channel signals, respectively, with the pair of surround channel signals, so as to produce a pair of mixed surround channel signals; providing the pair of mixed surround channel signals, respectively, to a pair of surround speakers.

Description

MULTI-CHANNEL AUDIO PROCESSING METHOD, SYSTEM AND STEREO APPARATUS TECHNICAL FIELD
This present disclosure relates to a method for processing multi-channel audio signals, a system for processing multi-channel audio signals and a stereo apparatus comprising such a system.
BACKGROUND
With the development of multi-channel surround sound technologies like Dolby Atmos and DTS: X, multi-channel speaker systems are becoming increasingly popular for consumers. These technologies have their own multi-channel audio encoding techniques that provide multi-channel audio signals, each of which is intended to be provided to and played back by a respective speaker of the multi-channel speaker systems, to provide a good spatial audio resolution as well as a good immersive surround sound experience.
Multi-channel speaker systems are normally named after their speakers or audio channels, such as 5.1/7.1/9.1/5.1.2/7.1.2/9.1.2/5.1.4/7.1.4/9.1.4 speaker systems. For example, a 5.1.2 speaker system is a multi-channel speaker system, with the “5” referring to left, right, center, left surround and right surround speakers and their five corresponding channels, and the “1” referring to a woofer speaker and its corresponding channel, and the “2” referring to left top and right top speakers and their two corresponding channels. Similarly, a 5.1.4 speaker system is a multi-channel speaker system, with the “5” referring to left, right, center, left surround and right surround speakers and their five corresponding channels, and the “1” referring to a low frequency effect speaker and its corresponding channel, and the “4” referring to left top front, right top front, left top rear and right top rear speakers and their four corresponding channels.
With the four top speakers or top channels in the multi-channel speaker system, such as a 5.1.4 or 7.1.4 speaker system, the speaker system can better reproduce  height effects and thus provide a better immersive surround sound experience. For example, in some movie scenarios, such as those having a helicopter flying therearound, the 5.1.4 or 7.1.4 speaker system will be able to achieve the full 360-degree surround experience, namely the helicopter completes the full circle with pinpoint accuracy. On the other hand, with 5.1.2 or 7.1.2 speaker systems, due to the lack of the top rear speaker pair, the speaker system can only complete the front 180-degree surround sound. As a result, the helicopter only flies left and right in front of the listener, rather than flies a full circle around the listener as in the 5.1.4/7.1.4 channel speaker systems. Similarly, speaker systems without top speakers such as 5.1/7.1 channel speaker systems may downmix the top channels to the front and surround channels, and thus may not produce the height effects and may deteriorate the surround sound experience.
Therefore, there is a need to achieve a height effect, such as a 360-degree surround height effect, to provide a better surround sound experience, by using a speaker system that does not have a full four top speakers.
There have been attempts to achieve a better spatial audio by means of virtual sound features. Those features are commonly used to compensate for the missing speakers for the 2D listening positions to enhance the surround sound experience. The virtual sound features are achieved by algorithms based on the crosstalk cancellation, which relies on phase alignment at a cost of reduced sweet spot. Namely, when the listener is away from the sweet spot, the virtual effect degrades significantly. In some cases, this will further lead to phase misalignment which results in a deteriorated sound experience and spatial accuracy.
Therefore, there is a need for a better spatial audio by means of virtual sound features, with improved sound effect even when the listener is away from the sweet spot.
SUMMARY OF THE INVENTION
According to one aspect of the disclosure, a multi-channel audio processing method is provided. The multi-channel audio processing method comprises: receiving  multi-channel audio signals from an external audio source, the multi-channel audio signals comprises a pair of surround channel signals and a pair of top channel signals; applying a crosstalk cancellation process in consideration of a head-related transfer function to the pair of top channel signals, so as to produce a pair of processed top channel signals, the head-related transfer function is configured to provide an elevation angle; mixing the pair of processed top channel signals, respectively, with the pair of surround channel signals, so as to produce a pair of mixed surround channel signals; providing the pair of mixed surround channel signals, respectively, to a pair of surround speakers.
According to another aspect of the disclosure, a multi-channel audio processing system is provided. The multi-channel audio processing system comprises a processor for performing a multi-channel audio processing method.
According to another aspect of the disclosure, a stereo apparatus is provided. The stereo apparatus comprises: an audio source; a speaker system comprising a plurality of speakers, wherein the plurality of speaker comprises a pair of surround speakers; and a multi-channel audio processing system, the multi-channel audio processing system being configured to receive multi-channel audio signals from the audio source.
Other systems, method, features and advantages of the disclosure will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the disclosure, and be protected by the following claims.
BRIEF DESCRIPTION OF THE DRAWINGS
The disclosure can be better understood with reference to the flowing drawings and description. The components in the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure. Moreover, in the figures, like reference numerals designate corresponding parts throughout the different views.
Fig. 1 shows a schematic view of a stereo apparatus 100 according to one or more  embodiments of the present disclosure;
Fig. 2 is a block diagram showing an exemplary crosstalk cancellation (CTC) process;
Fig. 3 is a schematic view showing the elevated sound effect of the head related transfer function;
Fig. 4 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more embodiments of the present disclosure;
Fig. 5 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more alternative embodiments of the present disclosure;
Fig. 6 shows an exemplary calibration process according to one or more embodiments of the present disclosure;
Fig. 7 shows a schematic view about the calculation of the angle b in Fig. 6; and
Fig. 8 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more further embodiments of the present disclosure.
BRIEF DESCRIPTION OF THE PREFERRED EMBODIMENTS
Hereinafter, several embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.
As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms “comprises” , “comprising” , “includes” and/or “including” , as used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” and the symbol “/” are meant to include any and all combinations of one or more of the associated listed items. Additionally, while the terms first, second etc. may be used herein to describe various elements, components, steps or calculations, these elements, components, steps or calculations should not be limited  by these terms, rather these terms are only used to distinguish one element, component, step or calculation from another. For example, a first component could be termed a second component, similarly, a first calculation could be termed a second calculation; similarly a first step could be termed a second step; all without departing from the scope of this disclosure.
To clarify the use in the pending claims and to hereby provide notice to the public, the phrases “at least one of <A>, <B>, ... and <N>” or “at least one of <A>, <B>, ... <N>, or combinations thereof” are defined by the Applicant in the broadest sense, superseding any other implied definitions herebefore or hereinafter unless expressly asserted by the Applicant to the contrary, to mean one or more elements selected from the group comprising A, B, ... and N, that is to say, any combination of one or more of the elements A, B, ... or N including any one element alone or in combination with one or more of the other elements which may also include, in combination, additional elements not listed.
The present disclosure provides for a multi-channel audio processing method, a multi-channel audio processing system and a stereo apparatus comprising a multi-channel audio processing system. The multi-channel audio processing method comprises: receiving multi-channel audio signals comprising a pair of surround channel signals and a pair of top channel signals; applying a crosstalk cancellation process and a head-related transfer function to the pair of top channel signals, so as to produce a pair of processed top channel signals; mixing the pair of processed top channel signals with the pair of surround channel signals, and providing the pair of mixed surround channel signals, respectively, to a pair of surround speakers. The head-related transfer function is configured to provide an elevation of 30-60 degrees.
The pair of mixed surround channel signals contains a pair of surround channel signals and a pair of processed top channel signals, and is provided to and played by the pair of surround speakers. The sounds produced by playing back the pair of surround channel signals (hereafter referred to as surround channel sound) are coming from the physical positions of the pair of surround speakers. On the other hand, since the top channel signals have been applied with the crosstalk cancellation process and  the head-related transfer function, it has an elevated sound effect as if the sounds produced by playing back the pair of processed top channel signals (hereafter referred to as top channel sounds) are coming from elevated positions relative to the physical positions of the surround speakers. That is, the method of the present disclosure may provide a pair of virtual top speakers at an elevated position relative to the physical positions of the surround speakers.
In one or more embodiments, the method of the present disclosure may provide a pair of virtual top rear speakers for a 5.1.2 or 7.1.2 channel speaker system. That is, the present disclosure may provide a sound effect of a 5.1.4 or 7.1.4 channel speaker system by using a 5.1.2 or 7.1.2 channel speaker system. Thus, the present disclosure may have an advantage of providing a better immersive surround sound experience with reduced cost.
In one or more embodiments of the present disclosure, the method of the present disclosure may be used to retrofit an existing speaker system that does not include a pair of top rear speakers, such as a 5.1.2 or 7.1.2 speaker system, to provide a sound effect of a 5.1.4 or 7.1.4 channel speaker system without rewiring or adding new speakers. This may be particularly advantageous when the existing 5.1.2 or 7.1.2 speaker system is provided in a room where redecoration or rewiring is undesirable.
In one or more embodiments of the present disclosure, the method of the present disclosure may be applied to a 5.1 or 7.1 channel speaker system to provide a pair of virtual top speakers, to provide a sound effect of a 5.1.2 or 7.1.2 speaker system. In one or more embodiments of the present disclosure, the method of the present disclosure may be applied to a 5.1 or 7.1 channel speaker system to provide two pair of virtual top speakers, to provide a sound effect of a 5.1.4 or 7.1.4 speaker system. Moreover, the present disclosure may be applied to either wired speaker system or wireless speaker system.
In one or more further embodiments of the present disclosure, the method further comprises: calibrating the crosstalk cancellation process by adjusting parameters of the crosstalk cancellation process. Calibrating the crosstalk cancellation process may be performed automatically. Calibrating the crosstalk cancellation process may be  performed periodically and/or upon an actuation of the user.
With the calibration, the crosstalk cancellation process may be adjusted to accommodate an actual position of the listener so that the listener may still enjoy an improved sound effect even when he or she is away from the original sweet spot of the speaker system. The calibration may be performed automatically and conveniently upon an actuation of the user.
Fig. 1 shows a schematic view of a stereo apparatus 100 according to one or more embodiments of the present disclosure. The stereo apparatus 100 comprises a 5.1.2 speaker system. The 5.1.2 speaker system comprises a left speaker 112, a right speaker 114, a center speaker 116, a woofer speaker 118, a left surround speaker 122, a right surround speaker 124 (the left surround speaker 122 and the right surround speaker 124 may be collectively referred to as a pair of surround speaker) , a left top front speaker 132 and a right top front speaker 134. The stereo apparatus 100 further comprises an audio source (not shown in Fig. 1) for providing multi-channel audio signals each for a respective speaker. The stereo apparatus 100 further comprises a multi-channel audio processing system (not shown in Fig. 1) configured to receive audio signals from the audio source. The multi-channel audio processing system is configured to receive at least a left surround channel signal, a right surround channel signal (the left surround channel signal and the right surround channel signal may be collectively referred to as a pair of surround channel signals) , a left top rear channel signal and a right top rear channel signal (the a left top rear channel signal and the right top rear channel signal may be collectively referred to as a pair of top rear channel signals) from the audio source. The multi-channel audio processing system applies a virtual algorithm including a crosstalk cancellation process and a head-related transfer function to the left top rear channel signal and the right top rear channel signal and then mix them respectively with the left surround channel signal and the right surround channel signal before providing the mixed audio signals respectively to the left surround speaker and the right surround speaker. The virtual algorithm is configured to provide an elevated sound effect so that the sound is coming from the ceiling, i.e., as if there is a pair of top rear speakers. That is, the  virtual algorithm functions to provide a pair of virtual top rear speakers, i.e., a virtual left top rear speaker 142 and a virtual right top rear speaker 144.
Thus, although the stereo apparatus 100 shown in Fig. 1 has only two physical top speakers, i.e., the left top front speaker 132 and the right top front speaker 134, the stereo apparatus 100 can produce a sound effect of four top speakers, i.e., the left top front speaker 132 and the right top front speaker 134 and the virtual left top rear speaker 142 and the virtual right top rear speaker 144. Hence, the present disclosure may achieve a 360-degree surround height effect.
As shown Fig. 1, the multi-channel audio processing system and method of the present disclosure is used in conjunction with a 5.1.2 speaker system. However, the present disclosure is not limited thereto. In one or more embodiments of the present disclosure, the multi-channel audio processing system and method of the present disclosure may be applied to any suitable speaker system, such as a 7.1.2, 9.1.2, 11.1.2, 5.1, 7.1, 9.1 or 11.1 channel speaker system.
Fig. 2 is a block diagram showing an exemplary crosstalk cancellation (CTC) process. The input of the process may be any suitable audio input. For example, in the embodiment shown in Fig. 1, the input may be the left top rear channel signal and the right top rear channel signal. C stands for a transfer function between the speaker (s) and the listener. As shown, the input is applied with H, which stands for a crosstalk cancellation function. The crosstalk cancellation function may be defined as below.
H= [C HC]  -1C H
Wherein the superscript H stands for a conjugate transpose operation, the superscript -1 stands for an inverse operation.
A head related transfer function (HRTF) may be utilized in connection with the crosstalk cancellation process to provide or enhance the elevated sound effect. A modified transfer function in consideration of the HRTF may be rewritten as follows.
H=C HRTF [C HC]  -1C H
where C HRTF stands for a measured HRTF configured to provide an elevation angle. The C HRTF may be measured by using two microphones positioned in a dummy  head. The measuring method for a C HRTF is known in the art and the detailed description thereof is omitted. The C HRTF may also be obtained by a numerical simulation.
Fig. 3 is a schematic view showing the elevated sound effect of the head related transfer function. The plane of Fig. 3 is substantially perpendicular to the plane of Fig. 1. As shown in Fig. 3, the virtual left top rear speaker 142 has an elevation angle α relative to the left surround speaker 122. That is, an elevation angle (an angle in a vertical direction) α is formed between the line from the listener 352 to the virtual left top rear speaker 142 and the line from the listener 352 to the left surround speaker 122. Similarly, the virtual right top rear speaker 144 also has an elevation angle relative to the right surround speaker 124. That is, the head-related transfer function is configured to provide an elevation angle α.
In one or more embodiments of the present disclosure, the elevation angle may be 30-60 degrees. In one or more embodiments of the present disclosure, the elevation angle may be about 60 degrees. In the embodiments shown in Fig. 1, the HRTF mainly functions to achieve an elevated sound effect. However, the present disclosure is not limited thereto. In one or more further embodiments of the present disclosure, the HRTF may also function to achieve a surround angle, i.e., an angle in the plane of Fig. 1, in addition to the elevation angle.
Fig. 4 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more embodiments of the present disclosure. As shown, the process receives multi-channel audio signals from an external audio source. The multi-channel audio signals comprises a left surround (Ls) channel signal, a right surround (Rs) channel signal, a left top rear (Ltr) channel signal and a right top rear (Rtr) channel signal. Then the process applies a crosstalk cancellation process and a head-related transfer function to the left top rear channel signal and the right top rear channel signal. Then the processed left top rear channel signal and the processed right top rear channel signal is mixed with the left surround channel signal and the right surround channel signal, respectively. Then the mixed surround channel signals are provided to the left surround (Ls) speaker and the right surround (Rs) speaker  respectively. In one or more embodiments of the present disclosure, the left surround channel signal and the right surround channel signal are delayed to synchronize with processed left top rear channel signal and the processed right top rear channel signal respectively. Although shown in Fig. 4, the step of delay is optional in the present disclosure. In one or more embodiments of the present invention, the step of delay may be omitted. The block “CTC” and the block “HRTF” in Fig. 4 together means the transfer function applied to the Ltr and Rtr channel signals, and may be referred to collectively as a combination of a crosstalk cancellation process and a head-related transfer function or a crosstalk cancellation process in consideration of a head-related transfer function.
Fig. 5 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more alternative embodiments of the present disclosure. As shown, the multi-channel audio processing method shown in Fig. 5 further comprises a calibration process of calibrating the crosstalk cancellation process by adjusting parameters of the crosstalk cancellation process. In one or more embodiments of the present disclosure, the adjusted parameters comprises a distance and an angle of an intended listener relative to the pair of the surround speakers. The multi-channel audio processing method shown in Fig. 5 is similar to that shown in Fig. 4 except the calibration process, and detailed description thereof is omitted.
Fig. 6 shows an exemplary calibration process according to one or more embodiments of the present disclosure. The calibration process may be performed automatically upon an actuation of the user. In one or more embodiments of the present disclosure, the calibration process may be started when a button, such as a button on a remote control, is pressed down by a user.
The calibration process may be configured to obtain the distances and angles of an intended listener 652 relative to the  surround speakers  122, 124, such as the distances Dis_Ls, Dis_Rs, Dis_LsRs and the angles c, d shown in Fig. 6, and then calibrate the crosstalk cancellation process by using the obtained distances and angles, so as to accommodate the position of the intended listener 652. In this way, the listener 652 may enjoy an improved sound effect even when he or she is away from  an original sweet spot of the speaker system. Once the calibration process begins, each of the  surround speakers  122, 124 plays a sweep test signal and hence emits a sweep sound, and the  microphones  662, 662 receives the sweep sounds. Then the calibration process may obtain or calculate a time period it takes for the sound to travel from the  speakers  122, 124 to the  microphone  662, 664, and the time difference between the time when the two  microphones  662, 664 receives the sound from one speaker. The calibration process then may obtain or calculate the distances and angles of an intended listener 652 relative to the  surround speakers  122, 124 based on the obtained time period and time difference, as well as a preset listening distance Dis_C.
In one or more embodiments of the present disclosure, the distance Dis_L, the distance Dis_R, the angle a and the angle b shown in Fig. 6 may be obtained or calculated by using the time period it takes for the sound to travel from the  speakers  122, 124 to the  microphone  662, 664 and the time difference between the time when the two  microphones  662, 664 receives the sound of a speaker. The angles of the intended listener 652 relative to the  surround speakers  122, 124 may be obtained or calculated from the distance Dis_L, the distance Dis_R, the angle a and the angle b. For example, the distance Dis_R may be obtained or calculated by multiplying a sound speed and the time period it takes for the sound to travel from the speakers 124 to the  microphone  662, 664. The angle b may be obtained or calculated by using the time difference between the time when the two  microphones  662, 664 receives the sound from the speaker 124 (the time difference corresponding to Δd) and the distance D between the two  microphones  662, 664, as shown in Fig. 7. The Dis_L and angle a may be obtained in a similar way. Then, the distances Dis_Ls, Dis_Rs, Dis_LsRs and the angles c, d may be obtained or calculated from Dis_L, Dis_R, the angles a, b and the preset listening distance Dis_C by using a geometry method.
In one or more embodiments of the present disclosure, the listening distance Dis_C is a distance value preset by the user. In one or more other embodiments, the listening distance Dis_C may be obtained or calculated in a way similar to the distance Dis_R. In short, a speaker may be placed at the position of the intended listener, and then speaker may emit a sweep sound and the  microphones  662, 664  receives the sweep sound. Then the listening distance Dis_C may be obtained or calculated in a similar way as the distance Dis_R.
The calibration process of the present disclosure can be started by the user very easily, such as by a simple press down action of the user on a button. The calibration procedure may be performed completed automatically without any user’s intervention. Thus, the user can start the calibration process whenever he or she wants, so that he or she may still be able to enjoy an improved sound effect even when he or she has changed his or her position.
Fig. 8 shows an audio signal processing flowchart of the multi-channel audio processing method according to one or more further embodiments of the present disclosure. The multi-channel audio processing method shown in Fig. 8 is similar to that shown in Fig. 4 except that in the embodiments shown in Fig. 4, the process applies a crosstalk cancellation process and then applies a head-related transfer function to the pair of top rear channel signals, while in the embodiments shown in Fig. 8, the process applies a head-related transfer function and then applies a crosstalk cancellation process to the pair of top rear channel signals, and detailed description thereof is omitted. Although Figs. 4 and 8 shows embodiments of the combination of the crosstalk cancellation process and the head-related transfer function, the present disclosure is not limited thereto, and the crosstalk cancellation process and the head-related transfer function may be applied in any suitable combination. For example, the modified transfer function in consideration of the head-related transfer function may be defined as one of the followings.
H=C HRTF [C HC]  -1C H;
H= [C HC]  -1C HC HRTF;
H= [C HC]  -1C HRTFC H.
Moreover, the calibration process shown in Figs. 5-9 may be applied to any of the combination of the crosstalk cancellation process and the head-related transfer function.
According to some embodiments of the disclosure, the present disclosure can be  implemented as follows.
Item 1: a multi-channel audio processing method, comprising:
receiving multi-channel audio signals from an external audio source, the multi-channel audio signals comprises a pair of surround channel signals and a pair of top channel signals;
applying a crosstalk cancellation process in consideration of a head-related transfer function to the pair of top channel signals, to produce a pair of processed top channel signals, the head-related transfer function is configured to provide an elevation angle;
mixing the pair of processed top channel signals, respectively, with the pair of surround channel signals, to produce a pair of mixed surround channel signals;
providing the pair of mixed surround channel signals, respectively, to a pair of surround speakers.
Item 2: the multi-channel audio processing method according to Item 1, wherein the head-related transfer function is configured to provide an elevation of 30-60 degrees.
Item 3: the multi-channel audio processing method according to any of Items 1-2, wherein the crosstalk cancellation process in consideration of the head-related transfer function may be defined as one of the followings:
H=C HRTF [C HC]  -1C H;
H= [C HC]  -1C HC HRTF;
H= [C HC]  -1C HRTFC H,
wherein H stands for the crosstalk cancellation process in consideration of the head-related transfer function, C HRTF stands for the head-related transfer function, C stands for a transfer function between speaker (s) and a listener, the superscript H stands for a conjugate transpose operation, and the superscript -1 stands for an inverse operation.
Item 4: the multi-channel audio processing method according to any of Items 1-3, further comprising:
calibrating the crosstalk cancellation process by adjusting parameters of the crosstalk cancellation process, wherein the parameters comprises a distance and an angle of an intended listener position relative to the pair of the surround speakers.
Item 5: the multi-channel audio processing method according to any of Items 1-4, wherein calibrating the crosstalk cancellation process is performed automatically.
Item 6: the multi-channel audio processing method according to any of Items 1-5, wherein calibrating the crosstalk cancellation process is performed upon an actuation of the user.
Item 7: the multi-channel audio processing method according to any of Items 1-6, wherein multi-channel audio signals comprises 5.1.4 or 7.1.4 channel audio signals, and the pair of top channel signals is a pair of top rear channel signals.
Item 8: the multi-channel audio processing method according to any of Items 1-7, further comprising:
delaying the pair of surround channel signals to synchronize with the pair of processed top channel signals prior to the mixing.
Item 9: a multi-channel audio processing system, comprising a processor for performing a method according to any of Items 1-8.
Item 10: A stereo apparatus, comprising: an audio source; a speaker system comprising a plurality of speakers, wherein the plurality of speaker comprises a pair of surround speakers; and a multi-channel audio processing system according to Item 9, the multi-channel audio processing system being configured to receive multi-channel audio signals from the audio source.
Item 11: the stereo apparatus according to Item 10, wherein the speaker system is a 5.1.2 speaker system or a 7.1.2 speaker system,
wherein the pair of top channel signals is a pair of top rear channel signals, and the speaker system does not comprises a top rear speaker.
Aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc. ) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit, ” “module” or “system. ”
The present disclosure may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , a static random access memory (SRAM) , a portable compact disc read-only memory (CD-ROM) , a digital versatile disk (DVD) , a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable) , or electrical signals transmitted through a wire.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a  particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function (s) . In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
Systems and methods have been described in general terms as an aid to understanding details of the disclosure. In some instances, well-known structures, materials, and/or operations have not been specifically shown or described in detail to avoid obscuring aspects of the disclosure. In other instances, specific details have been given in order to provide a thorough understanding of the disclosure. One skilled in the relevant art will recognize that the disclosure may be embodied in other specific forms, for example to adapt to a particular system or apparatus or situation or material or component, without departing from the spirit or essential characteristics thereof. Therefore, the disclosures and descriptions herein are intended to be illustrative, but not limiting, of the scope of the disclosure. Accordingly, the disclosure is not to be restricted except in light of the attached claims and their equivalents.

Claims (11)

  1. A multi-channel audio processing method, comprising:
    receiving multi-channel audio signals from an external audio source, the multi-channel audio signals comprises a pair of surround channel signals and a pair of top channel signals;
    applying a crosstalk cancellation process in consideration of a head-related transfer function to the pair of top channel signals, to produce a pair of processed top channel signals, the head-related transfer function is configured to provide an elevation angle;
    mixing the pair of processed top channel signals, respectively, with the pair of surround channel signals, to produce a pair of mixed surround channel signals;
    providing the pair of mixed surround channel signals, respectively, to a pair of surround speakers.
  2. The multi-channel audio processing method according to claim 1, wherein the head-related transfer function is configured to provide an elevation angle of 30-60 degrees.
  3. The multi-channel audio processing method according to claim 1, wherein the crosstalk cancellation process in consideration of the head-related transfer function may be defined as one of the followings:
    H＝C HRTF［C HC］ -1C H;
    H=[C HC] -1C HC HRTF；
    H＝[C HC] -1C HRTFC H，
    wherein H stands for the crosstalk cancellation process in consideration of the head-related transfer function, C HRTF stands for the head-related transfer function, C stands for a transfer function between speaker (s) and a listener, the superscript  H stands for a conjugate transpose operation, and the superscript  -1 stands for an inverse  operation.
  4. The multi-channel audio processing method according to claim 1, further comprising:
    calibrating the crosstalk cancellation process by adjusting parameters of the crosstalk cancellation process, wherein the parameters comprises a distance and an angle of an intended listener position relative to the pair of the surround speakers.
  5. The multi-channel audio processing method according to any of claims 1-4, wherein calibrating the crosstalk cancellation process is performed automatically.
  6. The multi-channel audio processing method according to claim 5, wherein calibrating the crosstalk cancellation process is performed upon an actuation of the user.
  7. The multi-channel audio processing method according to any of claims 1-4, wherein multi-channel audio signals comprises 5.1.4 or 7.1.4 channel audio signals, and the pair of top channel signals is a pair of top rear channel signals.
  8. The multi-channel audio processing method according to any of claims 1-4, further comprising:
    delaying the pair of surround channel signals to synchronize with the pair of processed top channel signals prior to the mixing.
  9. A multi-channel audio processing system, comprising a processor for performing a method according to any of claims 1-8.
  10. A stereo apparatus, comprising:
    an audio source;
    a speaker system comprising a plurality of speakers, wherein the plurality of  speaker comprises a pair of surround speakers; and
    a multi-channel audio processing system according to claim 9, the multi-channel audio processing system being configured to receive multi-channel audio signals from the audio source.
  11. The stereo apparatus according to claim 10, wherein the speaker system is a 5.1.2 speaker system or a 7.1.2 speaker system,
    wherein the pair of top channel signals is a pair of top rear channel signals, and the speaker system does not comprises a top rear speaker.
PCT/CN2021/117633 2021-09-10 2021-09-10 Multi-channel audio processing method, system and stereo apparatus Ceased WO2023035218A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/CN2021/117633 WO2023035218A1 (en) 2021-09-10 2021-09-10 Multi-channel audio processing method, system and stereo apparatus
CN202180101821.2A CN117859348A (en) 2021-09-10 2021-09-10 Multi-channel audio processing method, system and stereo device
EP21956401.0A EP4399883A4 (en) 2021-09-10 2021-09-10 MULTICHANNEL AUDIO PROCESSING METHOD, SYSTEM AND STEREO APPARATUS
US18/690,697 US20240388862A1 (en) 2021-09-10 2021-09-10 Multi-channel audio processing method, system and stereo apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/117633 WO2023035218A1 (en) 2021-09-10 2021-09-10 Multi-channel audio processing method, system and stereo apparatus

Publications (1)

Publication Number Publication Date
WO2023035218A1 true WO2023035218A1 (en) 2023-03-16

Family

ID=85507157

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/117633 Ceased WO2023035218A1 (en) 2021-09-10 2021-09-10 Multi-channel audio processing method, system and stereo apparatus

Country Status (4)

Country Link
US (1) US20240388862A1 (en)
EP (1) EP4399883A4 (en)
CN (1) CN117859348A (en)
WO (1) WO2023035218A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050135643A1 (en) * 2003-12-17 2005-06-23 Joon-Hyun Lee Apparatus and method of reproducing virtual sound
US20050271213A1 (en) * 2004-06-04 2005-12-08 Kim Sun-Min Apparatus and method of reproducing wide stereo sound
US20050281408A1 (en) * 2004-06-16 2005-12-22 Kim Sun-Min Apparatus and method of reproducing a 7.1 channel sound
US20070154019A1 (en) * 2005-12-22 2007-07-05 Samsung Electronics Co., Ltd. Apparatus and method of reproducing virtual sound of two channels based on listener's position
CN109983785A (en) * 2016-11-29 2019-07-05 三星电子株式会社 Electronic device and its control method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102358283B1 (en) * 2016-05-06 2022-02-04 디티에스, 인코포레이티드 Immersive Audio Playback System
GB2569214B (en) * 2017-10-13 2021-11-24 Dolby Laboratories Licensing Corp Systems and methods for providing an immersive listening experience in a limited area using a rear sound bar
US12225367B2 (en) * 2019-03-06 2025-02-11 Harman International Industries, Incorporated Virtual height and surround effect in soundbar without up-firing and surround speakers

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050135643A1 (en) * 2003-12-17 2005-06-23 Joon-Hyun Lee Apparatus and method of reproducing virtual sound
US20050271213A1 (en) * 2004-06-04 2005-12-08 Kim Sun-Min Apparatus and method of reproducing wide stereo sound
US20050281408A1 (en) * 2004-06-16 2005-12-22 Kim Sun-Min Apparatus and method of reproducing a 7.1 channel sound
US20070154019A1 (en) * 2005-12-22 2007-07-05 Samsung Electronics Co., Ltd. Apparatus and method of reproducing virtual sound of two channels based on listener's position
CN109983785A (en) * 2016-11-29 2019-07-05 三星电子株式会社 Electronic device and its control method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4399883A4 *

Also Published As

Publication number Publication date
EP4399883A4 (en) 2025-04-30
CN117859348A (en) 2024-04-09
EP4399883A1 (en) 2024-07-17
US20240388862A1 (en) 2024-11-21

Similar Documents

Publication Publication Date Title
US7123731B2 (en) System and method for optimization of three-dimensional audio
US10341799B2 (en) Impedance matching filters and equalization for headphone surround rendering
US8831231B2 (en) Audio signal processing device and audio signal processing method
US9154896B2 (en) Audio spatialization and environment simulation
US10142761B2 (en) Structural modeling of the head related impulse response
RU2540774C2 (en) Method and apparatus for playing back stereophonic sound
JP5496235B2 (en) Improved reproduction of multiple audio channels
US20170070838A1 (en) Audio Signal Processing Device and Method for Reproducing a Binaural Signal
JP7285967B2 (en) foveated audio rendering
JP5757945B2 (en) Loudspeaker system for reproducing multi-channel sound with improved sound image
AU2001239516A1 (en) System and method for optimization of three-dimensional audio
JP2017513382A (en) Acoustic signal rendering method, apparatus, and computer-readable recording medium
US20170289724A1 (en) Rendering audio objects in a reproduction environment that includes surround and/or height speakers
JP6896626B2 (en) Systems and methods for generating 3D audio with externalized head through headphones
Glasgal 360° localization via 4. x RACE processing
US20180317003A1 (en) Three-dimensional sound reproduction method and device
US9872121B1 (en) Method and system of processing 5.1-channel signals for stereo replay using binaural corner impulse response
WO2019198314A1 (en) Audio processing device, audio processing method, and program
WO2023035218A1 (en) Multi-channel audio processing method, system and stereo apparatus
JP2017183779A (en) Localization method of sound reproduced from speaker and sound image localization apparatus used therefor
JP2007028066A (en) Audio reproducing system
JP2013176170A (en) Reproduction device and reproduction method
Kuhlen et al. A true spatial sound system for CAVE-like displays using four loudspeakers
Toole Direction and space–the final frontiers
Glasgal Achieving Physiological Realism in Music Recording and Reproduction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21956401

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180101821.2

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 18690697

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2021956401

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021956401

Country of ref document: EP

Effective date: 20240410