WO2024252597A1 - Dispositif de commande de directivité pour réseau de microphones, procédé de commande de directivité et programme - Google Patents

Dispositif de commande de directivité pour réseau de microphones, procédé de commande de directivité et programme Download PDF

Info

Publication number
WO2024252597A1
WO2024252597A1 PCT/JP2023/021257 JP2023021257W WO2024252597A1 WO 2024252597 A1 WO2024252597 A1 WO 2024252597A1 JP 2023021257 W JP2023021257 W JP 2023021257W WO 2024252597 A1 WO2024252597 A1 WO 2024252597A1
Authority
WO
WIPO (PCT)
Prior art keywords
directivity
directivity control
directional characteristic
processing unit
filter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/021257
Other languages
English (en)
Japanese (ja)
Inventor
健太 今泉
公孝 堤
真二 深津
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NTT Inc
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Priority to JP2025525559A priority Critical patent/JPWO2024252597A1/ja
Priority to PCT/JP2023/021257 priority patent/WO2024252597A1/fr
Publication of WO2024252597A1 publication Critical patent/WO2024252597A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
    • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
    • H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers

Definitions

  • One aspect of the present invention relates to a microphone array directivity control device, a directivity control method, and a program.
  • Non-Patent Document 1 proposes a technology for arbitrarily setting desired directional characteristics for a microphone array.
  • a beamformer is designed in which the generated directional characteristic has a directional characteristic in a direction other than the target direction, called a side lobe. As a result, it is not possible to pick up sound in the target direction with high directivity.
  • the objective of this invention is to provide a technology that generates an output acoustic signal that is highly directional and controlled from an input acoustic signal input from a microphone array, so that the sound in the desired target direction is highly directional.
  • the directivity control device has a directional characteristic determination processing unit that determines a desired directional characteristic from a desired sound pickup direction, a directional characteristic correction processing unit that corrects the desired directional characteristic using spherical harmonic function expansion, a directional control filter design processing unit that designs a directional control filter corresponding to each microphone based on the corrected desired directional characteristic, and a convolution calculation processing unit that performs a convolution calculation of the audio signal and the directional control filter to generate an output audio signal whose directivity is controlled to the desired directional characteristic.
  • One aspect of the present invention is a directivity control method for controlling the directivity of an input audio signal input from a microphone array having multiple microphones.
  • the directivity control method includes the steps of determining a desired directional characteristic from a desired sound pickup direction, correcting the desired directional characteristic using spherical harmonic function expansion, designing a directional control filter corresponding to each microphone based on the corrected desired directional characteristic, and performing a convolution operation of the audio signal and the directional control filter to generate an output audio signal whose directivity is controlled to the desired directional characteristic.
  • One aspect of the present invention is a directivity control program that controls the directivity of an input sound signal input from a microphone array having multiple microphones.
  • the directivity control program causes a computer having a processor and a storage device to execute at least some of the functions of the processing performed by each processing unit of the above-mentioned directivity control device.
  • a technology can be provided that generates an output audio signal in which sound from a desired target direction is controlled with high directivity from an input audio signal input from a microphone array.
  • FIG. 1 is a diagram showing an example of the configuration of a sound collection system including a directivity control device according to an embodiment of the present invention.
  • FIG. 2 is a block diagram showing an example of a hardware configuration of the directivity control device shown in FIG.
  • FIG. 3 is a block diagram showing an example of a software configuration of the directivity control device shown in FIG.
  • FIG. 4 is a flowchart showing an example of a processing procedure and processing contents performed by the directivity control device shown in FIG.
  • FIG. 5 is a diagram showing an example of an observation system for explaining a design method of a filter for directivity control using the least squares method.
  • FIG. 6 is a diagram showing a polar coordinate system.
  • FIG. 7 is a diagram showing spherical harmonics up to the third order.
  • FIG. 1 is a diagram showing the overall configuration of a sound collection system including a directivity control device according to an embodiment of the present invention.
  • the sound collection system includes a microphone array MC, a directivity control device 1, and an input/output device 2 connected to the directivity control device 1.
  • the microphone array MC has a plurality of microphones MC 1 to MC M.
  • the microphones MC 1 to MC M are arranged at equal intervals on the same circumference.
  • the microphones MC 1 to MC M are arranged so that their respective sound collection directions are oriented radially.
  • the microphones MC 1 to MC M of the microphone array MC convert sounds input from their respective sound collection directions into electrical signals, and output them to the directivity control device 1 as input acoustic signals IS 1 to IS M.
  • the directivity control device 1 is a device that generates an output audio signal OS in which sound in a desired target direction is controlled with high directivity by performing a convolution operation between an input audio signal IS input from each microphone MC1 to MCM of a microphone array MC and a directivity control filter corresponding to each microphone.
  • Directivity control device 1 2 and 3 are block diagrams showing the hardware and software configurations, respectively, of the directivity control device 1. As shown in FIG.
  • the directional control device 1 includes a control unit 10 that uses a hardware processor such as a central processing unit (CPU).
  • a storage unit having a program storage unit 20 and a data storage unit 30, an input/output interface (hereinafter, the interface will be referred to as an I/F) 40 to which an input/output device 2 is connected, and an acoustic signal I/F 50 are connected to the control unit 10 via a bus 60.
  • the acoustic signal I/F 50 is used to receive input acoustic signals IS (IS 1 to IS M ) output by each microphone MC 1 to MC M of the microphone array MC, and to output an output acoustic signal OS in which sound in a desired target direction is controlled with high directivity.
  • the program storage unit 20 is configured, for example, by combining a non-volatile memory such as a HDD (Hard Disk Drive) or SSD (Solid State Drive) as a storage medium that can be written to and read from at any time, and a non-volatile memory such as a ROM (Read Only Memory), and stores middleware such as an OS (Operating System) as well as programs necessary to execute various control processes according to one embodiment.
  • a non-volatile memory such as a HDD (Hard Disk Drive) or SSD (Solid State Drive) as a storage medium that can be written to and read from at any time
  • a non-volatile memory such as a ROM (Read Only Memory)
  • middleware such as an OS (Operating System)
  • OS Operating System
  • the data storage unit 30 is, for example, a combination of a non-volatile memory such as an HDD or SSD, which can be written to and read from at any time, and a volatile memory such as a RAM (Random Access Memory), as a storage medium, and includes a directional characteristic storage unit 31 and a filter storage unit 32 as the main storage units required to implement one embodiment.
  • a non-volatile memory such as an HDD or SSD
  • a volatile memory such as a RAM (Random Access Memory)
  • RAM Random Access Memory
  • the directional characteristic memory unit 31 is used to store the desired directional characteristic determined by the control unit 10.
  • the filter memory unit 32 is used to store the directional control filter calculated by the control unit 10.
  • the control unit 10 includes, as processing functions necessary to implement one embodiment, a directional characteristic determination processing unit 11, a directional characteristic correction processing unit 12, a directivity control filter design processing unit 13, and a convolution calculation processing unit 14. All of these processing units 11 to 14 are realized by causing the hardware processor of the control unit 10 to execute a program stored in the program storage unit 20.
  • the directional characteristic determination processing unit 11 receives the sound pickup direction signal DS, which indicates the desired sound pickup direction, input from the input/output device 2 via the input/output I/F 40, determines the directional characteristic of the microphone array MC, and stores the determined desired directional characteristic in the directional characteristic storage unit 31.
  • the directional characteristic correction processing unit 12 performs a process to correct the desired directional characteristic determined by the directional characteristic determination processing unit 11 using spherical harmonic function expansion by t-design.
  • the directivity control filter design processing unit 13 performs processing to design a directivity control filter based on the desired directivity corrected by the directivity correction processing unit 12, and stores the directivity control filter calculated by this processing in the filter storage unit 32.
  • the convolution calculation processing unit 14 performs a convolution calculation between the input acoustic signal IS (IS 1 to IS M ) input via the acoustic signal I/F 50 and the directivity control filter stored in the filter memory unit 32, and generates an output acoustic signal OS in which sound in the desired target direction is controlled with high directivity.
  • control points are placed around a microphone array in which multiple microphones are lined up, and a directional control filter is designed based on the characteristics of transmission from the microphones to the control points. For example, multiple microphones are placed at equal intervals on the same circumference.
  • the directional control filter is a filter that controls the direction in which each microphone picks up sound or the direction in which it does not pick up sound.
  • a typical method for designing a directional control filter is to use the least squares method.
  • Figure 5 shows an example of an observation system to explain the design method for a directional characteristic filter using the least squares method.
  • M (1 ⁇ m ⁇ M) indicates the number of microphones
  • Q (1 ⁇ q ⁇ Q) indicates the number of control points.
  • W m ( ⁇ ) is a filter corresponding to the m-th microphone
  • G qm ( ⁇ ) is a transfer function from the m-th microphone MC m to the q-th control point CP q .
  • G( ⁇ ) is a transfer function matrix of Q rows and M columns that stores the transfer functions G qm ( ⁇ ) from each microphone MC 1 to MC M to each control point CP 1 to CP Q.
  • the transfer function G qm ( ⁇ ) is given by the following equation.
  • w m ( ⁇ ) represents the filter coefficient corresponding to the m-th microphone.
  • the superscript * represents the complex conjugate. If the filter gain is large, the input signal also becomes larger proportionally, and the noise mixed in the picked-up signal is amplified, causing a decrease in performance. In response to this, there is a method of deriving a filter that controls directivity by using a penalty term, which will be described later, for the objective function that derives the filter. In this case, the sum of squares of the filter coefficients is used as the penalty term to suppress the filter gain.
  • ⁇ ( ⁇ ) is a regularization parameter that controls the relative weight between the loss term
  • W( ⁇ ) is obtained by solving the minimization problem for w( ⁇ ).
  • I is a unit matrix with M rows and M columns, and M is the number of microphones.
  • a representative method of directional control is a method of minimizing the error between the desired directional characteristic and the reproduced directional characteristic.
  • the target direction is often set to 1 and other directions to 0, but since this is a directional characteristic that is difficult to reproduce and this method aims to minimize the overall directional characteristic, it is difficult to control the shape of the side lobe.
  • the desired directional characteristic is replaced with a window function to suppress side lobes, which are directional characteristics in directions other than the target direction. Because a window function is set as the desired directional characteristic, the beam width of the directional characteristic in the target direction becomes wider, but it has been confirmed that the side lobes can be suppressed.
  • Y n,m ( ⁇ , ⁇ ) is a spherical harmonic function
  • a n,m ( ⁇ ) is its expansion coefficient, which is defined by the following equation.
  • Equation (12) is called the spherical harmonic expansion.
  • Y n,m ( ⁇ , ⁇ ) * is the complex conjugate of the spherical harmonic function.
  • the spherical harmonic expansion coefficients can be derived using control points spaced equally ( ⁇ , ⁇ ) using the following formula, which is an approximation of formula (10).
  • ⁇ q is a correction coefficient.
  • Equation (12) when expressing the integral as a discrete sum, approximation is introduced, so accurate expansion coefficients cannot be obtained.
  • Fig. 4 is a flowchart showing the processing procedure and processing contents of the directivity control of the microphone array MC executed by the directivity control device 1.
  • a filter is designed to realize desired directivity reproduction by solving the approximate gradient method with the sum of squares of the error between the desired directional characteristic corrected using the spherical harmonic function expansion of t-design and the reproduced directional characteristic as the target function, and an indicator function such that the filter coefficient falls within an arbitrary range.
  • (5-1) Determination of Desired Directivity
  • a system administrator or a user inputs a desired sound collection direction to be reproduced in the microphone array MC to the input/output device 2.
  • the input/output device 2 generates a sound collection direction signal DS representing the desired sound collection direction and sends it to the directivity control device 1.
  • the control unit 10 of the directivity control device 1 receives the sound collection direction signal DS from the input/output device 2 via the input/output I/F 40, and determines a desired directional characteristic from the sound collection direction signal DS under the control of the directional characteristic determination processing unit 11 in step S11.
  • the desired directional characteristic corresponds to a signal observed at a control point CP q arranged by t-design.
  • the control unit 10 of the directivity control device 1 stores the desired directional characteristic determined by the directional characteristic determination processing unit 11 in the directional characteristic storage unit 31.
  • step S12 the directional characteristic correction processing unit 12 reads the desired directional characteristic from the directional characteristic storage unit 31 and corrects the desired directional characteristic. For this reason, the directional characteristic correction processing unit 12 calculates spherical harmonic expansion coefficients A n,m for the input directional characteristic using the maximum order N max according to the following formula. N max is uniquely determined at the time of designing t-design.
  • step S13 the control unit 10 of the directivity control device 1 calculates a filter for directivity control by solving an optimization problem of the objective function expressed by equation (3) under the objective function set based on the desired directional characteristic corrected by the directional characteristic correction processing unit 12 and the range constraint of the microphone gain under the control of the directivity control filter design processing unit 13.
  • the optimization problem is defined as follows.
  • ⁇ is the update rate and prox represents the proximity operator.
  • the update formula for the resulting algorithm is expressed as follows:
  • k is the number of updates
  • Tmax is the maximum number of updates
  • the algorithm stops when the number of updates k exceeds the maximum number of updates Tmax .
  • the control unit 10 of the directivity control device 1 stores the directivity control filter calculated by the directivity control filter design processing unit 13 in the filter storage unit 32.
  • control unit 10 of the directivity control device 1 executes the process of directivity control of the microphone array MC as follows.
  • the control unit 10 of the directivity control device 1 receives the input acoustic signal IS from the microphone array MC via the acoustic signal I/F 50. At this time, if the input acoustic signal IS is an analog signal, it is converted into a digital signal by the acoustic signal I/F 50.
  • the control unit 10 of the directivity control device 1 performs a convolution calculation between the input acoustic signal IS received by the acoustic signal I/F 50 and the directivity control filter stored in the filter storage unit 32. As a result, an acoustic signal is generated in which sound in the desired target direction is controlled with high directivity. The generated acoustic signal is converted into an analog signal by the acoustic signal I/F 50 and then output as the output acoustic signal OS.
  • the directivity control device 1 is a device that generates an output audio signal OS in which sound in a desired target direction is controlled with high directivity by performing a convolution operation between the input audio signals IS1 to ISM input from each microphone MC1 to MCM of the microphone array MC and a directivity control filter corresponding to each microphone.
  • the directional characteristic determination processing unit 11 determines the desired directional characteristic from the input sound pickup direction.
  • the desired directional characteristic is corrected using spherical harmonic expansion by t-design.
  • the directivity control filter design processing unit 13 designs a directivity control filter that achieves the desired directivity reproduction by solving the proximity gradient method with an indicator function that causes the filter coefficients to fall within a given range as the target function.
  • the convolution calculation processing unit 14 performs a convolution calculation between the input sound signal IS and the calculated directivity control filter to generate an output sound signal in which the sound in the desired target direction is controlled with high directivity.
  • the present invention is not limited to the above embodiment.
  • the directional characteristic correction process, the directional control filter design process, and the convolution calculation process executed by the directivity control device 1 are all realized by having a hardware processor (CPU) execute a program.
  • CPU hardware processor
  • some or all of these functions may be realized by using an integrated circuit configured for a specific application, such as an ASIC (Application Specific Integrated Circuit) or a DMC (Digital Signal Processor).
  • this invention is not limited to the above-described embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention.
  • various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

Landscapes

  • Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

Un aspect de la présente invention concerne un dispositif de commande de directivité qui commande la directivité d'un signal acoustique entré à partir d'un réseau de microphones ayant une pluralité de microphones. Ce dispositif de commande de directivité comprend : une unité de traitement de détermination de caractéristique de directivité qui détermine une caractéristique de directivité souhaitée à partir d'une direction de collecte de son souhaitée ; une unité de traitement de correction de caractéristique de directivité qui corrige la caractéristique de directivité souhaitée en utilisant une expansion de fonction harmonique sphérique ; une unité de traitement de conception de filtre de commande de directivité qui conçoit un filtre de commande de directivité correspondant à chacun des microphones sur la base de la caractéristique de directivité souhaitée corrigée ; et une unité de traitement d'opération de convolution qui effectue des opérations de convolution du signal acoustique d'entrée et du filtre de commande de directivité, et génère un signal acoustique de sortie dont la directivité est commandée selon la caractéristique de directivité souhaitée.
PCT/JP2023/021257 2023-06-07 2023-06-07 Dispositif de commande de directivité pour réseau de microphones, procédé de commande de directivité et programme Ceased WO2024252597A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2025525559A JPWO2024252597A1 (fr) 2023-06-07 2023-06-07
PCT/JP2023/021257 WO2024252597A1 (fr) 2023-06-07 2023-06-07 Dispositif de commande de directivité pour réseau de microphones, procédé de commande de directivité et programme

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/021257 WO2024252597A1 (fr) 2023-06-07 2023-06-07 Dispositif de commande de directivité pour réseau de microphones, procédé de commande de directivité et programme

Publications (1)

Publication Number Publication Date
WO2024252597A1 true WO2024252597A1 (fr) 2024-12-12

Family

ID=93795615

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/021257 Ceased WO2024252597A1 (fr) 2023-06-07 2023-06-07 Dispositif de commande de directivité pour réseau de microphones, procédé de commande de directivité et programme

Country Status (2)

Country Link
JP (1) JPWO2024252597A1 (fr)
WO (1) WO2024252597A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012523731A (ja) * 2009-04-09 2012-10-04 エヌティーエヌユー テクノロジー トランスファー エーエス センサーアレイに最適なモーダルビームフォーマ
JP2019075616A (ja) * 2017-10-12 2019-05-16 日本電信電話株式会社 音場収録装置及び音場収録方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012523731A (ja) * 2009-04-09 2012-10-04 エヌティーエヌユー テクノロジー トランスファー エーエス センサーアレイに最適なモーダルビームフォーマ
JP2019075616A (ja) * 2017-10-12 2019-05-16 日本電信電話株式会社 音場収録装置及び音場収録方法

Also Published As

Publication number Publication date
JPWO2024252597A1 (fr) 2024-12-12

Similar Documents

Publication Publication Date Title
CN110557710B (zh) 具有语音控制的低复杂度多声道智能扩音器
JP6968376B2 (ja) ステレオ仮想バス拡張
CN103037290B (zh) 音频处理装置以及音频处理方法
US20110286601A1 (en) Audio signal processing device and audio signal processing method
JP6893986B2 (ja) ブライトゾーンおよびダークゾーンに関して最適化された音声前置補償フィルタ
CN109196884A (zh) 声音再现系统
EP3651355B1 (fr) Essai de puissance et procédé d'amélioration de la qualité sonore
JP2011010183A (ja) 楽曲再生システム、携帯端末装置および楽曲再生プログラム
WO2024037189A9 (fr) Procédé et appareil d'étalonnage d'image acoustique
JP2012169895A (ja) 多重極スピーカ群とその配置方法と、音響信号出力装置とその方法と、その方法を用いたアクティブノイズコントロール装置と音場再生装置と、それらの方法とプログラム
WO2024252597A1 (fr) Dispositif de commande de directivité pour réseau de microphones, procédé de commande de directivité et programme
JP2016092562A (ja) 音声処理装置および方法、並びにプログラム
TWI501657B (zh) 電子音訊裝置
JP7616363B2 (ja) エリア再生システムとその再生制御装置、方法およびプログラム
JPWO2018167921A1 (ja) 信号処理装置
JP7803421B2 (ja) エリア再生のための再生制御装置、再生制御方法、およびプログラム
US11875774B2 (en) Sound image localization device, sound image localization method, and program
US20230336913A1 (en) Acoustic processing device, method, and program
US20250220349A1 (en) Tuning of multiband audio systems executing crosstalk cancellation
JP6699280B2 (ja) 音響再生装置
CN110740404A (zh) 一种音频相关性的处理方法及音频处理装置
JP7260821B2 (ja) 信号処理装置、信号処理方法および信号処理プログラム
RU2852710C2 (ru) Кластеризация аудиообъектов
JPH1051890A (ja) オーディオ信号伝送回路
CN120786223A (zh) 耳机透传控制方法、耳机和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23940698

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025525559

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE