JP6233023B2 - Acoustic processing apparatus, acoustic processing method, and acoustic processing program - Google Patents

Description

  The present invention relates to an acoustic processing device, an acoustic processing method, and an acoustic processing program.

  By using the head related impulse response (HRIR), which shows the temporal characteristics of the sound from the human head and pinna to the left and right ears, the sound can be heard from any direction. There is technology that makes people feel as if they are heard. This kind of technology is called sound image localization technology or stereophonic technology.

  There are individual differences in the head impulse response. Therefore, a head impulse response in an arbitrary direction is obtained by measuring a head impulse response using each of a plurality of sound sources provided around an individual's head and interpolating the measured plurality of head impulse responses with each other. A technique for obtaining a response has been proposed (see, for example, Patent Document 1). In addition, there is a method that uses a common head impulse response obtained by statistical learning for head impulse responses measured and accumulated for a plurality of persons for interpolation between individual head impulse responses measured for a plurality of directions. It has been proposed (see, for example, Patent Documents 2 and 3).

JP 2000-166000 A Special table 2008-527821 JP 2010-45489 A

  In the conventional technology for generating a head impulse response in an arbitrary direction by interpolating individual head impulse responses measured in a plurality of directions, for example, a plurality of sound sources are arranged at equal intervals around the individual head. In the state, an individual's head impulse response is measured. Therefore, in this type of technology, in order to measure an individual's head impulse response, a large-scale measuring device is often provided, and the measurement is repeated for each of the sound sources provided around the head. The measurement work of the part impulse response is also complicated.

  By the way, for example, in order to realize a service using sound image localization technology for a large number of persons gathered at an exhibition, etc., a head impulse in an arbitrary direction corresponding to each of the large number of persons. It is desirable to have a response.

  However, preparing a large-scale measuring device for measuring the head impulse response corresponding to each of a large number of people and repeating complicated operations may cause restrictions on the place where the measuring device is installed and time constraints. Therefore, realization is difficult.

  The acoustic processing device, the acoustic processing method, and the acoustic processing program of the present disclosure are intended to provide a technique for realizing a good sound image localization using a head impulse response measured for each individual in a part of a direction. .

According to one aspect, the sound processing device performs predetermined processing based on an impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head. A prediction unit for predicting a delay time of an impulse response when the sound reaches the head from the second direction outside the range, and a reference impulse response delay modeled in advance for the sound from the second direction A correction unit that corrects the time according to the delay time predicted by the prediction unit , and the prediction unit is a positional relationship between the head and the sound source installed in the first direction when measuring the impulse response. Based on the positional relationship specified by the specifying unit that specifies the positional relationship in which the delay time of the sound that arrives from the sound source becomes the delay time of the measured impulse response, the sound from the second direction to the head Predict if will reach Having a calculation unit for calculating a delay time.

According to another aspect, the sound processing method is based on an impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head. A prediction step for predicting a delay time of the impulse response when the sound reaches the head from the second direction outside the range, and a delay of the reference impulse response modeled in advance for the sound from the second direction A correction step that corrects the time according to the delay time predicted in the prediction step , and the prediction step is a positional relationship between the head and the sound source installed in the first direction when measuring the impulse response. Based on the positional relationship that specifies the positional relationship in which the delay time of the sound that arrives from the sound source becomes the measured delay time of the impulse response, and the positional relationship that is identified by the identifying step, the sound from the second direction to the head Reach It has a calculation step of calculating a delay time which is predicted if the.

According to another aspect, the sound processing program is based on an impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head. A prediction step for predicting a delay time of the impulse response when the sound reaches the head from the second direction outside the range, and a delay of the reference impulse response modeled in advance for the sound from the second direction And a correction step for correcting the time according to the delay time predicted in the prediction step. The computer executes a process including a correction step, and the prediction step is installed in the first direction when measuring the head and the impulse response. As a positional relationship with the sound source, based on the positional relationship identified by the identifying step and the identifying step that identifies the positional relationship in which the delay time of the sound that arrives from the sound source becomes the measured delay time of the impulse response Having a calculation step of calculating a delay time which is expected if the acoustic reaches the head from the second direction.

  The sound processing apparatus, sound processing method, and sound processing program of the present invention can realize a good sound image localization by using a head impulse response measured for each individual in some directions.

It is a figure which shows one Embodiment of a sound processing apparatus. It is a figure which shows the example of the range which measures an individual head impulse response with the measuring apparatus shown in FIG. It is a figure which shows the example of an individual head impulse response. It is a figure which shows the example of the relationship between delay time and the direction of a sound source. It is a figure which shows the example of correction | amendment of the delay time of a common head impulse response. It is a figure which shows operation | movement of the sound processing apparatus shown in FIG. It is a figure which shows the example of the positional relationship of the microphone shown in FIG. 2, and a sound source. It is a figure which shows another embodiment of a sound processing apparatus. It is a figure which shows the example of the delay time calculated by the calculation part shown in FIG. It is a figure which shows another embodiment of a sound processing apparatus. It is a figure which shows the example of the weight set by the weight setting part shown in FIG. It is a figure which shows another embodiment of a sound processing apparatus. It is a figure which shows the example of the positional relationship of the person shown in FIG. 12, and the exhibit in an exhibition hall. It is a figure which shows an example of the hardware constitutions of a sound processing apparatus. It is a figure which shows the hardware structural example of the position detection apparatus shown in FIG. It is a figure which shows operation | movement of the measuring device shown in FIG. It is a figure which shows operation | movement of the acoustic AR apparatus shown in FIG.

  Hereinafter, embodiments will be described with reference to the drawings. In the following, a technique of using a combination of a head impulse response measured for each individual in a direction included in a predetermined measurement range and a common head impulse response prepared in advance for other directions not included in the measurement range Is explained.

  Here, when comparing the head impulse response in various directions indicated by an angle with respect to the orientation of the person's head, the head impulse response in front of the person's head is The individual difference is larger than the head impulse response. Therefore, in the following, the head impulse response measured for each individual for a predetermined range including the orientation of the person's head and the range not measured for each individual are modeled in advance by measurement using a dummy head, etc. A case of combining with the head impulse response will be described.

  FIG. 1 shows an embodiment of a sound processing apparatus. The sound processing apparatus 10 illustrated in FIG. 1 includes a prediction unit 11 and a correction unit 12. Further, the measuring apparatus EQ shown in FIG. 1 generates a head impulse response unique to the person Q1 in a first direction which is a plurality of directions set inside a predetermined measurement range R described with reference to FIG. Measurement is performed, and the head impulse response of the person Q1 obtained by the measurement is passed to the sound processing apparatus 10. In addition, the storage device SD illustrated in FIG. 1 is obtained by performing measurement using a dummy head or the like in the second direction, which is a plurality of directions set outside the measurement range R by the measurement device EQ. The information indicating the head impulse response is stored. The storage device SD may be provided as a component independent of the sound processing device 10 or may be included in the sound processing device 10.

  In the following description, the head impulse response unique to the person Q1 obtained by measurement by the measuring device EQ is referred to as an individual head impulse response. Further, another head impulse response indicated by information stored in the storage device SD is referred to as a common head impulse response. The common head impulse response is not limited to the head impulse response obtained by measurement using a dummy head or the like, but is a head modeled in advance with respect to the sound from the second direction set outside the measurement range R. Any partial impulse response may be used. For example, the common head impulse response may be a head impulse response modeled by learning a head impulse response obtained by measurement for a large number of persons. The measurement of the individual head impulse response and the measurement of the common head impulse response by the measurement device EQ will be described later with reference to FIG.

  In the acoustic processing device 10 illustrated in FIG. 1, the prediction unit 11 receives individual head impulse responses measured for each of the first directions by the measurement device EQ. Based on the received individual head impulse response, the prediction unit 11 performs a prediction process to be described later with reference to FIGS. 3 and 4, and performs a plurality of second directions set outside the measurement range R by the measurement device EQ. The delay time when the sound reaches the person Q1 from each is predicted. In the prediction unit 11, the delay time predicted for each second direction is passed to the correction unit 12. The correction unit 12 performs a correction process to be described later with reference to FIG. 5, thereby predicting the delay time of the common head impulse response stored in the storage device SD corresponding to each second direction in the same direction. Adjust to the delay time. The common head impulse response whose delay time is corrected by the correction unit 12 and the measured individual head impulse response are passed to an acoustic AR (Augmented Reality) device ARC described below. The individual head impulse response passed to the acoustic AR device ARC and the corrected common head impulse response are used in combination in a sound image localization process that is a process of localizing a sound image in an arbitrary direction.

  The acoustic AR device ARC includes a control unit CNT, an audio database DB1, and an acoustic processing unit SP built in the terminal device UE that can be carried by a person Q1, such as a smart phone or a tablet terminal. The control unit CNT is connected to the voice database DB1, and the control unit CNT can acquire the voice information stored in the voice database DB1. The control unit CNT and the acoustic processing unit SP are connected by a communication path using, for example, a wireless LAN (Local Area Network). The acoustic processing unit SP has a function of generating an acoustic signal based on audio information received from the control unit CNT.

  For example, the control unit CNT associates the individual head impulse response or the corrected common head impulse response passed from the sound processing device 10 for each direction in which the sound image is localized by the sound image localization processing. For example, the control unit CNT internally stores information indicating the correspondence between each of the first directions set inside the predetermined measurement range R and the individual head impulse response obtained by the measurement in the first direction. It memorizes in the memory etc. In addition, the control unit CNT corrects each of the second directions set outside the measurement range R and the corrected common head obtained by correcting the delay time of the common head impulse response corresponding to the second direction. Information indicating a correspondence relationship with the head impulse response is stored in an internal memory or the like.

  When the control unit CNT passes the sound information acquired from the sound database DB1 to the sound processing unit SP, the control unit CNT outputs the individual head impulse response stored corresponding to the direction in which the sound image is localized or the corrected common head impulse response. The indicated information is read from an internal memory or the like. Then, the control unit CNT uses the information read from the internal memory or the like as information indicating a head impulse response to be applied to the acoustic signal generated from the speech information, together with the speech information acquired from the speech database DB1, to the acoustic processing unit SP. hand over.

  The acoustic processing unit SP generates an acoustic signal based on the audio information received from the control unit CNT. In addition, the acoustic processing unit SP performs a convolution process between the head impulse response passed from the control unit CNT and the acoustic signal generated from the voice information using a built-in filter, and the acoustic signal after the convolution processing is converted into the person Q1. Is output by the earphones EPL and EPR attached to the ears.

  That is, the acoustic AR device ARC shown in FIG. 1 is measured in a state in which sound is caused to reach the person Q1 from the same first direction to generate sound that localizes the sound image in the first direction with respect to the person Q1. Individual head impulse responses are used. Then, the acoustic AR device ARC corrects the delay time of the common head impulse response modeled with respect to the sound from the second direction to generate sound that localizes the sound image in the second direction with respect to the person Q1. The corrected common head impulse response obtained by the above is used.

  The terminal device UE is not limited to a smart phone or a tablet-type terminal, and can be carried by a person Q1, and may be any device including an acoustic processing unit SP for causing the earphones EPL and EPR to output stereo sound. A mobile phone or a portable game machine may be used. Further, the control unit CNT of the acoustic AR device ARC may be included in the terminal device UE, and the acoustic processing device 10 may include the control unit CNT and the acoustic processing unit SP of the acoustic AR device ARC.

  Next, prior to description of the functions and operations of the prediction unit 11 and the correction unit 12 included in the acoustic processing device 10, a method of measuring an individual head impulse response by the measurement device EQ will be described.

  FIG. 2 shows an example of a range in which the individual head impulse response is measured by the measuring apparatus EQ shown in FIG. 2 that are equivalent to the elements shown in FIG. 1 are denoted by the same reference numerals and description of the elements may be omitted.

  Microphones MCL and MCR are attached to the ears EL and ER of the person Q1 shown in FIG. The outputs of the microphones MCL and MCR are connected to the measuring device EQ. The measuring device EQ has a function of generating a TSP (Time stretched Pulse) signal that is a signal for measuring an impulse response, and inputs the generated TSP signal to the speaker S1. In the example of FIG. 2, the speaker S <b> 1 is installed at a distance D from the person Q <b> 1 in the direction of the angle θ <b> 1 with reference to the direction Dir indicating the front direction of the head of the person Q <b> 1. In FIG. 2, a circle S ′ indicated by a dotted line indicates an example of a sound source used for measurement of a common head impulse response, which will be described later with reference to FIG. 4 together with FIG. 2.

  The measuring device EQ receives, for example, sound that has reached the person Q1 from the speaker S1 as sound signals obtained by the microphones MCL and MCR. Then, the impulse response indicated by the received acoustic signal is passed to the acoustic processing device 10 as an individual head impulse response in the direction indicated by the angle θ1 from the head direction Dir of the person Q1. That is, the measuring device EQ measures the individual head impulse response.

  Similarly, the measuring apparatus EQ changes the individual head while changing the angle θ1 at which the speaker S1 is installed in the measurement range R indicated by a fan shape having a central angle 2φ with the head direction Dir of the person Q1 as the central axis. Measure impulse response. For example, the measuring device EQ sequentially generates sound corresponding to the TSP signal in each of the plurality of speakers S1 installed at the positions indicated by the angles θ1 at which the measurement is performed on the fan-shaped arc indicating the measurement range R. . Then, the measuring device EQ uses the individual head impulse for each of the first directions from the acoustic signal indicating the sound that has reached the person Q1 from the first direction (for example, the direction of the angle θ1) corresponding to the position of each speaker S1. Ask for a response. The measurement range R shown in FIG. 2 is an example of a measurement range in which an angle with respect to the front direction Dir of the person Q1 is within a predetermined range, and the direction indicated by the angle θ1 is the direction of the measurement range R. It is an example of the several 1st direction set inside. Further, the sector-shaped central angle 2φ indicating the measurement range R is, for example, an angle smaller than an angle of 180 degrees, and is preferably set to about 120 to 150 degrees.

  The measurement of the individual head impulse response for each direction included in the measurement range R shown in FIG. 2 is performed in a smaller space than when the head impulse response is measured for the 360-degree direction around the person Q1. be able to. For this reason, compared with the case where the head impulse response is measured in the 360-degree direction around the person Q1, the number of speakers S1 and the number of wires connecting the speakers S1 and the measuring device EQ can be reduced. Measurement time can be shortened.

  Next, a method for correcting the delay time of the common head impulse response using the delay time of the individual head impulse response measured for a predetermined range based on the head of the person Q1 will be described.

  The prediction unit 11 shown in FIG. 1 receives sound from each of the plurality of second directions set outside the measurement range R shown in FIG. 2 based on the individual head impulse response received from the measurement device EQ. Predict the delay time of the impulse response.

  The prediction unit 11 specifies a delay time when sound reaches the head of the person Q1 from each of the first directions based on the individual head impulse responses of the first directions. For example, in the individual head impulse response received from the measurement apparatus EQ, the prediction unit 11 sets the time from when the sound is generated until the amplitude of the waveform becomes a predetermined threshold or more as the delay time. Note that the threshold value used for specifying the delay time is preferably set to a value equivalent to the threshold value used for determining the noise component in the acoustic signal, for example.

  FIG. 3 shows an example of an individual head impulse response. The coordinate axis t shown in FIG. 3 indicates the time from the time when the sound is generated, and the coordinate axis P indicates the sound pressure. Here, as the time when the sound is generated, it is desirable to use the time when the TSP signal is passed from the measuring apparatus EQ shown in FIG. 2 to the speaker S1.

  The example of FIG. 3 shows an individual head impulse response obtained from an acoustic signal obtained by either the microphone MCL or MCR when the sound reaches the head of the person Q1 from the direction indicated by the angle θ1 in FIG. The waveform is shown. In the example of FIG. 3, the delay time of the individual head impulse response in the direction indicated by the angle θ1 in FIG. 2 is the first time that the waveform showing the individual head impulse response from the origin (t = 0) of the coordinate axis t has the threshold value Thp. It is indicated by the time δp (θ1) until it exceeds. In the example of FIG. 3, the individual head impulse response obtained for one ear of the person Q1 is shown, and the individual head impulse response obtained for the other ear is not shown.

  From the relationship between each first direction (for example, θ1 in FIG. 2) set inside the measurement range R shown in FIG. 2 and the delay time δp (θ1) of the individual head impulse response, the prediction unit 11 The delay time of the impulse response to the sound from other directions outside the measurement range R is predicted.

  FIG. 4 shows an example of the relationship between the delay time and the direction of the sound source. 4, the coordinate axis θ represents the direction of the sound source with reference to the head direction Dir of the person Q1 shown in FIG. 2, and the coordinate axis t represents the delay time shown in FIG. In the example of FIG. 4, the angle measured clockwise from the front direction Dir of the head of the person Q1 shown in FIG. 2 is shown as a positive value on the coordinate axis θ, and the head direction Dir of the person Q1. The angle measured counterclockwise from is indicated as a negative value on the coordinate axis θ. That is, the measurement range R shown in FIG. 2 corresponds to a range from the angle “−φ” to the angle “+ φ” on the coordinate axis θ shown in FIG.

  In the example of FIG. 4, each of the black circles represents a case where sound reaches one ear of the person Q1 from a sound source arranged in each of the plurality of first directions set in the measurement range R shown in FIG. The delay time which the individual head impulse response measured about is shown. For example, the black circle Pm (θ1) is the individual head impulse measured when the sound source (for example, the speaker S1) is in the direction of the angle θ1 with respect to the head direction Dir of the person Q1 shown in FIG. Indicates the delay time that appears in the response. In FIG. 4, the illustration of the relationship between the delay time indicated by the individual head impulse response measured when the sound reaches the other ear of the person Q1 and the direction of the sound source is omitted.

  The prediction unit 11 illustrated in FIG. 1 estimates the relationship between the direction of the sound source and the delay time by, for example, obtaining a curve CV that approximates the distribution of the plurality of black circles illustrated in FIG. And the prediction part 11 is assumed when performing the measurement which has arrange | positioned the sound source in each of the 2nd direction set outside the measurement range R shown in FIG. 2 based on the curve CV which shows the estimated relationship. The delay time indicated by the individual head impulse response is predicted. Note that, in the prediction unit 11, a method for obtaining a delay time predicted when sound arrives from each direction set outside the measurement range R from the delay time indicated by the measured individual head impulse response is shown in FIG. It is not restricted to the method of calculating | requiring the curve CV shown in 4. FIG. For example, the prediction unit 11 may estimate the relationship between the direction of the sound source and the delay time using a method described later with reference to FIGS.

  In the storage device SD shown in FIG. 1, information indicating the common head impulse response measured in advance using a dummy head or the like for each of the second directions set outside the measurement range R shown in FIG. Is remembered. The common head impulse response is not limited to the head impulse response measured using a dummy head. For example, the common head impulse response causes sound to reach the head of a person different from the person Q1 shown in FIG. 2 from each of the second directions set outside the measurement range R shown in FIG. It may be a head impulse response measured in a state. In FIG. 2, a circle S ′ indicated by a dotted line indicates an example of a sound source used for measurement of the common head impulse response.

  Here, the distance D between the head of the person Q1 shown in FIG. 2 and the speaker S1, which is the sound source used for the measurement of the individual head impulse response, and the dummy head and the common head impulse response were measured. The distance D ′ to the sound source S ′ may not exactly match. This is because it is difficult to position the midpoint of the line connecting both ears of the person Q1 and the midpoint of the line connecting both ears of the dummy head at the center of the sector indicating the measurement range R. Similarly, the distance between both ears of the person Q1 and the distance between the two microphones attached to the dummy head for the measurement of the common head impulse response may not exactly match. Therefore, the delay time predicted for each of the second directions set outside the measurement range R shown in FIG. 2 and the common head impulse stored in the storage device SD when there is a sound source in the corresponding direction. The delay time indicated by the response may not match. For example, the sound source in the direction indicated by the same angle θ2 as the delay time τ (θ2) predicted from the curve CV shown in FIG. 4 in the direction of the angle θ2 (θ2 <−φ) from the head direction Dir of the person Q1. It does not necessarily match the delay time indicated by the common head impulse response when S ′ is present.

  Therefore, the correction unit 12 illustrated in FIG. 1 generates a prediction unit in response to the common head impulse response stored in the storage device SD for each of the plurality of second directions set outside the measurement range R illustrated in FIG. 11 is performed to indicate the delay time obtained for the corresponding direction.

  FIG. 5 shows an example of correcting the delay time of the common head impulse response. In FIG. 5, the coordinate axis t represents the passage of time in the common head impulse response, and the coordinate axis P represents the sound pressure.

  FIG. 5A shows the position of the head of the person Q1 shown in FIG. 2 with the front direction of the dummy head aligned with the direction Dir of the head of the person Q1, and the angle with the head direction Dir. It is an example of a head impulse response measured in a state where sound is made to reach from a direction intersecting at θ2. That is, the head impulse response shown in FIG. 5A is an example of the common head impulse response stored in correspondence with the angle θ2 in the storage device SD shown in FIG. Note that the direction indicated by the angle θ2 is one of the second directions set outside the measurement range R shown in FIG. The delay time of the common head impulse response shown in FIG. 5A is indicated by a time δc (θ2) when the waveform representing the common head impulse response first exceeds the threshold Thp.

  FIG. 5B shows an example of the corrected common head impulse response obtained by the correction unit 12 shown in FIG. That is, the correction unit 12 obtains the corrected common head impulse response shown in FIG. 5B by correcting the delay time of the common head impulse response shown in FIG.

  The correction unit 12 illustrated in FIG. 1 predicts each of the second directions by the prediction unit 11 by moving the common head impulse response held in each of the second directions in the storage device SD in the time axis direction. To the specified delay time.

  For example, the correction unit 12 has the delay time τ (θ2) predicted from the relationship shown in FIG. 4 and the common shown in FIG. 5A with respect to the direction of the angle θ2 from the head direction Dir shown in FIG. A difference dτ from the delay time δc (θ2) of the head impulse response is obtained. Then, the correction unit 12 moves the common head impulse response in the direction indicated by the angle θ2 on the coordinate axis t so that the difference dτ is eliminated. By performing the correction described above, the correction unit 12 substantially equalizes the elapsed time until the waveform of the common head impulse response after movement exceeds the threshold Thp and the delay time predicted by the prediction unit 11. To do.

  FIG. 6 shows the operation of the sound processing apparatus 10 shown in FIG. The process of step S301 to step S303 shown in FIG. 6 shows the operation of the sound processing apparatus 10 shown in FIG. Further, the processing of each step shown in FIG. 6 is performed by using an acoustic for realizing sound image localization in an arbitrary direction using an individual head impulse response measured for an individual and a common head impulse response prepared in advance. The example of a processing method and a sound processing program is shown. For example, the process illustrated in FIG. 6 is realized by a processor installed in the sound processing apparatus 10 executing a sound processing program. Note that the processing shown in FIG. 6 may be executed by hardware mounted on the sound processing apparatus 10.

  In step S301, the acoustic processing device 10 illustrated in FIG. 1, for example, the individual heads measured by the measurement device EQ for each of the plurality of first directions set inside the measurement range R illustrated in FIG. Receive impulse response.

  In step S302, the prediction unit 11 illustrated in FIG. 1 is obtained when measurement is performed for a plurality of second directions set outside the measurement range R illustrated in FIG. 2 from the measured individual head impulse response. The delay time indicated by the individual head impulse response is predicted.

  In step S303, the correction unit 12 illustrated in FIG. 1 performs a correction so that the delay time indicated by the common head impulse response corresponding to each of the second directions approaches the delay time predicted in the process of step S302.

  The common head impulse response whose delay time has been corrected in the process of step S303 described above is applied to the acoustic AR device ARC shown in FIG. 1 together with the individual head impulse response received from the measurement apparatus EQ in the process of step S301. Passed.

  The acoustic AR device ARC then generates individual head impulses measured with respect to the sound from the same direction to generate sound that localizes the sound image in the first direction set inside the measurement range R shown in FIG. Use a response. The acoustic AR device ARC also generates a delay time of the common head impulse response modeled for the sound from the same direction in generating the sound that localizes the sound image in the second direction set outside the measurement range R. The corrected common head impulse response obtained by correcting is used.

  That is, in the acoustic AR device ARC shown in FIG. 1, the delay time of the head impulse response used for generating the sound that localizes the sound image in the second direction is the delay time predicted by the prediction unit 11 in the same second direction. Is almost the same. Here, the delay time predicted by the prediction unit 11 is measured in a state where sound is made to reach the person Q1 from another speaker S1 ′ installed on the extension of the arc in which the speaker S1 shown in FIG. 2 is arranged. It is almost equivalent to the delay time of the head impulse response.

  That is, the sound processing device 10 shown in FIG. 1 reproduces the delay time indicated by the individual head impulse response in the corresponding direction in the sound generated by the sound AR device ARC in order to localize the sound image in an arbitrary direction. can do. Therefore, by using the sound processing device 10 shown in FIG. 1, in terms of the delay time of the sound to be heard by the person Q1, good sound image localization equivalent to the case where individual head impulse responses are measured in all directions Can be realized.

  Here, the human perceives the direction of the sound source corresponding to the heard sound based on the time difference of the sound heard by both ears. Therefore, in the sound that localizes the sound image in an arbitrary direction, even when the relative position between the person Q1 and the sound source changes by reproducing the delay time indicated by the individual head impulse response in the corresponding direction, the person Q1 The sound that does not give an unnatural impression can be heard.

  In addition, as described with reference to FIG. 2, the space used for measuring the individual head impulse response in each direction included in the measurement range R is a space used for measurement about 360 degrees around the person Q1. Smaller than. Furthermore, by measuring the individual head impulse response for each of a plurality of ranges obtained by dividing the measurement range R, it is possible to reduce the space prepared for measuring the individual head impulse response. For example, it can be rotated in a box-shaped booth with a rectangular bottom that is circumscribed by a figure that is obtained by dividing the inner angle of the sector shape indicating the measurement range R into n (n is an integer of 2 or more). An individual head impulse response can be measured by arranging a chair and a plurality of speakers facing each other. In this case, the individual head impulse response in each direction included in the measurement range R in FIG. 2 is measured by changing the relative position between the person Q1 seated on the chair and the plurality of speakers and repeating the measurement process n times. can do.

  As described above, the measurement of the individual head impulse response used in the sound processing apparatus 10 shown in FIG. 1 can be realized without using a large-scale facility as used in the prior art. It is. Therefore, for example, a booth for measuring the individual head impulse response is provided in a corner of a venue such as an exhibition, and the individual head impulse response is measured for each of a large number of people gathered at the exhibition. Is possible. Then, using the individual head impulse response obtained by measurement for each of a large number of persons, it is possible to provide a service using sound image localization technology to each person. Note that, for example, an acoustic AR system that provides a service using the sound image localization technology to the person Q1 using the acoustic processing device 10 illustrated in FIG. 1 will be described later with reference to FIGS.

  Next, the prediction unit 11 shown in FIG. 1 estimates the positional relationship between the microphone used for the measurement and the sound source, so that the direction of the sound source based on the head direction Dir of the person Q1 and the individual head A method for estimating the relationship with the delay time indicated by the partial impulse response will be described.

  FIG. 7 shows an example of the positional relationship between the microphones MCL and MCR and the speaker S1 shown in FIG. Of the elements shown in FIG. 7, elements equivalent to those shown in FIG. 2 are denoted by the same reference numerals and description of the elements may be omitted. Moreover, the speaker S1 shown in FIG. 7 is an example of a sound source used for measurement of an individual head impulse response.

  In FIG. 7, a line segment DL and a line segment DR indicate two sides sandwiching the apex corresponding to the speaker S1 among the triangular sides obtained by connecting the speaker S1 and the two microphones MCL and MCR to each other. That is, the length | DL | of the line segment DL shown in FIG. 7 indicates the distance from the speaker S1 to the microphone MCL, and the length | DR | of the line segment DR indicates the distance from the speaker S1 to the microphone MCR. . A line segment D indicates a line segment obtained by connecting the midpoint Qc of the line segment connecting the two microphones MCL and MCR and the speaker S1. The length Y of the line segment D indicates the distance from the midpoint Qc of the line segment W connecting the two microphones MCL and MCR to the speaker S1, and the length X of the line segment W indicates the two microphones MCL, The distance between MCR is shown. That is, the length Y of the line segment D indicates the distance from the midpoint of the line segment connecting both ears of the person Q1 to the speaker S1, and the length X of the line segment W indicates the distance between both ears of the person Q1. Indicates.

  When the speaker S1 is in the direction of the angle θ with respect to the front direction Dir of the head of the person Q1, the distance | DL | and the distance | DR | from the speaker S1 to the microphones MCL and MCR are It is represented by the formula (1) and the formula (2) as functions. In equations (1) and (2), the symbol Y indicates the distance from the midpoint Qc of the line segment W to the speaker S1, and the symbol X indicates the distance X between the two microphones MCL and MCR.

そして、スピーカＳ１で発生した音響をマイクロホンＭＣＬ，ＭＣＲで受けた際に得られる音響信号から頭部インパルス応答を求めた場合に、求めた頭部インパルス応答に現れる遅延時間ＴＬ，ＴＲは、式(３)、式(４)で示される。なお、式(３)および式(４)において、符号ＤＬ(θ)は、角度θの関数として式(１)で表されるスピーカＳ１からマイクロホンＭＣＬまでの距離を示す。また、符号ＤＲ(θ)は、角度θの関数として式(２)で表されるスピーカＳ１からマイクロホンＭＣＲまでの距離を示す。そして、符号Ｖは、空気中の音速を示し、符号Ｃは、図１に示した計測装置ＥＱによる頭部インパルス応答の計測処理にかかる処理時間などを含む固定のオフセット時間を示す。

When the head impulse response is obtained from the acoustic signal obtained when the sound generated by the speaker S1 is received by the microphones MCL and MCR, the delay times TL and TR appearing in the obtained head impulse response are expressed by the following formulas ( 3) and expressed by equation (4). In equations (3) and (4), symbol DL (θ) indicates the distance from speaker S1 to microphone MCL represented by equation (1) as a function of angle θ. Further, the symbol DR (θ) indicates the distance from the speaker S1 to the microphone MCR expressed by the equation (2) as a function of the angle θ. Reference sign V indicates the speed of sound in the air, and reference sign C indicates a fixed offset time including a processing time required for the measurement processing of the head impulse response by the measuring device EQ shown in FIG.

式(１)から式(４)に示した関係を用いれば、計測装置ＥＱを用いて複数の方向について計測された個別頭部インパルス応答のそれぞれに現れた遅延時間に基づいて、図７に示した距離Ｘ，距離Ｙと式(３)、(４)に示したオフセット時間Ｃとを推定することができる。ここで、図７に示した距離Ｘ，距離Ｙと式(３)、(４)に示したオフセット時間Ｃとは、人物Ｑ１の個別頭部インパルス応答を計測した際に固有のパラメータであり、計測装置ＥＱによる人物Ｑ１についての計測を特徴付ける計測条件である。即ち、音響処理装置１０の予測部１１において上述の式(１)から式(４)に示した関係を用いることで、一部の方向について計測された個別頭部インパルス応答から、計測時における人物Ｑ１の両耳と音源との位置関係を含む計測条件を推定することができる。そして、推定された計測条件に基づいて、計測装置ＥＱによる計測が行われていない任意の方向について、人物Ｑ１の個別頭部インパルス応答が示すと予想される遅延時間を求めることができる。

Using the relationships shown in Equations (1) to (4), based on the delay times appearing in the individual head impulse responses measured in a plurality of directions using the measuring device EQ, shown in FIG. The distances X and Y and the offset time C shown in equations (3) and (4) can be estimated. Here, the distance X and the distance Y shown in FIG. 7 and the offset time C shown in the equations (3) and (4) are unique parameters when the individual head impulse response of the person Q1 is measured, This is a measurement condition that characterizes the measurement of the person Q1 by the measurement device EQ. That is, the prediction unit 11 of the sound processing device 10 uses the relationship shown in the above formulas (1) to (4), so that the person at the time of measurement can be obtained from the individual head impulse responses measured in some directions. Measurement conditions including the positional relationship between both ears of Q1 and the sound source can be estimated. Then, based on the estimated measurement conditions, it is possible to obtain a delay time that is expected to be indicated by the individual head impulse response of the person Q1 in an arbitrary direction in which measurement by the measurement device EQ is not performed.

  FIG. 8 shows another embodiment of the sound processing apparatus 10. 8 that are equivalent to the components shown in FIG. 1 are denoted by the same reference numerals and description of the components may be omitted.

  The sound processing apparatus 10 illustrated in FIG. 8 includes a storage device SD and a generation unit 13 in addition to the prediction unit 11 and the correction unit 12. In the example of FIG. 8, individual head impulse response (Individual HRIR) PIR obtained by measurement in each of the first directions by the measurement device EQ corresponds to, for example, each of the first directions. It is stored in the storage device SD. The storage device SD stores a common head impulse response (common HRIR) CIR prepared in advance corresponding to each of the second directions, and the correction unit 12 accesses the storage device SD, The common head impulse response CIR is acquired. The common head impulse response (corrected HRIR) AIR corrected by the correcting unit 12 is stored in the storage device SD.

  The generation unit 13 illustrated in FIG. 8 includes, for example, a setting unit 131, a selection unit 132, an acoustic processing unit SP, and a storage unit MEM. The acoustic processing unit SP illustrated in FIG. 8 is, for example, hardware mounted on the terminal device UE. Moreover, the memory | storage part MEM is implement | achieved using some memory built in the terminal device UE. And the selection part 132 is implement | achieved by performing the application program mentioned later using FIG. 17 by the processor mounted in the terminal device UE, for example. The setting unit 131 is connected to the terminal device UE via a network NW such as a wireless LAN, for example, and can access the storage unit MEM.

  In the generation unit 13 illustrated in FIG. 8, the setting unit 131 is obtained by measurement in the first direction corresponding to each of the first directions set inside the measurement range R illustrated in FIG. 2. The individual head impulse response is stored in the storage unit MEM. Further, the setting unit 131 corresponds to each of the second directions set outside the measurement range R, and after correction is obtained by correcting the delay time of the common head impulse response in the second direction. Is stored in the storage unit MEM.

  For example, the selection unit 132 receives information indicating the direction in which the sound image is localized from the server device SV via the network NW, and the individual head impulse stored in the storage unit MEM corresponding to the direction indicated by the received information. Read the response or corrected common head impulse response. Then, the selection unit 132 uses the read individual head impulse response or the corrected common head impulse response as an impulse response used for generating sound for localizing a sound image in the direction indicated by the information from the server device SV. It passes to processing section SP.

  In addition, the acoustic information stored in the voice database DB1 is read by, for example, the server device SV, and passed to the acoustic processing unit SP via the network NW. Then, the acoustic processing unit SP performs the convolution process between the acoustic signal generated from the acoustic information passed from the server device SV and the impulse response passed from the selection unit 132, thereby being indicated by the information from the server device SV. The sound that localizes the sound image in the selected direction is generated.

  That is, the generation unit 13 illustrated in FIG. 8 generates sound that localizes the sound image in the second direction using the common head impulse response whose delay time is corrected by the correction unit 12. And the production | generation part 13 produces | generates the sound which localizes a sound image in a 1st direction using the separate head impulse response measured by the measuring apparatus EQ. Further, the server device SV and the selection unit 132 illustrated in FIG. 8 perform a function corresponding to the control unit CNT of the acoustic AR device ARC illustrated in FIG.

  In the acoustic processing apparatus 10 illustrated in FIG. 8, the prediction unit 11 includes a specifying unit 111 and a calculation unit 112. The identifying unit 111 measures the delay time of the sound reaching from each sound source as the positional relationship between the head of the person Q1 and the sound source installed in each first direction when measuring the individual head impulse response. The positional relationship that is the delay time of each individual head impulse response is specified. The calculation unit 112 calculates a delay time predicted when sound reaches the head of the person Q1 from each of the second directions based on the specified measurement conditions, and calculates the delay calculated for each of the second directions. The time is passed to the correction unit 12.

  The specifying unit 111 uses, for example, the distance X and the distance shown in FIG. 7 as the measurement conditions when the individual head impulse response is measured using the relationship shown in the above equations (1) to (4). Y and the offset time C shown in equations (3) and (4) are obtained. Here, the error included in the delay time tL (θ) obtained for the left ear in the measurement in the first direction indicated by the angle θ shown in FIG. 7 is represented by the delay time tL (θ) and Equation (3). This is indicated by the difference from the required delay time TL (θ). Similarly, the error included in the delay time tR (θ) obtained for the right ear in the measurement in each of the first directions indicated by the angle θ is the delay obtained by the delay time tR (θ) and Equation (4). It is indicated by the difference from the time TR (θ). Therefore, the specifying unit 111 sets, for example, a parameter that minimizes the error sum of squares E shown in Expression (5) when the angle θ shown in FIG. 7 changes in the range of the value −φ to the value φ. As shown in FIG. 7, the distance X, the distance Y, and the offset time C shown in FIG. In the equation (5), the range from the value −φ shown as the lower limit of the range of the angle θ to the value φ shown as the upper limit of the angle θ corresponds to the inside of the measurement range R shown in FIG.

そして、算出部１１２は、特定部１１１によって特定されたパラメータと上述の式（１）〜式（４）とを用いることで、図２に示した計測範囲Ｒの外側に設定された複数の第２方向のそれぞれにおいて予測される遅延時間を算出する。つまり、算出部１１２は、特定されたパラメータＸ，Ｙと第２方向を示す角度θとを式（１）を代入することで、個別頭部インパルス応答の計測条件が再現された場合に、計測範囲Ｒの外側に設置された音源から人物Ｑ１の左耳までの距離ＤＬ（θ）を求める。同様に、算出部１１２は、特定されたパラメータＸ，Ｙと第２方向を示す角度θとを式（２）に代入することで、個別頭部インパルス応答の計測条件が再現された場合に、計測範囲Ｒの外側に設置された音源から人物Ｑ１の右耳までの距離ＤＲ（θ）を求める。そして、算出部１１２は、式（１）を用いて求めた距離ＤＬ（θ）と特定されたパラメータＣとを式（３）に代入することで、個別頭部インパルス応答の計測条件が再現された状態で、第２方向からの音響に対して得られるインパルス応答の遅延時間を算出する。同様に、算出部１１２は、式（２）を用いて求めた距離ＤＲ（θ）と特定されたパラメータＣとを式（４）に代入することで、個別頭部インパルス応答の計測条件が再現された状態で、第２方向からの音響に対して得られるインパルス応答の遅延時間を算出する。

Then, the calculation unit 112 uses the parameters specified by the specifying unit 111 and the above-described equations (1) to (4), so that a plurality of second values set outside the measurement range R shown in FIG. A delay time predicted in each of the two directions is calculated. That is, the calculation unit 112 performs measurement when the measurement conditions of the individual head impulse response are reproduced by substituting the specified parameters X and Y and the angle θ indicating the second direction with Expression (1). A distance DL (θ) from a sound source installed outside the range R to the left ear of the person Q1 is obtained. Similarly, the calculation unit 112 substitutes the specified parameters X and Y and the angle θ indicating the second direction in the equation (2), so that when the measurement condition of the individual head impulse response is reproduced, The distance DR (θ) from the sound source installed outside the measurement range R to the right ear of the person Q1 is obtained. Then, the calculation unit 112 substitutes the distance DL (θ) obtained using the equation (1) and the identified parameter C into the equation (3), thereby reproducing the measurement conditions of the individual head impulse response. In this state, the delay time of the impulse response obtained for the sound from the second direction is calculated. Similarly, the calculation unit 112 reproduces the measurement condition of the individual head impulse response by substituting the distance DR (θ) obtained using Expression (2) and the identified parameter C into Expression (4). In this state, the delay time of the impulse response obtained for the sound from the second direction is calculated.

  FIG. 9 shows an example of the delay time calculated by the calculation unit 112 shown in FIG. Of the elements shown in FIG. 9, elements equivalent to those shown in FIG. 4 are denoted by the same reference numerals and description of the constituent elements may be omitted.

  FIG. 9A is specified by the specifying unit 111 based on the delay time of the individual head impulse response obtained by the measurement by the measurement apparatus EQ shown in FIG. 8 and the equations (1) to (5). An example of the delay time calculated by the calculation unit 112 when parameters are used is shown. FIG. 9B shows an example of a delay time calculated by the calculation unit 112 when using a parameter specified by another specifying unit 111a described later with reference to FIG.

  First, an example of FIG. 9A will be described. Each black circle shown in FIG. 9A indicates a delay time of the individual head impulse response obtained for each of the plurality of first directions included in the measurement range R by the measurement apparatus EQ. Further, the curve CVa shown in FIG. 9A shows the delay time calculated from the equation (3) or the equation (4) in which the parameter for minimizing the square sum E of the error shown in the equation (5) is substituted. A change corresponding to the angle θ is shown. Each white circle shown in FIG. 9A represents a delay calculated by the calculation unit 112 in the second direction corresponding to each of the common head impulse responses stored in the storage device SD shown in FIG. Show time.

  Here, in the square sum of errors E expressed by the above equation (5), an equal weight is given to all delay times obtained by measurement. Therefore, the parameter set obtained by the specifying unit 111 using Expression (5) is a parameter set that gives a curve CVa that approximates the distribution of all the black circles shown in FIG. However, if the delay time in the boundary direction indicating the inner and outer boundaries of the measurement range R is calculated using the set of parameters that give the curve Cva and the equations (1) to (4), the delay obtained by the measurement is calculated. There may be a difference between the time and the calculated delay time. For example, in the example of FIG. 9A, the delay time Pm (−φ) appearing in the individual head impulse response measured for the angle θ = −φ indicating the boundary direction of the measurement range R and the set of the specified parameters There is a difference d from the delay time Ta (−φ) calculated using

  The difference d shown in the example of FIG. 9A is caused by an error included in the individual head impulse response measured with respect to the sound from each of the first directions set inside the measurement range R. When such a difference d occurs, in the vicinity of the boundary of the measurement range R, the delay time indicated by the individual head impulse response obtained by measurement and the delay time used for correcting the common head impulse response are smooth. The connection is lost. For example, in the example of FIG. 9A, the delay time Ta (−θ3) calculated for the angle θ3 indicating the second direction set near the boundary of the measurement range R and the delay obtained by the measurement for the boundary direction. A gap dA having the same size as the difference d is generated between the time Pm (−φ). When such a gap dA occurs, when the direction in which the sound image is localized near the boundary of the measurement range R changes, an unnatural silence time occurs in the sound to be heard by the person Q1, or sequentially The sound that is supposed to be heard by the user may be heard in an overlapping manner.

  The gap dA shown in FIG. 9A is a least square method in which the same weight is given to all delay times obtained by measurement by using the parameter set specified by the specifying unit 111a shown in FIG. Compared to the case where the parameters specified in (1) are used, the size can be reduced.

  FIG. 10 shows another embodiment of the sound processing apparatus 10. 10 that are equivalent to the components shown in FIG. 1 or FIG. 8 are denoted by the same reference numerals and description of the components may be omitted.

  The sound processing apparatus of FIG. 10 includes a specifying unit 111a instead of the specifying unit 111 of the sound processing apparatus 10 illustrated in FIG. The identifying unit 111a illustrated in FIG. 10 includes an analysis unit 113 that obtains the parameters X, Y, and C shown in the above formulas (1) to (4) by regression analysis using the minimum two information, and the analysis unit And a weight setting unit 114 for setting weights used in the regression analysis by 113.

  For example, the analysis unit 113 obtains parameters X, Y, and C that minimize the square sum E ′ of errors represented by the following equation (6) instead of the above equation (5). In Equation (6), the symbol W (θ) represents the delay time error of the individual head impulse response measured for the sound from the direction of the angle θ with reference to the front direction Dir of the person Q1. The weight set by the weight setting unit 14 is shown.

重み設定部１４は、例えば、角度θの下限（θ＝−φ）および上限（θ＝φ）に設定された第１方向からの音響に対して計測された個別頭部インパルス応答の遅延時間の誤差に他の第１方向よりも大きい重みを設定する。

For example, the weight setting unit 14 determines the delay time of the individual head impulse response measured for the sound from the first direction set to the lower limit (θ = −φ) and the upper limit (θ = φ) of the angle θ. A larger weight than the other first direction is set for the error.

  FIG. 11 shows an example of weights set by the weight setting unit 114 shown in FIG. In FIG. 11, the coordinate axis θ indicates the direction of the sound source with reference to the head direction Dir of the person Q1 shown in FIG. 2, and the coordinate axis W indicates the magnitude of the value set as the weight. In the example of FIG. 4, the angle measured clockwise from the front direction Dir of the head of the person Q1 shown in FIG. 2 is shown as a positive value on the coordinate axis θ, and the head direction Dir of the person Q1. The angle measured counterclockwise from is indicated as a negative value on the coordinate axis θ. That is, the measurement range R shown in FIG. 2 corresponds to a range from the angle “−φ” to the angle “+ φ” on the coordinate axis θ shown in FIG.

  Each of FIG. 11A and FIG. 11B shows an example of the weight set as the function W (θ) of the angle θ in the equation (6) by the weight setting unit 114 shown in FIG.

  The weight W (θ) shown in FIG. 11A is the individual head impulse response obtained by the measurement in the first direction set at the lower limit (θ = −φ) and the upper limit (θ = φ) of the angle θ. Is given a weight W1. On the other hand, the weight W (θ) gives the weight W2 having a value smaller than the weight W1 to the error in the delay time of the individual head impulse response obtained by the measurement in the other first direction.

  The analysis unit 113 illustrated in FIG. 10 obtains parameters X, Y, and C that minimize the sum of squares of the error represented by Expression (6) in which the weight W (θ) illustrated in FIG. Thus, a parameter that minimizes the error indicated by the term including the delay times tL (−φ) and tR (−φ) is obtained. Accordingly, as shown in FIG. 9B, the delay time shown when the curve CVb shown by the set of parameters obtained by the analysis unit 113 is an angle θ = −φ and the measured individual head impulse response. The difference from the delay time can be made smaller than the difference d shown in FIG.

  In the example of FIG. 9B, the calculation unit 112 uses the delay time Pm (−φ) of the individual head impulse response measured in the direction of the angle θ = −φ and the specified parameters X, Y, and C. Is substantially equal to the delay time Tb (−φ) calculated by. Accordingly, the gap dB between the delay time Tb (−θ3) calculated by the calculation unit 112 and the delay time Pm (−φ) obtained by the measurement for the angle θ3 shown in FIG. The gap dA is smaller than that shown in FIG.

  That is, by using the parameters specified by the specifying unit 111a shown in FIG. 10, even when there is an error in the measurement of the individual head impulse response, the delay time of the individual head impulse response and the corrected common head impulse The response delay time can be smoothly connected.

  Therefore, in the sound processing apparatus 10 having the specifying unit 111a illustrated in FIG. 10, the direction of the sound source is smooth in the vicinity of the boundary of the measurement range R even when there is an error in the measurement of the individual head impulse response by the generation unit 13. The sound that changes to can be given to the person Q1.

  Note that the weight W (θ) set by the weight setting unit 114 is not limited to the weight W (θ) shown in FIG. 11A, but is an error in the delay time of the individual head impulse response obtained by measurement. Any weight W (θ) may be used as long as it is closer to the boundary of the measurement range R. For example, as shown in FIG. 11B, the weight setting unit 114 changes the value set as the weight stepwise according to the difference between the angle θ and the angle φ or the angle −φ. May be set.

  The weight W (θ) shown in FIG. 11B is the individual head measured for the sound from the first direction set in a range where the angle θ is larger than the angle −φ + η and smaller than the angle φ−η. A weight having a predetermined value W2 is set for the error of the delay time of the partial impulse response. On the other hand, the weight W (θ) shown in FIG. 11B is the error of the delay time of the individual head impulse response measured for the sound from the first direction where the angle θ is the angle −φ or the angle φ. In addition, a weight having a value W1 larger than the value W2 is set. The weight W (θ) shown in FIG. 11B is the delay time of the individual head impulse response measured for the sound from the first direction where the angle θ is the angle −φ + η or the angle φ−η. Is set to a weight having a value W3 smaller than the value W1 and larger than the value W2.

  Further, the weight setting unit 114 is not limited to the example of FIGS. 11A and 11B, and the analysis unit 113 uses the weight W (θ) set by dividing the value set as the weight into four or more levels. A weight may be set.

  In addition, the method used to determine the parameters X, Y, and C in the specifying unit 111a is not limited to the weighted least square method. For example, the specifying unit 111a is configured to have a delay time of the individual head impulse response in the first direction close to the boundary of the measurement range R, rather than a weight for the delay time of the individual head impulse response in the first direction away from the boundary What is necessary is just to obtain | require the parameters X, Y, and C by weighting which gives a big weight.

  The sound processing apparatus 10 described above is useful for realizing a guidance system that makes it possible for a visitor to an exhibition hall or the like to recognize audio information explaining an exhibit so that it can be heard from the direction of the exhibit. is there.

  FIG. 12 shows another embodiment of the sound processing apparatus 10. Note that, among the components shown in FIG. 12, components equivalent to those shown in FIG. 1 or FIG. 8 are denoted by the same reference numerals and description of the components may be omitted.

  The acoustic processing device 10 and the acoustic AR device ARC shown in FIG. 12 are included in a guidance system GS that uses a sound image localization technique to guide visitors to an exhibition hall or the like using audio information.

  The acoustic AR device ARC shown in FIG. 12 includes an exhibition database DB2 and an orientation specifying unit DRD in addition to the server device SV, the voice database DB1, the selection unit 132, and the acoustic processing unit SP shown in FIG. Yes.

  The server apparatus SV shown in FIG. 12 is connected to each of the audio database DB1 and the exhibition database DB2, and the server apparatus SV can access information stored in the audio database DB1 and the exhibition database DB2. In the exhibition database DB2, information indicating the positions of the exhibits arranged in the exhibition hall HL described later with reference to FIG. 13 is accumulated. In the audio database DB1 shown in FIG. 12, audio information for explaining each of the exhibits arranged in the exhibition hall HL is accumulated.

  Further, the orientation specifying unit DRD shown in FIG. 12 is realized, for example, by executing an application program described later with reference to FIG. 17 by a processor mounted on the terminal device UE. The direction specifying unit DRD is connected to the position detection device HMD mounted on the head of the person Q1 by, for example, a wireless communication path using a short-range wireless communication technology, and information obtained by the position detection device HMD Receive. The orientation specifying unit DRD is connected to the server apparatus SV via the network NW, and refers to information stored in the exhibition database DB2 by making an inquiry to the server apparatus SV.

  The position detection device HMD illustrated in FIG. 12 acquires information indicating the position of the head of the person Q1 and the front direction Dir of the person Q1 by performing processing described later with reference to FIG.

  Next, the function and operation of the position detection device HMD shown in FIG. 12 and the function and operation of each component included in the acoustic AR device ARC will be described with reference to FIG.

  FIG. 13 shows an example of the positional relationship between the person Q1 shown in FIG. 12 and the exhibits in the exhibition hall HL. Of the elements shown in FIG. 13, elements equivalent to those shown in FIG. 12 are denoted by the same reference numerals and description of the constituent elements may be omitted.

  The example of FIG. 13 shows a case where two exhibits indicated by capsule-shaped figures Exh1 and Exh2 and signs indicated by circular Anc1 and Anc2 are installed in an exhibition hall indicated by a rectangular area HL. . Each of the signs Anc1 and Anc2 is associated with each of the exhibits Exh1 and Exh2, and has a function of transmitting identification information indicating the corresponding exhibits Exh1 and Exh2, for example, using infrared rays. . In FIG. 13, each of the sectors AR1 and AR2 indicated by broken lines indicates a range where infrared rays or the like indicating identification information transmitted by the signs Anc1 and Anc2 reach.

  In the example of FIG. 13, the exhibit Exh1 and the sign Anc1 are arranged at one of the corners of the exhibition hall HL, and the exhibit Exh2 and the sign Anc2 are arranged at one of the other corners of the exhibition hall HL. Yes. Note that three or more exhibits and signs associated with the exhibits may be arranged in the exhibition hall HL.

  In addition, the audio database DB1 illustrated in FIG. 13 stores, for example, audio information that describes the contents of the exhibits Exh1 and Exh2, corresponding to the identification information indicating the exhibits Exh1 and Exh2. For example, the exhibition database DB2 stores information indicating the positions of the exhibits Exh1 and Exh2 in the exhibition hall HL corresponding to the identification information of the exhibits Exh1 and Exh2.

  The position detection device HMD shown in FIG. 13 has a function of receiving identification information transmitted by the signs Anc1 and Anc2, and a function of detecting the position of the person Q1 and the front direction Dir of the person Q1 by a gyro sensor or the like. have. Further, the position detection device HMD has a function of transmitting the received identification information and information indicating the position of the person Q1 and the front direction Dir of the head to the terminal device UE using a short-range wireless communication technology or the like. Yes. Note that, in FIG. 13, illustration of a communication path by the short-range wireless communication technology set between the position detection device HMD and the terminal device UE is omitted. The functions and operations of the gyro sensor included in the position detection device HMD will be described later with reference to FIGS. 15 and 17.

  In the example of FIG. 13, the terminal device UE is connected to the network NW via an access point AP installed on the wall of the exhibition hall HL and the like, and is connected to the server device SV and the sound processing device 10 via the network NW. Information can be exchanged between them.

  The orientation specifying unit DRD included in the terminal device UE illustrated in FIG. 12 includes, for example, the identification information received by the position detection device HMD and the person detected by the position detection device from the position detection device HMD every predetermined time. Information indicating the position of Q1 and the head direction Dir of the person Q1 is received. Here, the identification information received by the position detection device HMD indicates that the person Q1 is staying in the areas AR1 and AR2 shown in FIG. 13, that is, the exhibit closest to the person Q1 is the exhibits Exh1 and Exh2. It indicates which one of them. In the example of FIG. 13, since the person Q1 stays in the area AR1, the position detection device HMD receives the identification information transmitted from the sign Anc1, and uses the received identification information as the position of the head of the person Q1 and Along with information indicating the direction Dir, the information is passed to the direction specifying unit DRD.

  The direction specifying unit DRD obtains information indicating the position of the exhibit Exh1 from the exhibition database DB2, for example, by making an inquiry to the server device SV based on the identification information received from the position detection device HMD. Then, based on the information indicating the position of the person Q1 and the head direction Dir received from the orientation specifying unit DRD and the information indicating the position of the exhibit Exh1, the exhibit based on the head direction Dir of the person Q1. An angle indicating the direction of Exh1 is obtained. Here, the direction of the exhibit Exh1 based on the head direction Dir of the person Q1 indicates the direction in which the sound image corresponding to the audio information describing the exhibit Exh1 is localized by the sound image localization process by the acoustic AR device ARC. .

  That is, for example, every time the orientation specifying unit DRD receives information indicating the exhibit nearest to the person Q1 from the position detection device HMD and information indicating the head direction Dir of the person Q1, a sound image localization process is performed. An angle θ indicating the orientation direction is obtained. Then, the azimuth specifying unit DRD passes the obtained angle θ to the selection unit 132 shown in FIG. 12 as information for designating the head impulse response used for the convolution processing by the acoustic processing unit SP.

  When the information indicating the angle θ indicating the direction in which the sound image is localized is received from the direction specifying unit DRD, the selection unit 132 stores the individual head impulse response stored in the storage unit MEM corresponding to the angle θ or the corrected head Read the common head impulse response. Then, the selection unit 132 passes the read individual head impulse response or the corrected common head impulse response to the acoustic processing unit SP as a head impulse response used for convolution for sound image localization processing.

  For example, when the server apparatus SV receives an inquiry based on the identification information indicating the exhibition unit Exh1 from the direction specifying unit DRD, the server device SV recognizes that the person Q1 is staying in the area AR1 corresponding to the exhibit Exh1. . In this case, the server apparatus SV reads out the voice information stored in the voice database DB1 corresponding to the identification information indicating Exh1 to be exhibited, and uses the read voice information as the acoustic processing unit of the terminal apparatus UE via the network NW. Pass to SP.

  Therefore, the acoustic processing unit SP exhibits the exhibits when the direction of the angle θ with respect to the head direction Dir of the person Q1 is the first direction set inside the measurement range R shown in FIG. The acoustic signal generated from the audio information of Exh1 is convolved with the individual head impulse response. On the other hand, when the direction of the angle θ is the second direction set outside the measurement range R shown in FIG. 2, the acoustic processing unit SP performs the correction with the acoustic signal generated from the audio information of the exhibit Exh1. Is convolved with the common head impulse response.

  As described with reference to FIG. 9, the individual head impulse response stored in the storage unit MEM or the corrected common head impulse response for each angle with respect to the head direction Dir of the person Q1 is an angle of The delay time which changes smoothly according to the change is shown. Therefore, according to the change of the angle θ obtained by the orientation specifying unit DRD, for example, the head impulse response used for the convolution processing in the speech processing unit SP is the individual head impulse response and the corrected common head impulse response. The continuity of the delay time is maintained even if it is switched between.

  That is, the acoustic AR device ARC shown in FIG. 12 has, for example, a sound image for localizing a change in the relative position of the person Q1 and the exhibit Exh1 moving in the exhibition hall HL shown in FIG. The position can be reflected smoothly. That is, the guidance system GS shown in FIG. 12 uses an individual head impulse response measured within a predetermined range in the front direction of the head of the person Q1 and a common head impulse response prepared in advance. It is possible to provide voice guidance by localizing a virtual sound image in the direction of.

  As described with reference to FIG. 2, by limiting the measurement by the measurement device EQ to a part of the direction including the front direction of the head of the person Q1, for example, individual heads for a large number of persons visiting an exhibition hall or the like Measurement of the impulse response is possible. Therefore, the guidance system GS shown in FIG. 12 has substantially the same naturalness at a lower cost than a case where individual head impulse responses are individually measured in all directions for a large number of persons visiting an exhibition hall or the like. Can provide a service for localizing sound images.

  The sound processing device 10 disclosed herein can be realized using a computer device or the like.

  FIG. 14 shows an example of the hardware configuration of the sound processing apparatus 10. Note that among the components shown in FIG. 14, components equivalent to those shown in FIG. 12 are denoted by the same reference numerals and description of the components may be omitted.

  The computer device 20 includes a processor 21, a memory 22, a hard disk device 23, a network interface 24, an audio interface 25, and an acoustic signal generation unit 26. The processor 21, the memory 22, the hard disk device 23, the network interface 24, the audio interface 25, and the acoustic signal generation unit 26 illustrated in FIG. 14 are connected to each other via a bus. The computer device 20 is connected to the network NW via the network interface 24, and can exchange data with each of the server device SV and the terminal device UE via the network. The computer device 20 is connected via an audio interface 25 to a plurality of speakers SPK and microphones MCL and MCR attached to both ears of the person Q1.

  In FIG. 14, a processor 21, a memory 22, a hard disk device 23, a network interface 24, and an audio interface 25 are included in the sound processing device 10. The processor 21, the memory 22, the acoustic signal generation unit 26, and the audio interface 25 are included in the measurement device EQ.

  The memory 22 shown in FIG. 14 stores the operating system of the computer device 20. Furthermore, the memory 22 stores an application program for the processor 21 to execute the acoustic processing shown in FIG. Further, the memory 22 further stores an application program for executing the measurement process for measuring the individual head impulse response described with reference to FIG. Note that the application program for executing the acoustic processing and the application program for executing the measurement processing shown in FIG. 6 can be recorded and distributed on a storage medium such as an optical disc, or can be distributed via the network NW. Can also be distributed. For example, the application program for acoustic processing and the application program for measurement processing illustrated in FIG. 6 may be downloaded from the server apparatus SV via the network interface 24. The downloaded application program is stored in the memory 22 or the hard disk device 23, so that it can be executed by the processor 21. Note that the application program for acoustic processing desirably includes information indicating a common head impulse response measured using a dummy head or the like.

  The processor 21 performs the functions of the prediction unit 11 and the correction unit 12 illustrated in FIG. 1 by executing an application program for acoustic processing stored in the memory 22. Further, the processor 21 controls the operations of the acoustic signal generation unit 26 and the audio interface 25 based on an application program for measurement processing stored in the memory 22, so that the function of the measurement device EQ illustrated in FIG. Fulfill. The function of the acoustic signal generation unit 26 included in the measurement device EQ and the operation of the measurement device EQ will be described later with reference to FIG.

  The terminal device UE illustrated in FIG. 14 includes a processor 31, a memory 32, a network interface 33, an acoustic processing unit SP, and a short-range wireless communication interface 34. The processor 31, the memory 32, the network interface 33, the acoustic processing unit SP, and the short-range wireless communication interface 34 illustrated in FIG. 14 are connected to each other via a bus. The terminal device UE is connected to the network NW via the network interface 33, and can exchange data with each of the server device SV and the sound processing device 10 via the network NW. The terminal device UE is connected to the position detection device HMD attached to the head of the person Q1 'via the short-range wireless communication interface 34. In addition, the acoustic processing unit SP is connected to each of the earphones EPL and EPR attached to both ears of the person Q1 '. The sound signal generated by the sound processing unit SP is output as sound by the earphones EPL and EPR, and the sound output by the earphones EPL and EPR is heard by the person Q1 '. The person Q1 'shown in FIG. 14 is the same person as the person Q1 whose individual head impulse response has been measured by the measuring device EQ. The hardware configuration of the position detection device HMD will be described later with reference to FIG.

  In the terminal device UE illustrated in FIG. 14, the processor 31, the memory 32, the network interface 33, the acoustic processing unit SP, and the short-range wireless communication interface 34 are included in the acoustic AR device ARC.

  The memory 32 illustrated in FIG. 14 includes an application program for the processor 31 to execute the acoustic AR process for providing the service using the sound image localization technology to the person Q1 ′ together with the operating system of the terminal device UE. Storing. The application program for executing the acoustic AR process can be recorded and distributed on a storage medium such as a memory card, or can be distributed via the network NW. For example, an application program for acoustic AR processing may be downloaded from the server device SV via the network interface 34. The downloaded application program is stored in the memory 32 so that the processor 31 can execute it.

  Then, the processor 31 performs the functions of the selection unit 132 and the orientation specifying unit DRD shown in FIG. 12 by executing an application program for the acoustic AR process stored in the memory 32.

  FIG. 15 shows a hardware configuration example of the position detection device HMD shown in FIG. Of the elements shown in FIG. 15, elements equivalent to those shown in FIG. 12 are denoted by the same reference numerals and description of the constituent elements may be omitted.

  The position detection device HMD shown in FIG. 15 includes a processor 41, a short-range wireless communication interface 42, an infrared sensor 43, a gyro sensor 44, and an acceleration sensor 45. In the position detection device HMD shown in FIG. 15, the processor 41 is connected to each of the short-range wireless transmission / reception unit 42, the infrared sensor 43, the gyro sensor 44, and the acceleration sensor 45. The position detection device HMD is connected to the terminal device UE via the short-range wireless communication interface 42. The infrared sensor 43 has a function of receiving identification information indicated by infrared rays emitted from the signs Anc1 and Anc2 shown in FIG.

  The memory built in the processor 41 detects the position of the head of the person Q1 ′ and the front direction Dir (FIG. 13) of the person Q1 ′ based on the measurement results obtained by the gyro sensor 44 and the acceleration sensor 45, respectively. A program for position detection processing is stored. Further, the processor 41 detects the position and orientation of the head of the person Q1 ′ by executing a program for position detection processing stored in the built-in memory, and the detected position and orientation are short-range wireless communication. It transmits to the terminal device UE through the interface 42. The operation of the position detection device HMD will be described later with reference to FIG.

  FIG. 16 shows the operation of the measurement apparatus EQ shown in FIG. Each process of step S321 to step S326 illustrated in FIG. 16 is an example of a process included in the application program for the measurement process stored in the memory 22 illustrated in FIG. In addition, each processing of step S321 to step S326 is executed by the processor 21 of the computer apparatus 20 shown in FIG.

  The example of FIG. 16 divides the sector-shaped inner angle indicating the measurement range R in FIG. 2 into n (n is an integer of 2 or more), and a plurality of first directions set inside each of the divided measurement ranges. An example of a technique for sequentially measuring individual head impulse responses will be described. In this case, each of the plurality of speakers SPK shown in FIG. 14 has, for example, a fan-shaped arc with a central angle 2φ / n centered on the rotation center of the rotatable chair on which the person Q1 is seated (m is 2). It is installed at the position where it is equally divided. In the example of FIG. 14, illustration of a chair on which the person Q1 is seated and a mechanism for rotating the chair by the processor 21 are omitted.

  Then, the processor 21 rotates the chair on which the person Q1 is seated to match the arc in which the speaker SPK is arranged with the fan-shaped arc corresponding to the selection range that is one of the divided measurement ranges. Then, the processing described below is started. In this case, the direction of the line segment that connects each of the m speakers SPK and the rotation center of the chair indicates each of the m first directions set inside the selection range.

  In step S321 illustrated in FIG. 16, the processor 21 instructs the acoustic signal generation unit 26 to generate a TSP signal, thereby causing each of the plurality of speakers SPK to sequentially output the sound corresponding to the TSP signal. The TSP signal generated by the acoustic signal generation unit 26 is sequentially delivered to each of the plurality of speakers SPK via the audio interface 25, for example. The speaker SPK that has received the TSP signal from the audio interface 25 outputs sound corresponding to the TSP signal.

  In step S322, the processor 21 receives an acoustic signal generated by the microphones MCL and MCR when the sound output in the process of step S321 reaches the head of the person Q1, and receives the received acoustic signal in the memory 22 or the hard disk. It is held in the device 23. Acoustic signals obtained by the microphones MCL and MCR are passed to the processor 21 via the audio interface 25. In step S321, the processor 21 holds, in the memory 22 or the like, acoustic signals received from the microphones MCL and MCR via the audio interface 25 during a period until a predetermined time elapses from the time when the generation of the TSP signal is instructed. Let Here, the processor 21 receives the acoustic signal received from the microphones MCL and MCR as the sound from the first direction corresponding to the speaker SPK that outputs the sound among the plurality of first directions set inside the selection range. It is stored in the memory 22 or the like.

  In step S323, the processor 21 determines whether or not the measurement of the selection range is completed based on whether or not the acoustic signals indicating the sounds received from all the first directions set inside the selection range are held. judge.

  If there is a first direction that does not yet hold an acoustic signal among the plurality of first directions set inside the selection range, the processor 21 performs step S321 according to the negative determination route (NO) of step S323. Return to the process. In this case, the processor 21 causes the speaker SPK corresponding to the new first direction to output the sound corresponding to the TSP signal in step S321.

  When the output of the sound for all the first directions set inside the selection range is completed by repeating the processes in steps S321 to S323 (Yes determination in step S323 (YES)), the processor 21 The process proceeds to step S324.

  In step S324, the processor 21 determines whether or not the measurement for all of the divided measurement ranges has been completed, that is, whether or not the measurement for the measurement range R illustrated in FIG. 2 has been completed.

  If the measurement for any of the divided measurement ranges has not yet been completed (No at Step S324 (NO)), the processor 21 proceeds to the process at Step S325.

  In step S325, the processor 21 performs a positioning process for measuring the selected range using the divided measurement range for which measurement has not been completed among the n divided measurement ranges as the selection range. The processor 21 changes the relative position between the head of the person Q1 and the plurality of speakers SPK, for example, by rotating the chair on which the person Q1 is seated, and newly creates a range adjacent to the selected range where the measurement has been completed. Select a suitable range. Here, the processor 21 indicates that the angle at which the processor 21 rotates the chair each time measurement for one of the divided measurement ranges is completed is, for example, an angle obtained by dividing the angle 2φ illustrated in FIG. 2 into n equal parts. It is.

  On the other hand, when it is determined that the measurement for all of the divided measurement ranges has been completed (Yes in step S324 (YES)), the processor 21 proceeds to the process of step S326.

  In step S326, the processor 21 obtains an individual head impulse response for each of both ears of the person Q1 based on the acoustic signal stored in the memory 22 or the like corresponding to each of the first directions. Further, the processor 21 stores information indicating the individual head impulse response obtained in each of the first directions in the memory 22 or the hard disk device 23. Information indicating the individual head impulse response stored in the memory 22 or the hard disk device 23 is used when the application program for the acoustic processing shown in FIG. 6 is executed.

  Note that the method of obtaining the individual head impulse responses in the first direction is not limited to the method of obtaining all at once in step S326 illustrated in FIG. For example, every time the processor 21 acquires the acoustic signal corresponding to each of the first directions in the process of step S322, the processor 21 may obtain the individual head impulse response for the first direction from the acquired acoustic signals.

  After the measurement process described above is completed, the processor 21 executes the application program for the acoustic process shown in FIG. In the process of executing the application program for acoustic processing, the processor 21 performs, for example, the individual head impulse response obtained by the measurement processing for each first direction as described with reference to FIGS. Thus, the delay time for each second direction is predicted. Then, the processor 21 corrects the delay time of the common head impulse response held in the memory 22 or the hard disk device 23 corresponding to each second direction, using the predicted delay time. Thereafter, the processor 21 sends information indicating the individual head impulse response obtained for each first direction by the measurement process and information indicating the corrected common head impulse response obtained for each second direction to the network NW. To the terminal device UE and stored in the memory 32. As described above, in the process of executing the application program for acoustic processing, the information stored in the memory 32 of the terminal device UE is stored in the application program for acoustic AR processing by the processor 31 of the terminal device UE. Used when is executed.

  FIG. 17 shows the operation of the acoustic AR device ARC shown in FIG. Each process of step S331 to step S337 illustrated in FIG. 17 is an example of a process included in the application program for the acoustic AR process. In addition, each processing of step S331 to step S335 is performed by the processor 31 of the terminal device UE, for example, about several milliseconds to several tens of milliseconds after the person Q1 ′ enters the exhibition hall HL shown in FIG. It is executed every time a predetermined time set in the elapses.

  In step S331, the processor 31 detects the position and orientation of the head of the person Q1 ′ detected by the position detection device HMD and the one closest to the person Q1 ′ among the exhibits Exh1 and Exh2 shown in FIG. Information indicating For example, the processor 31 requests the position detection device HMD to transmit measurement results obtained by the gyro sensor 44, the acceleration sensor 45, and the infrared sensor 43 via the short-range wireless communication interface 35. The request from the processor 31 is passed to the processor 41 of the position detection device HMD via the short-range wireless communication interface 42 shown in FIG. The processor 41 receives the angular velocity measurement result from the gyro sensor 44 and the acceleration measurement result from the acceleration sensor 45 based on the request passed from the processor 31. Further, the processor 41 receives identification information indicated by infrared rays that reach the infrared sensor 43 from one of the signs Anc1 and Anc2 shown in FIG. Then, the processor 41 transmits the identification information obtained by the infrared sensor 43 together with the information indicating the measurement results of the angular velocity and acceleration to the terminal device UE via the short-range wireless communication interface 42. Information transmitted by the processor 41 of the position detection device HMD is passed to the processor 31 via the short-range wireless communication interface 35 of the terminal device UE. Then, the processor 31 calculates the position and orientation of the head of the person Q1 'based on the acceleration and angular velocity included in the received information. Further, the processor 31 uses the identification information included in the received information as information indicating one of the exhibits Exh1 and Exh2 that is closest to the person Q1 '. Note that the process of calculating the position and orientation of the head of the person Q1 'may be executed by the processor 41 of the position detection device HMD.

  In step S332, the processor 31 determines whether or not the identification information received in the process of step S331 has changed from the identification information received when the process shown in FIG. 17 was executed previously.

  When the process shown in FIG. 17 is executed for the first time or when identification information different from the previous one is received in the process of step S331, the processor 21 proceeds to the process of step S333 as an affirmative determination (YES) of step S332. On the other hand, when the identification information received in the process of step S331 and the previously received identification information are the same (No determination in step S332 (NO)), the processor 31 does not perform the process of step S333, but performs the step The process proceeds to S334.

  In step S333, the processor 31 makes an inquiry to the server apparatus SV shown in FIG. 14 based on the new identification information received in the process of step S331, so that the position of the exhibit indicated by the identification information is guided by voice. Is acquired as the position of the target to be provided.

  In step S334, the processor 31 determines the guidance target based on the orientation of the head of the person Q1 ′ based on the position of the exhibit to be guided and the position and orientation of the head of the person Q1 ′. Calculate the direction of the exhibit. For example, based on the information indicating the position of the exhibit Exh1 and the information indicating the position and orientation of the head of the person Q1, the processor 31 performs the head direction Dir of the person Q1 and the head of the person Q1 shown in FIG. And the angle θ at which the line connecting the exhibition Exh1 intersects is calculated. Then, the processor 31 passes information indicating the calculated angle θ to the sound processing unit SP illustrated in FIG. 14 as information indicating the direction of the sound image to be localized with reference to the front direction Dir of the head of the person Q1 ′. .

  In step S335, the processor 31 receives from the server device SV a part of the audio information stored in the audio database DB1 corresponding to the exhibit to be guided. For example, every time the processing shown in FIG. 17 is executed, the processor 31 sequentially receives audio information divided for each amount to be reproduced at a time equivalent to the time interval, and receives the received audio information as an audio processing unit SP. To pass.

  In step S336, the processor 31 compares the head impulse response for each ear corresponding to the direction indicated by the information received in step S334 and the acoustic signal generated for each ear from the voice information received in step S335. The speech processing unit SP is caused to execute the convolution process. For example, when the angle θ calculated in the process of step S334 indicates one of the first directions, the sound processing unit SP stores each ear held in the memory 32 corresponding to the first direction indicated by the angle θ. The convolution processing of the individual head impulse response and the acoustic signal is executed. On the other hand, when the angle θ calculated in the process of step S334 indicates one of the second directions, the sound processing unit SP performs the correction after being stored in the memory 32 corresponding to the second direction indicated by the angle θ. The convolution processing of the common head impulse response and the acoustic signal is executed.

  In step S337, the processor 31 outputs the acoustic signal generated for each of both ears of the person Q1 in the process of step S336 from the acoustic processing unit SP via the earphones EPL and EPR, and causes the person Q1 to listen.

  As described above, the processor 31 of the terminal device UE illustrated in FIG. 14 executes the processing of steps S331 to S337 at predetermined time intervals, thereby realizing the acoustic AR device ARC illustrated in FIG. be able to. That is, the acoustic AR device ARC shown in FIG. 14 uses the individual head impulse response or the corrected common head impulse response stored in the memory 32 of the terminal device UE corresponding to each direction by the acoustic processing device 10. Sound image localization processing.

  As a result, the acoustic AR device ARC corresponds to the exhibit Exh1, for example, as sound from the direction of the exhibit Exh1 based on the head direction Dir of the person Q1 moving in the exhibition hall HL shown in FIG. The person Q1 can listen to the sound generated from the sound information. That is, the acoustic AR device ARC shown in FIG. 14 provides a guidance service for providing information by voice explaining the exhibits Exh1, Exh2, etc. as a service using the sound image localization technology for the person Q1 moving in the exhibition hall HL. Can be realized.

  From the above detailed description, features and advantages of the embodiment will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and changes. Therefore, there is no intention to limit the scope of the inventive embodiments to those described above, and appropriate modifications and equivalents included in the scope disclosed in the embodiments can be used.

Regarding the above description, the following items are further disclosed.
(Supplementary note 1) Based on the impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head, A prediction unit that predicts a delay time of an impulse response when sound reaches the head from two directions;
A correction unit that corrects a delay time of a reference impulse response modeled in advance with respect to the sound from the second direction in accordance with the delay time predicted by the prediction unit;
An acoustic processing apparatus comprising:
(Supplementary note 2) In the sound processing apparatus according to supplementary note 1,
A generation unit that generates sound for localizing the sound image in the second direction using a reference impulse response whose delay time is corrected by the correction unit;
An acoustic processing apparatus comprising:
(Supplementary Note 3) In the sound processing apparatus according to Supplementary Note 1 or Supplementary Note 2,
The prediction unit
As the positional relationship between the head and the sound source installed in the first direction when measuring the impulse response, the positional relationship in which the delay time of the sound reaching from the sound source becomes the delay time of the measured impulse response A specific part to identify;
A sound processing apparatus comprising: a calculation unit configured to calculate a delay time predicted when sound reaches the head from the second direction based on the positional relationship specified by the specifying unit.
(Supplementary Note 4) In the sound processing apparatus according to Supplementary Note 3,
The specific part is:
For identifying the positional relationship, a weight for a delay time of an impulse response measured when sound reaches the head from a first direction close to a boundary of the predetermined range among the plurality of first directions, A sound processing apparatus using a weight larger than a weight for a delay time of an impulse response measured when sound reaches the head from a first direction away from a boundary.
(Supplementary note 5) In the sound processing device according to supplementary note 3,
The specific part is:
By performing a regression analysis in which each of the delay times of the impulse response obtained by measurement in a plurality of first directions including the first direction close to the boundary is set to have a greater weight closer to the boundary, the position A sound processing apparatus characterized by specifying a relationship.
(Appendix 6) Based on the impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head, Predict the delay time of the impulse response when sound reaches the head from two directions,
Correcting the delay time of the reference impulse response modeled in advance for the sound from the second direction in accordance with the delay time predicted by the prediction unit;
The acoustic processing method characterized by the above-mentioned.
(Appendix 7) Based on the impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head, Predict the delay time of the impulse response when sound reaches the head from two directions,
Correcting the delay time of the reference impulse response modeled in advance for the sound from the second direction in accordance with the delay time predicted by the prediction unit;
An acoustic processing program for causing a computer to execute processing.

DESCRIPTION OF SYMBOLS 10 ... Sound processing apparatus; 11 ... Prediction part 11 ... Correction part; 13 ... Generation part; 111, 111a ... Identification part; 112 ... Calculation part; 131 ... Setting part; 132 ... Selection part; , 41 ... Processor; 22, 32 ... Memory; 23 ... Hard disk device; 24, 33 ... Network interface; 25 ... Audio interface; 26 ... Acoustic signal generator; 35, 42 ... Short-range wireless communication interface; 44 ... gyro sensor; 45 ... acceleration sensor; EQ ... measuring device; SD ... storage device; ARC ... acoustic AR (Augmented Reality) device; MEM ... storage unit; CNT ... control unit; DB1 ... voice database; Exhibit database; UE ... Terminal device; SP ... Acoustic processing unit; SV ... Server device; DRD ... Direction specifying unit; CL, MCR ... microphone; EPL, EPR ... earphone; HMD ... position detection device; NW ... network; Q1 ... person; Exh1, Exh2 ... exhibit; Anc1, Anc2 ... sign; SPK ... speaker

Claims (5)

Based on an impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head, from the second direction outside the predetermined range, A prediction unit for predicting a delay time of an impulse response when sound reaches the head;
The delay time of the impulse response of the pre-modeled criteria for sound from the second direction, and a correcting unit for correcting in accordance with the delay time predicted by the prediction unit,
The prediction unit
As the positional relationship between the head and the sound source installed in the first direction when measuring the impulse response, the positional relationship in which the delay time of the sound reaching from the sound source becomes the delay time of the measured impulse response A specific part to identify;
A calculating unit that calculates a delay time predicted when sound reaches the head from the second direction based on the positional relationship specified by the specifying unit.
A sound processing apparatus. The sound processing apparatus according to claim 1,
Using the impulse response of the reference delay time is corrected by the correction unit, Ru includes a generator for generating a sound to localize a sound image in the second direction
A sound processing apparatus. In the sound processing apparatus according to claim 1 or 2 ,
The specific part is:
For identifying the positional relationship, a weight for a delay time of an impulse response measured when sound reaches the head from a first direction close to a boundary of the predetermined range among the plurality of first directions, A sound processing apparatus using a weight larger than a weight for a delay time of an impulse response measured when sound reaches the head from a first direction away from a boundary. Based on an impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head, from the second direction outside the predetermined range, A prediction step for predicting a delay time of an impulse response when sound reaches the head;
A correction step of correcting a delay time of a reference impulse response pre-modeled with respect to the sound from the second direction in accordance with the delay time predicted in the prediction step ,
The prediction step includes
As the positional relationship between the head and the sound source installed in the first direction when measuring the impulse response, the positional relationship in which the delay time of the sound reaching from the sound source becomes the delay time of the measured impulse response A specific process to identify;
A calculation step of calculating a delay time that is predicted when sound reaches the head from the second direction based on the positional relationship specified by the specifying step. . Based on an impulse response measured when sound reaches the head from each of a plurality of first directions within a predetermined range in the forward direction of the head, from the second direction outside the predetermined range, A prediction step for predicting a delay time of an impulse response when sound reaches the head;
Causing the computer to execute a process including a correction step of correcting a delay time of a reference impulse response modeled in advance with respect to the sound from the second direction in accordance with the delay time predicted in the prediction step. ,
The prediction step includes
As the positional relationship between the head and the sound source installed in the first direction when measuring the impulse response, the positional relationship in which the delay time of the sound reaching from the sound source becomes the delay time of the measured impulse response A specific process to identify;
A calculation step of calculating a delay time predicted when sound reaches the head from the second direction based on the positional relationship specified by the specifying step.
An acoustic processing program characterized by that.

Images