JPH0223878B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a voice detection method for detecting human voice.

With the development of computers, voice input devices have been proposed. This voice input device stores the characteristics of a voice wave in advance in a voice storage section, and recognizes the input voice by comparing the characteristics of the newly input voice wave with the stored values in the voice storage section.

However, conventional speech recognition is not fully satisfactory in terms of recognition rate and recognition time, because it identifies and recognizes phonemes even in speech waves in which nasal and oral cavity outputs are superimposed.

By the way, when producing a voiced sound, the vocal cord vibration waves are divided by the velum into those directed to the nasal cavity and those directed to the oral cavity, and are output from the nostrils and lips after being modified by transmission characteristics based on the differences in the shapes of the nasal cavity and oral cavity. be done.
At this time, the shape of the nasal cavity does not change, but the velum of the mouth moves, and the shape of the oral cavity changes due to the movement of teeth, tongue, etc.
Lips open and close. Therefore, the vocal cord vibration waves generated from the vocal cords and divided into the oral cavity and nasal cavity are modified in various ways depending on the shape of the passage from the nostrils to the lips, so the output waveforms from both nostrils and lips are distinct. will be different.

Furthermore, even when producing an unvoiced sound, the nasal cavity output is extremely small, only the oral cavity output exists, and the output waveforms of both are clearly different.

This invention was made by focusing on the fact that the sound waves emitted from the oral cavity and the nasal cavity exhibit clearly different waveforms, and aims to make recognition of phonemes extremely easy.

In order to achieve this objective, the sound wave detection method of the present invention detects the sound output from the oral cavity at the same time as the sound output from the nasal cavity, and calculates the difference in rise time between the nasal cavity output energy curve and the oral cavity output energy curve. 1, and the second feature is the ratio of the slope of the normalized nasal output energy curve and the normalized oral cavity output energy curve at the rise of the oral cavity output energy curve.
The S plane and the ratio of the nasal cavity output energy to the pharyngeal cavity passage energy at the rise of the pharyngeal cavity passage energy curve are the first feature quantity, and the value of the maximum slope of the time change curve of the ratio during phonation is the second feature. The first feature quantity is the O-Δ plane for discrimination, which is the quantity, and the noise-removed differential zero-crossing number obtained by removing the differential zero-crossing number of silent sections in the oral cavity output audio signal, and the noise-removed differential zero-crossing number and the nasal cavity output. Create a discrimination N-D plane with the rise time difference of the energy curve as the second feature, and based on the distribution of the emitted audio waves on each of these discrimination planes, (step 1) O-Δ plane Separate (/N/) on the top, (step 2) On the N-D plane (/S-line phoneme/),
(/affricate/), (/fricative/), (/the phoneme of the line/), (/the phoneme of the line M/, /the phoneme of the line N/, /
Separate each phoneme and phoneme group (phoneme in the G line /, phoneme in the /DA line /, / phoneme in the B line /), and (step 3) on the O-Δ plane (phoneme in the /Y line/, /RA line). Separate the phoneme group /, phoneme of /Wa line/) on the D-S plane (phoneme of /Ka line/, /plosive phoneme/ of /Ta line, /phoneme of /P line/)
Separate the phoneme group of (step 5) On the N-D plane (/phoneme in row A/)
The above phoneme groups or phonemes are classified by a series of steps consisting of the above steps in which phonemes are identified.

In the audio wave detection method according to the present invention configured as described above, first, a preliminary experiment based on the vocalizations of an unspecified number of people is carried out to determine whether the D-S plane, the O
- The Δ plane and the ND plane are determined, and the utterances of an unspecified person are classified into phoneme groups or phonemes based on the planes, and the speech is recognized.

Therefore, according to the present invention, the voice is recognized based on the two clearly different waveforms of the nasal cavity and oral cavity outputs, so that voice recognition is extremely accurate and easy. , when processing by computer, the recognition rate, recognition time, and storage capacity are significantly improved.

Embodiments of the present invention will be described below with reference to the accompanying drawings.

As shown in FIG. 1, from a head arm 2 fixed to a human head 1, an arm 4 supporting microphones 3m and 3n at its tip extends in front of the face, the position of which is adjustable. The microphone 3n faces the nostril to detect the nasal cavity output of sound, and the microphone 3m faces the mouth to detect the oral cavity output of sound. Note that it is desirable that either one of the microphones 3n and 3m is provided so that its position can be adjusted. Further, it is desirable to provide a shielding plate 5 between the microphones 3m and 3n to separate the nasal cavity output and the oral cavity output, so that both outputs are inputted to each microphone 3n and 3m without being mixed.

The signals from each of the microphones 3m and 3n are inputted to an AD converter 7 via a switch 6 such as a multiplexer, as shown in FIG. 2, converted into digital signals, and inputted to a computer 8. The computer 8 performs recognition processing of the emitted voice based on the outputs of both the microphones 3m and 3n.

This recognition process can be performed by various means, for example, as follows.

This means uses the energy curves N ^E m and N ^E of the audio output detected by the microphones 3 m and 3 n
Based on n, the following D, S, (En/Eo) _p ′, Δ
Calculate the parameters (En/Eo), CR, and DT,
Depending on each parameter, the D-S plane, (En/Eo) _p −
The Δ(En/Eo) plane (0-Δ plane) and the CR-DT plane (ND plane) are obtained, and vocalizations are identified based on these planes.

D: Rise time difference between N ^E n and N ^E m S: Slope ratio at the rise of N ^E n and N ^E m En: Energy passing through the nasal cavity Eo: Energy passing through the pharyngeal cavity (En/Eo) _p : En/Eo Value at the starting point of the curve Δ(En/Eo): Maximum slope value of the En/Eo curve CR: Noise removal difference zero crossing number DT: Noise removal difference zero crossing number and the rise time difference of N ^E n Next, each of the above planes This section describes the creation of ``phoneme'' and the phonological identification based on it.

(i) D-S plane Figure 3 a-l shows the sounds (a) /a/, (f) /ka/,
(sa) /sa/, (ta) /ta/, (na) /na/,
(ha) /ha/, (ma) /ma/, (ya) /ya/,
(la) /ta/, (wa) /wa/, (pa) /pa/,
These are the energy curves N ^E m and N ^E n obtained by detecting (n)/N/ with microphones 3m and 3n, and are energy time change curves obtained by normalizing the observed acoustic wave energy by the maximum value. be.

In this energy curve, /a/, /
In ha/, N ^E n and N ^E m rise simultaneously, and N ^E n and N ^E m change in the same way during utterance. /
In ka/, /ta/, /pa/, a peak due to burst airflow appears at the rise of N ^E m, and N ^E m is N ^E n
I'm getting up faster. In /sa/, a small value (arrow) appears in N ^E m in the /s/ interval. In /na/, /ma/, N ^E n rises earlier than N ^E m, and N ^E n starts to increase at the same time as N ^E m starts increasing.
starts to decrease. For /ya/, /ra/, and /wa/, N ^E n and N ^E m rise almost simultaneously, but the slope at the time of rise is larger for N ^E n than N ^E m. The oral cavity output is extremely small at /N/, and the energy curve of indoor noise appears at N ^E m.

The delay time D and slope ratio S defined by the following equations (1) and (2) are calculated as parameters representing the characteristics of the energy curve of each phoneme.

D=t _op −t _np ………(1) S=N ^E n (t _n3 ) − N ^E n (t _np )/N ^E m (t _n3 ) − N ^E
m (t _np )......(2) However, t _op and t _np are the times when N ^E n and N ^E m exceed the 5% point of their maximum values for the first time, and t _n3 is an arbitrary time from t _np , e.g. , 19.2 msec later. formula
(1) is the rise time difference between N ^E n and N ^E m, and equation (2) is N ^E
It represents the ratio of the slope of N ^E n and N ^E m at the time of rise of m.

Figure 4 shows the two parameters of 10 types of monosyllables excluding /sa/ and /N/ in Figure 3 calculated and plotted on the D-S plane with the initial letters of the vocalizations, such as /ta/.
is plotted with T, and the audio samples are isolated vocalizations by 10 men. In addition, /sa/
In the /s/ interval, the variation in the N ^E m value was large, making the detection of t _np unstable, and in the /N/ interval, the oral cavity output was extremely small, so t _np could not be determined, so these were excluded. In the drawing,
When S>3.0, it is plotted at the position of S=3.0.

According to this figure, the delay time D of /a/ and /ha/ is small, and the slope ratio S is distributed around 1.0. For /ka/, /tm/, /pa/, D>0 and S
is small. . Also, for /na/ and /ma/, D<0,
S<0. For /ya/, /ra/, and /wa/, D is small and S is larger than other phoneme groups. In this phoneme, there are speech samples in which N ^E m is slightly later than the rising edge of N ^E n (D<0), but these are
The outline shapes of N ^E n and N ^E m were almost the same as those in Figure 3, but the first peak of the N ^E n curve was earlier;
Since the slope of N ^E n is calculated near the top of the peak, it is thought that the slope ratio S becomes slightly smaller (arrow in Figure 4).

From the above, it can be understood that by determining D and S, each phoneme can be classified into several types of groups.

( _ii ) O _- Δ plane As _shown in FIG. , microphone 3
The energy observed by n, 3m is E _o (in) (t),
When E _n (in) (t), first, the observed value E _o (in)
(t), E _n (in) (t) to estimate the time change curve of E _o (t)/E _p (t).

Assuming that energy is lossless within the vocal tract, equation (3) holds true.

E _p (t) = E _o (t) + E _n (t) ...... (3) Also, if C _o and C _n are the proportions of the radiated energy that enter each microphone, then E _o (in) (t )=C _o E _o (t) E _n (in) (t)=C _n E _n (t) ......(4) From equations (3) and (4), C _o E _p (t) = E _o (in) (t) + (C _o /C _n ) E _n (in) (t) ......(5) is obtained.

Here, calculation of C _o /C _n is a problem, but for example, one microphone is placed in the opening at one end of the cylinder, and the opening at the other end covers the mouth and nose to detect E _p and the first microphone. Detect E _n and E _o by the means shown in the figure, and E _p
It is sufficient to calculate C _o and C _n such that =E _n +E _o . _Cn ,
Since C _o changes depending on the position of microphone 3m and 3n,
It is better to fix E _p , E _n , and E _o and compare the average values of multiple times.

Based on C _o /C _n thus obtained, the following equation (6) is obtained.

E _o /E _p = E _o (t) / E _p (t) = E _o (in) (t) / E _o (in) (t) + (C _o /C _n ) E _n (in
)(t) ......(6) Figure 5 shows the above equation for the vocal sound /a/...
The time change curve of E _o /E _p in (6) is shown.

The following equation is defined as a parameter representing the characteristics of this E _o /E _p curve.

(E _o / E _p ) _p = E _o (to) / E _p (to) ………(7) Δ (E _o / E _p ) = Max [E _o (t) / E _p (t) −E _o (t)′/E _p (t)′] ………(8) However, Max means the maximum value within the hook. Further, t _p is the time when E _p exceeds the 15% point of the maximum value for the first time, and t' is the time after a certain time, for example, 19.2 msec, from t _p .
Equation (7) calculates the leftmost value of the E _o /E _p curve, and Equation (8)
represents the maximum slope of the curve. FIG. 6 shows the above two parameters of the 12 types of monosyllables uttered in isolation by the 10 men shown in FIG. 5, plotted on the O-Δ plane using the initial letters of the utterances.
(In Figure 6, there are phonemes plotted for only 5 phonetic samples to make the diagram easier to read, but the other 5 phonemes have a similar distribution.) According to this figure, 12 types of monosyllables (/ a/,/
ka/, /sa/, /ta/, /ha/, /pa/),
(/na/, /ma/), (/ya/, /ra/, /
It can be classified into four groups: wa/) and (/N/).

Note that /∫a/, / on this plan view
Although za/, /ga/, /da/, and /ba/ are not treated, the distribution of /∫a/ is the same as /sa/, and /
The distribution of za/, /ga/, /da/, /ba/ is /
This is because although it is similar to na/, the distribution is wide and difficult to classify.

(iii) N-D plane This plane is used for fricatives such as /s/, /z/, /
It identifies affricates such as t∫i/ and /tsu/. First, one of its parameters, the number of noise rejected differential zero crossovers,
This section describes differential zero crossing rate.

For example, in the oral cavity output speech waveform of /su/ shown in FIG. 7, when the oral cavity output speech signal at a certain point is {xi}, the noise removal difference zero crossing number is
CR is defined by the following equation (9).

{(xi+1−xi)(xi−xi−1)}<0 and {|
If xi + 1 | threshold or | xi | > threshold or | xi − 1 | > threshold, then sample point i
Assuming that there is one noise removal difference zero crossing in , this is the sum of these zero crossings within a certain interval.

(9) In Figure 8 a to f, /u/, /su/, /
CR of vocalizations of zu/, /tsu/, /hu/, /nu/
shows the relationship between and time.

Next, as another parameter, the following equation (10)
The ^rise time difference DT (Delay
time).

Delay time=t _op −z _pHowever , when Delay time<0, Delay time=t _op −t _np (10) Here, the rising times of NEn and NEm are t _op and
t _np , the rise of the noise removal difference zero crossing number is defined as the time z _p when the number of crossings exceeds 9 times for the first time. FIGS. 8a to 8f show normalized time change curves N ^E n and N ^E m of nasal cavity and oral cavity output energy.

Figure 9 shows the noise removal differential zero crossing number [NR-D.
ZCR] (CR) is the noise removal difference zero crossing number on the vertical axis.
Each monosyllable is plotted by the initial letter of the uttered sound for each subsequent vowel on a plane (ND plane) with the delay time between CR and N ^E n as the horizontal axis.

According to this diagram, the monosyllables /s/, /∫/, /
z/, /h/, (/n/, /m/, /g/, /
d/, /b/) and vowels.

In this figure, /ka/, /ta/, /
Although pa/, /ya/, /ra/, /wa/, and /N/ are not treated, the distribution of /ka/, /ta/, and /pa/ is between /a/ and /ha/. , /ya／, /
The distribution of ra/ and /wa/ is the same as that of /a/, which is inconvenient for classification, and /N/ cannot be plotted on the ND plane since there is no oral output.

The method for creating each plan view has been described above, and next, an example of an algorithm for phoneme recognition using these planes will be shown.

An example of this algorithm is the phonemes /a/, /
ka/, /sa/, /ta/, /na/, /ha/, /
ma/, /ya/, /ra/, wa/, /pa/, /
∫a/, /za/, /ga/, /da/, /ba/, /
This is to identify N/, and is carried out according to the identification flowchart shown in FIG.

In step 1, /N/ is separated and identified on the O-Δ plane. This is because the distribution of /N/ on the O-Δ plane is very prominent, and it is appropriate to first separate it from other phonemes.

In step 2, /sa/, /∫a/, / on the N-D plane
za/, (/ma/, /na/, /ga/, /da/, /
Separate and identify the four groups of ba/).

In step 3, on the O-Δ plane again (/ya/, /
ra/, /wa/) is separated and identified.

In step 4, on the D-S plane (/a/, /ha/)
and (/ka/, /ta/, /pa/) are separated.

In step 5, /a/ and /ha/ are again on the N-D plane.
Identification is performed.

The 17 types of phonemes are classified into 9 groups through the above 5 steps of processing. The reason for this five-stage configuration is that a method is used to first separate and identify phonemes that are significantly separated from other phoneme groups on each plane.

Above, the vowel and the following vowel were /a/, that is, the phonetic identification of the A vowel string (a, ka, sa, ta...), but the vowel /a/ was replaced with the vowel /e/ or /o/.
By replacing the following vowel /a/ with the following vowel /e/ or /o/, the phoneme groups or phonemes of the E vowel string and the O vowel string can be classified in the same way. Also, the vowel /a/ can be replaced with the vowel /i/ or /.
Replace the following vowel /a/ with the following vowel /
If the affricate /t∫i/ and /tsu/ are replaced with i/ or /u/ and the phonemes of /t∫i/ and /tsu/ are separated in step 2, each phoneme group or phoneme of the i vowel string and the u vowel string can be classified in the same way. be able to. It can be confirmed from FIG. 9 that this affricate can be classified (in the figure, T is an affricate, b/t∫i/ in the figure, and /tsu/ in c in the figure).

Within each phoneme group classified into multiple groups in this way, each phoneme is identified using a conventionally well-known recognition method, such as a recognition method based on the centroid frequency, peak frequency, and valley frequency of the spectrum and their temporal changes. Identify the phonology and make the final judgment.

In the above embodiment, a shielding plate was provided to separate the nasal cavity output and the oral cavity output, but in the case where the shielding plate is not provided, the energy curves N ^E m and N ^E n as shown in FIG. 11, for example. , connect the 35% point and 15% point of the maximum value with a straight line, and make corrections such as setting the intersection of this straight line and the time axis as t _op , (t _np ), and calculate D-S.
What is necessary is to create a plane, an O-Δ plane, and an N-D plane.

In the above-mentioned speech recognition, more accurate identification can be achieved by providing a detector using a camera or the like that detects mouth movements and performing detection in combination with this detector and the detection method of the present invention.

[Brief explanation of drawings]

FIG. 1 is an explanatory diagram showing an example of this invention, FIG. 2 is a control block diagram using this invention, and FIG. 3 a
~l is a graph showing time and energy distribution, the fourth
The figure is a graph showing the D-S plane, and Figures a to l are
A graph showing the time change curve of E _o /E _p , Fig. 6 is a graph showing the O-Δ plane, Fig. 7 is an oral output speech waveform diagram of /su/, and Fig. 8 a to f are nasal cavity and oral cavity outputs. Graphs of the normalized energy time change curve and the time change curve of the noise removal difference zero crossover number of the oral cavity output, Figures 9a to 9e are graphs showing the N-D plane, and Figure 10 is a flow chart showing an example of speech recognition. Chart,
FIG. 11 is a graph showing an example of correction. 3m, 3n...Microphone, 4...Support arm.

Claims (1)

[Scope of Claims] 1. Detect the audio output from the oral cavity at the same time as the audio output from the nasal cavity, and set the rise time difference between the nasal cavity output energy curve and the oral cavity output energy curve as the first feature quantity, and set the difference in the rise time of the nasal cavity output energy curve and the oral cavity output energy curve Discrimination D- whose second feature is the ratio of the slopes of the normalized nasal output energy curve and the normalized oral cavity output energy curve at the time of rising.
The S plane and the ratio of the nasal cavity output energy to the pharyngeal cavity passage energy at the rise of the pharyngeal cavity passage energy curve are the first feature quantity, and the value of the maximum slope of the time change curve of the ratio during phonation is the second feature. The first feature quantity is the O-Δ plane for discrimination, which is the quantity, and the noise-removed differential zero-crossing number obtained by removing the differential zero-crossing number of silent sections in the oral cavity output audio signal, and the noise-removed differential zero-crossing number and the nasal cavity output. Create a discrimination N-D plane with the rise time difference of the energy curve as the second feature, and based on the distribution of the emitted audio waves on each of these discrimination planes, (step 1) O-Δ plane Separate (/N/) on the top, (step 2) On the N-D plane (/S-line phoneme/),
(/affricate/), (/fricative/), (/the phoneme of the line/), (/the phoneme of the line M/, /the phoneme of the line N/, /
Separate each phoneme and phoneme group (phoneme in the G line /, phoneme in the /DA line /, / phoneme in the B line /), and (step 3) on the O-Δ plane (phoneme in the /Y line/, /RA line). Separate the phoneme group /, phoneme of /Wa line/) on the D-S plane (phoneme of /Ka line/, /plosive phoneme/ of /Ta line, /phoneme of /P line/)
Separate the phoneme group of (step 5) On the N-D plane (/phoneme in row A/)
and (/phoneme in line C/) A speech wave detection method characterized in that each of the phoneme groups or phonemes is classified by a series of steps consisting of the steps described above, in which phonemes are identified.